Reading and writing to a JSON-like file

[Yincognito]

I have a large (over 5 MB) UTF-8 (in Notepad++) encoded "text" file that I would like to (selectively) read from and write to, while also quickly iterating over its "records" in Rainmeter. The file structure is similar to a JSON/Javascript one, like this (classes are more like "objects", but you get the idea):

Code: Select all

MasterClass
{
Class1 : ClassName1 {
 Field1: Value1
 Field2: Value2
}

Class2 : ClassName2 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
}

Class3 : ClassName3 {
 Field1: Value1
}

Class4 : ClassName4 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
 Field4: Value4
 Field5: Value5
}

Class5 : ClassName5 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
 Field4: Value4
}

}

Now I tried to work it out by using the standard Rainmeter tools, like reading the file with a WebParser measure, iterating through its records by means of dynamic regex quantifiers in String measures, and simulating editing records through InputText measures on top of regular String meters (I can share the code or the original file, but it isn't that important in this context, as what I'm looking after are alternatives, see below):

File Records Manipulation.jpg

The problems are that:
- the size of the file makes iterating through the records relatively slow (I mean, it's fast, but it's not instantaneous like, say, for this mini sample above), with around 236000 lines to parse and around 16000 "classes"/"objects" to deal with
- the particularities of Rainmeter, while enough for reading files and iterating through its records, makes writing the changes back to the same file a bit problematic, partly because of the default INI format, partly because of the overall size, and partly because using external tools like CMD, Powershell, VBScript or others have various drawbacks of their own, like speed, the use of regex, etc.

So, my question is, what alternatives exist to achieve this goal (reading such a file, iterating through its records, selectively change them, and writing the changes back to that file)? Are the already mentioned alternatives using external programs feasible for this scenario, or can it be done fast in Lua? If the latter, can you point me in the right direction or share a minimal script that can do it on the mini sample above (I would probably be able to read and write from/to file in Lua, but - preferably - placing those records in tree-like arrays/tables is a mistery for me at this moment)? Obviously, if that's the case, you can move the topic to the Lua section eventually.

P.S. If anyone wonders, this is about semi-automatically editing a decrypted savegame, instead of doing the operations manually in Notepad++.

[Yincognito]

Nevermind, I solved this using Lua and its very fast tables. It works really well, a bit of CPU spike at the beginning due to the large file, but once the table is populated there is 0 lag whether it's about reading, iterating or writing back the modified elements to the file.

For future reference (not that anyone else but me was interested in it or able to provide a response), assuming a file structure like this test.txt located in the skin's @Resources folder:

Code: Select all

Group1
{
Class1 : ClassName1 {
 Field1: Value1
 Field2: Value2
}

Class2 : ClassName2 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
}

Class3 : ClassName3 {
 Field1: Value1
}

Class4 : ClassName4 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
 Field4: Value4
 Field5: Value5
}

Class5 : ClassName5 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
 Field4: Value4
}

}

Group2
{
Class1 : ClassName1 {
 Field1: Value1
 Field2: Value2
 Field3: Value3
}

Class2 : ClassName2 {
 Field1: Value1
}

Class3 : ClassName3 {
 Field1: Value1
 Field2: Value2
}

}

and a script measure like this somewhere in the skin:

Code: Select all

[AutosaveGame]
Measure=Script
ScriptFile=#@#AutosaveGame.lua
File=#AutosavePath#test.txt
UpdateDivider=-1
OnUpdateAction=[&AutosaveGame:GetItems([#GroupIndex],[#ClassIndex],[#FieldIndex],[#ValueIndex])]...some measure and meter updates and a redraw...
DynamicVariables=1

then this AutosaveGame.lua file, placed in the @Resources folder of the skin will do the job:

Code: Select all

function Initialize()
end

function Update()
  ReadAutosave(SELF:GetOption('File'))
  return items
end

function ReadAutosave(inputfile)
  local file = assert(io.open(inputfile, 'r'), 'Unable to open ' .. inputfile)
  local contents = file:read('*a')
  file:close()
  local gi = 1
  items = {}
  for grouptext in contents:gmatch('.-%b{}') do
    local ci, groupname, groupbody = 1, grouptext:match('^%s*(.-)%s*{(.*)}%s*$')
    items[gi] = {groupname, {}}
    for classtext in groupbody:gmatch('.-%b{}') do
      local fi, classname, classbody = 1, classtext:match('^%s*(.-)%s*{(.*)}%s*$')
      items[gi][2][ci] = {classname, {}}
      for fieldtext in classbody:gmatch('[^\n]-:[^\n]*') do
        local vi, fieldname, fieldbody = 1, fieldtext:match('^%s*(.-)%s*:%s*(.*)%s*$')
        items[gi][2][ci][2][fi] = {fieldname, {}}
        for valuetext in fieldbody:gmatch('.-$') do
          local valuename, valuebody = valuetext:match('^%s*(.*)(.-)%s*$')
          items[gi][2][ci][2][fi][2][vi] = {valuename}
          vi = vi + 1
        end
        fi = fi + 1
      end
      ci = ci + 1
    end
    gi = gi + 1
  end
  return true
end

function WriteAutosave(outputfile)
  local lines = {}
  for gi = 1, #items do
    lines[#lines + 1] = ('%s\n{'):format(items[gi][1])
    for ci = 1, #items[gi][2] do
      lines[#lines + 1] = ('%s {'):format(items[gi][2][ci][1])
      for fi = 1, #items[gi][2][ci][2] do
        for vi = 1, #items[gi][2][ci][2][fi][2][1] do
          lines[#lines + 1] = (' %s: %s'):format(items[gi][2][ci][2][fi][1], items[gi][2][ci][2][fi][2][vi][1])
        end
      end
      lines[#lines+1] = ('%s'):format('}\n')
    end
    if gi == #items then lines[#lines+1] = ('%s'):format('}') else lines[#lines+1] = ('%s'):format('}\n') end
  end
  local file = assert(io.open(outputfile, 'w'), 'Unable to save ' .. outputfile)
  file:write(table.concat(lines, '\n'))
  file:close()
  return true
end

function GetItems(groupindex, classindex, fieldindex, valueindex)
  Group = items[groupindex][1] or ''
  Class = items[groupindex][2][classindex][1] or ''
  Field = items[groupindex][2][classindex][2][fieldindex][1] or ''
  Value = items[groupindex][2][classindex][2][fieldindex][2][valueindex][1] or ''
  GroupCount = #items
  ClassCount = #items[groupindex][2]
  FieldCount = #items[groupindex][2][classindex][2]
  ValueCount = #items[groupindex][2][classindex][2][fieldindex][2][1]
  return true
end

This is somewhat similar to the Read INI and Write INI snippets from the Rainmeter manual, but personally I couldn't use them right away to make my version as I didn't get much from the Lua jargon at the beginning, so I "independently" (well, checking lots and lots of StackOverflow codes and reading from the Lua manuals) arrived to a similar implementation. I'm happy with the result and with the fact that I can now reasonably write small pieces of Lua to help native Rainmeter in various tasks.

That being said, I would still use Lua instead of native Rainmeter only when there is no other (feasible) choice. That didn't (and won't) change. This was one of these instances, and I'm glad I took the journey, I even started to like Lua a bit...