[BUG] @Include recursion tree is flawed

[Jeff]

This one is gonna be a bit hard to wrap your head around but the example should show the "bug"
We have a folder with the files

MainSkin.ini
└A.inc
└B.inc
└C.inc

In the skin, the meters are color coded to represent it easier, A is RED, C is GREEN, B is BLUE

MainSkin.ini

Code: Select all

[Rainmeter]
@IncludeA=A.inc
@IncludeB=B.inc

A.inc

Code: Select all

[MeterA]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=FF0000
@IncludeC=C.inc

[MeterA2]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=FF0000

B.inc

Code: Select all

[MeterB]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=0000FF

C.inc

Code: Select all

[MeterC1]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=00FF00
[MeterC2]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=00FF00
[MeterC3]
Meter=Image
X=0R
Y=0R
W=50
H=50
SolidColor=00FF00

On the assumption that the Docs saying You may also include files within files. Once again, the ordering is determined by placement: when any file includes another file, the new contents are added within its own sections, immediately after the section where the statement is made. is correct, with the way we have written this, let's go over how the @Includes should work, @Include by @Include.

First let's start with the code as is

Code: Select all

[Rainmeter]
@IncludeA=A.inc
@IncludeB=B.inc

First the @IncludeA is loaded, so the skin now becomes

Code: Select all

[Rainmeter]
[MeterA]
@IncludeC=C.inc
[MeterA2]
@IncludeB=B.inc

After that, it sees that @IncludeC exists, so it gets placed between the meters

Code: Select all

[Rainmeter]
[MeterA]
[MeterC1]
[MeterC2]
[MeterC3]
[MeterA2]
@IncludeB=B.inc

Finally, it sees @IncludeB, so it adds it at the end

Code: Select all

[Rainmeter]
[MeterA]
[MeterC1]
[MeterC2]
[MeterC3]
[MeterA2]
[MeterB]

So now let's put them as boxes next to each other with how it should look
ACCCAB
The Rainmeter log debugs the loads for those files in that order too, ACB

And that's how it should work by the logic imposed by the wording on the docs and what humans would assume with recursive actions of this kind... except it dosen't, that's why I made this report, when Rainmeter gets to

Code: Select all

[Rainmeter]
[MeterA]
@IncludeC=C.inc
@IncludeB=B.inc
; [MeterA2] is in limbo 

for some reason, Rainmeter breaks human logic at this step, it sees that there's an @IncludeC, so [MeterA2] is put into limbo and from the "we have read in A.inc these following sections" list, the @IncludeC triggers "in A.inc we have these sections we have no idea about, assume they are new sections to be placed after every @Include is done" for every single other thing in A.inc, Rainmeter is no longer gonna take it's time dealing with [MeterA2] at this step

This ends up being what Rainmeter currently dose right now

Code: Select all

[Rainmeter]
[MeterA]
[MeterC1]
[MeterC2]
[MeterC3]
[MeterB]
[MeterA2]

which is ACCCBA

I wanted to give a suggestion here but I genuinely have no idea how to fix this. When I found this out my very first reaction was a loud WHYYYYYY???? and laughing at my screen.
Most likely binary tree where the sections are resolved first and the @Includes get their branches later can be a solution for this? (If my assumption that Rainmeter reads @Includes line by line is correct and dosen't read sections before @Includes, I blame Microsoft if it reads sections before @Includes). Maybe make an agent for @Include and for every file that gets @Included, spawn an agent in them too instead of a global "we'll add them later" list?
I know the solution is to just include [MeterB] in MainSkin.ini after the 2 @Includes, but the logic for the implementation is still flawed and it's what I wanted to point out, the docs are too correct in saying When Rainmeter reaches an @include line, it reads the given file, placing all unique options into any existing sections, then places the remainder of the included file after the section in which it was called.

And to give credits, this was submitted by Gabe on Discord

[SilverAzide]

There does seem to be a flaw someplace, as the meters do get ordered incorrectly per the docs.

That said, your interpretation of how Rainmeter files get parsed is completely wrong. Understanding how they do get parsed won't resolve the issue, but the vision of some magical Rainmeter recursive super-file-parser needs a little reality check.

Rainmeter uses "standard" Windows .INI format files (.INI type files existed before Windows, but they only follow a general agreed-upon convention, not some RFC/ISO standard). These files are read using ancient been-around-since-1.0 Windows APIs, GetPrivateProfileSectionNames, GetPrivateProfileSection, GetPrivateProfileString, etc.

The key thing to note here is that the "@include" syntax is purely a Rainmeter invention. The Win32 API does not have any mechanism to read nested .INI files. None. Zilch. Nada.

So, what really happens when Rainmeter reads your .INI files?

First, Rainmeter will pass the name of the top level skin .INI to GetPrivateProfileSectionNames. This eats the entire file in one bite. No magical line-at-a-time parsing or recursion. The return of this function is simple list of all the section names in the file. So, in your example, this is simply a one-element list containing "Rainmeter". if you had a more typical file, you'd get a longer list of all the "[...]" section names in the file.

Next, Rainmeter will loop through each item in the section name list and call GetPrivateProfileSection to read the contents of the section. Again, this eats the entire section in one bite. What you get back from this is an array of all the lines in the section (the "key=value" lines).

You'll notice we have not read any other files yet. This where the Rainmeter magic happens. As the code chews through each section in the list of section names, it checks each line in the section looking for "@key=value" entries. If it finds one in a particular section, it will recursively call the "INI reading routine" and start again with the new .INI (which in your example, is "A.inc". The entire A.inc file is read and the list of A's sections is obtained. It is keeping track of where it is in the list of sections, so it able to insert the A.inc section name list into the "MainSkin.ini" section list, after the [Rainmeter] section. The code then reads the each of the A sections, encounters the "@include=C.ini", and the loop repeats again.

After C.inc is read and the sections inserted into section list, and the remainder of the A.inc sections processed and found to have no more includes, then we bubble back up to the top. This is where the second include file in the top level skin is encountered, B.ini. Again we read the whole file, read each section looking for includes, and finally insert all the "B" sections into the section list.

This is where I think there could be a minor error in the insertion code, where it is off by one and inserting the sections 1-element too high in the section list. Possibly it has gotten slightly lost due to multiple includes in a single section, with some of those includes having more includes.

In fact, if you nest a D.inc into the C.ini file, the B.inc sections will be inserted even further off up the list. So yeah, there is a glitch someplace.

This isn't a solution, but as a work-around you can force the correct order by moving the line @IncludeB=B.inc into its own section, thus allowing the code to be able to clearly identify the proper insertion point.

P.S.: There also seems to be a glitch if the last section of an included file contains an @include line. In this case the insertion point gets wildly thrown off (like inserts the file's sections at the beginning of the main section list).

[Yincognito]

Nice catch, and nice explanation by both of you.

I suspect the above is a slightly more complex form of the much simpler (and probably better known by experienced users) example below...

[SkinFolder]\Variables.inc:

Code: Select all

[Variables]
SomeVariable=7

[SkinFolder]\Skin.ini:

Code: Select all

[Variables]
@IncludeVariables=Variables.inc

[Dummy]
Meter=Image
W=100
H=100
SolidColor=0,0,0,255

Comment out any of the [Variables] lines (the section lines, that is) from either the main .ini or the included .inc, for example...

[SkinFolder]\Variables.inc:

Code: Select all

;[Variables]
SomeVariable=7

[SkinFolder]\Skin.ini:

Code: Select all

[Variables]
@IncludeVariables=Variables.inc

[Dummy]
Meter=Image
W=100
H=100
SolidColor=0,0,0,255

and watch the log: you'll have no variable called SomeVariable whose value is 7 - in other words, you have to duplicate the [Variables] section line in both the .ini and the .inc files. So, as SilverAzide mentioned, the @includes are indeed dependent on the [section] lines and their layout. Which is, of course, counterintuitive at times, considering that one of the particularities of an .ini syntax is that "all section names in a skin must be unique" per the manual, which, in the simple example I posted above, "should" in theory, cancel or ignore one of the [Variables] sections (from the .inc or from the .ini), given that they have the same section name - which obviously doesn't happen when it comes to @includes.