How to properly escape strings containing magic/meta characters when using string.match/gmatch in for loops in Lua?

[Crest]

So I have a script that defines a string like:

Code: Select all

myTable = {
    myString = "Here is (something)"
}

This string gets used later in the script via a bang to change a meter's text.

The issue is that various meta characters like parentheses, square brackets and asterisks cause the entire string to not get parsed on the INI side. So in the current version of the script I've simply removed them from the string.

I've looked through the site but couldn't find how to escape the characters in this scenario. I've tried `\`, raw unicode character reference, wrapping with asterisks, different quoting types but still fails to parse with them present.

Is there a way?

---

Edit: title renamed from: How to escape meta characters when passing string from Lua to Rainmeter?

[Yincognito]

So I have a script that defines a string like:

Code: Select all

myTable = {
    myString = "Here is (something)"
}

This string gets used later in the script via a bang to change a meter's text.

The issue is that various meta characters like parentheses, square brackets and asterisks cause the entire string to not get parsed on the INI side. So in the current version of the script I've simply removed them from the string.

I've looked through the site but couldn't find how to escape the characters in this scenario. I've tried `\`, raw unicode character reference, wrapping with asterisks, different quoting types but still fails to parse with them present.

Is there a way?

There's always a way, but it would help to provide more details about the issue, like where you're using that string in the skin and how you "parse" it. If by any chance you're using it as a regex pattern, there is a parameter for a measure section variable:
https://docs.rainmeter.net/manual/variables/section-variables/#EscapeRegExp
that can automatically escape these chars. If it's about something else, like I said, a few relevant details (or maybe a very short sample replicating the issue) would be useful to find a solution.

[Crest]

Sure, here's a minimal test case that shows the behavior. The initial value of this example shows a functioning string, while to produce the unexpected behavior change the tab.some table string in the Lua to contain eg. parentheses/square brackets then refresh the skin (which will then output nothing on the INI side).

Maybe it is the gmatch pattern.

Skin.ini

Code: Select all

[Rainmeter]
AccurateText=1
DynamicWindowSize=1
OnRefreshAction=[&Script:ShowText()][!Update]
Update=1000

[Script]
Measure=Script
ScriptFile=Script.lua

[TextMeasure]
Measure=String
DynamicVariables=1

[Text]
Meter=String
X=0
Y=0
W=300
H=100
FontColor=200,200,200
SolidColor=0,0,0
FontSize=12
Text=
AntiAlias=1
Padding=10,10,10,10
DynamicVariables=1

Script.lua

Code: Select all

function Initialize()
    tab = {
        some = "Here is something"
    }
end

function ShowText()
    SKIN:Bang("!SetOption","TextMeasure","String",tab.some)
    SKIN:Bang("!Update")
    local textMeas = SKIN:GetMeasure("TextMeasure")
    local textMeasVal = textMeas:GetStringValue()

    for line in textMeasVal:gmatch("[^\n]+") do
        for key in pairs(tab) do
            if line:match(tab[key]) then
                output(key)
            end
        end
    end


    return 0
end

function output(key)
    SKIN:Bang("!SetOption", "Text", "Text", tab[key])
end

[Yincognito]

Sure, here's a minimal test case that shows the behavior. The initial value of this example shows a functioning string, while to produce the unexpected behavior change the tab.some table string in the Lua to contain eg. parentheses/square brackets then refresh the skin (which will then output nothing on the INI side).

Maybe it is the gmatch pattern.

Skin.ini

Code: Select all

[Rainmeter]
AccurateText=1
DynamicWindowSize=1
OnRefreshAction=[&Script:ShowText()][!Update]
Update=1000

[Script]
Measure=Script
ScriptFile=Script.lua

[TextMeasure]
Measure=String
DynamicVariables=1

[Text]
Meter=String
X=0
Y=0
W=300
H=100
FontColor=200,200,200
SolidColor=0,0,0
FontSize=12
Text=
AntiAlias=1
Padding=10,10,10,10
DynamicVariables=1

Script.lua

Code: Select all

function Initialize()
    tab = {
        some = "Here is something"
    }
end

function ShowText()
    SKIN:Bang("!SetOption","TextMeasure","String",tab.some)
    SKIN:Bang("!Update")
    local textMeas = SKIN:GetMeasure("TextMeasure")
    local textMeasVal = textMeas:GetStringValue()

    for line in textMeasVal:gmatch("[^\n]+") do
        for key in pairs(tab) do
            if line:match(tab[key]) then
                output(key)
            end
        end
    end


    return 0
end

function output(key)
    SKIN:Bang("!SetOption", "Text", "Text", tab[key])
end

Thanks. I'm on my phone so can't test it, but the gmatch() pattern seems fine, according to the Lua manual (see string patterns, and how to escape magic characters):
https://www.lua.org/manual/5.1/

Have you tried using print() at each step of the process in the script to see if the relevant values of the variables you manipulate are the expected ones at the Rainmeter log? This often helps in debugging your process. Also, you don't seem to !Update in the output() function, maybe that plays a part in the issue you're experiencing - albeit from what I can tell, the script currently "should" set the measure value to tab.some / tab[some] twice, which is a bit redundant, unless it's just for testing purposes.

[Crest]

Thanks. I'm on my phone so can't test it, but the gmatch() pattern seems fine, according to the Lua manual (see string patterns, and how to escape magic characters):

Mmm, the issue seems to be that gmatch (or any similar Lua pattern matching function) requires escaping of magic characters for the input string. The problem is escaping them similarly fails to produce the intended result.

Here's an updated Script.lua demonstrating this issue. An escape function has been added which escapes all non-alphanumeric chars besides new lines and spaces.

A print on line 20 shows that both the escaped 'line' loop var and the escaped string.match pattern appear identical in the Rainmeter log. And yet despite this the string.match doesn't evaluate true (and thus no output occurs). If anyone knows what is failing and if there's a fix it'd be appreciated.

Code: Select all

function Initialize()
    tab = {
        some = "Here is (something"
    }
end

function Esc(str)
    local esc = str:gsub("([^%w^\n^%s])", "%%%1")
    return esc
end

function ShowText()
    SKIN:Bang("!SetOption","TextMeasure","String",tab.some)
    SKIN:Bang("!Update")
    local textMeas = SKIN:GetMeasure("TextMeasure")
    local textMeasVal = textMeas:GetStringValue()

    for line in string.gmatch(Esc(textMeasVal),"[^\n]+") do
        for key in pairs(tab) do
            print(line,Esc(tab[key]))
            if string.match(line,Esc(tab[key])) then
                output(key)
            end
        end
    end

    return 0
end

function output(key)
    SKIN:Bang("!SetOption", "Text", "Text", tab[key])
end

Also, you don't seem to !Update in the output() function, maybe that plays a part in the issue you're experiencing - albeit from what I can tell, the script currently "should" set the measure value to tab.some / tab[some] twice, which is a bit redundant, unless it's just for testing purposes.

Yeah, was just a simplified test case pared down from a larger script, while trying to debug what the issue was, rather than representative of the normal skin. Also I'd tested with an update bang but it makes no difference for this issue.

[Yincognito]

This is untested since I didn't get to my laptop yet, but I believe that you need to escape each of the 12 magic characters i.e. ^$()%.[]*+-? (possibly other punctuation ones too, not sure yet) by preceding every one of them with % in the first parameter of your gsub() function, so:

Code: Select all

str:gsub("([^%w^\n^%s])", "%%%1")

should be something like:

Code: Select all

str:gsub("([%^%$%(%)%%%.%[%]%*%+%-%?])", "%%%1")

If it works, feel free to add any other character needing escaping (escaping it as well by preceding it with the % symbol, e.g. %\ to escape the backslash, just to be on the safe side) to the set between those [ and ] square brackets.

P.S. Alternatively, if you were to pass the literal string directly from Lua (as opposed to passing it as a variable or parameter), you could have used literals in the bracketed form, e.g. [[...]] where string goes instead of the ... part, see:
https://www.lua.org/pil/2.4.html

[Crest]

This is untested since I didn't get to my laptop yet, but I believe that you need to escape each of the 12 magic characters i.e. ^$()%.[]*+-? (possibly other punctuation ones too, not sure yet) by preceding every one of them with % in the first parameter of your gsub() function, so:

Tested but unfortunately same result. The prior method similarly escaped all those characters except (curiously) for caret.

P.S. Alternatively, if you were to pass the literal string directly from Lua (as opposed to passing it as a variable or parameter), you could have used literals in the bracketed form, e.g. [[...]] where string goes instead of the ... part, see:
https://www.lua.org/pil/2.4.html

Will keep in mind, though yeah in this case will be reading variables.

[Yincognito]

Tested but unfortunately same result. The prior method similarly escaped all those characters except (curiously) for caret.

I'll take a look at it in a few moments on my laptop. It's not curious that the the caret wasn't escaped, since in your version, it isn't preceded by % not to mention that you had it more than once in the set.

[Yincognito]

Tested but unfortunately same result.

Took a little longer than expected, but this works for me:

Code: Select all

function Initialize()
  tab = {some = 'Here is (something'}
end

function Esc(str)
  --local esc = string.format('%q', str)
  local esc = str:gsub('([%^%$%(%)%%%.%[%]%*%+%-%?])', '%%%1')
  return esc
end

function Output(key)
  SKIN:Bang('!SetOption', 'Text', 'Text', tab[key])
  SKIN:Bang('!Update')
end

function ShowText()
  print(Esc(tab.some))
  SKIN:Bang('!SetOption', 'TextMeasure', 'String', tab.some)
  SKIN:Bang('!Update')
  local txt = SKIN:GetMeasure('TextMeasure'):GetStringValue()
  for line in txt:gmatch('[^\r\n]+') do
    for key, val in pairs(tab) do
      print(key, val, line)
      if line:match(Esc(val)) then print('matched'); Output(key) else print('not matched') end
    end
  end
  return 0
end

I let the string.format('%q', ...) alternative there since it looks much simpler, but although it does escape "almost" everything as it should and advertised, it doesn't produce the expected result even if I use Esc(line) in the if statement.

What I initially recommended works though, and as I believe it should happen, only the literal pattern (i.e. val or tab[some]) needs escaping, nothing else. There is a catch though - I tried escaping stuff like \n or \s after in the main string, and except literally using \\n and \\s there or declaring it as tab = {some = [[Here is (something]]}, I got mixed results: it either didn't work (tried local esc = str:gsub('([%^%$%(%)%%%.%[%]%*%+%-%?])', '%%%1'):gsub('\\', '\\\\')), or it partially work and it kept the \ but not the n afterwards. You can use " aka double quotes in the string without having to escape them, but not ' aka apostrophes / single quotes, though you can invert that if you want.

[Crest]

Took a little longer than expected, but this works for me:

Appreciate the sleuthing! The observation about the escaping only being required for the string.match pattern and not also the for loop line var for string.gmatch was key. Thanks for taking the time to help, I'm sure will be useful for others as well.

Will update the topic title. I suppose given this turned out to be a Lua specific issue it'd probably be better suited in the Lua script sub-forum, if anyone feels like moving it.