It is currently September 20th, 2020, 9:16 pm

Lua not accepting umlauts from InputText Plugin

Discuss the use of Lua in Script measures.
TheBlubb
Posts: 6
Joined: August 25th, 2015, 2:22 pm

Lua not accepting umlauts from InputText Plugin

Post by TheBlubb »

Hello everyone!

My problem is the following: I have got a game launcher with an edit button using InputText.dll, which sends a function to a LUA script, which changes lines from a variables.txt file.

The problem appears when trying to type in german umlauts (ä, ö, ü) and sending those to my script. About rainmeter gives me two errors:

Code: Select all

Script: File not valid
Script: EditGames.lua:1: unexpected symbol near 'þ'
The script works perfectly when using "normal" characters.

I call the function like this in rainmeter:

Code: Select all

[MeasureInputText]
DynamicVariables=1
Measure=Plugin
Plugin=InputText
SolidColor=0, 0, 0
FontColor=f2f2f2
FontFace=Futura-CondensedLight
FontSize=18
X=5
Y=400
H=30
W=240
DefaultValue=""
Command1=[!SetVariable GameNumber "$UserInput$"][!Update] DefaultValue="Enter game number"
Command2=[!SetVariable NewName "$UserInput$"][!Update] DefaultValue="Enter new name"
Command3=[!SetVariable LeftAction "$UserInput$"][!Update] DefaultValue="Enter new game path"
Command4=[!SetVariable RightAction "$UserInput$"][!Update] DefaultValue="Enter new rightclick action"

[EditGames]
Measure=Script
ScriptFile=#@#EditGames.lua

;Setting the Variables via inputText.dll
[Edit]
Meter=Image
ImageTint=888888
ImageName=#@#Skyrim_Edit_Button.png
x=110
Y=885
H=25
Y=25
AntiAlias=1
MouseOverAction=[!SetOption Edit ImageTint f2f2f2][!Update]
MouseLeaveAction=[!SetOption Edit ImageTint 888888][!Update]
LeftMouseUpAction=[!HideMeterGroup AppButton][!ShowMeterGroup Edit][!CommandMeasure MeasureInputText "ExecuteBatch 1-4"][!Update]

;calling the function
[ReplaceButton]
Meter=Image
ImageTint=888888
ImageName=#@#ReplaceButton.png
x=30R
Y=r
H=20
Y=20
AntiAlias=1
MouseOverAction=[!SetOption ReplaceButton ImageTint f2f2f2][!Update]
MouseLeaveAction=[!SetOption ReplaceButton ImageTint 888888][!Update]
LeftMouseUpAction=[!CommandMeasure "EditGames" "replaceLine(#GameNumber#, #NewName#, #LeftAction#, #RightAction#)"][!Update][!Redraw][!Refresh]
Group = Edit
Hidden=1
The script is as following:

Code: Select all


fileName = "C:\Censored\Filepath\variables.txt"
local file = io.open(fileName, "r")
local sContents = file:read("*all")
tContents={}
for line in io.lines(fileName) do table.insert (tContents, line) end

function save(t, txt)
	local f = io.open(txt, "w")
	io.output(f)
	for i = 1, #t do io.write(t[i].."\n") end
end

function open(t, txt)
	local file = io.open(txt, "r")
	local sContents = file:read("*all")
	t={}
	for line in io.lines(txt) do table.insert (t, line) end
	return(t)
end

function replaceLine(gameNumber, NewName, NewLeft, NewRight)
	lineNumber=gameNumber*4-2
	open(tContents, fileName)
	if not gameNumber then print("No number given") return end
	if not NewName then print("No name given") return end
	if not NewLeft then print("No nath given")return end
	if not NewRight then NewRight="" end
	for i=1, 3, 1 do
		table.remove(tContents, lineNumber)
	end
	table.insert(tContents, lineNumber, "Game"..gameNumber.."Name=\""..NewName.."\"")
	table.insert(tContents, lineNumber+1, "Game"..gameNumber.."Path=\""..NewLeft.."\"")
	table.insert(tContents, lineNumber+2, "Game"..gameNumber.."PathRight=\""..NewRight.."\"")
	save(tContents, FileName)
end
I'm really new to using LUA so i have no clue what i could try to solve the problem.

Thanks in advance.

TheBlubb
User avatar
jsmorley
Developer
Posts: 21372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Lua not accepting umlauts from InputText Plugin

Post by jsmorley »

TheBlubb
Posts: 6
Joined: August 25th, 2015, 2:22 pm

Re: Lua not accepting umlauts from InputText Plugin

Post by TheBlubb »

Thank you. I tried that and it changed the symbol in the error message to:

Code: Select all

Script: EditGames.luaattempt to call table value
Script: EditGames.lua:1: ')' expected near '�'
I changed the encoding of the games.ini to ANSI, the lua script in notepad++ to UCS-2 LE and the .txt file to Unicode via windows editor (UTF 8 didn't work - rainmeter only showed #Game1Name# instead of the value set inside the .txt)

it also seems like the script is only working by itself when using ANSI, otherwise the Variables.txt consists only of " [] "; Umlauts give me the same error as before even when running the script by itself.

Maybe I'm doing the encoding wrong?
User avatar
jsmorley
Developer
Posts: 21372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Lua not accepting umlauts from InputText Plugin

Post by jsmorley »

TheBlubb wrote:Thank you. I tried that and it changed the symbol in the error message to:

Code: Select all

Script: EditGames.luaattempt to call table value
Script: EditGames.lua:1: ')' expected near '�'
I changed the encoding of the games.ini to ANSI, the lua script in notepad++ to UCS-2 LE and the .txt file to Unicode via windows editor (UTF 8 didn't work - rainmeter only showed #Game1Name# instead of the value set inside the .txt)

it also seems like the script is only working by itself when using ANSI, otherwise the Variables.txt consists only of " [] "; Umlauts give me the same error as before even when running the script by itself.

Maybe I'm doing the encoding wrong?
Any text file you want to read in Lua MUST be UTF-8. You might have to create a new empty file, save it as UTF-8 w/BOM, then open it and copy/paste the text from the old UTF-16 file into it. It doesn't really work to just convert UTF-16 to UTF-8.
TheBlubb
Posts: 6
Joined: August 25th, 2015, 2:22 pm

Re: Lua not accepting umlauts from InputText Plugin

Post by TheBlubb »

Thanks for the answer.

I did as you said; Saved a new empty file with the windows text editor as UTF-8, opened it and copied the contents of the old .txt file into it. I also created the skin.ini and the script file anew. the script is unicode and the .ini first ANSI, then UTF-8, then Unicode, but the skin now looks like this. The script unfortunately still doesn't support umlauts.
User avatar
jsmorley
Developer
Posts: 21372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Lua not accepting umlauts from InputText Plugin

Post by jsmorley »

Can you zip up the entire skin folder and host it somewhere so I can download it?
TheBlubb
Posts: 6
Joined: August 25th, 2015, 2:22 pm

Re: Lua not accepting umlauts from InputText Plugin

Post by TheBlubb »

Here you are and thank you very much for your help.
Link
The games skin starts hidden, to show it you have to click "Games" in the launcher skin.
User avatar
jsmorley
Developer
Posts: 21372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Lua not accepting umlauts from InputText Plugin

Post by jsmorley »

This is just not going to work for you.

The problem is sorta with Lua. It's file input capabilities are UTF-8 only. That means it can only "read" a UTF-8 encoded text file. I'll not get into why this is, other than to point out that Lua was and is intended to be entirely platform agnostic. UTF-8 continues to be the lingua franca of the cross-platform world.

So as I wrote in that "Unicode" Tips & Tricks article in the documentation, if you are going to create, write to, and save a text file with Lua, you must start by opening the new file and writing FileHandle:write('\239\187\191') or FileHandle:write('') to it, which is how you encode a text file with a UTF-8 "BOM".

You only need to do this if you are going to be writing characters to the file that are not in the first 127 characters of the ASCII table, basically English letters, numbers, and punctuation. ANSI and UTF-8 without BOM are exactly the same thing for the first 127 characters of the ASCII table. If you are going to write umlauts or any other character that is either in the codepage-supported "extended ASCII" set from 128 to 255, or real Unicode, you must create the file as UTF-8 w/BOM.

Then you can write any characters, including the full range of Unicode, and save the file. If you open it again in Lua, it will be properly read since it is UTF-8 w/BOM, which is what it wants.

If you encode the file as ANSI, or even UTF-8 without BOM, and write characters to it that are not in the first 127 characters of the ASCII set, Lua will not properly write to, or read the file.

Ok, so let's say we change your Lua, and have it make the new file UTF-8 w/BOM before we write all those entries you have saved to a table to it:

function save(t, txt)
local f = io.open(txt, "w")
io.output(f)
f:write('\239\187\191')
for i = 1, #t do io.write(t.."\n") end
end

That will work ok up to that point. The Lua will open and read the file into a table, will create a brand-new copy of the file as UTF-8 w/BOM, write the table full of stuff to it, and close it.

Now comes the issue we can't really solve.

Rainmeter on the other hand will REFUSE to read a file encoded as UTF-8 w/BOM. If you @Include a file in a skin, that file must be ANSI or UTF-16. So if I use your edit skin feature, and add a new entry to the list, when the Games skin is refreshed it hasn't read any of the variables from the file and the entire thing displays #Game1Name# etc.

I can't think of any way that you can use Lua to "manage" @Include files in Rainmeter. It's just a rock and a hard place. Lua insists on UTF-8 and Rainmeter hates UTF-8.

It's perfectly fine to have InputText create data that you capture with Lua, and write that to a text file. Lua will be able to write, and later read, the file just fine, as long as the file is, or you tell it to make it, UTF-8. However, it's not possible to then have Rainmeter read that file, either as a skin, or as an @Include file in a skin. It just won't.

I think you will need a new strategy, one that doesn't tie the @Include files to Lua. I know it's not what you want, but if it were me, I'd be tempted to do away with all the InputText, and just have the "edit" button open the Variables_G.txt file in their default editor. Have that file be encoded in UTF-16 Little Endian. Then they edit, add what they want, save the file and refresh the Games skin.

Either that, or find some way to get Lua out of the mix entirely. Have the InputText measures just use !WriteKeyValue to add things to the Variables_G.txt file, which is encoded in UTF-16 Little Endian. Get all that UTF-8 Lua issue out of the way.
TheBlubb
Posts: 6
Joined: August 25th, 2015, 2:22 pm

Re: Lua not accepting umlauts from InputText Plugin

Post by TheBlubb »

That's a shame, I will look for a workaround.
Thank you very much for your help, I really appreciate it.
User avatar
jsmorley
Developer
Posts: 21372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Lua not accepting umlauts from InputText Plugin

Post by jsmorley »

TheBlubb wrote:That's a shame, I will look for a workaround.
Thank you very much for your help, I really appreciate it.
Glad to help, sorry I wasn't able to really do so.

One thing to consider is the !WriteKeyValue can easily write to an existing ini-formatted text file. It will change or add Key=Value in existing sections. The only thing it really can't do is "remove" sections or values, although you could have it set things you want to remove to a value of "" and then have Rainmeter ignore "entries" that have a "title" of "".

What I'm getting at is that while you might not want to let the user add or remove from the total number of entries in the variables file, you could just set up a default file with say 20 entries or whatever maximum is reasonable, and set them all to "". You can then just let them specify a number, say "5" and have !WriteKeyValue write values to GameTile5 / GamePath5 or whatever those variables are called. They can modify any of the 20 maximum entries any way they like, and you just ignore ones they don't change from the default "".

Seems to me there is something there that could potentially get rid of the need for Lua. You are currently really just using Lua to mange the ".ini" format properly, which !WriteKeyValue already knows how to do.