It is currently November 30th, 2022, 1:33 am

Unicode cmd, read UTF16 problem

Discuss the use of Lua in Script measures.
jorm
Posts: 3
Joined: November 16th, 2022, 4:07 pm

Unicode cmd, read UTF16 problem

Post by jorm »

Hello!

I'm writing a file via cmd:

Code: Select all

@echo off
chcp 65001

echo Fred > Output.txt
echo Barney >> Output.txt
This is referenced in a script plugin measure:

Code: Select all

[MeasureRun]
Measure=Plugin
Plugin=RunCommand
Parameter=#CURRENTPATH#CreateOutput.cmd
State=Hide
OutputType=ANSI
OutputFile=#CURRENTPATH#CreateOutput.log
Now I have some lua:

Code: Select all

local p = assert(io.open('Output.txt', 'r'))
content = p:read('*all')
print('File content: ' .. content)
p:close()
The Rainmeter.log only shows:

Code: Select all

DBUG (10:31:34.003) : File content: F
In short, it only reads the first character. The lua file is UTF16 LE; the Output.txt also. If I change Output.txt to UTF8 manually, the lua can read the full file, but I can't make the cmd file output UTF8. I also tried

Code: Select all

Parameter=cmd /u /c #CURRENTPATH#CreateOutput.cmd
but the Output.txt is still UTF16 after one run.

I also tried to write everything to stdout and read from the Log, since that is ansi or UTF8 according to the specified OutputType. But when I do that, only the first letter of my echo goes into the log. Of course, the batch output is perfectly fine when I run it from windows.

(I gotta say, once again I wished Unicode had never happened... so often it seems to be more trouble than it's worth, and I'd gladly give up my äöü. I don't need them that much, really.)

Did I miss something? I also went through os.execute() (annoyingly flashes a cmd window, otherwise fine) and tried os.popen() for a while until I realised it's not implemented, alas.
This is Windows 10 and Rainmeter 4.5.16.3687 (64-bit).
User avatar
SilverAzide
Rainmeter Sage
Posts: 2174
Joined: March 23rd, 2015, 5:26 pm

Re: Unicode cmd, read UTF16 problem

Post by SilverAzide »

As you noticed, Lua can't read or write Unicode (UTF-16). And your batch file will always output UTF-16 since you are switching to code page 65001. Instead of using the command processor, would you be willing to switch whatever you are doing to PowerShell instead? PowerShell can encode output however you like. See this for more, for example. Your PS command line can also do things like this:

[Console]::OutputEncoding = New-Object -TypeName System.Text.UTF8Encoding; your additional commands here...
Gadgets Wiki GitHub More Gadgets...
jorm
Posts: 3
Joined: November 16th, 2022, 4:07 pm

Re: Unicode cmd, read UTF16 problem

Post by jorm »

Code page 65001 was used as an attempt to force UTF8 output (65001 should be UTF8). And I thought attempts to do that with Powershell might end up the same, tbh. But it's worth a try. Thanks for the hint!