⭐ Weather.com - Parsing the V3 JSON

[jsmorley]

Yeah, that's my only advice as well. So, looks like there's a need for a "percent encoding" thing for strings. Can Lua turn Unicode text (char array?) into a percent-encoded string?

No, Lua isn't going to help at all. It is just spectacularly bad at Unicode.

[jsmorley]

I am going to push Brian to take a look at this issue, to see what if anything can be done in WebParser.

I'm not sure what the root cause is, but I'm guessing that modern browsers might well do this:

https://www.url-encode-decode.com/

Under the covers.

OR

Perhaps WebParser is not sending the URL as UTF-8, and maybe that can be changed.

We will see if Brian can spot anything.

[SilverAzide]

OK!

[jsmorley]

This simple skin will demonstrate the problem:

Code: Select all

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1

[Variables]
Var1=Wrocław

[MeasureSite]
Measure=WebParser
URL=https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json
Flags=Resync | NoCookies
UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
RegExp=(?siU)^(.*)$
StringIndex=1

[MeterDisplay]
Meter=String
MeasureName=MeasureSite
FontSize=11
FontWeight=400
FontColor=255,255,255,255
SolidColor=47,47,47,255
Padding=5,5,5,5
AntiAlias=1
Text=Result : %1
LeftMouseUpAction=[https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json]

[SilverAzide]

This simple skin will demonstrate the problem:

I didn't try modding this example (just spit-balling here), but if anyone wants to brute force it, you can use curl in PowerShell to grab the JSON and parse that.

Something like:

Code: Select all

[MeasureGetLocationData]
Measure=Plugin
Plugin=RunCommand
Program=powershell.exe
Parameter="-NoProfile -ExecutionPolicy Bypass -NonInteractive -Command "[Console]::OutputEncoding = New-Object -TypeName System.Text.UnicodeEncoding; (curl -Uri 'https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json').Content"
OutputType=UTF16
State=Hide
Timeout=-1
FinishAction=[...do stuff here...]

[jsmorley]

I didn't try modding this example (just spit-balling here), but if anyone wants to brute force it, you can use curl in PowerShell to grab the JSON and parse that.

Something like:

Code: Select all

[MeasureGetLocationData]
Measure=Plugin
Plugin=RunCommand
Program=powershell.exe
Parameter="-NoProfile -ExecutionPolicy Bypass -NonInteractive -Command "[Console]::OutputEncoding = New-Object -TypeName System.Text.UnicodeEncoding; (curl -Uri 'https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json').Content"
OutputType=UTF16
State=Hide
Timeout=-1
FinishAction=[...do stuff here...]

Yeah, I'd prefer to avoid that if possible, as that would be multiple steps. Read the results with PoweShell, have that write to a file, and have WebParser read that file to do the parsing into measures.

I'd like to understand, and if possible fix, this issue with URL's and WebParser. This limitation isn't just in this skin, but presumably any skin that passes parameters to a URL.

[SilverAzide]

I'd like to understand, and if possible fix, this issue with URL's and WebParser. This limitation isn't just in this skin, but presumably any skin that passes parameters to a URL.

Yes. I'm a bit shocked this hasn't cropped up before now...

[jsmorley]

Yes. I'm a bit shocked this hasn't cropped up before now...

We were just saying that in IRC

[SilverAzide]

But I have found the results from openstreetmap to be a bit hinky in general. I get really puzzling results a lot, and if you search for a location in a particular country, like "Moscow, Russia", you get results in Russian / Cyrillic, which is a bit annoying.

Welp, I've got a small gift for you!

In your location skin, add the lines needed to include the WeatherComJSONVariables.inc file. Then, in your main WebParser, do this:

Code: Select all

[MeasureLocations]
...
Header="Accept-Language: #Language#"
...

Then, you'll get the following. On the left is "en-US", on the right is "ru-RU", which is the default if you don't specify.

weathercode.jpg

[jsmorley]

Welp, I've got a small gift for you!

In your location skin, add the lines needed to include the WeatherComJSONVariables.inc file. Then, in your main WebParser, do this:

Code: Select all

[MeasureLocations]
...
Header="Accept-Language: #Language#"
...

Then, you'll get the following. On the left is "en-US", on the right is "ru-RU", which is the default if you don't specify.

weathercode.jpg

Huh... That's cool indeed! Thanks! Still some Cyrillic here and there, but a lot less...