It is currently October 22nd, 2021, 7:48 am

⭐ Weather.com - Parsing the V3 JSON - NEW!

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 4th, 2020, 5:02 pm Yeah, that's my only advice as well. So, looks like there's a need for a "percent encoding" thing for strings. Can Lua turn Unicode text (char array?) into a percent-encoded string?
No, Lua isn't going to help at all. It is just spectacularly bad at Unicode.
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

I am going to push Brian to take a look at this issue, to see what if anything can be done in WebParser.

I'm not sure what the root cause is, but I'm guessing that modern browsers might well do this:

https://www.url-encode-decode.com/

Under the covers.

OR

Perhaps WebParser is not sending the URL as UTF-8, and maybe that can be changed.

We will see if Brian can spot anything.
User avatar
SilverAzide
Rainmeter Sage
Posts: 1616
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

jsmorley wrote: August 4th, 2020, 5:05 pm
OK!
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

This simple skin will demonstrate the problem:

Code: Select all

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1

[Variables]
Var1=Wrocław

[MeasureSite]
Measure=WebParser
URL=https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json
Flags=Resync | NoCookies
UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
RegExp=(?siU)^(.*)$
StringIndex=1

[MeterDisplay]
Meter=String
MeasureName=MeasureSite
FontSize=11
FontWeight=400
FontColor=255,255,255,255
SolidColor=47,47,47,255
Padding=5,5,5,5
AntiAlias=1
Text=Result : %1
LeftMouseUpAction=[https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json]
User avatar
SilverAzide
Rainmeter Sage
Posts: 1616
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

jsmorley wrote: August 4th, 2020, 5:44 pm This simple skin will demonstrate the problem:
I didn't try modding this example (just spit-balling here), but if anyone wants to brute force it, you can use curl in PowerShell to grab the JSON and parse that.

Something like:

Code: Select all

[MeasureGetLocationData]
Measure=Plugin
Plugin=RunCommand
Program=powershell.exe
Parameter="-NoProfile -ExecutionPolicy Bypass -NonInteractive -Command "[Console]::OutputEncoding = New-Object -TypeName System.Text.UnicodeEncoding; (curl -Uri 'https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json').Content"
OutputType=UTF16
State=Hide
Timeout=-1
FinishAction=[...do stuff here...]
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 4th, 2020, 5:46 pm I didn't try modding this example (just spit-balling here), but if anyone wants to brute force it, you can use curl in PowerShell to grab the JSON and parse that.

Something like:

Code: Select all

[MeasureGetLocationData]
Measure=Plugin
Plugin=RunCommand
Program=powershell.exe
Parameter="-NoProfile -ExecutionPolicy Bypass -NonInteractive -Command "[Console]::OutputEncoding = New-Object -TypeName System.Text.UnicodeEncoding; (curl -Uri 'https://nominatim.openstreetmap.org/search.php?q=#Var1#&format=json').Content"
OutputType=UTF16
State=Hide
Timeout=-1
FinishAction=[...do stuff here...]
Yeah, I'd prefer to avoid that if possible, as that would be multiple steps. Read the results with PoweShell, have that write to a file, and have WebParser read that file to do the parsing into measures.

I'd like to understand, and if possible fix, this issue with URL's and WebParser. This limitation isn't just in this skin, but presumably any skin that passes parameters to a URL.
User avatar
SilverAzide
Rainmeter Sage
Posts: 1616
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

jsmorley wrote: August 4th, 2020, 5:57 pm I'd like to understand, and if possible fix, this issue with URL's and WebParser. This limitation isn't just in this skin, but presumably any skin that passes parameters to a URL.
Yes. I'm a bit shocked this hasn't cropped up before now...
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 4th, 2020, 6:08 pm Yes. I'm a bit shocked this hasn't cropped up before now...
We were just saying that in IRC
User avatar
SilverAzide
Rainmeter Sage
Posts: 1616
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

jsmorley wrote: August 4th, 2020, 4:45 pmBut I have found the results from openstreetmap to be a bit hinky in general. I get really puzzling results a lot, and if you search for a location in a particular country, like "Moscow, Russia", you get results in Russian / Cyrillic, which is a bit annoying.
Welp, I've got a small gift for you! :)

In your location skin, add the lines needed to include the WeatherComJSONVariables.inc file. Then, in your main WebParser, do this:

Code: Select all

[MeasureLocations]
...
Header="Accept-Language: #Language#"
...
Then, you'll get the following. On the left is "en-US", on the right is "ru-RU", which is the default if you don't specify.
weathercode.jpg
You do not have the required permissions to view the files attached to this post.
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 22372
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 4th, 2020, 6:44 pm Welp, I've got a small gift for you! :)

In your location skin, add the lines needed to include the WeatherComJSONVariables.inc file. Then, in your main WebParser, do this:

Code: Select all

[MeasureLocations]
...
Header="Accept-Language: #Language#"
...
Then, you'll get the following. On the left is "en-US", on the right is "ru-RU", which is the default if you don't specify.

weathercode.jpg
Huh... That's cool indeed! Thanks! Still some Cyrillic here and there, but a lot less...