It is currently September 21st, 2020, 1:29 pm

⭐ Weather.com - Parsing the V3 JSON - NEW!

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 21376
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 4th, 2020, 4:17 pm OK, I need a little assistance with an issue illustrated by this skin but not directly weather.com related.

Can someone try using the location search feature in this skin and try to search for the text Wrocław? Take note of the "accented-L" symbol, it's not a latin letter L. The WebParser is returning nothing, and I can't figure out why. If you open your browser and use the exact string that WebParser is using (https://nominatim.openstreetmap.org/search.php?q=Wrocław&format=json), it is working perfectly. Searching for https://nominatim.openstreetmap.org/search.php?q=Wroc%C5%82aw&format=json also works, of course, but that's not the URL that's going out... I tried a custom header of "Accepts-Language: pl-PL", but that didn't help (the header is mentioned in the OSM API docs).

Using the latin "L" works, but that's not what normal users would enter. Is there some behind the scenes Unicode handling going on in the browsers but not in WebParser?
This has been addressed in the latest r3401 beta of Rainmeter.

https://forum.rainmeter.net/viewtopic.php?f=129&t=35885
User avatar
Yincognito
Posts: 2565
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by Yincognito »

jsmorley wrote: August 20th, 2020, 12:52 pm This has been addressed in the latest r3401 beta of Rainmeter.

https://forum.rainmeter.net/viewtopic.php?f=129&t=35885
Excellent. :thumbup: I guess it took quite a bit of work to get it done - I for one am grateful for that. :rosegift:
User avatar
SilverAzide
Posts: 954
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

jsmorley wrote: August 20th, 2020, 12:52 pm This has been addressed in the latest r3401 beta of Rainmeter.
Awesome! Thank you guys for all your hard work! :rosegift: :thumbup:
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 21376
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 20th, 2020, 1:47 pm Awesome! Thank you guys for all your hard work! :rosegift: :thumbup:


1.jpg

2.jpg
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Posts: 2565
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by Yincognito »

Something I discovered recently, given the fact that I helped on some "radar" skins here on the forum. One can access weather.com radar as well, using something like (the supported formats are JPG and PNG, no GIF apparently):

Code: Select all

https://api.weather.com/v2/maps/dynamic?geocode=47.5,6.0&h=320&w=568&lod=5&product=twcRadarHcMosaic&map=light&format=png&language=en&apiKey=<ApiKey>&a=0
dynamic.png
Apparently, changing the a= parameter is doing something, for bigger differences.

The idea was to have something like:

Code: Select all

https://weather.com/weather/radar/interactive/l/Fowler+IN?canonicalCityId=7eb5f0887b7c498cb1a1b03f43da73b0e7336ef0d0e55150c0cf5e7634e260c7
(play with the Layer menu on the left or else you won't see much) that can be grabbed with WebParser and displayed. Sure, from the notes on the page it seems it's working only for the western hemisphere and only during daylight hours (whose daylight, I wonder?), but it's still something.

Also, I used my own image scraping Chrome extension to further investigate the matter and it appears one can get those images, albeit piece by piece (i.e. one for each map section):
Weather Radar.jpg
aka...

Code: Select all

https://api.mapbox.com/styles/v1/weather/cjza0fpcf54z71cnty86ulfhj/tiles/5/7/11?access_token=pk.eyJ1Ijoid2VhdGhlciIsImEiOiJjamxncnYwMmYxYXgxM3BzNWEyNHUzbmhhIn0.UqrZwn6qTa6WJwdHjPVUfA

Code: Select all

https://api.mapbox.com/styles/v1/weather/cjza0fpcf54z71cnty86ulfhj/tiles/5/8/11?access_token=pk.eyJ1Ijoid2VhdGhlciIsImEiOiJjamxncnYwMmYxYXgxM3BzNWEyNHUzbmhhIn0.UqrZwn6qTa6WJwdHjPVUfA

Code: Select all

https://api.weather.com/v3/TileServer/tile?product=satradFcst&ts=1598046300&fts=1598047200&xyz=16:24:6&apiKey=<ApiKey>

Code: Select all

https://api.weather.com/v3/TileServer/tile?product=satradFcst&ts=1598046300&fts=1598047200&xyz=16:23:6&apiKey=<ApiKey>
and so on.

Maybe others will find more about it, if they are interested.
You do not have the required permissions to view the files attached to this post.
MikeG621
Posts: 81
Joined: March 18th, 2013, 1:59 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by MikeG621 »

I don't think this is a problem with the JSON (which I finally rolled into my skins, just don't have the time to maintain my scraper anymore), but has anyone else seen the alert SeverityCode values be non-zero while the Severity/Text/Descriptions are still blank?
User avatar
SilverAzide
Posts: 954
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by SilverAzide »

MikeG621 wrote: August 24th, 2020, 2:13 pm I don't think this is a problem with the JSON (which I finally rolled into my skins, just don't have the time to maintain my scraper anymore), but has anyone else seen the alert SeverityCode values be non-zero while the Severity/Text/Descriptions are still blank?
Yes, I have seen this as well. I just noticed it recently. If you have a location that has at least one alert, and then on the next update all alerts are gone, the severity codes will retain their previous numeric values. I guess it is because none of the measures can be refreshed since the whole alert section is missing, but I'm not sure why the values don't go to zero. Perhaps @JSMorley can chime in on why this would be. It is rather tedious to find a location with an expiring alert to try this out, but it is doable.
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 21376
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

SilverAzide wrote: August 24th, 2020, 2:29 pm Yes, I have seen this as well. I just noticed it recently. If you have a location that has at least one alert, and then on the next update all alerts are gone, the severity codes will retain their previous numeric values. I guess it is because none of the measures can be refreshed since the whole alert section is missing, but I'm not sure why the values don't go to zero. Perhaps @JSMorley can chime in on why this would be. It is rather tedious to find a location with an expiring alert to try this out, but it is doable.
Not quite sure why this would be honestly. The [@AlertsParent] parent measure will always return some value, and although if there are no alerts present, no child measures will be populated, the RegExp won't "fail", as they are all lookaheads.

I wonder if both the "string" and "number" values are being kept in this case?
MikeG621
Posts: 81
Joined: March 18th, 2013, 1:59 pm

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by MikeG621 »

jsmorley wrote: August 24th, 2020, 2:47 pmI wonder if both the "string" and "number" values are being kept in this case?
It's keeping just the number value, the string is blank.
json.PNG
You do not have the required permissions to view the files attached to this post.
User avatar
jsmorley
Developer
Posts: 21376
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON - NEW!

Post by jsmorley »

MikeG621 wrote: August 24th, 2020, 9:23 pm It's keeping just the number value, the string is blank.
json.PNG
Hm... So it looks like the propensity of WebParser to "keep" existing values when no values are returned on a subsequent update might be getting in the middle of this, since when there are no alerts, literally nothing is returned. I think "number" values are more susceptible to this in WebParser.

I think we might need to send a [!CommandMeasure MeasureName "Reset"] just before we go out to look for alerts each time, but I need to chew on where to put that to make sense. You can't !CommandMeasure a measure that doesn't exist, or you get log blight. So for the many folks who don't use the Alerts .inc file, that becomes a factor.

I will explore a bit, but this tends to lead me to think that the .inc for Alerts might need to actually go out to the site itself, rather than just depending on the "super parent" contained in WeatherComJSONMeasures.inc. Then I can safely have a "super parent" measure that first does a "reset" on the "not as super parent" alert measure, followed by an "update" of that parent. In effect starting fresh and clean on each iteration of UpdateRate.