It is currently September 23rd, 2020, 4:33 pm

⭐ Weather.com - Parsing the V3 JSON - NEW!

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the JSON

Post by SilverAzide »

OnyxBlack wrote: June 3rd, 2020, 6:31 pm Fixed, should work with all locales now (based on language set in the Variables file)

Unit types have been reenabled.
Excellent, and in the nick of time too. Just got hit with V3 JSON again today for the first time since the first day this started (clearing the cookies has fixed it for me). Great work!!!
Gadgets Wiki GitHub More Gadgets...
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the JSON

Post by SilverAzide »

OnyxBlack wrote: June 3rd, 2020, 6:31 pm Fixed, should work with all locales now (based on language set in the Variables file)

Unit types have been reenabled.
I think the "Narrative" regex is busted... if there is a comma in the text, it's chopping off everything before the comma. :( It's also grabbing the wrong narratives; instead of grabbing the ones in the "daypart" section, it's grabbing some from the data section and some from the 15 day forecast?
Last edited by SilverAzide on June 3rd, 2020, 9:10 pm, edited 1 time in total.
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 2589
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the JSON

Post by Yincognito »

SilverAzide wrote: June 3rd, 2020, 8:05 pm I think the "Narrative" regex is busted... if there is a comma in the text, it's chopping off everything before the comma. :(
But should be a comma in the narrative though? In other words, have you seen instances where there is a comma in those texts (e.g. phrase, narrative, etc)?
OnyxBlack
Posts: 19
Joined: June 3rd, 2020, 10:06 am

Re: ⭐ Weather.com - Parsing the JSON

Post by OnyxBlack »

SilverAzide wrote: June 3rd, 2020, 8:05 pm I think the "Narrative" regex is busted... if there is a comma in the text, it's chopping off everything before the comma. :(
True, if a value contains a comma, the regex will not work (or match prematurely). I hadn't thought of that, then again I have yet to encounter a comma in a value that is not a delimiter. On first thought I assumed it would suffice to simply match with the surrounding "" quotation marks, but then there are these pesky Nulls that don't fit that pattern. Not sure if "narrative" suffers from occasional nulls though.
Thoughts?
User avatar
Yincognito
Posts: 2589
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the JSON

Post by Yincognito »

OnyxBlack wrote: June 3rd, 2020, 8:32 pm[...] but then there are these pesky Nulls that don't fit that pattern. Not sure if "narrative" suffers from occasional nulls though.
Thoughts?
And also numbers, they are not enclosed by quotes - not in the narrative, but in general. The possibility of a comma existing in a string value would complicate a bit the separation and parsing of those values indeed. Let's see what SilverAzide has to say about actually seeing a comma in one of those strings. Me, haven't seen them there either.
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the JSON

Post by SilverAzide »

Yincognito wrote: June 3rd, 2020, 8:40 pm And also numbers, they are not enclosed by quotes - not in the narrative, but in general. The possibility of a comma existing in a string value would complicate a bit the separation and parsing of those values indeed. Let's see what SilverAzide has to say about actually seeing a comma in one of those strings. Me, haven't seen them there either.
Here's a narrative string I got..

"Scattered thunderstorms early, then cloudy skies after midnight. A few storms may be severe. Low around 70F. W winds at 10 to 20 mph, decreasing to less than 5 mph. Chance of rain 50%."

The regex is giving "decreasing to less than 5 mph. Chance of rain 50%." and chopping off everything else. :( Notice this one has TWO commas.

It's also grabbing the wrong narratives; instead of grabbing the ones in the "daypart" section, it's grabbing some from the data section and some from the 15 day forecast?
Gadgets Wiki GitHub More Gadgets...
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the JSON

Post by SilverAzide »

OnyxBlack wrote: June 3rd, 2020, 8:32 pm True, if a value contains a comma, the regex will not work (or match prematurely). I hadn't thought of that, then again I have yet to encounter a comma in a value that is not a delimiter. On first thought I assumed it would suffice to simply match with the surrounding "" quotation marks, but then there are these pesky Nulls that don't fit that pattern. Not sure if "narrative" suffers from occasional nulls though.
Thoughts?
Yep, there's nulls in there (when it's night)...

Code: Select all

"narrative": [
    null,
    "Scattered thunderstorms early, then cloudy skies after midnight. A few storms may be severe. Low around 70F. W winds at 10 to 20 mph, decreasing to less than 5 mph. Chance of rain 50%.",
    "Partly cloudy early with thunderstorms becoming likely during the afternoon. High 89F. Winds WSW at 5 to 10 mph. Chance of rain 80%.",
    "Scattered thunderstorms in the evening, with mostly cloudy skies overnight. Low 68F. Winds light and variable. Chance of rain 60%.",
    "Sun and clouds mixed with a slight chance of thunderstorms during the afternoon. High 88F. Winds SW at 5 to 10 mph. Chance of rain 30%.",
    "Partly cloudy. Low 69F. Winds light and variable.",
    "Partly cloudy with isolated thunderstorms possible. High 86F. Winds WNW at 10 to 15 mph. Chance of rain 30%.",
    "A mostly clear sky. Low 61F. Winds NW at 5 to 10 mph.",
    "Sunny along with a few clouds. High near 80F. Winds NNW at 5 to 10 mph.",
    "Partly cloudy. Low 54F. Winds N at 5 to 10 mph.",
    "Sunshine. High around 80F. Winds light and variable.",
    "A mostly clear sky. Low 57F. Winds light and variable.",
    "A mainly sunny sky. High 86F. Winds SSE at 5 to 10 mph.",
    "A few clouds from time to time. Low 64F. Winds SSE at 5 to 10 mph.",
    "A few showers in the morning with scattered thunderstorms arriving in the afternoon. High 83F. Winds S at 10 to 15 mph. Chance of rain 50%.",
    "Mostly cloudy with showers and a few thunderstorms. Low 67F. Winds S at 10 to 15 mph. Chance of rain 50%."
]
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 2589
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the JSON

Post by Yincognito »

Yep. For reference only, the JSON specification might be of some help. As for dealing with the comma thing, there are at least two possibilities, although I hoped that they wouldn't be necessary: simple character sets or lookarounds.

EDIT: By the way, JSON doesn't allow trailing commas in values/strings, which means the role of a comma as a delimiter is based on how it's positioned regarding to quotes. Generally, a comma should be immediately (bar spacing) preceded or followed by a quote, a number, a bracket or null (in theory, it could even be another comma, but that's quite unlikely) to be considered a delimiter. Again, that's where a stable, Rainmeter team supported JSON parser would be of help. Sure, regex is much more flexible than a parser, but there are times when the latter is easier to use, and no one says one should exclude the other.
Last edited by Yincognito on June 3rd, 2020, 9:39 pm, edited 1 time in total.
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the JSON

Post by SilverAzide »

Yincognito wrote: June 3rd, 2020, 9:27 pm Yep. For reference only, the JSON specification might be of some help. As for dealing with the comma thing, there are at least two possibilities, although I hoped that they wouldn't be necessary: simple character sets or lookarounds.
Regexps make my brain hurt... but just a dumb question for the experts. For this V3 JSON, wouldn't it be a lot simpler to grab the data from these arrays with something like (pseudo-code!!)
RegExp="arrayName:[(".*"),(".*"),(".*"),(".*"),(".*"),(".*"),(".*")]"

Since the arrays are known lengths, you'd know the exact number of captures to use. :confused:
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 2589
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the JSON

Post by Yincognito »

SilverAzide wrote: June 3rd, 2020, 9:37 pm Regexps make my brain hurt... but just a dumb question for the experts. For this V3 JSON, wouldn't it be a lot simpler to grab the data from these arrays with something like (pseudo-code!!)
RegExp="arrayName:[(".*"),(".*"),(".*"),(".*"),(".*"),(".*"),(".*")]"

Since the arrays are known lengths, you'd know the exact number of captures to use. :confused:
Yeah, not mine - I like them, they're easy to get, it's only some characters used to express other characters, after all. :D
Regarding arrays, I prefer an "iteration" based on regex quantifiers instead of hardcoding the entire array, like explained here. As a side note, you used a 7 day example above, but I would like to use the 15 day forecast (since my skin is "scrollable", so I could use there any number of days and display the currently scrolled one), and in that case some arrays are of 7 elements, some are of 15 elements. Not a huge problem, since a formula based on day and day part index can be used, but just saying...

P.S. Using that kind of iteration though is somewhat tricky, as it is better suited for String measures instead of WebParser ones, as the former are much friendlier to dynamic variables in the regex related options than the latter (which would require a [!CommandMeasure MeasureName "Update"], which has the potential to ruin things up by unnecessarily polling the site if a ForceReload=1 option is used in the WebParser parent).