It is currently September 25th, 2020, 9:45 am

⭐ Weather.com - Parsing the V3 JSON - NEW!

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 21387
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

OnyxBlack wrote: June 19th, 2020, 11:34 pm:confused:
Trust me, I'm 99% in love with it... :thumbup:
User avatar
jsmorley
Developer
Posts: 21387
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

For instance OnyxBlack, I think that for a "forecast day", this:
RegExp=(?siU)"getSunV3DailyForecastUrlConfig":.*"duration:15day;.*"dayOfWeek":\[(?:.*[,|\]]){0}(.*)[,|\]].*"narrative":\[(?:.*[,|\]]){0}(.*)[,|\]].*"qpf":\[(?:.*[,|\]]){0}(.*)[,|\]].*"qpfSnow":\[(?:.*[,|\]]){0}(.*)[,|\]].*"sunriseTimeLocal":\[(?:.*[,|\]]){0}(.*)[,|\]].*"sunsetTimeLocal":\[(?:.*[,|\]]){0}(.*)[,|\]].*"temperatureMax":\[(?:.*[,|\]]){0}(.*)[,|\]].*"temperatureMin":\[(?:.*[,|\]]){0}(.*)[,|\]].*"cloudCover":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"dayOrNight":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"daypartName":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"iconCode":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"narrative":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"precipChance":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"precipType":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"qpf":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"qpfSnow":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"relativeHumidity":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"snowRange":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperature":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperatureHeatIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperatureWindChill":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"thunderCategory":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"thunderIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"uvDescription":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"uvIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windDirection":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windDirectionCardinal":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windPhrase":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windSpeed":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"wxPhraseLong":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"wxPhraseShort":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*
Would work just as well, and be significantly simpler, than:
RegExp=(?siU)"getSunV3DailyForecastUrlConfig":.*"duration:15day;.*"dayOfWeek":\s*\[(?:".*",\s*){0}(".*").*"narrative":\s*\[(?:".*",\s*){0}(".*").*"qpf":\s*\[(?:.*,\s*){0}(.*),.*"qpfSnow":\s*\[(?:.*,\s*){0}(.*),.*"sunriseTimeLocal":\s*\[(?:".*",\s*){0}(".*").*"sunsetTimeLocal":\s*\[(?:".*",\s*){0}(".*").*"temperatureMax":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),.*"temperatureMin":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),.*"cloudCover":\s*\[(?:.*,\s*){0}(.*),\s*(.*)(?:,|\]).*"dayOrNight":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"daypartName":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"iconCode":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"narrative":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"precipChance":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"precipType":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"qpf":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"qpfSnow":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"relativeHumidity":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"snowRange":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"temperature":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"temperatureHeatIndex":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"temperatureWindChill":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"thunderCategory":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"thunderIndex":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"uvDescription":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"uvIndex":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"windDirection":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"windDirectionCardinal":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"windPhrase":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"windSpeed":\s*\[(?:null,\s*|.*,\s*){0}(null|.*),\s*(null|.*)(?:,|\]).*"wxPhraseLong":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\]).*"wxPhraseShort":\s*\[(?:null,\s*|".*",\s*){0}(null|".*"),\s*(null|".*")(?:,|\])
The main difference(s) are:

1) I'm not worrying about the distinction between a "string" and a number. I let the #CommonSubstitute# take care of that.
2) I don't try to deal with the null value. Again, the #CommonSubstitute# changes that to an empty string.
3) I don't think you need the non-capturing group at the end of each sequence. I think just looking for a character set of [,|\]], comma OR right-bracket will do the trick.
4) I don't test for spaces where there just aren't any in this particular JSON.
User avatar
Yincognito
Posts: 2612
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

jsmorley wrote: June 19th, 2020, 11:40 pm Trust me, I'm 99% in love with it... :thumbup:
LMAO. Regarding capturing the entire value without bothering if it contains quotes or not, I already posted somewhere around here a "safe" solution, but just in case it is "lost" in the ton of posts in these threads, here it is:

Code: Select all

Data=(?:[^"\{\[\]\}]*|(?:(?>\\"|[^"])*"){2}+)
Stop=(?>,|\{|\[|\]|\}|$)
And I use it like (latitude sample):

Code: Select all

"[^"]*latitude[^"]*":(#Data#)#Stop#
The only thing that sometimes needs to be replaced by another pattern is the Stop variable, as the "right bound" of the value can sometimes be, say, a [,\]] (in the case of JSON arrays). The Data variable works in all cases, but it obviously needs to be followed by a "bound" of some kind. For example, this is the part where I get the N-th data in such an array:

Code: Select all

(?(?=\[)\[(?:#Data#[,\]]){0,#MomentIndex#}+)(#Data#)#Stop#
Even if not relevant for the topic, in case you wonder what #MomentIndex# is, is just (DayNumber*2+DayPartNumber), where DayNumber is the day number (starting at 0) and DayPartNumber is 0 if day and 1 if night.
User avatar
jsmorley
Developer
Posts: 21387
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

It really doesn't matter if the value contains quotes or not. I don't care.

IconName:[null,"Sunny","Cloudy"]
IconCode:[null,34,22]

Are both handled just fine with what I posted. I get whatever is between the commas, and then use a Substitute (#CommonSubstitute#) on all child measures that strip off any starting and ending quotes, and turn the static value null into an empty string.

That's all that's needed.
User avatar
jsmorley
Developer
Posts: 21387
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

I know you are looking to build something that in theory won't break if they change the JSON, but I'm just not going to go on that journey with you... ;-)

If they change the JSON again, it won't be something simple like the order of the fields, or adding a space where there wasn't one before. If they change it again, I'm going to have to just get a dump, look at it, and rework the RegExp. I'm resigned to that, as I think trying to predict how they might change it is doomed to failure, and can only end in tears.
User avatar
jsmorley
Developer
Posts: 21387
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

Ok...

Edit June 19, 2020 - Simplified the RegExp for "forecast" somewhat. This is just to make the regular expression a bit simpler and hopefully easier to follow, and doesn't impact any functionality. Also, re-ordered the measures in the "current" section, to make it a bit easier to deal with in About/Skins.

Get the .rmskin in the first post of this thread
User avatar
SilverAzide
Posts: 955
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by SilverAzide »

OnyxBlack wrote: June 19th, 2020, 11:34 pm:confused:
jsmorley wrote: June 19th, 2020, 11:40 pm Trust me, I'm 99% in love with it... :thumbup:
...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 2612
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

jsmorley wrote: June 20th, 2020, 12:40 amI know you are looking to build something that in theory won't break if they change the JSON, but I'm just not going to go on that journey with you... ;-)
I know, I know. :lol: That's ok, I'm just going to enjoy the result alone. ;-) I'm never swayed or discouraged by the fact that the rest choose a different path than mine, so no worries.
jsmorley wrote: June 20th, 2020, 12:40 amIf they change the JSON again, it won't be something simple like the order of the fields, or adding a space where there wasn't one before. If they change it again, I'm going to have to just get a dump, look at it, and rework the RegExp. I'm resigned to that, as I think trying to predict how they might change it is doomed to failure, and can only end in tears.
Yeah, you used that "end in tears" a couple of times when we had such conversations, and apparently it never ended in tears for me when you said it would. :D We'll just have to wait and see, when it happens, now won't we? I, for once, am ready - way readier than the last time this occured. Are you ready to rework the regex at every minor change they could make? 8-)
User avatar
eclectic-tech
Rainmeter Sage
Posts: 4115
Joined: April 12th, 2012, 9:40 pm
Location: Cedar Point, Ohio, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by eclectic-tech »

SilverAzide wrote: June 20th, 2020, 1:25 am ...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
DITTO! :rosegift:
User avatar
Yincognito
Posts: 2612
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

SilverAzide wrote: June 20th, 2020, 1:25 am ...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
I'm in love with the fact that OnyxBlack used the type of quantifiers that saved my regexes from a ton of otherwise impossible situations over the past years. Mine are a bit different, like {0,N}+ instead of just {N}. I used (and still use) them in all kinds of places in my skins, not just in the weather one. They are simply invaluable for (dynamic or not) iteration over a known or unknown quantity of items.

My sincere congratulations! Not just to him, but to everyone involved. Including you, SilverAzide. :D