It is currently July 7th, 2020, 3:59 am

⭐ Weather.com - Parsing the V3 JSON

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 20995
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

Edit June 18, 2020 - Revised the WeatherComJSONMoon.inc file, to include the new measure for Moon Phase Days/Age, as well as extending it to support the entire 15 days included in the JSON. I have also created a WeatherComJSONMoon7Day.inc file, which will only parse 7 days, if you don't want or need the full 15.

Get the rmskin in the first post of this thread.
nikko
Posts: 22
Joined: December 5th, 2017, 5:58 pm

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by nikko »

hello. please explain somebody what this things do here RegExp=(?siU)"getSunV3DailyForecastUrlConfig":.*dayOfWeek":\s*\[(?:".*",\s*){0}(".*").*
thanx
SCR
Posts: 51
Joined: April 15th, 2015, 11:13 pm

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by SCR »

jsmorley wrote:
June 18th, 2020, 6:46 pm
Edit June 18, 2020 - Revised the WeatherComJSONMoon.inc file, to include the new measure for Moon Phase Days/Age, as well as extending it to support the entire 15 days included in the JSON. I have also created a WeatherComJSONMoon7Day.inc file, which will only parse 7 days, if you don't want or need the full 15.

Get the rmskin in the first post of this thread.
Thanks JSMorley. The new "Moon Phase Days/Age" is nice.
User avatar
Yincognito
Posts: 1866
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

nikko wrote:
June 19th, 2020, 7:42 pm
hello. please explain somebody what this things do here RegExp=(?siU)"getSunV3DailyForecastUrlConfig":.*dayOfWeek":\s*\[(?:".*",\s*){0}(".*").*
thanx
Technically, it looks in a dot-all (i.e. including newline characters), case-insensitive and ungreedy (i.e. taking as less characters as possible when using *) fashion (by using the (?siU) regex flags) for:
- the string "getSunV3DailyForecastUrlConfig":
- followed by any character, any number of times (i.e. the .*)
- followed by the string dayOfWeek":
- followed by any number of spaces (i.e. \s*)
- followed by a [ (i.e. \[, as [ needs to be escaped preceding it with \ in regexp, since is a reserved character)
- followed by any quote enclosed string that is succeeded by a comma and any number of spaces, all of it taken 0 times (i.e. (?:".*",\s*){0}; of course, taking a string 0 times means no character at all, so the 0 quantifier was probably used in order to be consistent with a similar place where the quantifier is different from 0)
- followed by any quote enclosed string that can be then referenced as a capture, like \1, \2, etc. (i.e. (".*"))
- followed by any number of characters any number of times (i.e. .*)

In simple, non technical terms, it looks for the value of the first day of week in the daily forecast section of the weather.com JSON being parsed in the skin.

Hint: changing the quantifier above from, say, {0} to {1} will instead look for the value of the second day of week in the same section of the JSON (since a comma followed enclosed string is basically how a value is stored in a JSON array, like, for example, ["Sun", "Mon", "Tue"]). Further increasing the quantifier will "iterate" through the rest of the day of week values in the array.
Last edited by Yincognito on June 19th, 2020, 9:27 pm, edited 1 time in total.
User avatar
jsmorley
Developer
Posts: 20995
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

Yincognito wrote:
June 19th, 2020, 9:10 pm
Technically, it looks in a dot-all (i.e. including newline characters), case-insensitive and ungreedy (i.e. taking as less characters as possible when using *) fashion (by using the (?siU) regex flags) for:
- the string "getSunV3DailyForecastUrlConfig":
- followed by any character, any number of times (i.e. the .*)
- followed by the string dayOfWeek":
- followed by any number of spaces (i.e. \s*)
- followed by a [ (i.e. \[, as [ needs to be escaped preceding it with \ in regexp, since is a reserved character)
- followed by any quote enclosed string that is succeeded by a comma and any number of spaces, all of it taken 0 times (i.e. (?:".*",\s*){0}; of course, taking a string 0 times means no character at all, so the 0 quantifier was used to be consistent with a similar place where the quantifier is different from 0)
- followed by any quote enclosed string that can be then referenced as a capture, like \1, \2, etc. (i.e. (".*"))
- followed by any number of characters any number of times (i.e. .*)

In simple, non technical terms, it looks for the value of the first day of week in the daily forecast section of the weather.com JSON being parsed in the skin.
Right, (?: is called a Non-Capturing Group. In this case, it is used to "skip over" some number of entries in a "series" of matches without capturing them. If you have a series like "dog","cat","fish","bird", and you want to know what the third entry in the series is, you would would use a non-capturing group to match on ".*", (any characters between "quotes" and followed by a comma, and do it {2} times. Then you capture the next instance of ".*", and Bob's Your Uncle. Saying {0} simply means "don't skip any, get the first one.".
nikko
Posts: 22
Joined: December 5th, 2017, 5:58 pm

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by nikko »

tnanx y.
and this please bit longer...
"cloudCover":\s*\[(?:.*,\s*){2}(.*),\s*(.*)(?:,|\]).*
User avatar
Yincognito
Posts: 1866
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

jsmorley wrote:
June 19th, 2020, 9:25 pm
Right, (?: is called a Non-Capturing Group. In this case, it is used to "skip over" some number of entries in a "series" of matches without capturing them. If you have a series like "dog","cat","fish","bird", and you want to know what the third entry in the series is, you would would use a non-capturing group to match on ".*", (any characters between "quotes" and followed by a comma, and do it {2} times. Then you capture the next instance of ".*", and Bob's Your Uncle.
Yep, it looks like we both completed the missing bits in my original reply, as I was editing my post at the same time as you posted yours. :D
User avatar
Yincognito
Posts: 1866
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by Yincognito »

nikko wrote:
June 19th, 2020, 9:27 pm
tnanx y.
and this please bit longer...
"cloudCover":\s*\[(?:.*,\s*){2}(.*),\s*(.*)(?:,|\]).*
I believe this is getting the 3rd and the 4th value from the cloud cover array, which are then probably referenced using \1 and \2 in the following code...
User avatar
jsmorley
Developer
Posts: 20995
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »

nikko wrote:
June 19th, 2020, 9:27 pm
tnanx y.
and this please bit longer...
"cloudCover":\s*\[(?:.*,\s*){2}(.*),\s*(.*)(?:,|\]).*
One of these "series" in the JSON will look like this:

"animalType":["dog","cat","fish","bird"]

In this case, you want to capture a "pair" of the values, as cloudCover has two entries per day, one "day" and one "night" so you skip over {2} times, and get the next two instances. In this case the "third" and "fourth".

To finish, you test, with a non-capturing group again, to see if the last instance you captured ends with a , or a ]. I'm not entirely convinced you need to use a non-capturing group for this, but it doesn't hurt.

The \s (white space) test is just to be on the safe side, in case the series looks like ["dog", "cat", "fish", "bird"]. So you test for spaces zero or more times.

It's part of what made it take some thought and care to get the RegExp right with this JSON, while all of the values are groups into these "series" based on the type of value, and each includes all 15 days worth, in some cases it is a single value for the "day", and thus 15, and in some cases it is for day,night,day,night... and thus 30. Also, strings are enclosed in "quotes", and numbers are not.

While it looks complicated, and let's face it, it is, the entire point of doing it this way is to make it as easy as possible to copy and paste the same RegExp over and over again for each of the 15 days, only changing the value in {n} from {0} to {14} to reflect which "day" you are going after. Any other approach would be hideously hand-crafted, and really long, RegExp options for each day.

I'm not entirely 100% in love with the RegExp that OnyxBlack came up with for this V3 JSON initially, as I while I think the non-capturing groups are a terrific idea, there are some other subtle complexities in the RegExp that I would probably have avoided. For instance, in my old V2 version, I didn't worry about, and differentiate, whether a value was a string in "quotes", or a number without them. I just captured the entire value, and used Substitute to strip off any quotes. This simplified handling the value of null you get when there is no data, without quotes, whether or not the other values in the series are strings or numbers.

I may tweak the RegExp's at some point, but man, I'm sorta sick of looking at weather code in general at the moment... I'm in the house with the air conditioning on, and don't give a tinker's damn if it's raining outside.

You are probably sorry you asked about now.. ;-)
OnyxBlack
Posts: 19
Joined: June 3rd, 2020, 10:06 am

Re: ⭐ Weather.com - Parsing the V3 JSON

Post by OnyxBlack »

jsmorley wrote:
June 19th, 2020, 9:34 pm
I'm not entirely 100% in love with the RegExp that OnyxBlack came up with for this V3 JSON initially...
:confused: