I know you are looking to build something that in theory won't break if they change the JSON, but I'm just not going to go on that journey with you...
If they change the JSON again, it won't be something simple like the order of the fields, or adding a space where there wasn't one before. If they change it again, I'm going to have to just get a dump, look at it, and rework the RegExp. I'm resigned to that, as I think trying to predict how they might change it is doomed to failure, and can only end in tears.
It is currently September 20th, 2024, 9:41 am
⭐ Weather.com - Parsing the V3 JSON
-
- Developer
- Posts: 22747
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
-
- Developer
- Posts: 22747
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: ⭐ Weather.com - Parsing the V3 JSON
Ok...
Edit June 19, 2020 - Simplified the RegExp for "forecast" somewhat. This is just to make the regular expression a bit simpler and hopefully easier to follow, and doesn't impact any functionality. Also, re-ordered the measures in the "current" section, to make it a bit easier to deal with in About/Skins.
Get the .rmskin in the first post of this thread
Edit June 19, 2020 - Simplified the RegExp for "forecast" somewhat. This is just to make the regular expression a bit simpler and hopefully easier to follow, and doesn't impact any functionality. Also, re-ordered the measures in the "current" section, to make it a bit easier to deal with in About/Skins.
Get the .rmskin in the first post of this thread
-
- Rainmeter Sage
- Posts: 2738
- Joined: March 23rd, 2015, 5:26 pm
Re: ⭐ Weather.com - Parsing the V3 JSON
...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
-
- Rainmeter Sage
- Posts: 8188
- Joined: February 27th, 2015, 2:38 pm
- Location: Terra Yincognita
Re: ⭐ Weather.com - Parsing the V3 JSON
I know, I know. That's ok, I'm just going to enjoy the result alone. I'm never swayed or discouraged by the fact that the rest choose a different path than mine, so no worries.
Yeah, you used that "end in tears" a couple of times when we had such conversations, and apparently it never ended in tears for me when you said it would. We'll just have to wait and see, when it happens, now won't we? I, for once, am ready - way readier than the last time this occured. Are you ready to rework the regex at every minor change they could make?jsmorley wrote: ↑June 20th, 2020, 12:40 amIf they change the JSON again, it won't be something simple like the order of the fields, or adding a space where there wasn't one before. If they change it again, I'm going to have to just get a dump, look at it, and rework the RegExp. I'm resigned to that, as I think trying to predict how they might change it is doomed to failure, and can only end in tears.
-
- Rainmeter Sage
- Posts: 5528
- Joined: April 12th, 2012, 9:40 pm
- Location: Cedar Point, Ohio, USA
Re: ⭐ Weather.com - Parsing the V3 JSON
DITTO!SilverAzide wrote: ↑June 20th, 2020, 1:25 am ...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
:: My DA Gallery :: Rainmeter DA Gallery :: Rainmeter Workshops :: Rainmeter Documentation :: BBCode Guide ::
-
- Rainmeter Sage
- Posts: 8188
- Joined: February 27th, 2015, 2:38 pm
- Location: Terra Yincognita
Re: ⭐ Weather.com - Parsing the V3 JSON
I'm in love with the fact that OnyxBlack used the type of quantifiers that saved my regexes from a ton of otherwise impossible situations over the past years. Mine are a bit different, like {0,N}+ instead of just {N}. I used (and still use) them in all kinds of places in my skins, not just in the weather one. They are simply invaluable for (dynamic or not) iteration over a known or unknown quantity of items.SilverAzide wrote: ↑June 20th, 2020, 1:25 am ...And I'm 110% in love with the fact that it all seems stable and working again. Massive thanks!
My sincere congratulations! Not just to him, but to everyone involved. Including you, SilverAzide.
-
- Developer
- Posts: 22747
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: ⭐ Weather.com - Parsing the V3 JSON
I'll cross that bridge when I come to it. Actually, the JSON code is not complicated at all, and pretty easy to read. The RegExp gets a little complicated, due to using Non-Capturing Groups in order to be able to just create the RegExp for one "day" and then just simply copy/paste it to all the others. However, the way I have worked this each element like IconCode:, windPhrase:, sunriseTimeLocal: and each and every other one all use exactly the same regular expression to capture them, so moving them around at least should be trivial. In that case, I just need to move the names around.Yincognito wrote: ↑June 20th, 2020, 1:31 am I know, I know. That's ok, I'm just going to enjoy the result alone. I'm never swayed or discouraged by the fact that the rest choose a different path than mine, so no worries.
Yeah, you used that "end in tears" a couple of times when we had such conversations, and apparently it never ended in tears for me when you said it would. We'll just have to wait and see, when it happens, now won't we? I, for once, am ready - way readier than the last time this occured. Are you ready to rework the regex at every minor change they could make?
We will see how it goes. It's been stable for 2-3 weeks now, and I'm hopeful they are done for a while. If they decide to roll out a V4 at some point, I'll just quietly scream and get at it.
-
- Rainmeter Sage
- Posts: 8188
- Joined: February 27th, 2015, 2:38 pm
- Location: Terra Yincognita
Re: ⭐ Weather.com - Parsing the V3 JSON
Yep, there are some easily changeable things in your structure as well, so it's probably better than last time from this point of view. The V4 has a long way to go, I wouldn't worry about that for a couple of years. I mean, they just unified things on V3 (like some of us suspected right from the start), they won't switch to V4 yet, especially since I never saw it used for weather stuff yet - and I have looked in quite a lot of places, LOL.jsmorley wrote: ↑June 20th, 2020, 1:55 am I'll cross that bridge when I come to it. Actually, the JSON code is not complicated at all, and pretty easy to read. The RegExp gets a little complicated, due to using Non-Capturing Groups in order to be able to just create the RegExp for one "day" and then just simply copy/paste it to all the others. However, the way I have worked this each element like IconCode:, windPhrase:, sunriseTimeLocal: and each and every other one all use exactly the same regular expression to capture them, so moving them around at least should be trivial. In that case, I just need to move the names around.
We will see how it goes. It's been stable for 2-3 weeks now, and I'm hopeful they are done for a while. If they decide to roll out a V4 at some point, I'll just quietly scream and get at it.
So yeah, hold back your quiet scream on this for the time being - if changes happen, they won't be about V4. But changes will happen, as it's an IBM business for some time, and they probably want to put their mark on stuff and tweak them for their own benefit.
-
- Developer
- Posts: 22747
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: ⭐ Weather.com - Parsing the V3 JSON
So this was about as simple as I could make the regular expression to capture one "pair" of values for a day, the "day" and "night" values:
"iconCode":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]]
What it says is:
Search for the value type "iconCode":
Search for an \[ open bracket
Start a Non-Capturing Group (?:
Search for and skip over without capturing
Any number of any character .*
Followed by a character set [
Of a comma , OR | a close bracket \]
End the character set ]
End the Non-Capturing Group )
Do this skip-over 0 {0} times
Capture any number of any character (.*)
End the capture when a comma , is reached
Capture any number of any character (.*)
End the capture when a characters set [
Of a comma , OR | a close bracket \] is reached.
End the character set ]
This should properly account for the JSON format of:
"iconCode":[null,29,4,27,38,29,4,29,38,29,38,47,30,29,34,33,30,29,30,29,37,47,38,47,38,47,30,47,38,47]
Since the values are enclosed in [brackets], and are comma delimited, EXCEPT for the last one, the "night" part of the 15th day. Thus the reliance on [,|\]], a character set of either a comma , OR | a close bracket ] to end the searches/captures.
Again, Strings are enclosed in "quotes" and numbers are not, but that doesn't matter at all. Deal with the "quotes" later.
The number of {times} you skip-over without capturing gets you to the particular "day" you are interested in. So you skip to the first value for the given day, the "daytime" value, then capture it and the next one, the "nighttime" value.
Note that the first 8 types of data are for the entire day, and so there are only 15 entries each, not 30. Just don't treat those as "pairs". Just get one per day. Starting with cloudCover: it switches to "day" and "night" and there are 30 entries, in "pairs".
And you end up with:
"iconCode":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]]
What it says is:
Search for the value type "iconCode":
Search for an \[ open bracket
Start a Non-Capturing Group (?:
Search for and skip over without capturing
Any number of any character .*
Followed by a character set [
Of a comma , OR | a close bracket \]
End the character set ]
End the Non-Capturing Group )
Do this skip-over 0 {0} times
Capture any number of any character (.*)
End the capture when a comma , is reached
Capture any number of any character (.*)
End the capture when a characters set [
Of a comma , OR | a close bracket \] is reached.
End the character set ]
This should properly account for the JSON format of:
"iconCode":[null,29,4,27,38,29,4,29,38,29,38,47,30,29,34,33,30,29,30,29,37,47,38,47,38,47,30,47,38,47]
Since the values are enclosed in [brackets], and are comma delimited, EXCEPT for the last one, the "night" part of the 15th day. Thus the reliance on [,|\]], a character set of either a comma , OR | a close bracket ] to end the searches/captures.
Again, Strings are enclosed in "quotes" and numbers are not, but that doesn't matter at all. Deal with the "quotes" later.
The number of {times} you skip-over without capturing gets you to the particular "day" you are interested in. So you skip to the first value for the given day, the "daytime" value, then capture it and the next one, the "nighttime" value.
Note that the first 8 types of data are for the entire day, and so there are only 15 entries each, not 30. Just don't treat those as "pairs". Just get one per day. Starting with cloudCover: it switches to "day" and "night" and there are 30 entries, in "pairs".
And you end up with:
To get all the values, both day and night, for "today".RegExp=(?siU)"getSunV3DailyForecastUrlConfig":.*"duration:15day;.*"dayOfWeek":\[(?:.*[,|\]]){0}(.*)[,|\]].*"narrative":\[(?:.*[,|\]]){0}(.*)[,|\]].*"qpf":\[(?:.*[,|\]]){0}(.*)[,|\]].*"qpfSnow":\[(?:.*[,|\]]){0}(.*)[,|\]].*"sunriseTimeLocal":\[(?:.*[,|\]]){0}(.*)[,|\]].*"sunsetTimeLocal":\[(?:.*[,|\]]){0}(.*)[,|\]].*"temperatureMax":\[(?:.*[,|\]]){0}(.*)[,|\]].*"temperatureMin":\[(?:.*[,|\]]){0}(.*)[,|\]].*"cloudCover":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"dayOrNight":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"daypartName":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"iconCode":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"narrative":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"precipChance":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"precipType":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"qpf":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"qpfSnow":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"relativeHumidity":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"snowRange":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperature":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperatureHeatIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"temperatureWindChill":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"thunderCategory":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"thunderIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"uvDescription":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"uvIndex":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windDirection":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windDirectionCardinal":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windPhrase":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"windSpeed":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"wxPhraseLong":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*"wxPhraseShort":\[(?:.*[,|\]]){0}(.*),(.*)[,|\]].*
-
- Posts: 139
- Joined: September 27th, 2017, 6:32 pm
- Location: Netherlands
Re: ⭐ Weather.com - Parsing the V3 JSON
Adding "?temp=" to the URL makes it possible to manually choose the units
Possible values for the parameter are "c", "f", "m", or "e".
I found out by clicking the little "weather.com" link below the widget when searching for "weather" on Google.
Possible values for the parameter are "c", "f", "m", or "e".
I found out by clicking the little "weather.com" link below the widget when searching for "weather" on Google.