⭐ Weather.com - Parsing the V3 JSON

[jsmorley]

Seems to me that I need to stop using those "pairs" entirely, and just individually capture each value in the "array" as a separate entity, and then just pass the day/night indicator, "D" or "N" as a part of the values.

If the very first indicator is "N" instead of "D", then I just disable the final measure so I don't get an error, or use a lookahead on it.

[Yincognito]

I'd prefer to keep the approach the same from the standpoint of the end user, where "today" just returns null values that can be reacted to with IfMatch. This is going to require some thought on how I force in null values for "today" and then push the others forward by one, and deal with the missing data at the end of the parsing.

As long as this doesn't hang Rainmeter due to empty capture (shouldn't happen for WebParser though, only for String measures), it should be a starting point:

Code: Select all

RegExp=(?siU)(?|.*"dayInd":\["D".*"grassPollenIndex":\[(?:.*[,|\]]){0}(.*),|())

Tested with a string similar to what pbutler6 posted.

EDIT: Just read your last posts, and yes, I concur. Treating the field values individually is best. I already do that, but well, our approaches are a bit different, so it isn't relevant.

[jsmorley]

Yeah, I'm starting to think that the best approach, although it's a bit different for the skin author than the other data is to:

capture each individual element of each "array" as a separate entity. Add captures for the date, day/night indicator (D/N), and the day name value (today/tonight/tomorrow/tomorrow night/thursday/thursday night) and create measures for those.

I also think I will change this to a 14day version of the .inc, so it will always return 0-26 values and won't ever "run out of" data. It might start with "day" and end with "night" or start with "night" and end with "day", but it will be a consistent number of values.

I wish there was a V3 version of this, although I don't see one. The old V2 version of this stuff was I think tailored to the "cards" that used to be on the actual website, that worked like that. During the day it started with "today/tonight" and at night it started with "tonight/tomorrow". I'm betting a V3 version this data would be indexed the same as the other V3 stuff.

[Yincognito]

Yeah, I'm starting to think that the best approach, although it's a bit different for the skin author than the other data is to:

capture each individual element of each "array" as a separate entity. Add captures for the date, day/night indicator (D/N), and the day name value (today/tonight/tomorrow/tomorrow night/thursday/thursday night) and create measures for those.

Have at least the 30th capture be a (?(?= lookahead) so if they decide to use the full 15day .inc file, they don't get an error.

I wish there was a V3 version of this, although I don't see one. The old V2 version of this stuff was I think tailored to the "cards" that used to be on the actual website, that worked like that. During the day it started with "today/tonight" and at night it started with "tonight/tomorrow". I'm betting a V3 version this data would be indexed the same as the other V3 stuff.

I don't use lookaheads for this, I just take how many groups are available through the (?<group>){0,N}+ approach:

Code: Select all

(?siU)"grassPollenIndex":\[(?:.*[,|\]]){0,29}+([^"\{\[\]\}]*)[,|\]]

which automatically makes non existing captures blank, but then I use it on String measures and have empty capture failsafe mechanisms, like replacing stuff with _\1_ or <\1> instead of simply \1 (so that the result isn't empty, even if the capture is), clearing the garbage afterwards. You'd probably be more comfortable with lookaheads though - I just shared this as an alternative, that's all.

Yeah, I tried the V3 approach on pollen, but just couldn't nail the right section name (couldn't use V1 so I settled for V2). As I said, it takes some attempts and permutations (if the thing is available in V3 for this API Key, that is) to find the correct URL syntax, as the documentation, besides being "confidential", is quite sparse on these things...

I'm not sure it would change things if a V3 was available and known by us. Maybe it's a pollen thing, after all.

[jsmorley]

Capturing the values that have distinct "day/night" components was mostly done in "pairs" like that for the purpose of organizing the measures into "days" in the .inc file, so they were displayed that way in About / Skins. It just made things a bit easier to follow when creating a skin.

There is no other particular value in pairing up the values like that.

[jsmorley]

As to any lookaheads, I'm now leaning toward this:

I also think I will change this to a 14day version of the .inc, so it will always return 0-26 values and won't ever "run out of" data. It might start with "day" and end with "night" or start with "night" and end with "day", but it will be a consistent number of values.

[Yincognito]

As to any lookaheads, I'm now leaning toward this:

As you wish. Personally, I don't mind non-existing values at all, I just substitute ^$ with a default no data string in each WebParser child, e.g. "(?i)(?:^$|^N\/A$|^null$)":"Not Available", and be done with it.

In fact, I intentionally set my "forecast days" variable to 16 (i.e. greater than 15) just to make sure making stuff blank / default works in my implementation. But then, as you no doubt noticed, my methods aren't suited for anyone, so in the end it's each skin designer's choice how to handle such scenarios.

[jsmorley]

As you wish. Personally, I don't mind non-existing values at all, I just substitute ^$ with a default no data string in each WebParser child, e.g. "(?i)(?:^$|^N\/A$|^null$)":"Not Available", and be done with it.

In fact, I intentionally set my "forecast days" variable to 16 (i.e. greater than 15) just to make sure making stuff blank / default works in my implementation. But then, as you no doubt noticed, my methods aren't suited for anyone, so in the end it's each skin designer's choice how to handle such scenarios.

Yes, but it's not about the final result, it's about the error in the log when you attempt to parse a value that doesn't exist. If I try to parse that 30th value from the array, which will be there 1/2 of each day, that will be a problem. A lookahead solves that, but I don't see that it helps the user to have an "empty" 30th value, when in fact it is the expected 1st value that is really missing.

[jsmorley]

In case anyone here hasn't tumbled to this yet, if you are writing a skin based on these Pollen .inc files, STOP. I should have something new by tomorrow sometime, but it won't in any way be the same.

[jsmorley]

So I'm looking at keeping it simple.

Code: Select all

; =================================================================================================
; WeatherComJSONPollen.inc by JSMorley
; "Weather.com - Parsing the JSON" (https://forum.rainmeter.net/viewtopic.php?f=118&t=34628)
;
; Requires that WeatherComJSONVariables.inc be @Included in the skin before this file.
;
; Updated August 4, 2020
; ==================================================================================================

[@PollenSuperParent]
Measure=WebParser
URL=https://api.weather.com/v2/indices/pollen/daypart/15day?geocode=#Latitude#,#Longitude#&language=#Language#&format=json&apiKey=#APIKey#
UpdateRate=#UpdateRate#
Flags=Resync | NoCookies
UserAgent=#UserAgent#
LogSubstringErrors=0
StringIndex=1
RegExp=(?siU)^(.*)$
FinishAction=[!EnableMeasureGroup Parents]

; ====================================================
; Create Parent measures for the various data elements
; ====================================================

[@PollenValidDateStampsParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"fcstValidLocal":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)
FinishAction=[!EnableMeasureGroup Times]

[@PollenDayPartIndicatorParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"dayInd":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)
FinishAction=[!EnableMeasureGroup Times]

[@PollenDayPartNameParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"daypartName":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenGrassIndexParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"grassPollenIndex":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenGrassCategoryParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"grassPollenCategory":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenTreeIndexParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"treePollenIndex":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenTreeCategoryParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"treePollenCategory":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenRagweedIndexParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"ragweedPollenIndex":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

[@PollenRagweedCategoryParent]
Measure=WebParser
Group=Parents
Disabled=1
URL=[@PollenSuperParent]
RegExp=(?siU)"ragweedPollenCategory":\[(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*),(.*)

; ====================================================
; Part 1
; ====================================================

[@PollenPart1ValidDateStamp]
Measure=WebParser
URL=[@PollenValidDateStampsParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1ValidWeekDay]
Measure=Time
Group=Times
Disabled=1
TimeStamp=[@PollenPart1ValidDateStamp]
DynamicVariables=1
TimeStampFormat=%Y-%m-%dT%H:%M:%S
FormatLocale=#Language#
Format=%A

[@PollenPart1DayPartIndicator]
Measure=WebParser
URL=[@PollenDayPartIndicatorParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1DayPartName]
Measure=WebParser
URL=[@PollenDayPartNameParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1GrassIndex]
Measure=WebParser
URL=[@PollenGrassIndexParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1GrassCategory]
Measure=WebParser
URL=[@PollenGrassCategoryParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1TreeIndex]
Measure=WebParser
URL=[@PollenTreeIndexParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1TreeCategory]
Measure=WebParser
URL=[@PollenTreeCategoryParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1RagweedIndex]
Measure=WebParser
URL=[@PollenRagweedIndexParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart1RagweedCategory]
Measure=WebParser
URL=[@PollenRagweedCategoryParent]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

; ====================================================
; Part 2
; ====================================================

[@PollenPart2ValidDateStamp]
Measure=WebParser
URL=[@PollenValidDateStampsParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2ValidWeekDay]
Measure=Time
Group=Times
Disabled=1
TimeStamp=[@PollenPart2ValidDateStamp]
DynamicVariables=1
TimeStampFormat=%Y-%m-%dT%H:%M:%S
FormatLocale=#Language#
Format=%A

[@PollenPart2DayPartIndicator]
Measure=WebParser
URL=[@PollenDayPartIndicatorParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2DayPartName]
Measure=WebParser
URL=[@PollenDayPartNameParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2GrassIndex]
Measure=WebParser
URL=[@PollenGrassIndexParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2GrassCategory]
Measure=WebParser
URL=[@PollenGrassCategoryParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2TreeIndex]
Measure=WebParser
URL=[@PollenTreeIndexParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2TreeCategory]
Measure=WebParser
URL=[@PollenTreeCategoryParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2RagweedIndex]
Measure=WebParser
URL=[@PollenRagweedIndexParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[@PollenPart2RagweedCategory]
Measure=WebParser
URL=[@PollenRagweedCategoryParent]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

All those "child" measures will be repeated 29 times. That way the missing 30th index at night won't factor in.

1.jpg