⭐ Weather.com - Parsing the V3 JSON

[jsmorley]

I don't follow. To me "when the parent hasn't even updated yet" is pretty much the same as "JUST BEFORE the parent measure updates the values", since children are a function of the parent. I understand the subtle difference in what you said, but from a practical point of view, it shouldn't make any difference: once that child measure is 0, it can't be set back to the "past" value of 4 or whatever if we're talking about a leftover from the previous WebParser update, can it? I mean, unless that non zero value is set (or kept) because it remains there in the weather.com source itself (which would be unfortunate), the following parent update will just skip both the number and the string value of the child measure since "if there are no alerts present, no child measures will be populated", like you said earlier.

Yeah, what I'm saying is that if you have some measure that is unrelated to the parent measure for the alerts, you need to be careful when it does the enable/disable or "Reset", as you don't want all the child measures to display as "zero" while they wait for the alert parent measure to actually execute. What I'm saying is that you really want one single process that:

1) Reset all child values to an empty string and a zero number.
2) Update the parent measure
3) The parent measure will populate all the child measures

I don't see any way to have that be a single, connected series of actions, that ensures they are done in the right order at the right time.

[Brian]

Just to chime in here. It looks as if the old Webparser plugin would default the "number" value to 0 on every update. This was omitted (probably by accident) when we converted the plugin to a measure.

Took a few years to catch!

If you look at the committed code from when we switched the plugin to a measure, you will see the plugin used to default the "number" value to 0.
https://github.com/rainmeter/rainmeter/blob/6ae599238785dcd5676db1b0b16b3f954ecd167a/Plugins/PluginWebParser/WebParser.cpp#L759-L827

However, in the same commit, you can see the "new" measure code does not default the number value to "0".
https://github.com/rainmeter/rainmeter/blob/1f8b3e0434eef8a0fa196e4510accf2d27e18939/Library/MeasureWebParser.cpp#L378-L441

This will be fixed in the next beta.

-Brian

[jsmorley]

Just to chime in here. It looks as if the old Webparser plugin would default the "number" value to 0 on every update. This was omitted (probably by accident) when we converted the plugin to a measure.

Took a few years to catch!

If you look at the committed code from when we switched the plugin to a measure, you will see the plugin used to default the "number" value to 0.
https://github.com/rainmeter/rainmeter/blob/6ae599238785dcd5676db1b0b16b3f954ecd167a/Plugins/PluginWebParser/WebParser.cpp#L759-L827

However, in the same commit, you can see the "new" measure code does not default the number value to "0".
https://github.com/rainmeter/rainmeter/blob/1f8b3e0434eef8a0fa196e4510accf2d27e18939/Library/MeasureWebParser.cpp#L378-L441

This will be fixed in the next beta.

-Brian

Ah great. I'll get that out later this morning. That's good news actually, as the "work around" for this behavior was a bad Rube Goldberg. The mouse kept getting out of the trap...

[jsmorley]

Beta r3404 is out to address this.

[Yincognito]

Beta r3404 is out to address this.

It seems changes in weather.com result in an increased number of Rainmeter updates and fixes - who would have thought?

[SilverAzide]

As of this moment there are 9 simultaneous weather alerts in Lake Charles, LA. I've never seen that many at once. I had to tweak the JSONAlerts file to get them all. Time to get out of town, folks! On the upside, Beta r3404 is working perfectly now.

WxMeter Alert Severe 9a.png

WxMeter Alert Severe 9.png

[Yincognito]

As of this moment there are 9 simultaneous weather alerts in Lake Charles, LA. I've never seen that many at once. I had to tweak the JSONAlerts file to get them all. Time to get out of town, folks! On the upside, Beta r3404 is working perfectly now.

WxMeter Alert Severe 9a.pngWxMeter Alert Severe 9.png

Thanks for mentioning it - even though I don't use the alerts (I use qualifierPhrase instead, it's simpler to manipulate and such events reflect there as well), I'm interested in how the source looks like during extreme situations (e.g. max number of alerts, max number of chars in a phrase, various formats for the snow amount, that kind of things).

[Cariboudjan]

Could the RegExp for Weather.com be located on Rainmeter.net? That way authors can parse Rainmeter.net for the RegExp to use to parse Weather.com with a double webparser.

Then authors do not need to keep updating their skins. When jsmorley updates the xml on Rainmeter.net, it will filter out to all of the Rainmeter skins out in the wild and start working again without any change to the skin itself.

I know I know it would mean that all weather Rainmeter skins would be constantly parsing Rainmeter.net. But it would be nice.

Maybe the template for the webparser could only parse on skin load, writekeyvalue the result to itself, and use the value when skin refreshed to reduce server pinging. That way the majority of pinging would only occur when most users restart their PCs or exit Game Mode.

Actually........ Why not include the weather.com RegExp as a built-in variable or something. That way if you want to get Weather.com working again you just need to update Rainmeter...

[Yincognito]

Could the RegExp for Weather.com be located on Rainmeter.net? That way authors can parse Rainmeter.net for the RegExp to use to parse Weather.com with a double webparser.

Then authors do not need to keep updating their skins. When jsmorley updates the xml on Rainmeter.net, it will filter out to all of the Rainmeter skins out in the wild and start working again without any change to the skin itself.

I know I know it would mean that all weather Rainmeter skins would be constantly parsing Rainmeter.net. But it would be nice.

Maybe the template for the webparser could only parse on skin load, writekeyvalue the result to itself, and use the value when skin refreshed to reduce server pinging. That way the majority of pinging would only occur when most users restart their PCs or exit Game Mode.

Actually........ Why not include the weather.com RegExp as a built-in variable or something. That way if you want to get Weather.com working again you just need to update Rainmeter...

This approach has already been suggested in various forms already by some users - I believe even this thread has some similar suggestions somewhere. The answer has been pretty straight forward - just saying. Maybe you're already aware of it, but if you aren't, this has already been discussed.

[nek]

I found something, tiny things about regular expressions in the
WeatherComJSON_August 4, 2020.rmskin > WeatherComJSONMeasures.inc | Updated July 31, 2020

There is a code RegExp=(?siU)^(.*)$ for WebParser measure, but RegExp=(?s)^(.*)$ is fewer steps and faster to parse.

Code: Select all

[@EntireSiteSuperParent]
Measure=WebParser
URL=#URLSite#
UpdateRate=#UpdateRate#
Flags=Resync | NoCookies
UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
DecodeCharacterReference=1
LogSubstringErrors=0
RegExp=(?siU)^(.*)$
StringIndex=1
FinishAction=[!EnableMeasureGroup Parents][!Update]

---

What is the best Regular Expressions for WebParser measure to fetch the whole contents?

I have tested some regular expressions and here are the results.

1. Testing tool
https://regex101.com/
Regex101 settings for a WebParser RegExp FLAVOR PCRE(PHP <7.3) FUNCTION Match REGEX FLAGS none

regex101.WebParser.RegExp.png

FYI: Regex101 settings for a Measures Substitute FLAVOR PCRE(PHP <7.3) FUNCTION Substitution REGEX FLAGS g

regex101.Measure.Substitute.png

2. Sample text (300 byte, LF)

Code: Select all

Rainmeter allows you to display customizable skins on your desktop, from hardware usage meters to fully functional audio visualizers.

You are only limited by your imagination and creativity.


Rainmeter is open source software distributed free of charge under the terms of the GNU GPL v2 license.
xyz

3. Results

RegExp.match.steps.png

(?siU)
s modifier: single line. Dot matches newline characters
i modifier: insensitive. Case insensitive match (ignores case of [a-zA-Z])
U modifier: Ungreedy. The match becomes lazy by default. Now a ? following a quantifier makes it greedy

As we can see (?s)^(.*)$ is fewer steps than (?siU)^(.*)$.
(?siU) is great options to parse a part of contents like RegExp=(?siU)"timeZoneAbbreviation":"(.*)", but not suitable to get whole contents.
Basically, child measures of WebPaser are able to access the whole contents with parent WebParser measure's RegExp=(?s)^.*$ (non-captured) and get rid of the StringIndex=1.

Code: Select all

[@EntireSiteSuperParent]
Measure=WebParser
URL=#URLSite#
UpdateRate=#UpdateRate#
Flags=Resync | NoCookies
UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
DecodeCharacterReference=1
LogSubstringErrors=0
RegExp=(?s)^.*$
;RegExp=(?siU)^(.*)$
;StringIndex=1
FinishAction=[!EnableMeasureGroup Parents][!Update]

4. .* and .+ are greedy by default

RegExp.greedy.png

5. References
PCRE Regex Cheatsheet
Resources for regular expressions - Rainmeter Docs
WebParser measure - Rainmeter Docs