It is currently May 26th, 2020, 8:41 pm

⭐ weather.com - Parsing the JSON

Our most popular Tips and Tricks from the Rainmeter Team and others
nbtc971
Posts: 3
Joined: May 22nd, 2020, 6:10 am

Re: ⭐ weather.com - Parsing the JSON

Post by nbtc971 »

Yeah, weather isn't loading for me either.
dvo
Posts: 780
Joined: February 7th, 2016, 6:08 am

Re: ⭐ weather.com - Parsing the JSON

Post by dvo »

in NL he's down to... hopefully it jumps back...wait and see ;-) so it seems not be a local problem... :o
User avatar
Yincognito
Posts: 1627
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ weather.com - Parsing the JSON

Post by Yincognito »

Yep, just turned on my computer today and still no data received in the weather's WebParser. Just tried using various UserAgent strings and it didn't work, but I also tried downloading the file using Debug=2 option and it did download the page source (which can be then parsed).

I don't recommend using the aforementioned "workaround" just yet, maybe someone comes up with a more convenient alternative in the meantime. The "good news" is that there is a workaround for this, albeit a bit inconvenient.
User avatar
xenium
Posts: 491
Joined: January 4th, 2018, 9:52 pm

Re: ⭐ weather.com - Parsing the JSON

Post by xenium »

It works for me.
But WeatherComCodes doesn't work.
I need to manually edit the WeatherComJSONVariables.inc file
Capture.PNG
Capture2.PNG
Edit:
My skins that use the HTML version work for me.
But for other users they don't work, although their locations work for me
I don't know what could be causing this issue
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Posts: 1627
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ weather.com - Parsing the JSON

Post by Yincognito »

xenium wrote:
May 22nd, 2020, 8:38 am
It works for me.
But WeatherComCodes doesn't work.
I need to manually edit the WeatherComJSONVariables.inc file
Capture.PNG
Capture2.PNG

Edit:
My skins that use the HTML version work for me.
But for other users they don't work, although their locations work for me
I don't know what could be causing this issue
Yeah, found a way to get all the data, just having trouble processing it further (the same regexp error). Will explore some workarounds to handle this.
User avatar
Yincognito
Posts: 1627
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ weather.com - Parsing the JSON

Post by Yincognito »

Ok, so after examining and formatting the WebParserDump.txt that I was able to download using the Debug=2 option and wondering why the data seemed to be available but the regex failed, it appears that they indeed changed their format ... but only for accessing stuff from "outside" a browser. In other words, while in the browser the page source would appear the same as before, what is "served" to the WebParser measure is significantly different.

Long story short, this should work for you - let me know if it doesn't (replace the latitude and longitude values with your own):

Code: Select all

[Variables]
Latitude=0
Longitude=0

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1
BackgroundMode=2
SolidColor=47,47,47,255

---Measures---

[MeasureWebParser]
Measure=WebParser
Url=https://weather.com/en-GB/weather/today/l/#Latitude#,#Longitude#
;Debug=2
RegExp=^(?siU).*"getSunV3CurrentObservationsUrlConfig":\{.*"temperature":(.*),.*$
UpdateRate=600

[MeasureTemperature]
Measure=WebParser
Url=[MeasureWebParser]
StringIndex=1

---Meters---

[MeterTemperature]
Meter=STRING
X=0
Y=0
FontFace=Consolas
FontColor=255,255,255,255
SolidColor=47,47,47,255
Padding=5,5,5,5
FontSize=16
AntiAlias=1
MeasureName=MeasureTemperature
Text="Temperature = %1 °C"
Basically, what you should do to build an "updated" regex is to set Debug=2 in your WebParser (you can uncomment it in the skin sample above, if you like) just one time in order to get how the source served to WebParser looks like in WebParserDump.txt, optionally format it for clarity in a clone file and then set your regex(es) accordingly.

Or ... you could wait for jsmorley to do that for you, of course. :D
User avatar
jsmorley
Developer
Posts: 20637
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

I am very hesitant to jump to any conclusions, and certainly hesitant to try and "fix" an issue that at least for me, doesn't exist.


1.jpg


Over the years, I have found it pays to have a little patience with "glitches" with weather.com.
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Posts: 1627
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ weather.com - Parsing the JSON

Post by Yincognito »

jsmorley wrote:
May 22nd, 2020, 10:59 am
I am very hesitant to jump to any conclusions, and certainly hesitant to try and "fix" an issue that at least for me, doesn't exist.



1.jpg
Yes, that would be the wise choice indeed - you never know if they're going to revert this or not. Another option, if possible, of course, would be to add those "differences" as optional patterns (either using ? or the | or character) to the current regexes, so that it would work with either format. The latter, obviously, if the two formats aren't critically different from one another.

EDIT: Simple example, based on the sample skin I posted above:

Code: Select all

RegExp=^(?siU).*"(vt1observation|getSunV3CurrentObservationsUrlConfig)":\{.*"temperature":(.*),.*$
This should work with either format. Not sure if it's possible for all the fields, but then, it's just an example, after all.

EDIT 2: While I believe the above "or" method will work for every field, there is another issue that makes it unfunctional: since the names of the fields changed and they are ordered alphabetically in the weather.com source, this means that the order of the fields in the "new" regex(es) would have to change as well. In other words, if this change becomes definitive, one would have to rebuild the regex(es) completely - the "old" regex(es) do help as they can be used as a base for the "new" regex(es), but it's both the names and the order of the names that would have to be "updated" for everything to work flawlessly.

EDIT 3: This is where a stable and supported by the Rainmeter team JSON parser would come handy... 8-) Just change the names of the fields, without bothering with their order in the JSON source. Sure, this can be done using regex as well, but you'd have to waste resources by parsing the source string multiple times, once for each field (assuming they're unique).