Weather.com JSON not loading correctly

[Yincognito]

Oh, I was not referring to that part, I was referring to the actual units of measure ("F", "C", "in", "mm", "mph", "kph", etc.). But you're right, it's not too big of a deal. More of an annoyance...

Indeed.

[nek]

FYI: HTTP request (User-Agent, without Cookies) and response from weather.com

1. Environment

Windows 10 Pro 1909 64-bit (build 18363) - Japanese (1041)
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Rainmeter 4.4.0.3338 beta (64-bit), portable installed.

2. Target URL

https://weather.com/en-US/weather/today/l/42.31,-71.11

3. Results (2020-05-29 / May 29th, 2020)

(1) No User-Agent specified.

Code: Select all

curl.exe -s -I -A "" --url "https://weather.com/en-US/weather/today/l/42.31,-71.11"

## Request Header ##
HEAD /en-US/weather/today/l/42.31,-71.11 HTTP/1.1
Host: weather.com
Accept: */*
## Response Header ##
HTTP/1.1 200 OK
Set-Cookie: ci=TWC-Connection-Speed=4G&TWC-Locale-Group=US&TWC-Device-Class=desktop
TWC-Device-Class: desktop

(2) Rainmeter WebParser Measure default.

Code: Select all

curl.exe -s -I -A "Rainmeter WebParser plugin" --url "https://weather.com/en-US/weather/today/l/42.31,-71.11"

## Request Header ##
HEAD /en-US/weather/today/l/42.31,-71.11 HTTP/1.1
Host: weather.com
User-Agent: Rainmeter WebParser plugin
Accept: */*
## Response Header ##
HTTP/1.1 200 OK
Set-Cookie: ci=TWC-Connection-Speed=4G&TWC-Locale-Group=US&TWC-Device-Class=desktop
TWC-Device-Class: desktop

(3) Web browser for Desktop

Code: Select all

curl.exe -s -I -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" --url "https://weather.com/en-US/weather/today/l/42.31,-71.11"

## Request Header ##
HEAD /en-US/weather/today/l/42.31,-71.11 HTTP/1.1
Host: weather.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0
Accept: */*
## Response Header ##
HTTP/1.1 200 OK
Set-Cookie: ci=TWC-Connection-Speed=4G&TWC-Locale-Group=US&TWC-Device-Class=desktop
TWC-Device-Class: desktop

(4) Web browser for Mobile

Code: Select all

curl.exe -s -I -A "Mozilla/5.0 (iPhone; CPU iPhone OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.4 Mobile/15E148 Safari/604.1" --url "https://weather.com/en-US/weather/today/l/42.31,-71.11"

## Request Header ##
HEAD /en-US/weather/today/l/42.31,-71.11 HTTP/1.1
Host: weather.com
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.4 Mobile/15E148 Safari/604.1
Accept: */*
## Response Header##
HTTP/1.1 200 OK
Set-Cookie: ci=TWC-Connection-Speed=4G&TWC-Locale-Group=US&TWC-Device-Class=mobile
TWC-Device-Class: mobile

(5) Other Results.

### TWC-Device-Class=desktop ###
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 10.0; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36 Edg/83.0.478.37
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)

### TWC-Device-Class=mobile ###
Mozilla/5.0 (Linux; Android 10) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.83 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 10; Pixel 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Mobile Safari/537.36
Mozilla/5.0 (iPhone; CPU iPhone OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.4 Mobile/15E148 Safari/604.1

### TWC-Device-Class=bot ### Add: 2020-06-05 / June 5h, 2020
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

WebParser measure UserAgent
https://docs.rainmeter.net/manual/measures/webparser/#UserAgent
Latest user agents for Web Browsers & Operating Systems
https://www.whatismybrowser.com/guides/the-latest-user-agent/
HTTP headers
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

[Yincognito]

FYI: HTTP request (User-Agent, without Cookies) and response from weather.com

Nicely done.

The "ci" cookie is a session cookie (mine is something like TWC-Locale-Group=GLS+&X-Origin-Hint=Prod-IBM-LS&TWC-GeoIP-Country=RO&TWC-Privacy=gdpr, as my region is EU). Apart from that, I saved another 2 cookies (probably persistent ones) on my hard drive with the weather.com thing still working "normally". I might compared those with their version when the weather.com parsing will become broken again, to see if there's any difference.

That being said, it appears that visiting any page in IE has the potential to produce the issue, as mentioned here, so in the end it might not have anything to do with the cookies from weather.com at all...

Thing is, we can't avoid the IE stuff mixing into this, since WebParser relies on it.

[pul53dr1v3r]

just to mention that the issue can be reproduced wnever you delete IE data. For instance, even if a weather skin works and you delete the data, it may work or not. No rules.

And this is the most accurate way how to fix it based on my experience with the issue so far:
https://forum.rainmeter.net/viewtopic.php?f=27&p=177437#p177435

[Yincognito]

FYI: HTTP request (User-Agent, without Cookies) and response from weather.com

It appears that one can view the session cookies and other bowser data from weather.com right on their site, by choosing Data Rights at the bottom of their main page, then opening Request A Portable Copy Of Data, followed by clicking on the View Browser Data button.

The ci session cookie shown by the site is slightly different from what I saved a few days ago after applying the fix, but the weather skin still gets the "old" format (i.e. it's still working).

[xenium]

I don't have this new design either... UNLESS I change the user agent string to one for Android. If you set it to:
Mozilla/5.0 (Linux; Android 10; Pixel 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Mobile Safari/537.36
you'll get the new page.

Thank you very much SilverAzide
I used it to rewrite my Windows 10 Weather Enterprise skin to use JSON data from the new TWC website.
Anyway, my skins that use the HTML source from TWC work well for me

[Yincognito]

Thank you very much SilverAzide
I used it to rewrite my Windows 10 Weather Enterprise skin to use JSON data from the new TWC website.

Yep, that's probably the most convenient solution of them all (if weather.com consistently serves V3 to mobile clients, that is). Well done!

[jsmorley]

I will be updating my @Include files to support the "V3" JSON version from weather.com sometime tomorrow.

[DrRoboculous]

So what does this mean for users? Is there a code i'm supposed to copy & paste in all my skins? I've been trying to follow these threads for the last couple of weeks but, i'm not a coder or a skin-maker or anything; i'm just a Rainmeter user. I think I have a basic sort-of understanding of what the problem is but i don't understand what any of this means in terms of how to fix it. Any help? Thanks. Keep up the great work, everybody.

[By the way: you guys probably can't see the humor in the situation of a non-coder reading a message board and seeing a post that is just a wall of long lines of code followed by a response in human words, like "great work!" - but trust me, it's funny.]

[SilverAzide]

So what does this mean for users? Is there a code i'm supposed to copy & paste in all my skins? I've been trying to follow these threads for the last couple of weeks but, i'm not a coder or a skin-maker or anything; i'm just a Rainmeter user. I think I have a basic sort-of understanding of what the problem is but i don't understand what any of this means in terms of how to fix it. Any help? Thanks. Keep up the great work, everybody.

[By the way: you guys probably can't see the humor in the situation of a non-coder reading a message board and seeing a post that is just a wall of long lines of code followed by a response in human words, like "great work!" - but trust me, it's funny.]

Well, this all depends on the weather skin you are using. Assuming it was working up until recently (i.e., based on weather.com), AND assuming it was using the "weather templates" found on these forums, AND assuming the skin author hasn't already fixed it for you, then you can do this:

• Go to https://forum.rainmeter.net/viewtopic.php?f=118&t=34628
• Install the skin
• Go into the Weather.com\@Resources folder and copy the WeatherComJSONMeasures.inc file to your skin's @Resources folder, overwriting the one you have. (If your skin doesn't have this file, you are hosed.)
• Refresh everything and pray.