It is currently June 16th, 2021, 11:20 am

⭐ Weather.com

Weather skins
User avatar
SilverAzide
Rainmeter Sage
Posts: 1290
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com

Post by SilverAzide »

Yincognito wrote: June 10th, 2020, 11:35 am I didn't have any problems yet, but then, I'm not sure if I was online at the time of the incident(s) and my update rate of 1 hour makes it hard to notice a 5 minute issue. However, since it's obvious that weather.com is going through some changes / upgrades / whatever, such inconveniences might continue to happen intermitently for a while, until everything is settled on their end.
Today I'm getting the old "V2" JSON served up using the latest JSON template. Commenting out the [EntireSiteSuperParent] UserAgent string (which was there for insurance only) looks like a quick fix, at least for me.
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 21758
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

SilverAzide wrote: June 10th, 2020, 12:19 pm Today I'm getting the old "V2" JSON served up using the latest JSON template. Commenting out the [EntireSiteSuperParent] UserAgent string (which was there for insurance only) looks like a quick fix, at least for me.
Yeah, the site itself seems in a bit of a flux. I sometimes get the "old" HTML version and sometimes the "new" one when I refresh. I'm hoping things will settle down soon and we can stop obsessing about our weather skins...
User avatar
SilverAzide
Rainmeter Sage
Posts: 1290
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com

Post by SilverAzide »

jsmorley wrote: June 10th, 2020, 12:26 pmI'm hoping things will settle down soon and we can stop obsessing about our weather skins...
How many times can I click the "upvote" button on this comment? :)
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 3184
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

SilverAzide wrote: June 10th, 2020, 12:19 pm Today I'm getting the old "V2" JSON served up using the latest JSON template. Commenting out the [EntireSiteSuperParent] UserAgent string (which was there for insurance only) looks like a quick fix, at least for me.
Really? :jawdrop
I'm not, I'm getting V3, just checked it out using the Debug=2 option.

That being said, it looks like my system of getting the data irrespective of version, field names or their order in the JSON might be useful, after all - if the issue persists, that is. :sly:

Let me know if you or anyone is interested. And yes, it can be adapted to be used in modular .inc files (jsmorley's type), as it is variable based anyway. 8-)

P.S. By the way, I'm not using any user agent at all. Must have forgot to include it, as I did want to follow your trick by having a mobile user agent, after all...
Last edited by Yincognito on June 10th, 2020, 12:43 pm, edited 2 times in total.
User avatar
jsmorley
Developer
Posts: 21758
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

SilverAzide wrote: June 10th, 2020, 12:37 pm How many times can I click the "upvote" button on this comment? :)
I'm going to change my handle to "Captain Obvious"...
User avatar
jsmorley
Developer
Posts: 21758
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

Yincognito wrote: June 10th, 2020, 12:40 pm Really? :jawdrop
I'm not, I'm getting V3, just checked it out using the Debug=2 option.

That being said, it looks like my system of getting the data irrespective of version, field names or their order in the JSON might be useful, after all - if the issue persists, that is. :sly:

Let me know if you or anyone is interested. And yes, it can be adapted to be used in modular .inc files (jsmorley's type), as it is variable based anyway. 8-)

P.S. By the way, I'm not using any user agent at all. Must have forgot to include it, as I did want to follow your trick by having a mobile user agent, after all...
I'm hesitant to jump to an approach that is that complicated. While it might work just fine, and for 90% of users they are never going to look at how the sausage is made anyway, I do fear that if there is some minor change to the JSON, one that breaks both my stuff and yours, a much more complicated RegExp is going to be really hard for anyone but the most determined person to fix.

What would be nice at some point, is something, probably a plugin, that could take a dump of the site from WebParser, and extract just the JSON from it, formatting it into a text file with all the line breaks and indenting and all that, so it becomes MUCH more readable. Then use some JSON "reader" functionality to get fields by name, so it wouldn't matter about any "order" or other extraneous "formatting" in the original source. I guess my point is that this is JSON, although embedded in HTML in a weird and ugly way, and JSON is JSON. "Parsing" JSON like any other text file sorta defeats the purpose.

In any case, I refuse to accept that they are going to continue to maintain both V2 and V3 going forward. I certainly refuse to accept that this can just be random and arbitrary. My guess is that this is an IBM Company, running on the IBM Cloud, and they are having some issues with how they are "balancing" between servers as they roll out the final (for now) V3 version of this. It's fine for me today, and hopefully will continue to be so. This may not be a problem with weather.com at all, but rather a problem with IBM. I haven't trusted Big Blue since the OS/2 days. Any company that stands there staring while Bill Gates picks their pockets not once, but twice in 20 years is not really on top of their game.
User avatar
QuakeGuy
Posts: 30
Joined: February 3rd, 2020, 11:15 pm
Location: Germany

Re: ⭐ Weather.com

Post by QuakeGuy »

I also get the JSON V2 version on a rare basis since yesterday. Let's hope this issue settles the next days.
User avatar
Yincognito
Posts: 3184
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote: June 10th, 2020, 12:53 pm I'm hesitant to jump to an approach that is that complicated. While it might work just fine, and for 90% of users they are never going to look at how the sausage is made anyway, I do fear that if there is some minor change to the JSON, one that breaks both my stuff and yours, a much more complicated RegExp is going to be really hard for anyone but the most determined person to fix.

In any case, I refuse to accept that they are going to continue to maintain both V2 and V3 going forward. My guess is that this is an IBM Company, running on the IBM Cloud, and they are having some issues with how they are "balancing" between servers as they roll out the final (for now) V3 version of this. It's fine for me today, and hopefully will continue to be so.
Well, 90% of it isn't complicated at all, really - it's actually very simple. My focus was to create the regexes using minimal data, and I'm sure that you have an idea just how minimal is minimal to me, knowing how my suite looks like. :lol: As for breaking stuff, my approach, while not impossible to, is going to be very hard to break up, as the only way of doing that is either name the fields in a completely different way or if the number of nested subsections is going to be dramatically changed (quite unlikely, this one). Even so, the modifications required if such a thing happens are again ... minimal, mainly changing the "root" name of the field (I mean, how many ways you can call "temperature" in English, right?) or changing the number of brackets used to anchor the regex patterns in the quantifiers (i.e. {2} instead of {1} and such).

Only the daily forecast regex is a bit more complicated, but not by much if you understand what each part represents. It's just one regex though, used in all the daily forecast fields, and as always, things are controlled just by changing the regex quantifiers accordingly, nothing else needed.
jsmorley wrote: June 10th, 2020, 12:53 pmWhat would be nice at some point, is something, probably a plugin, that could take a dump of the site from WebParser, and extract just the JSON from it, formatting it into a text file with all the line breaks and indenting and all that, so it becomes MUCH more readable. Then use some JSON "reader" functionality to get fields by name, so it wouldn't matter about any "order" or other extraneous "formatting" in the original source. I guess my point is that this is JSON, although embedded in HTML in a weird and ugly way, and JSON is JSON. "Parsing" JSON like any other text file sorta defeats the purpose.
Yeah, a JSON parser plugin supported by the Rainmeter team would be nice, indeed. The thing is, even with a JSON parser plugin, you'd still have to use regex for changing field names. You can do a dailyforecast.data.daypart.temperature[5] (just an example) to get the temperature for the 3rd day's daytime, but if the root section is changed into daily_forecast, you'd still have to use regex to handle that. Even so, if the version fluctuates between V2 and V3 and the whole structure / field names change, an already set (i.e. hardcoded) JSON query in the JSON parser won't be of much help, but regex will, as it's much more flexible.

That being said, one can get only the JSON (i.e. without the site or the HTML), and only a specific version of it (V1, V2 or V3 as desired) easily - so a stable way to get the data, but that involves dealing with an API key, and as already discussed, we agreed we should avoid that, at least as long as another method is still usable. I made my approach to work in those circumstances as well, and even though I'm not using it, it's good to know it's there if things go seriously south.

P.S. I can share the details of my approach with you in private, if you want, and then it will be up to you to decide if it's too complicated, if it has any drawbacks, or anything else you might feel it's "shaky". Me, I don't believe it has many flaws (if any), but then, this is not just about me, in the end...
User avatar
jsmorley
Developer
Posts: 21758
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

Like I said, I refuse to believe that they intend to deliver two entirely incompatible JSON formats to users on a random and arbitrary basis going forward. Time will tell I guess. I just don't see the burning need to have a parser that is capable of reacting to V2 or V3 seamlessly, when what is more likely is that sometime in the not-too-distant future there will be a V4 that we have to deal with. I want to have an approach that makes it as simple as possible to change the RegExp, and create @Include .inc files that can just be "dropped in" to fix any number of skins without having to manhandle each of them individually.
User avatar
Yincognito
Posts: 3184
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote: June 10th, 2020, 1:49 pm Like I said, I refuse to believe that they intend to deliver two entirely incompatible JSON formats to users on a random and arbitrary basis going forward. Time will tell I guess. I just don't see the burning need to have a parser that is capable of reacting to V2 or V3 seamlessly, when what is more likely is that sometime in the not-too-distant future there will be a V4 that we have to deal with. I want to have an approach that makes it as simple as possible to change the RegExp, and create @Include .inc files that can just be "dropped in" to fix any number of skins without having to manhandle each of them individually.
As you wish. :confused: I'm not trying to force my approach on anyone, just trying to help, if possible. Weather.com switching from V2 to V3 or viceversa when delivering stuff has nothing to do with anyone's refusal to believe such a scenario is possible or not, since, as explained in another thread, this doesn't cost them anything, their Javascript engine will read either. They could deliver V1 (exaggerating a bit for the sake of the argument) for all I care and not have any problems with it, unlike us "scrapers", LOL.

P.S. A V4 is already available, it's just that it's used for other stuff on their site. I'm not even sure weather data can be queried using V4 at the moment. Also, just to clarify in case it wasn't properly understood, my method is fully compatible with your @Include .inc files approach, bar some minor adaptation work. Not sure why you have the impression that everything I do must be overly complicated, by the way. :rofl: