It is currently July 10th, 2020, 1:30 pm

⭐ Weather.com

Post your work-in-progress and completed skins to share and discuss.
User avatar
SilverAzide
Posts: 884
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Weather.com

Post by SilverAzide »

Excellent, thank you!
Gadgets DeviantArt More...
User avatar
Yincognito
Posts: 1893
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

SilverAzide wrote:
June 17th, 2020, 6:15 pm
To veer this thread slightly back toward the main topic... Should there be any concern with skins using this template about performance issues from having so many measures? The number of child measures is pretty ginormous now... something like 950, was it? It SEEMS okay, I've got 3 weather skins running all the time and Rainmeter is still using ~1.0% of my CPU, but I wasn't sure if there's any real benefit to cutting down the number of active measures (like organizing blocks of measures into groups, so if you only wanted a 5 day forecast instead of 15 days, you could turn off a group).
It's not a performance hit from the CPU usage point of view, like jsmorley pointed out, but it surely is a performance hit from the quantity of data being parsed / downloaded, that's for sure. Bottom line, getting stuff parsed is going to take longer than back in the wxdata days where a short xml was parsed in a matter of nanoseconds or something, LOL. My biggest concern regarding this is weather.com trying to stop or irreparably change the page source data to avoid heavy trafficking from it, including Rainmeter skins updating a couple of minutes away and downloading MB of data from all around the world. I saw a post on StackOverflow by an Indian guy who tried to get 100000 aggregate records using a (supposedly, legit) API key, and even if the data obtained in that way is way smaller than downloading the entire main page like we do, I reckon he's not the only one trying such overload on weather.com. I think I saw / read somewhere that one of the reasons they deprecated / retired wxdata was the traffic overload on it (compared to their "expectations" or "common sense recommendations") - hopefully this won't be the case with the main site, but I obviously can't guarantee it.
User avatar
jsmorley
Developer
Posts: 21010
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

It's one of the reasons I use a UserAgent string in my @Include file. That has two objectives for me. One, to ensure that what I get in the skin with Debug=2 and what I get in my browser are the same thing. There is some comfort in that. Two, I'd just as soon have the site see a connection from my browser every 10 minutes, rather than "Rainmeter WebParser Plugin".

I'm not overly concerned about weather.com getting slammed by WebParser. What the skin is doing is downloading exactly what any browser hitting the site would do, and trust me, that is a HUGELY popular site, and a few thousand Rainmeter users hitting it every 10 minutes is going to be line noise to it. I have no idea what their traffic on the site looks like, but I can well imagine that hundreds of thousands of people are on it any any given second.
User avatar
Yincognito
Posts: 1893
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote:
June 17th, 2020, 6:50 pm
It's one of the reasons I use a UserAgent string in my @Include file.
Yes. It's a good idea, and I thought about it as well - will apply it in my case ASAP as well. By the way, which user agent do you think it's the "safest" from the point of view of preferably getting the same data as without it?

P.S. I'm not necessarily talking about getting the same data as in the browser, I jut want to get the same data as without any user agent at all (i.e. just like Rainmeter's WebParser Plugin would do).
User avatar
jsmorley
Developer
Posts: 21010
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

Yincognito wrote:
June 17th, 2020, 6:56 pm
Yes. It's a good idea, and I thought about it as well - will apply it in my case ASAP as well. By the way, which user agent do you think it's the "safest" from the point of view of preferably getting the same data as without it?
WebParser DOES use a User Agent string, it is "Rainmeter WebParser Plugin".

Well, I just go to one of the many "What is my User Agent" websites and use the same as my Firefox browser uses. That is part of the point for me. As for trying to ensure you get the same thing that you would get with the default "Rainmeter WebParser Plugin", while not sending that particular string, I don't know. Might be able to just use "Four Score and Seven Years Ago" or something. The point presumably is to have the User Agent string be "unknown" to the site. Not sure I would do that, as I can well imagine that any "known" User Agent string would be flagged as a "browser" and that's good, they assume you are seeing their advertising. Any "unknown" User Agent string, including our default, would likely be flagged as an "application", and that might be more of a black mark in their eyes.
User avatar
Yincognito
Posts: 1893
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote:
June 17th, 2020, 7:01 pm
Well, I just go to one of the many "What is my User Agent" websites and use the same as my Firefox browser uses. That is part of the point for me. As for trying to ensure you get the same thing that you would get with the default "Rainmeter WebParser Plugin", while not sending that particular string, I don't know. Might be able to just use "Four Score and Seven Years Ago" or something. The point presumably is to have the User Agent string be "unknown" to the site. Not sure I would do that, as I can well imagine that any "known" User Agent string would be flagged as a "browser" and that's good, they assume you are seeing their advertising. Any "unknown" User Agent string, including our default, would likely be flagged as an "application", and that might be more of a black mark in their eyes.
Ok then, thanks! :thumbup:
User avatar
Yincognito
Posts: 1893
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote:
June 17th, 2020, 7:01 pm
As for trying to ensure you get the same thing that you would get with the default "Rainmeter WebParser Plugin", while not sending that particular string, I don't know.
Yeah, just applied setting the user agent and analyzed the results. Setting the user agent makes no difference in what is otherwise received by WebParser's Debug=2 option, the two files are identical, at 976 993 bytes. Saving the page source from the browser creates a similar UTF-8 document that is significantly different from the other two, at 1 306 773 bytes. It's not just the adds or whatever, it's also what I mentioned earlier in this or another thread ... the applicationId at the start of the document. I might be mistaken on this one, but if that's a way to detect if one is accessing weather.com using a browser or not (as the 2 WebParser documents had the same applicationId value - the same as the last time I checked it out using WebParser, i.e. ending in 989 -, different from what the browser produced), then I'm not sure setting the user agent is of much use.

Just curious, can you temporarily set a Debug=2 in your environment and then CTRL+F (i.e. find) the applicationId value in the downloaded WebParserDump.txt, to see if it also ends in 989? :???:
User avatar
jsmorley
Developer
Posts: 21010
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Weather.com

Post by jsmorley »

Yincognito wrote:
June 17th, 2020, 8:17 pm
Yeah, just applied setting the user agent and analyzed the results. Setting the user agent makes no difference in what is otherwise received by WebParser's Debug=2 option, the two files are identical, at 976 993 bytes. Saving the page source from the browser creates a similar UTF-8 document that is significantly different from the other two, at 1 306 773 bytes. It's not just the adds or whatever, it's also what I mentioned earlier in this or another thread ... the applicationId at the start of the document. I might be mistaken on this one, but if that's a way to detect if one is accessing weather.com using a browser or not (as the 2 WebParser documents had the same applicationId value - the same as the last time I checked it out using WebParser, i.e. ending in 989 -, different from what the browser produced), then I'm not sure setting the user agent is of much use.

Just curious, can you temporarily set a Debug=2 in your environment and then CTRL+F (i.e. find) the applicationId value in the downloaded WebParserDump.txt, to see if it also ends in 989? :???:
"applicationID":"296284162"

UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
User avatar
Yincognito
Posts: 1893
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: ⭐ Weather.com

Post by Yincognito »

jsmorley wrote:
June 18th, 2020, 12:00 am
"applicationID":"296284162"

UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
Yep. At first, I was surprised, because I expected to end in 989 in order to point out at Rainmeter (i.e. the same application), then tried with your default settings in your skin (i.e. 3 day forecast) and got something ending in 226, then changed to the 15 day forecast and got the same ending as you, aka 162 (I'm talking about endings here only to simplify the matters, as the latter means I got the same applicationId as you for the 15 day forecast, i.e. 296284162).

Then, just when I was preparing to reply to you that the applicationId is somehow "skin dependent" and not "Rainmeter dependent" ... I got the very same applicationId when dumping the WebParser result in my own skin. Then, the next time I tried got the 226 ending. :jawdrop So, eventually, I have no idea what this means, but my impression is still that weather.com has other means of differentiating between the browser and a specific application apart from reading the user agent (or maybe the applicationId for the broswer fluctuates in a similar manner as well?). I might be wrong with this or talk about something that isn't / won't be really relevant ... or not. We'll see. :confused:

Sorry to use you in this (so far) inconclusive test. I thought I got it all figured it out, but I guess I was mistaken.
User avatar
Active Colors
Moderator
Posts: 579
Joined: February 16th, 2012, 3:32 am

Re: ⭐ Weather.com

Post by Active Colors »

Thank you jsmorley for this skin and keeping it updated and other people for the common effort to keep this updated and simple for other users. Much love.