It is currently February 20th, 2020, 6:42 pm

⭐ weather.com - Some Tools for Parsing

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 20269
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Some Help With Parsing weather.com

Post by jsmorley »

Added a new @Include file WeatherComHourly.inc, which will provide measures for weather data hourly, for the next 12 hours.

This will start at the most recent observation time in the current hour, and then on the hour for the next 11 hours.

[@Hourly1Time]
[@Hourly1Icon]
[@Hourly1Day]
[@Hourly1Conditions]
[@Hourly1Temperature]
[@Hourly1TemperatureSymbol]
[@Hourly1Precipitation]
[@Hourly1PrecipitationSymbol]

...

[@Hourly12Time]
[@Hourly12Icon]
[@Hourly12Day]
[@Hourly12Conditions]
[@Hourly12Temperature]
[@Hourly12TemperatureSymbol]
[@Hourly12Precipitation]
[@Hourly12PrecipitationSymbol]

This caused small tweaks to:

WeatherComRegExp.inc
WeatherComVariables.inc


Get the new .rmskin in the first post of this thread.
User avatar
eclectic-tech
Rainmeter Sage
Posts: 3763
Joined: April 12th, 2012, 9:40 pm
Location: Cedar Point, Ohio, USA

Re: ⭐ Some Help With Parsing weather.com

Post by eclectic-tech »

Thanks for the update. :thumbup:

I will be posting an update of my WeatherData package (it's in an earlier post in this thread) which will use your include files.
That way everyone is on the same page :sly:

WeatherData only shows a Titlebar which allows users to look at any one, or all, of the include's measurenames and values in the Log,
This can be useful by having this skin loaded, and open the Log window's skin tab to 'WeatherData' to see the info while creating or editing weather skins.

EDIT: I forgot to mention that it also can display the information in 30+ languages :Whistle (Thanks to Xenium!)
User avatar
jsmorley
Developer
Posts: 20269
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Some Help With Parsing weather.com

Post by jsmorley »

eclectic-tech wrote:
January 22nd, 2020, 6:37 pm
Thanks for the update. :thumbup:

I will be posting an update of my WeatherData package (it's in an earlier post in this thread) which will use your include files.
That way everyone is on the same page :sly:

WeatherData only shows a Titlebar which allows users to look at any one, or all, of the include's measurenames and values in the Log,
This can be useful by having this skin loaded, and open the Log window's skin tab to 'WeatherData' to see the info while creating or editing weather skins.

EDIT: I forgot to mention that it also can display the information in 30+ languages :Whistle (Thanks to Xenium!)
Sounds cool. Let us know when it is done.

It would not be a bad thing if we had one common set of @Include files to talk about and share. Not saying what we have ended up with is perfect, but I do think it's a good start.

Languages are tricky to do. First off, there are far more than 30 language variants on weather.com. Second, at least for USA users, there are some variants that can be useful. For instance:

English with imperial units: en-US
English with metric units: en-GB, or en-CA, or en-AU
Latin America Spanish with imperial units: es-US
Latin America Spanish with metric units: es-MX

Then there are variants within a country, like:

Canada with English: en-CA
Canada with Canadian French: fr-CA

Generally speaking, if a user is in a particular country / locale, when they go to weather.com it will try to "figure it out", presumably based on their IP address. Then they can get the language code from the URL. If a user wants to force a particular language, they can change the country / language on the site, but unfortunately when they then use that and hit the site with WebParser, the units of measure will be automatic, based on the country / language. There is no way on the URL to force F when your country / language combination is C, or C when your country / language combination is F. You can on the site in your browser, but not via the URL.

It's also all-or-nothing. If your country / language combination is determined to be metric, ALL the values will be metric. There is no way to get Celsius temperatures, but wind speed in Miles per Hour or pressure in Inches of Mercury. Any non-standard combinations will require coding in the skin.

As noted above, there are some solutions for this in the USA, but if for some reason a user wanted the Russian language, but wanted imperial (F) units of measure, that is going to take some pretty involved Lua code to do. It can be particularly tricky with Barometric Pressure values, as not only are the values in different units of measure, but the values themselves can be formatted in a variety of ways. Some countries use , and . differently in numbers than what you might be used to. 1,301.5 mb/hPa might be any of 1,301.5 1.301,5 1 301,5. Try fr-FR for an example...

This is most often going to be fine. If a user is in Paris, they are almost certainly going to want French and metric. If they are in the USA, they are mostly likely going to want English and imperial. Just note that it can be tricky to have this be as flexible as you might like.
User avatar
SilverAzide
Posts: 690
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Some Help With Parsing weather.com

Post by SilverAzide »

Yes... the primary problem to solve is when you want to see units in a way that is DIFFERENT than your actual locale.

Getting the DEFAULT locale is actually very easy, and you don't need TWC to tell you. You can grab the user's locale from the registry, then take some action to set the language variable, etc. You can also handle cases for locales that are NOT supported. I do something like this:

Code: Select all

; LocaleName = "en-US"
[MeasureLocaleName]
Measure=Registry
RegHKey=HKEY_CURRENT_USER
RegKey=Control Panel\International
RegValue=LocaleName

; check if the machine locale is different from what is currently set
IfMatch=^#LocaleName#$
IfNotMatchAction=[!SetVariable LocaleName [MeasureLocaleName]][...do whatever else you need to do when the locale changes...]

; check if locale is supported by TWC
IfMatch2=ar-AE|ar-BH|ar-DZ|ar-EG|ar-ER|ar-IQ|ar-JO|ar-KM|ar-KW|ar-LB|ar-LY|ar-MA|ar-MR|ar-OM|ar-QA|ar-SA|ar-SD|ar-SO|ar-SY|ar-TD|ar-TN|bn-BD|ca-AD|ca-ES|cs-CZ|da-DK|de-AT|de-CH|de-DE|de-LI|el-CY|el-GR|en-AG|en-AS|en-AU|en-BB|en-BS|en-BZ|en-CA|en-CM|en-DM|en-FJ|en-FM|en-GB|en-GD|en-GH|en-GM|en-GY|en-IE|en-IN|en-JM|en-KE|en-KI|en-KN|en-LC|en-LR|en-LS|en-MH|en-MT|en-MU|en-NA|en-NG|en-NZ|en-PA|en-PH|en-PK|en-PW|en-RW|en-SB|en-SG|en-SL|en-SS|en-SZ|en-TO|en-TT|en-TV|en-TZ|en-UG|en-US|en-VC|en-VU|en-ZA|es-AR|es-BO|es-CL|es-CO|es-CR|es-DO|es-EC|es-ES|es-GQ|es-GT|es-HN|es-MX|es-NI|es-PA|es-PE|es-PY|es-SV|es-US|es-UY|es-VE|fa-IR|fi-FI|fr-AD|fr-BE|fr-BF|fr-BI|fr-BJ|fr-CA|fr-CD|fr-CF|fr-CG|fr-CI|fr-CM|fr-DJ|fr-DZ|fr-FR|fr-GA|fr-HT|fr-KM|fr-LU|fr-MA|fr-MC|fr-MG|fr-ML|fr-MU|fr-NE|fr-RW|fr-SN|fr-TD|fr-TG|fr-VU|he-IL|hi-IN|hr-BA|hr-HR|hu-HU|id-ID|it-IT|it-SM|it-VA|ja-JP|ko-KR|ms-BN|ms-MY|nl-BE|nl-NL|nl-SR|no-NO|pl-PL|pt-AO|pt-BR|pt-CV|pt-GW|pt-MZ|pt-PT|pt-ST|pt-TP|ro-RO|ru-BY|ru-EE|ru-KG|ru-RU|sk-SK|sv-SE|th-TH|tl-PH|tr-TR|uk-UA|ur-PK|vi-VN|zh-CN|zh-HK|zh-SG|zh-TW
IfMatchAction2=[...do whatever you need to do when the locale on your machine IS supported by TWC...]
IfNotMatchAction2=[...do some fallback for locales that are NOT supported by TWC...]
UpdateDivider=-1
The above locales were painstakingly grabbed from the TWC site a year or so ago, so they might have added more since then. I also disable the measure for users who wish to override the locale. This happens frequently for folks (ex-pats) who install the US version of Windows, but want the units for the country where they are actually located.

It is possible to convert those formatted numbers (like "1 301,5") to values that can be worked with. The thousands separator and decimal separators are also in the registry, so you could in theory feed everything into a Lua function (or Substitute regexp) to get an "unformatted" value and do the needed math on the result.
DeviantArt Gadgets More...
User avatar
balala
Rainmeter Sage
Posts: 9729
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: ⭐ Some Help With Parsing weather.com

Post by balala »

SilverAzide wrote:
January 23rd, 2020, 5:37 pm
The above locales were painstakingly grabbed from the TWC site a year or so ago, so they might have added more since then. I also disable the measure for users who wish to override the locale. This happens frequently for folks (ex-pats) who install the US version of Windows, but want the units for the country where they are actually located.

It is possible to convert those formatted numbers (like "1 301,5") to values that can be worked with. The thousands separator and decimal separators are also in the registry, so you could in theory feed everything into a Lua function (or Substitute regexp) to get an "unformatted" value and do the needed math on the result.
A somehow similar approach achieved in my recently launched Mirage suite. Right now it has three languages included: English, Romanian and Hungarian. Used in English, the decimal separator character is the dot (.) and the thousands separator is the comma (,), accordingly to the rules of the English language. Used in Romanian or Hungarian, the decimal separator character is the dot (,) and the thousands separator is the comma (.), accordingly to the rules of the both languages, Romanian and Hungarian.
This let me to discover an omission I did in one of the skins of suite (Mirage\Process\Process.ini). Unfortunately I forgot to add the two variables (DecimalSep and ThousandsSep) to this skin (and maybe to others, too?). Now will check and will correct this into an update. Along with fixing the not working Weather skin, which is in plan to be rewritten.
Thanks for guided my to this mistake...
User avatar
jsmorley
Developer
Posts: 20269
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Some Help With Parsing weather.com

Post by jsmorley »

My point with languages though is that sure, you can write a skin that uses any language and units of measure you like. What you can't do with weather.com is have a lot of flexibility in what the site sends you. If you don't like the resulting language strings and unit of measure values the site returns for the Language Code your user sets, you are going to have to individually intercept each bit of information, evaluate it, and change it to what you want.

Looking at how a user's Windows is set for locale stuff like number and time formatting is all well and good, but may well have no relationship to what weather.com sends. They will translate and format information as they see fit, based on their understanding of what is the standard for a given location code passed in the URL.

To each his own of course, but I'm not in the market to do other than let weather.com do almost all the work. The end-user can set any language code that is supported (which is virtually all of them as far as I can tell), and then my skins will display what the site returns.
User avatar
eclectic-tech
Rainmeter Sage
Posts: 3763
Joined: April 12th, 2012, 9:40 pm
Location: Cedar Point, Ohio, USA

Re: ⭐ Some Help With Parsing weather.com

Post by eclectic-tech »

jsmorley wrote:
January 23rd, 2020, 2:16 pm
Sounds cool. Let us know when it is done.
Done: https://forum.rainmeter.net/viewtopic.php?f=118&t=34470&start=10#p170558

@ JSMorley

Yes, language and measurements can present problems for end users when they do not want the default locale settings.

In my opinion, which you may disagree with, If an end users wants to change the defaults, that is something they can request.

@ SilverAzide

Thanks for the locale and numbers information. With that, it is a possibility, but I am happy to be able to offer the weather information in their language, with locale measurements, and handle changes on case-by-case basis.

That is as far as I intend go, given the complexity of international number systems. :p
User avatar
jsmorley
Developer
Posts: 20269
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ Some Help With Parsing weather.com

Post by jsmorley »

eclectic-tech wrote:
January 23rd, 2020, 8:39 pm
Done: https://forum.rainmeter.net/viewtopic.php?f=118&t=34470&start=10#p170558

@ JSMorley

Yes, language and measurements can present problems for end users when they do not want the default locale settings.

In my opinion, which you may disagree with, If an end users wants to change the defaults, that is something they can request.
Neat!

I don't disagree with that in the least. Using something like:

Text=[&LuaScript:ToggleTemperatureUnit('[&@CurrentTemperature]','[&@CurrentTemperatureUnit]')]

on a String meter is perfectly reasonable, but given that a weather skin may have dozens of temperature values displayed, it's probably overkill for the very small number of users who won't be happy with Language=en-GB to accomplish the same thing and more.

My point wasn't that it shouldn't or couldn't be done, but that it will always have to be "reactive". What you can't "proactively" do is tell weather.com "Send me the temperatures in celsius, the wind speed in mph, the barometer in millibars, and the times in 12-hour time, and send me all that using German.". The "German" bit of that controls everything...
User avatar
SilverAzide
Posts: 690
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ Some Help With Parsing weather.com

Post by SilverAzide »

jsmorley wrote:
January 19th, 2020, 1:36 am
Feel free to ask questions...
Hello... found a bug in the "next 36 hours" include file. Every [@CardNConditions] and [@CardNDetails] measure needs to have DecodeCharacterReference=1 added. This will fix non-English text, like Pluie dans l'après-midi / Chutes de neige.

Secondly, a question...
I am seeing text like the following when I set my locale to French (fr-FR):

Code: Select all

Nuageux. Maximales : 2 ºC. Vents O soufflant de 15 à 25 km\u002Fh.
I have no idea what is up with that "\u002F" stuff even with the DecodeCharacterReference=1 added, but when I try a Substitute to turn this style of markup to the Rainmeter-equivalent "[\x002F]" I just get the literal result of Nuageux. Maximales : 2 ºC. Vents O soufflant de 15 à 25 km[\x002F]h.. Perhaps my regexp is bad (I am the first to admit I stink at regexps). Brute force Substitute="\u002F":"/" works of course, but I don't want to do that for all 64000 codes, lol. Maybe the slash is being treated uniquely and I can skip the other 63999 values. ;)
DeviantArt Gadgets More...
User avatar
eclectic-tech
Rainmeter Sage
Posts: 3763
Joined: April 12th, 2012, 9:40 pm
Location: Cedar Point, Ohio, USA

Re: ⭐ Some Help With Parsing weather.com

Post by eclectic-tech »

SilverAzide wrote:
January 24th, 2020, 2:06 am
Hello... found a bug in the "next 36 hours" include file. Every [@CardNConditions] and [@CardNDetails] measure needs to have DecodeCharacterReference=1 added. This will fix non-English text, like Pluie dans l'après-midi / Chutes de neige.

Secondly, a question...
I am seeing text like the following when I set my locale to French (fr-FR):

Code: Select all

Nuageux. Maximales : 2 ºC. Vents O soufflant de 15 à 25 km\u002Fh.
I have no idea what is up with that "\u002F" stuff even with the DecodeCharacterReference=1 added, but when I try a Substitute to turn this style of markup to the Rainmeter-equivalent "[\x002F]" I just get the literal result of Nuageux. Maximales : 2 ºC. Vents O soufflant de 15 à 25 km[\x002F]h.. Perhaps my regexp is bad (I am the first to admit I stink at regexps). Brute force Substitute="\u002F":"/" works of course, but I don't want to do that for all 64000 codes, lol. Maybe the slash is being treated uniquely and I can skip the other 63999 values. ;)
Seeing the same here... what I find strange is the wind speed in km/h is captured and decoded fine, but this is not happening in the detail capture.

Not sure why it works in one place but not another (something in the site code?). I am going with a straight substution of just that one code, and see what else may appear in the future.

Good catch on the missing DecodeCharacterReference!