It is currently March 28th, 2024, 7:50 pm

⭐ Weather.com - Parsing the V3 JSON

Our most popular Tips and Tricks from the Rainmeter Team and others
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

⭐ Weather.com - Parsing the V3 JSON

Post by jsmorley »




NOTE: 🟢 This version corrects the @Include .inc files to reflect using the API JSON site.


WeatherComJSON_January 11, 2022.rmskin


1.jpg



This uses a series of @Include .inc files to eliminate having to figure out regular expression or even create any measures for your skin. It does all the hard work for you!

Then you can just concentrate on the Meters in your skin. All the measures you will need are already there! Just create the meters, and use the MeasureName= options you can easily get by looking at About/Skins to see what the measure names are that are populated. I tried to make the measure names as descriptive as possible, and if you look in the include files, I have done as much commenting as I can.

If you want to "overload" an included measure, for instance to use Substitute or put an IfCondition or IfMatch on the measure, just create a copy of the measure as a Measure=String or Measure=Calc in the skin, and dynamically use the value of the original measure as a [SectionVariable].

I strongly recommend that you DON'T alter WeatherComJSONMeasures.inc.
That just reduces its usefulness as something you can just plug into another new skin, and makes it very difficult indeed for you if I update it.


WeatherComJSONVariables.inc
This is where you set your Location latitude,longitude, units of measure and Language. Read it for help.
Always @Include this file.


WeatherComJSONMeasures.inc
This parses all 15 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures3Day.inc
This parses 3 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures5Day.inc
This parses 5 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures7Day.inc
This parses 7 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures10Day.inc
This parses 10 days of the JSON code and creates the measures you will need.
Always @Include one and only one of these files.

WeatherComJSONPollenCurrent.inc
This will create measures for the Grass / Tree / Ragweed Pollen levels for the current day
WeatherComJSONPollen29Parts.inc
This will create measures for the Grass / Tree / Ragweed Pollen levels for 29 parts of day and night values
WeatherComJSONPollen15Parts.inc
This will create measures for the Grass / Tree / Ragweed Pollen levels for 15 parts of day and night values
WeatherComJSONPollen7Parts.inc
This will create measures for the Grass / Tree / Ragweed Pollen levels for 7 parts of day and night values
It is optional to @Include one and only one of these files.

WeatherComJSONMoon.inc
This will create measures for the Moon Phase and MoonSet / MoonRise times for 15 days.
WeatherComJSONMoon7Day.inc
This will create measures for the Moon Phase and MoonSet / MoonRise times for 7 days.
It is optional to @Include one and only one of these files.

WeatherComJSONAlerts.inc
This will create measures for any severe weather alerts being returned.
It is optional to @Include this file.


WeatherComJSONLanguage.inc
This will allow you to translate the "labels" I use in my skin to any other language you want.
This is specific to this skin, and you may not need it for yours.

GeneralVariables.inc
This is where I set all the general font and color variables.
This is specific to this skin, and you may not need it for yours.



Once you @Include WeatherComJSONVariables.inc AND WeatherComJSONMeasures.inc in your skin, the following measures will be created. You can use About / Skins to see them AND the values they currently have, but here is a list for reference:
======================
Units of measure
======================
[@UnitsType] : This will be "m" or "e" or "h"
Note that "h" is for the hybrid units used in the UK.
[@UnitsTemperature] : This will be "C" or "F"
[@UnitsSpeed] : This will be "km/h" or "mph"
[@UnitsDistance] : This will be "km" or "mi"
[@UnitsAccumulation] : This will be "cm" or "in"
[@UnitsPrecipitation] : This will be "mm" or "in"
[@UnitsPressure] This will be "mb" or "in"

======================
Location information
======================
[@LocationLatitude] : i.e. 38.805
[@LocationLongitude] : i.e -77.047
[@LocationAdminDistrict]
[@LocationAdminDistrictCode]
These AdminDistrict values will vary in meaning depending on where you live.
In the US, this will be the "state". Other places it may be "region", or just empty.
[@LocationCountry] : i.e. United States of America
[@LocationCountryCode] : i.e. US
[@LocationName] : Location Display Name - Generally the "city"
[@LocationLanguageCode] : i.e. en-US
[@LocationTimeZoneAbbreviation] : i.e. EST

======================
Current weather information
======================
[@CurrentPressure] : Use @UnitsPressure
[@CurrentPressureChangeTrend] : Text "Steady", "Rising", "Falling"
[@CurrentPressureCode] : 0 - "Steady" 1 - "Rising" 2 - "Falling"
[@CurrentPressureChangeAmount] : Use @UnitsPressure
[@CurrentDewPoint]
[@CurrentFeelsLike]
[@CurrentWindGust] : Use @UnitsSpeed
[@CurrentHumidity] : Percentage
[@CurrentIcon]
[@CurrentObservationTimeStamp]
[@CurrentObservationDate]
Automatically uses defined #Lanaguage# (i.e. en-US) to translate Month/Day names.
[@CurrentObservationTime]
[@CurrentConditions] : This is the "long" version
[@CurrentConditionsMedium]
[@CurrentConditionsShort]
[@CurrentCloudCoverPhrase]
[@CurrentPrecipitationLast24Hours] : Use @UnitsPrecipitation
[@CurrentSnowDepth] : Use @UnitsAccumulation
[@CurrentTemperature]
[@CurrentTemperatureMaxSince7AM]
[@CurrentUVIndex] : A range of 1-10
[@CurrentUVDescription]
[@CurrentVisibilityDistance] : Use @UnitsDistance
[@CurrentWindSpeed] : Use @UnitsSpeed
[@CurrentWindDirectionCompass] : i.e. "NNW"
[@CurrentWindDirectionDegrees] : i.e. 220
[@CurrentSunriseTimeStamp]
[@CurrentSunriseTime]
[@CurrentSunSetTimeStamp]
[@CurrentSunsetTime]

======================
Today - Forecast for the entire day
======================
[@ForecastTodayDateString] : Full date string for Today - i.e. "2020-01-21T07:00:00-0400"
[@ForecastTodayDayLong] : Full day of the week i.e. Saturday
[@ForecastTodayDayShort] : i.e. Sat
[@ForecastTodayMonthLong] : i.e. "January"
[@ForecastTodayMonthShort] : i.e. "Jan"
[@ForecastTodayDayOfMonth] : 1-31
[@ForecastTodayNarative]
[@ForecastTodayHighTemperature]
[@ForecastTodayLowTemperature]
[@ForecastTodaySunriseTimeStamp]
[@ForecastTodaySunriseTime]
[@ForecastTodaySunsetTimeStamp]
[@ForecastTodaySunsetTime]
[@ForecastTodayQPFRain] : Quantitative Precipitation Forecasts - Use @UnitsPrecipitation
[@ForecastTodayQPFSnow] : Quantitative Precipitation Forecasts - Use @UnitsAccumulation

======================
Today - Forecast for "Day"
======================
[@ForecastTodayDayPart] : "Today"
[@ForecastTodayDayLetter] : "D"
[@ForecastTodayDayIcon]
[@ForecastTodayDayConditions]
[@ForecastTodayDayConditionsShort]
[@ForecastTodayDayTemperature]
[@ForecastTodayDayHeatIndex]
[@ForecastTodayDayWindChill]
[@ForecastTodayDayNarrative]
[@ForecastTodayDayHumidity]
[@ForecastTodayDayWindSpeed]
[@ForecastTodayDayWindDirectionDegrees] : i.e. 120
[@ForecastTodayDayWindDirectionCompass] : i.e. NNW
[@ForecastTodayDayWindPhrase] : i.e. "Winds SE at 5 to 10 mph."
[@ForecastTodayDayPrecipitationPercent] : Percent likely
[@ForecastTodayDayPrecipitationType] : "rain", "snow" or the generic "precip"
[@ForecastTodayDayPrecipitationAmount]
[@ForecastTodayDayThunderCategory]
[@ForecastTodayDayThunderIndex]
[@ForecastTodayDaySnowAmount] : Use @UnitsAccumulation
[@ForecastTodayDaySnowRange] : Text range, i.e. "2-5" - Use @UnitsAccumulation
[@ForecastTodayDayCloudPercent]
[@ForecastTodayDayUVIndex] : A range of 1-10
[@ForecastTodayDayUVIndexDescription]

Note: At some point in the afternoon, all "TodayDay" measures except
[@ForecastTodayDayLong] will return an empty string. If [@ForecastTodayDayTemperature]
is an empty string, then it is "Tonight". There can't be a "forecast" for a part of day
that has already gone by. You must react to that, and perhaps use "TodayNight"
measures in their place.

======================
Today - Forecast for "Night"
======================
[@ForecastTodayNightPart] : "Tonight"
[@ForecastTodayNightLetter] : "N"
[@ForecastTodayNightIcon]
[@ForecastTodayNightConditions]
[@ForecastTodayNightConditionsShort]
[@ForecastTodayNightTemperature]
[@ForecastTodayNightHeatIndex]
[@ForecastTodayNightWindChill]
[@ForecastTodayNightNarrative]
[@ForecastTodayNightHumidity]
[@ForecastTodayNightWindSpeed]
[@ForecastTodayNightWindDirectionDegrees] : i.e. 120
[@ForecastTodayNightWindDirectionCompass] : i.e. "NNW"
[@ForecastTodayNightWindPhrase] : i.e. "Winds SE at 5 to 10 mph."
[@ForecastTodayNightPrecipitationPercent] : Percent likely
[@ForecastTodayNightPrecipitationType] : "rain", "snow" or the generic "precip"
[@ForecastTodayNightPrecipitationAmount]
[@ForecastTodayNightThunderCategory]
[@ForecastTodayNightThunderIndex]
[@ForecastTodayNightSnowAmount] : Use @UnitsAccumulation
[@ForecastTodayNightSnowRange] : Text range, i.e. "2-5" - Use @UnitsAccumulation
[@ForecastTodayNightCloudPercent]
[@ForecastTodayNightUVIndex] : A range of 1-10
[@ForecastTodayNightUVIndexDescription]

======================
Day2 - Forecast for the entire day
======================
[@ForecastDay2DateString] : Full date string for Tomorrow - i.e. "2020-01-22T07:00:00-0400"
[@ForecastDay2DayLong] : Full day of the week i.e. Sunday
[@ForecastDay2DayShort] : i.e. Sun
[@ForecastDay2MonthLong] : i.e. "January"
[@ForecastDay2MonthShort] : i.e. "Jan"
[@ForecastDay2DayOfMonth] : 1-31
[@ForecastDay2Narative]
[@ForecastDay2HighTemperature]
[@ForecastDay2LowTemperature]
[@ForecastDay2SunriseTimeStamp]
[@ForecastDay2SunriseTime]
[@ForecastDay2SunsetTimeStamp]
[@ForecastDay2SunsetTime]
[@ForecastDay2QPFRain]
[@ForecastDay2QPFSnow]

======================
Day2 - Forecast for "Day"
======================
[@ForecastDay2DayPart] : "Tomorrow"
[@ForecastDay2DayLetter] : "D"
[@ForecastDay2DayIcon]
[@ForecastDay2DayConditions]
[@ForecastDay2DayConditionsShort]
[@ForecastDay2DayTemperature]
[@ForecastDay2DayHeatIndex]
[@ForecastDay2DayWindChill]
[@ForecastDay2DayNarrative]
[@ForecastDay2DayHumidity]
[@ForecastDay2DayWindSpeed]
[@ForecastDay2DayWindDirectionDegrees]
[@ForecastDay2DayWindDirectionCompass]
[@ForecastDay2DayWindPhrase]
[@ForecastDay2DayPrecipitationPercent]
[@ForecastDay2DayPrecipitationType]
[@ForecastDay2DayPrecipitationAmount]
[@ForecastDay2DayThunderCategory]
[@ForecastDay2DayThunderIndex]
[@ForecastDay2DaySnowAmount]
[@ForecastDay2DaySnowRange]
[@ForecastDay2DayCloudPercent]
[@ForecastDay2DayUVIndex]
[@ForecastDay2DayUVIndexDescription]

======================
Day2 - Forecast for "Night"
======================
[@ForecastDay2NightPart] : "Tomorrow night"
[@ForecastDay2NightLetter] : "N"
[@ForecastDay2NightIcon]
[@ForecastDay2NightConditions]
[@ForecastDay2NightConditionsShort]
[@ForecastDay2NightTemperature]
[@ForecastDay2NightHeatIndex]
[@ForecastDay2NightWindChill]
[@ForecastDay2NightNarrative]
[@ForecastDay2NightHumidity]
[@ForecastDay2NightWindSpeed]
[@ForecastDay2NightWindDirectionDegrees]
[@ForecastDay2NightWindDirectionCompass]
[@ForecastDay2NightWindPhrase]
[@ForecastDay2NightPrecipitationPercent]
[@ForecastDay2NightPrecipitationType]
[@ForecastDay2NightPrecipitationAmount]
[@ForecastDay2NightThunderCategory]
[@ForecastDay2NightThunderIndex]
[@ForecastDay2NightSnowAmount]
[@ForecastDay2NightSnowRange]
[@ForecastDay2NightCloudPercent]
[@ForecastDay2NightUVIndex]
[@ForecastDay2NightUVIndexDescription]

This pattern will repeat for Day3 - Day15

Edit January 11, 2022 Changed the API Key required.

Edit August 4, 2020 Added .inc files to provide Pollen level information.

Edit July 31, 2020 - IMPORTANT! This reworks entirely how the data is retrieved from weather.com, in order to address some ugly changes in the web site. This will now access the API data from weather.com to obtain data, and hopefully will be more stable over time. One critical change is that you no longer can use location "codes" with this, but must use the "latitude,longitude" for the desired location. See WeatherComJSONVariables.inc for details.

Edit June 20, 2020 - Tweaked the regular expression to account for the fact that there can be commas in several of the fields, like the "narrative" entries. Also added a missing WindPhrase field to the measures.

Edit June 19, 2020 - Simplified the RegExp for "forecast" somewhat. This is just to make the regular expression a bit simpler and hopefully easier to follow, and doesn't impact any functionality. Also, re-ordered the measures in the "current" section, to make it a bit easier to deal with in About/Skins.

Edit June 18, 2020 - Revised the WeatherComJSONMoon.inc file, to include the new measure for Moon Phase Days/Age, as well as extending it to support the entire 15 days included in the JSON. I have also created a WeatherComJSONMoon7Day.inc file, which will only parse 7 days, if you don't want or need the full 15.

Edit June 17, 2020 - Changed how the values for the "day short", "month long", "month short" and "day of month" are retrieved. This will hopefully correct an issue where the "day long" was out of sync with the other values for an hour or so right after midnight.

Edit June 16, 2020 - Created different versions of WeatherComJSONMeasures.inc based on the number of days you want to forecast.
If you use the full WeatherComJSONMeasures.inc file, that is going to create north of 900 measures in your skin, to support the entire 15 days included in the JSON. I have created versions parsing fewer days, so you don't have to wade through as many measures in About /Skins, many of which you may never use.

WeatherComJSONMeasures.inc
This parses all 15 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures3Day.inc
This parses 3 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures5Day.inc
This parses 5 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures7Day.inc
This parses 7 days of the JSON code and creates the measures you will need.
WeatherComJSONMeasures10Day.inc
This parses 10 days of the JSON code and creates the measures you will need.
Always @Include one and only one of these files.



Edit June 16, 2020 - Reworked the entire WeatherComJSONMeasures.inc file.
- Added information for all 15 days in the JSON, instead of the previous limit of 7 days.
- Reworked the order of the measures so it makes it easier to follow in About / Skins.
- Integrated daily Sunrise and Sunset into the main .inc file, and eliminated the redundant WeatherComJSONSun.inc file.
- Added some additional fields of information to the measures. See the updated MeasureNamesReference.txt for details.

Just drop the new .inc file(s) in your @Resources folder, and refresh your skin. The new .inc file is fully backwards compatible with your existing skins that use it.

The WeatherComJSONVariables.inc file has not changed in format, so you might want to save yours, to avoid having to reset your Language and LocationCode variables.

Edit: June 11, 2020 - Changed the WebParser super-parent measure to use the new NoCookies and Resync Flag options.

Edit: June 7, 2020 - Corrected the field used for "Current Conditions". A side effect of this is that there are three new measures added:

[@CurrentConditionsMedium]
[@CurrentConditionsShort]
[@CurrentCloudCoverPhrase]

Edit: June 5, 2020 - An entirely new "drop in" version of the @Include .inc files has been created by a collaborative effort of users OnyxBlack, Xenium, and SilverAzide. A TON of thanks to them. Simply replace:

WeatherComJSONMeasures.inc
WeatherComJSONMoon.inc
WeatherComJSONAlerts.inc

In your @Resources folder, refresh your skin, and it should work fine.
You do not have the required permissions to view the files attached to this post.
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

This is how I "react" to the fact that after some point in the afternoon all the "forecast" values for "today day" will be empty:

Code: Select all

[MeasureToggleAtNight]
Measure=String
Group=Weather
String=[@ForecastTodayDayTemperature]
DynamicVariables=1
IfMatch=^$
IfMatchAction=[!SetOption MeterTodayIcon MeasureName "@ForecastTodayNightIcon"][!SetOption MeterTodayName MeasureName "@ForecastTodayNightPart"][!SetOption MeterTodayHighLow MeasureName "@CurrentTemperatureMaxSince7AM"][!SetOption MeterTodayPrecipitation MeasureName "@ForecastTodayNightPrecipitationPercent"][!SetOption MeterTodayHighLow InlineSetting "Color | 160,160,160,255"]
IfNotMatchAction=[!SetOption MeterTodayIcon MeasureName "@ForecastTodayDayIcon"][!SetOption MeterTodayName MeasureName "@ForecastTodayDayPart"][!SetOption MeterTodayHighLow MeasureName "@ForecastTodayDayTemperature"][!SetOption MeterTodayPrecipitation MeasureName "@ForecastTodayDayPrecipitationPercent"][!SetOption MeterTodayHighLow InlineSetting "None"]
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

I'm going to let this percolate on my system for a few days, and if I don't find any issues with it, I will convert all my skins to use this JSON approach. There is no downside to using it instead of the HTML, and some possibility that it will have far greater longevity.
User avatar
eclectic-tech
Rainmeter Sage
Posts: 5382
Joined: April 12th, 2012, 9:40 pm
Location: Cedar Point, Ohio, USA

Re: ⭐ weather.com - Parsing the JSON

Post by eclectic-tech »

Gaff: You've done a man's job, sir. I guess you're through, huh?
Deckard: Finished.

Good work! :thumbup:
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

eclectic-tech wrote: February 8th, 2020, 4:16 am Gaff: You've done a man's job, sir. I guess you're through, huh?
Deckard: Finished.

Good work! :thumbup:
Thanks. There are several advantages and disadvantages to using the JSON.

Advantages:
- More reliable over time. This is a big one...
- Has several additional bits of information that the HTML doesn't.
- All the data, location/current/forecast/hourly are all in one place, one call to the site.
I have not bothered to parse the "hourly", as honestly, I just don't care...It's there though.
- Is FAR easier to parse, the regular expression is just falling off a log easy. You never need any "lookahead".
If the site doesn't have a particular bit of information it returns a literal string of null, rather than leaving it out.
There is really never "missing" data. I just RegExpSubstitute=1 / Substitute="^null$":"" and I have an empty string with no missing StringIndex.


Disadvantages:
- While changing the LanguageCode will make the site return the numeric values in the appropriate units of measure, and string values in the appropriate language, just as before, there isn't an easy way to get the "labels" like "Temperature/Wind/Pressure/Sunrise/Feels Like" and such that you will likely want in your skin. I worked around that by providing a file to allow you to "translate" those labels as desired. There are some translations of these labels in the JSON, but I wasn't happy with them. They are not nearly as "pithy" as the labels on the site HTML are, and there are just a billion of them to parse if you want to be exhaustive.
- Dates and Times are returned in an ISO standard format like 2020-02-07T23:10:51-0500. You really need one or more Time measures to pretty those up and return 12hr-24hr time as desired.
- High and Low temperatures are not distinctly returned as such. There is just a temperature for "day" and that can be treated as the "high" and a temperature for "night" and that can be treated as the "low".

Overall, the advantages greatly outweigh the disadvantages, for which there are ways around.
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

I'll add the "moon" times and phases at some point soon.
User avatar
SilverAzide
Rainmeter Sage
Posts: 2588
Joined: March 23rd, 2015, 5:26 pm

Re: ⭐ weather.com - Parsing the JSON

Post by SilverAzide »

Thank you! :great:
Gadgets Wiki GitHub More Gadgets...
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

SilverAzide wrote: February 8th, 2020, 4:43 pm Thank you! :great:
You are welcome. Unless I find some "gotcha" with this, I think we can be done with parsing the HTML, which has all kinds of risks.

Parsing any "website" that isn't a pure, static, RSS/ATOM/XML/JSON "feed" always has some risk, but it looks to me like they use the embedded JSON data to feed the "display" HTML, and them changing the HTML, which is almost certain, may not impact the underlying data. That's my hope anyway.
User avatar
jsmorley
Developer
Posts: 22628
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: ⭐ weather.com - Parsing the JSON

Post by jsmorley »

Added a measure for [@CurrentPressureCode], which to be honest I'm not sure what it represents, but perhaps a numeric code that indicates how quickly the barometer is "changing". Need to do some testing of this to be sure. I wanted to include it for completeness.

New .rmskin in the first post of this thread.
SCR
Posts: 60
Joined: April 15th, 2015, 11:13 pm

Re: ⭐ weather.com - Parsing the JSON

Post by SCR »

Late last night I was congratulating myself on completing the conversion of my weather skin. Feeling proud of myself that it worked, I went to sleep happy that the weather was once again on my screen and the job was done.

This morning as I settled in with my cup of coffee to read the latest and greatest from Rainmeter I was greeted with your post of your latest miracle.

Now I have to get back to another conversion project. However I don't mind a bit as I am now and forever addicted to Rainmeter. I have never had so much fun with any other software.

Besides, now that I'm in my late 70's Rainmeter keeps the gray matter in my head churning so you can add a health benefit to all the things Rainmeter can do. I don't understand a lot of it but it sure is fun to try and figure out.

Once again I offer my sincerest thanks for your hard work. Maybe not hard for you but near impossible for me. I appreciate all you do.