RegExp to fix Yahoo Weather (as of Feb 2023)

Post by **eclectic-tech** » February 15th, 2023, 3:28 am

I recently updated my skin for YAHOO.com Weather (uses YAHOO WEATHER website) to correct for changes made to their website about a month ago. THey removed most of the JSON data and replaced it with code. The only solution is to scrape the website to retrieve data.

I have seen several post on the forum that correct other skins that use YAHOO WEATHER as the source (Mond, Lumiero, etc.) and thought I would post the RegExp I developed to collect ALL current data.
Here is the RegExp:

Code: Select all

RegExp=(?siU).*<time class=.*, (.*)</time>.*<div class="My\(2px\).*60x60\/(.*).png.*<p class=.*>(.*)</p>.*class="temperature-forecast .*> <span class=.*>(.*)</span><span class=.*>(.*)</span>.*<tr class=.*forecast-item.*</button>(.*)<.*alt=(.*)>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<div id=module-weather-precipitation.*scope=col>(.*)<.*scope=col>(.*)<.*scope=col>(.*)<.*scope=col>(.*)<.*Fz\(1rem\)">(.*)<.*Fz\(1rem\)">(.*)<.*Fz\(1rem\)">(.*)<.*Fz\(1rem\)">(.*)<.*<div id=module-weather-wind-pressure.*<dd class=.*>(.*)<.*<dd class=.*>(.*)<.*<dd class=.*>(.*)<.*<dd class=.*>(.*)<.*<div id=module-weather-details.*<img class=.*60x60\/(.*).png.*<dd class=.*">(.*)&.*</dd><dd class=.*>(.*)&.*</dd>.*<dd>(.*)</dd>.*<dd.*>(.*)</dd>.*<dd.*>(.*)</dd>.*<dd>(.*)</dd>.*<p class=.*>(.*)</p>.*<p class=.*>(.*)</p>.*<div id=module-weather-sun-moon.*<dd>(.*)</dd>.*<time class=.*>(.*)</time><time class=.*>(.*)</time>.*data-meta='.*"city":"(.*)".*"county":"(.*)".*"state":"(.*)".*"country":"(.*)".*"lat":(.*),.*"lon":(.*),

This will capture the following 42 items; here is a list and StringIndex Numbers:

Code: Select all

Site: WWW.YAHOO.COM/NEWS/WEATHER/ 

INDEX => DATA			MEASURENAME
1 => 12:52 PM			.CurrentObservationTime
2 => clear_day@2x		.CurrentIcon
3 => Sunny			.CurrentConditions
4 => 48				.CurrentTemperatureF
5 => 9				.CurrentTemperatureC
6 => Tuesday			.TodayDayName
7 => "Mostly Cloudy"		.TodayConditions
8 => 0%				.TodayPrecipitationPercent
9 => 55Â°			.TodayHighTemperatureF
10 => 13Â°			.TodayHighTemperatureC
11 => 48Â°			.TodayLowTemperatureF
12 => 9Â°			.TodayLowTemperatureC
13 => Afternoon			.TodayPrecipitationFirstLabel
14 => Evening			.TodayPrecipitationSecondLabel
15 => Night			.TodayPrecipitationThirdLabel
16 => Overnight			.TodayPrecipitationFourthLabel
17 => 13%			.TodayPrecipitationPercentFirst
18 => 19%			.TodayPrecipitationPercentSecond
19 => 47%			.TodayPrecipitationPercentThird
20 => 51%			.TodayPrecipitationPercentFourth
21 => 6 mph SSE			.TodayWindSpeedF
22 => 10 km/h SSE		.TodayWindSpeedC
23 => 30.0 inches		.TodayPressureF
24 => 1016.9 millibars		.TodayPressureC
25 => clear_day@2x		.TodayIcon
26 => 52			.TodayFeelsLikeF
27 => 12			.TodayFeelsLikeC
28 => 55%			.TodayHumidity
29 => 12.99 miles		.TodayVisibilityDistanceF
30 => 20.90 km			.TodayVisibilityDistanceC
31 => 3 (Moderate)		.TodayUVIndex
32 => Narrative			.TodayDayNarrative
33 => Narrative			.TodayNightNarrative
34 => Waning Crescent		.TodayMoonPhase
35 => 7:25 AM			.TodaySunriseTime
36 => 6:01 PM			.TodaySunsetTime
37 => Oberlin			.LocationName
38 => Lorain			.LocationObservation
39 => Ohio			.LocationRegion
40 => United States		.LocationCountry
41 => 41.28767			.LocationLatitude
42 => -82.239487		.LocationLongitude

The MeasureNames are used in my skin; they will be different in other skins, but all you need to do to update other YAHOO WEATHER skins is to change the RegExp and update the StringIndex numbers in your measures that correspond to the data list above.
** Note that for Icons, Conditions, and Temperatures you will need to substitution to clean up your displayed info.

While I was doing the update, I also created a RegExp to capture the 10-day forecast. Here is the RegExp:

Code: Select all

RegExp=(?siU).*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<tr class=.*forecast-item.*</button>(.*)<.*<img class=.*60x60\/(.*).png.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>.*<dd class=".*">(.*)</dd>

This will capture 10 sets of the following 7 captures:

Code: Select all

; Returns for each day: (7 Indexes X 10 days = 70 StringIndexes)
INDEX => DESCRIPTION => DATA FORMAT 
1 => Day Name => Sunday
2 => Condition => "Partly Cloudy"
3 => Precip Percent => 0%
4 => High F => 45Â°
5 => High C => 8Â°
6 => Low F => 45Â°
7 =>Low C => 8Â°

** Note that for Conditions and Temperatures you will need to substitution to clean up your displayed info. This is due to two situations; Yahoo randomly does not quote conditions text and also because I use icons, by xenium, that match the icon names captured in Yahoo's code.

Scraping websites is not a reliable source for data due to the fact that most websites do annual/semi-annual updates to their code that will break this method.

Hopefully this info may help you if you need to use YAHOO WEATHER. I would recommend skins that parse the JSON data (use weather.com as the source) as they are a bit more reliable.