Backwards WebParsing?

[balala]

But Balala, he isn't interested in all the trains. Only the ones for a specific operator.

Exactly. That's why I used the IfMatch / IfMatchAction / IfNotMatchAction options on the measures which returns the operator.

And I think you will find that:

Item=(?(?=.*<d).*estination_name>(.*)</destination_name>.*<platform>.*</platform>.*<operator>(.*)</operator>)

Will NOT get you what you might expect if you for instance make it:

Item=(?(?=.*<d).*estination_name>(.*)</destination_name>.*<platform>.*</platform>.*<operator>#OperatorName#</operator>)

Whether you make it "greedy" (?si) or "ungreedy" (?siU) you will find that the fact that you have repeating search patterns, with .* (skip all chars) between them is going to cause problems.

Since we don't know the URL, I can check only the posted code, but with that I think my code works.

[jsmorley]

Exactly. That's why I used the IfMatch / IfMatchAction / IfNotMatchAction options on the measures which returns the operator.

Since we don't know the URL, I can check only the posted code, but with that I think my code works.

Well, I guess it can work to just return them all, setting some maximum limit to the number of trains, and then have meters that end up blank, but that is probably not how I would want to do it.

[balala]

Well, I guess it can work to just return them all, setting some maximum limit to the number of trains, and then have meters that end up blank, but that is probably not how I would want to do it.

Well, I'm also curious about your lua solution. Please post it when will be ready.

[jsmorley]

Well, I'm also curious about your lua solution. Please post it when will be ready.

Almost... Just tweaking some cosmetics...

[jsmorley]

TrainsPlanesAndAutomobiles_1.0.rmskin

1.png

Skin:

Code: Select all

[Rainmeter]
Update=1000
AccurateText=1
DynamicWindowSize=1

[MeasureSite]
Measure=Plugin
Plugin=WebParser
URL=file://#CURRENTPATH#Test.html
RegExp=(?siU)^(.*)$
StringIndex=1
UpdateRate=15
FinishAction=[!CommandMeasure MeasureScript "ParseSite()"]

[MeasureScript]
Measure=Script
ScriptFile=#CURRENTPATH#TrainsPlanesAndAutomobiles.lua
Operator=SN
MaxTrains=4
UpdateDivider=-1

[MeterDest1]
Meter=String
Group=All|Train1
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterAimed1]
Meter=String
Group=All|Train1
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterExpected1]
Meter=String
Group=All|Train1
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterDest2]
Meter=String
Group=All|Train2
Y=10R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterAimed2]
Meter=String
Group=All|Train2
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterExpected2]
Meter=String
Group=All|Train2
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterDest3]
Meter=String
Group=All|Train3
Y=10R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterAimed3]
Meter=String
Group=All|Train3
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterExpected3]
Meter=String
Group=All|Train3
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterDest4]
Meter=String
Group=All|Train4
Y=10R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterAimed4]
Meter=String
Group=All|Train4
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

[MeterExpected4]
Meter=String
Group=All|Train4
Y=0R
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
Hidden=1

Lua:

Code: Select all

function Initialize()

	inputMeasure = SKIN:GetMeasure('MeasureSite')
	trainPattern = '<update type=.-</update>'
	destinationPattern = '<destination_name>(.-)</destination_name>'
	aimedPattern = '<aimed_departure_time>(.-)</aimed_departure_time>'
	expectedPattern = '<expected_departure_time>(.-)</expected_departure_time>'
	
end
	
function Update()
		
end

function ParseSite()

	SKIN:Bang('!HideMeterGroup', 'All')
	
	operatorString = SELF:GetOption('Operator', 'None')
	maxTrains = tonumber(SELF:GetOption('MaxTrains', '1'))
	
	entireFeed = inputMeasure:GetStringValue()
	dummyString, trainCount = string.gsub(entireFeed, '<update type', '')
	startPos = 0
	meterNumber = 1
	
	for i = 1, trainCount do
	
		trainStart, trainEnd = string.find(entireFeed, trainPattern, startPos)
		oneTrain = string.sub(entireFeed, trainStart, trainEnd)
		destinationName = string.match(oneTrain, destinationPattern)
		aimedTime = string.match(oneTrain, aimedPattern)
		expectedTime = string.match(oneTrain, expectedPattern)
		operatorName = string.match(oneTrain, '<operator>('..operatorString..')</operator>')
		
		if operatorName then
			
			SKIN:Bang('!SetOption', 'MeterDest'..meterNumber, 'Text', destinationName)
			SKIN:Bang('!SetOption', 'MeterAimed'..meterNumber, 'Text', aimedTime)
			SKIN:Bang('!SetOption', 'MeterExpected'..meterNumber, 'Text', expectedTime)
			SKIN:Bang('!ShowMeterGroup', 'Train'..meterNumber)
			meterNumber = meterNumber + 1
			
		end
		
		if meterNumber > maxTrains then break end
		
		startPos = trainEnd + 1
		
	end
	
end

html:

Code: Select all

<station>
<station_name>Crystal Palace</station_name>
<request_time>2016-04-12T17:56:19</request_time>
<station_code>CYP</station_code>
<updates>
<update type="departure">
<mode>train</mode>
<service>24787000</service>
<train_uid>W64127</train_uid>
<origin_name>Norwood Junction</origin_name>
<destination_name>South Bermondsey</destination_name>
<platform>1</platform>
<operator>SN</operator>
<aimed_departure_time>17:55:00</aimed_departure_time>
<expected_departure_time>18:00:00</expected_departure_time>
<best_departure_estimate_mins>3</best_departure_estimate_mins>
<aimed_arrival_time>17:53:00</aimed_arrival_time>
<expected_arrival_time>17:58:00</expected_arrival_time>
<best_arrival_estimate_mins>1</best_arrival_estimate_mins>
<status>LATE</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24787000</service>
<train_uid>W63651</train_uid>
<origin_name>London Bridge</origin_name>
<destination_name>Beckenham Junction</destination_name>
<platform>2</platform>
<operator>SN</operator>
<aimed_departure_time>17:57:00</aimed_departure_time>
<expected_departure_time>18:00:00</expected_departure_time>
<best_departure_estimate_mins>3</best_departure_estimate_mins>
<aimed_arrival_time>17:57:00</aimed_arrival_time>
<expected_arrival_time>18:00:00</expected_arrival_time>
<best_arrival_estimate_mins>3</best_arrival_estimate_mins>
<status>LATE</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24782000</service>
<train_uid>W65388</train_uid>
<origin_name>Epsom</origin_name>
<destination_name>London Victoria</destination_name>
<platform>1</platform>
<operator>SN</operator>
<aimed_departure_time>17:58:00</aimed_departure_time>
<expected_departure_time>18:14:00</expected_departure_time>
<best_departure_estimate_mins>17</best_departure_estimate_mins>
<aimed_arrival_time>17:57:00</aimed_arrival_time>
<expected_arrival_time>18:13:00</expected_arrival_time>
<best_arrival_estimate_mins>16</best_arrival_estimate_mins>
<status>LATE</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24782000</service>
<train_uid>W65357</train_uid>
<origin_name>London Victoria</origin_name>
<destination_name>Sutton (Surrey)</destination_name>
<platform>2</platform>
<operator>SN</operator>
<aimed_departure_time>18:04:00</aimed_departure_time>
<expected_departure_time>18:06:00</expected_departure_time>
<best_departure_estimate_mins>9</best_departure_estimate_mins>
<aimed_arrival_time>18:03:00</aimed_arrival_time>
<expected_arrival_time>18:05:00</expected_arrival_time>
<best_arrival_estimate_mins>8</best_arrival_estimate_mins>
<status>LATE</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>22215003</service>
<train_uid>L42553</train_uid>
<origin_name>Crystal Palace</origin_name>
<destination_name>Highbury & Islington</destination_name>
<platform>5</platform>
<operator>LO</operator>
<aimed_departure_time>18:06:00</aimed_departure_time>
<expected_departure_time>18:06:00</expected_departure_time>
<best_departure_estimate_mins>9</best_departure_estimate_mins>
<aimed_arrival_time/>
<expected_arrival_time/>
<best_arrival_estimate_mins/>
<status>STARTS HERE</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24783000</service>
<train_uid>W63085</train_uid>
<origin_name>London Bridge</origin_name>
<destination_name>London Victoria</destination_name>
<platform>4</platform>
<operator>SN</operator>
<aimed_departure_time>18:14:00</aimed_departure_time>
<expected_departure_time>18:14:00</expected_departure_time>
<best_departure_estimate_mins>17</best_departure_estimate_mins>
<aimed_arrival_time>18:13:00</aimed_arrival_time>
<expected_arrival_time>18:13:00</expected_arrival_time>
<best_arrival_estimate_mins>16</best_arrival_estimate_mins>
<status>EARLY</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24787000</service>
<train_uid>W63656</train_uid>
<origin_name>Beckenham Junction</origin_name>
<destination_name>South Bermondsey</destination_name>
<platform>1</platform>
<operator>SN</operator>
<aimed_departure_time>18:17:00</aimed_departure_time>
<expected_departure_time>18:17:00</expected_departure_time>
<best_departure_estimate_mins>20</best_departure_estimate_mins>
<aimed_arrival_time>18:17:00</aimed_arrival_time>
<expected_arrival_time>18:17:00</expected_arrival_time>
<best_arrival_estimate_mins>20</best_arrival_estimate_mins>
<status>ON TIME</status>
<source>Network Rail</source>
</update>
<update type="departure">
<mode>train</mode>
<service>24782000</service>
<train_uid>W63098</train_uid>
<origin_name>London Victoria</origin_name>
<destination_name>London Bridge</destination_name>
<platform>6</platform>
<operator>SN</operator>
<aimed_departure_time>18:20:00</aimed_departure_time>
<expected_departure_time>18:20:00</expected_departure_time>
<best_departure_estimate_mins>23</best_departure_estimate_mins>
<aimed_arrival_time>18:19:00</aimed_arrival_time>
<expected_arrival_time>18:19:00</expected_arrival_time>
<best_arrival_estimate_mins>22</best_arrival_estimate_mins>
<status>ON TIME</status>
<source>Network Rail</source>
</update>
</updates>
</station>

Balala's approach might work, and maybe just as well, I'm not sure. I do 90% or more of all "parsing" of websites in Lua due to the power and flexibility it has, so I tend to jump there quickly...

There is no maximum amount that can be received and parsed from the site, the only thing you need to do to get more is to increase MaxTrains in the Script measure, and add some more meters. With the current MaxTrains=4, If there are 100 trains, and the operator you specify has the 2nd, 5th, 94th, and 98th, it will still work just fine.

I'm happy to explain any or all of this.

P.S. I have this set to get the local file every 15 seconds, so I could easily test it by changing the .html file. Don't leave your skin hitting that site every 15 seconds! Change UpdateRate to 600.

[balala]

Please let me one comment about this lua code: when you first posted about your intention to create a solution using a lua script, I thought it'll return as many results as exists, no matter how large this number is. But this code returns just 4 trains. Yes, I know this number can be easily extended (and just for a try I did), but the number of stations is still limited.

[jsmorley]

Please let me one comment about this lua code: when you first posted about your intention to create a solution using a lua script, I thought it'll return as many results as exists, no matter how large this number is. But this code returns just 4 trains. Yes, I know this number can be easily extended (and just for a try I did), but the number of stations is still limited.

I went entirely by the stated requirements of the user. He said he wanted the first four trains that belonged to a specific operator. That is what this does. It's easy to extend, but I just responded to the request.

There is no maximum amount that can be received and parsed from the site, the only thing you need to do to get more is to increase MaxTrains in the Script measure, and add some more meters. With the current MaxTrains=4, If there are 100 trains, and the operator you specify has the 2nd, 5th, 94th, and 98th, it will still work just fine.

[jsmorley]

Make no mistake, this does get ALL the trains from the feed, and parses each and every one of them. It will parse 1 or 1000, it doesn't care. It only sets the meter values if the operator matches, and only MaxTrains times, so you can make sure you have enough meters to hold MaxTrains worth of data. The only real limiting factor is the number of meters you create, but that is always going to be the case in a Rainmeter skin.

[balala]

I went entirely by the stated requirements of the user. He said he wanted the first four trains that belonged to a specific operator. That is what this does. It's easy to extend, but I just responded to the request.

Sorry, you're right, I missed this request and also missed that he needs near the destination, the aimed departure time and the expected departure time (it's my mistake!). It seems I'll have to rewrite my code, but I'll do this just tomorrow (because here is almost midnight).

[balala]

Make no mistake, this does get ALL the trains from the feed, and parses each and every one of them. It will parse 1 or 1000, it doesn't care. It only sets the meter values if the operator matches, and only MaxTrains times, so you can make sure you have enough meters to hold MaxTrains worth of data. The only real limiting factor is the number of meters you create, but that is always going to be the case in a Rainmeter skin.

Yes, for sure this is a great advantage of the lua code, as the number of data are not limited to certain number as it is with the plain Rainmeter code (where a set of WebParser measures, can return up to 100 data).