It is currently June 26th, 2019, 10:59 pm

Using WebParser in Lua Script Loop

Discuss the use of Lua in Script measures.
Wallboy
Posts: 62
Joined: October 1st, 2012, 4:53 am

Using WebParser in Lua Script Loop

Wallboy » August 11th, 2014, 2:12 am

I want to use a single WebParser measure to download a bunch of images with DownloadFile, and was wondering if it's ok to setup each image through SetOption on the measure and loop through all these in one go quickly. I was just wondering if I could run into any problems where an image hasn't been completed downloaded/parsed by the WebParser and the loop has already set and activated the measure for the next image/s.

For now I'm going to set it up where a FinishAction on the WebParser just calls that function again. That way I think I'll be clear of any problems with trying to change the Measures information too quickly before it's finished parsing/downloading the current image.

Though I'd like to use the first option and just download them all in one script call if it won't cause issues.

Thanks.
User avatar
killall-q
Posts: 285
Joined: August 14th, 2009, 8:04 am

Re: Using WebParser in Lua Script Loop

killall-q » August 11th, 2014, 3:28 am

Yes, you can capture the entire list of URLs in one string index, and parse it with a for loop in Lua. This is probably the more efficient method of parsing a large or unknown size list of similar items.

Code: Select all

for URL, title in string.gmatch(SKIN:GetMeasure('measureSite'):GetStringValue(), 'src="([^"]*)".-title="([%w%s]*)"') do
   URLList[#URLList + 1], titleList[#titleList + 1] = URL, title
end
Lua regex is different from the WebParser regex we know and love. For one, it is greedy, so you have to be careful about matching with .*. Use [] to restrict the characters to capture or use .- (as few as possible until the next pattern) instead.
Last edited by killall-q on August 11th, 2014, 4:31 am, edited 2 times in total.
Wallboy
Posts: 62
Joined: October 1st, 2012, 4:53 am

Wallboy » August 11th, 2014, 3:43 am

Well, the URL and DownloadFile is being set from SetOption from values I'm using in a table. My question isn't about capturing URLS in one index, but rather if WebParser can handle the speed of a loop in Lua changing it's DownloadFile, URL values, with a CommandMeasure Update in each loop.
User avatar
killall-q
Posts: 285
Joined: August 14th, 2009, 8:04 am

Re: Using WebParser in Lua Script Loop

killall-q » August 11th, 2014, 3:51 am

WebParser isn't the bottleneck, the internet connection is. Not being able to download the next image until the current image is finished will make things annoyingly slow. You can improve the situation with multiple measures downloading in batches.
Wallboy
Posts: 62
Joined: October 1st, 2012, 4:53 am

Re: Using WebParser in Lua Script Loop

Wallboy » August 11th, 2014, 5:05 am

Right, that's what I thought. I currently have it implemented and working with a single measure now using FinishAction to grab the next image through a script call. I just keep track of which image I'm currently on in the script using some global variables. Seems to be working great, and isn't dependent on the connection speed.