Using WebParser in Lua Script Loop

[Wallboy]

I want to use a single WebParser measure to download a bunch of images with DownloadFile, and was wondering if it's ok to setup each image through SetOption on the measure and loop through all these in one go quickly. I was just wondering if I could run into any problems where an image hasn't been completed downloaded/parsed by the WebParser and the loop has already set and activated the measure for the next image/s.

For now I'm going to set it up where a FinishAction on the WebParser just calls that function again. That way I think I'll be clear of any problems with trying to change the Measures information too quickly before it's finished parsing/downloading the current image.

Though I'd like to use the first option and just download them all in one script call if it won't cause issues.

Thanks.

[killall-q]

Yes, you can capture the entire list of URLs in one string index, and parse it with a for loop in Lua. This is probably the more efficient method of parsing a large or unknown size list of similar items.

Code: Select all

for URL, title in string.gmatch(SKIN:GetMeasure('measureSite'):GetStringValue(), 'src="([^"]*)".-title="([%w%s]*)"') do
   URLList[#URLList + 1], titleList[#titleList + 1] = URL, title
end

Lua regex is different from the WebParser regex we know and love. For one, it is greedy, so you have to be careful about matching with .*. Use [] to restrict the characters to capture or use .- (as few as possible until the next pattern) instead.

[Wallboy]

Well, the URL and DownloadFile is being set from SetOption from values I'm using in a table. My question isn't about capturing URLS in one index, but rather if WebParser can handle the speed of a loop in Lua changing it's DownloadFile, URL values, with a CommandMeasure Update in each loop.

[killall-q]

WebParser isn't the bottleneck, the internet connection is. Not being able to download the next image until the current image is finished will make things annoyingly slow. You can improve the situation with multiple measures downloading in batches.

[Wallboy]

Right, that's what I thought. I currently have it implemented and working with a single measure now using FinishAction to grab the next image through a script call. I just keep track of which image I'm currently on in the script using some global variables. Seems to be working great, and isn't dependent on the connection speed.