Recursive download method

Post by **kenz0** » February 16th, 2010, 8:34 pm

WebParser can get any information from specified webpage by the user. However, is it possible to get information from the linked page on the page?

Yes, It is possible if Rainmeter is version 1.2. (also beta or later)

-----------------------------------------------------------------

First example tries to extract URL of a top entry (Blog posts) on linked pages from the top of Google Trends.

Process 1: Access RSS of Google Trends and parse the top link.

Code: Select all

[SOURCE-0]
Measure=Plugin
Plugin=Plugins\WebParser.dll
UpdateRate=3600
Url=http://www.google.com/trends/hottrends/atom/hourly
RegExp="<a href=\"(.*?)\">(.*?)<"

Process 2: Download parsed link URL to DownloadFile's folder as "src1.html" by using new function "DownloadFile".

Code: Select all

[SOURCE-1]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=[SOURCE-0]
StringIndex=1
Download=1
DownloadFile=src1.html
FinishAction=!RainmeterEnableMeasure SOURCE-2

Process 3: When download of "src1.html" is finished, [SOURCE-2] section becomes enable, and the parsing of "src1.html" is started.

Code: Select all

[SOURCE-2]
Measure=Plugin
Plugin=Plugins\WebParser.dll
UpdateRate=1
Url="file://#CURRENTPATH#DownloadFile\src1.html"
RegExp="<a class=\"gs-title\" href=\"(.*?)\""
FinishAction=!RainmeterDisableMeasure SOURCE-2
Disabled=1

In order not to continue unnecessary parsing, it is important to make [SOURCE-2] section disable after completing parsing.

Finally: URL parsed by [SOURCE-2] section is referred to with [LINK] section.

Code: Select all

[LINK]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=[SOURCE-2]
StringIndex=1

Full code is here.

-----------------------------------------------------------------

Next example traces the link from RSS of customize.org and is the way to get an original wallpaper image.

Process 1: Access RSS feed and download first link to DownloadFile's folder as "src1.html" and then start [SOURCE-1] section.

Code: Select all

[SOURCE-0]
Measure=Plugin
Plugin=Plugins\WebParser.dll
UpdateRate=3600
Url=http://customize.org/feeds/category/wallpapers
RegExp="(?sU)<item>.*<link>(.*)<"
StringIndex=1
Download=1
DownloadFile=src1.html
FinishAction=!RainmeterEnableMeasure SOURCE-1

Process 2: Parse "src1.html" then extract URL of "Download" page.

Code: Select all

[SOURCE-1]
Measure=Plugin
Plugin=Plugins\WebParser.dll
UpdateRate=1
Url="file://#CURRENTPATH#DownloadFile\src1.html"
RegExp="(?sU)class=\"download\".*href=\"(.*)\""
Disabled=1

Process 3: Download "Download" page to DownloadFile's folder as "src2.html" and then start [WALLPAPER] section.

Code: Select all

[SOURCE-2]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=http://customize.org[SOURCE-1]
StringIndex=1
Download=1
DownloadFile=src2.html
FinishAction=!Execute [!RainmeterDisableMeasure SOURCE-1][!RainmeterEnableMeasure WALLPAPER]

Process 4: Parse "src2.html" and extract URL of original wallpaper, and then download it.

Code: Select all

[WALLPAPER]
Measure=Plugin
Plugin=Plugins\WebParser.dll
UpdateRate=1
Url="file://#CURRENTPATH#DownloadFile\src2.html"
RegExp="(?sU)wallpaper details<.*img src=\"(.*)\""
StringIndex=1
Download=1
DownloadFile=wp.jpg
FinishAction=!RainmeterDisableMeasure WALLPAPER
Disabled=1

Finally: Original wallpaper is downloaded to DownloadFile's folder as "wp.jpg".

Full code is here.

-----------------------------------------------------------------

The feature of this method is to parse it after downloading the source to a local folder. If such a processes is repeated step-by-step, it is possible to retrieve information even from the page of deep structure.

~Faradey~ · Post by **~Faradey~** » February 21st, 2010, 8:09 pm

This is very usefull information...i was thinking how to do it in version 1.1..and stuck in the middle

but with this i can finally finish one of my youtube skins \o/

Tnx a lot kenz0

Recursive download method

Recursive download method

Re: Recursive download method