It is currently March 29th, 2024, 9:30 am
WebParser value as URL for new WebParser
-
- Posts: 182
- Joined: April 10th, 2014, 12:31 am
- Location: Canada
WebParser value as URL for new WebParser
Hi. I have a WebParser StringIndex which returns a URL (https://weather.gc.ca/warnings/report_e.html?on61). How can I use this returned value as the URL in a new WebParser measure?
-
- Developer
- Posts: 22628
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: WebParser value as URL for new WebParser
Code: Select all
[FakeSite]
Measure=String
String=https://weather.gc.ca/warnings/report_e.html?on61
[MeasureGetReferredURL]
Measure=WebParser
URL=[&FakeSite]
RegExp=(?siU)^(.*)$
DynamicVariables=1
You are absolutely going to want a FinishAction on the very first parent measure, one that sends [!CommandMeasure SecondParentMeasure "Update"] to the second parent measure when it is done. If you don't, the second parent measure will initially fail as there is no value yest for the child measure it is asking for, and it won't try again for 10 minutes (UpdateRate) by default.
-
- Developer
- Posts: 22628
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: WebParser value as URL for new WebParser
In fact I'd probably use something like:
So I don't even set the value for URL on the second parent until the first parent is done. That will eliminate any bogus initial errors in the log as well.
Obviously you can't use FinishAction on a String measure, but this is just a non-working example, since I don't have enough of your code to really test properly.
Note that FinishAction goes on "parent" WebParser measures, not "child" WebParser measures.
So it's going to be:
[FirstParent]
FinishAction setting [*&FirstChild*] as the URL for [SecondParent]
[FirstChild]
URL=[FirstParent]
StringIndex=1
[SecondParent]
Code: Select all
[FakeSite]
Measure=String
String=https://weather.gc.ca/warnings/report_e.html?on61
FinishAction=[!SetOption MeasureGetReferredURL URL "[*&FakeSite*]"][!CommandMeasure MeasureGetReferredURL "Update"]
[MeasureGetReferredURL]
Measure=WebParser
RegExp=(?siU)^(.*)$
DynamicVariables=1
Obviously you can't use FinishAction on a String measure, but this is just a non-working example, since I don't have enough of your code to really test properly.
Note that FinishAction goes on "parent" WebParser measures, not "child" WebParser measures.
So it's going to be:
[FirstParent]
FinishAction setting [*&FirstChild*] as the URL for [SecondParent]
[FirstChild]
URL=[FirstParent]
StringIndex=1
[SecondParent]
-
- Posts: 182
- Joined: April 10th, 2014, 12:31 am
- Location: Canada
Re: WebParser value as URL for new WebParser
Thank you. So this is the main (first parent) measure, which reads and parses the main weather page:
Next, this measure reads and parses the Header group from the first measure (and its results will be used by a subsequent WebParser measure, so is it a parent or child?):
Now this measure obtains from the above measure, the full URL, and strips off the trailing part:
Finally, this is the new parent measure which hopes to use the URL from the preceding measure:
I wasn't sure where to place the FinishAction (it's shown here on the very first parent measure), but no matter what variations I try, Rainmeter About always shows [msrHeaderAlertPage] to be empty.
Code: Select all
[msrSite]
Measure=WebParser
URL=#PageURL#
RegExp=#regexBase#
UpdateRate=#SiteUpdateRate#
FinishAction=[!SetOption msrHeaderAlertPage URL "[*&msrHeaderAlertURL*]"][!CommandMeasure msrHeaderAlertPage "Update"]
Code: Select all
[msrHeaderGroup]
Measure=WebParser
URL=[msrSite]
RegExp=#regexHeader#
StringIndex=#IndexHeader#
Code: Select all
[msrHeaderAlertURL]
Measure=WebParser
URL=[msrHeaderGroup]
StringIndex=#HeaderIndexAlertURL#
RegExpSubstitute=1
Substitute="^(.*)#.*$":"\1"
Code: Select all
[msrHeaderAlertPage]
Measure=WebParser
;URL=[&msrHeaderAlertURL]
regexBase=(?siU).*<p>.*>(.*)<section
;StringIndex=1
DynamicVariables=1
-
- Developer
- Posts: 22628
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: WebParser value as URL for new WebParser
There is no such option as regexBase.
-
- Developer
- Posts: 22628
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: WebParser value as URL for new WebParser
I need the values set for all those #Variables#, so I can actually test and find the best solution.
-
- Posts: 182
- Joined: April 10th, 2014, 12:31 am
- Location: Canada
-
- Posts: 182
- Joined: April 10th, 2014, 12:31 am
- Location: Canada
Re: WebParser value as URL for new WebParser
Okay, here we go:
Code: Select all
[msrSite]
Measure=WebParser
URL=https://weather.gc.ca/city/pages/on-143_metric_e.html
RegExp=(?siU)property="mainContentOfPage"(.*)</ul>(.*)<h2>Forecast.*Forecast(.*)>(.*)Detailed Forecast(.*)Normals:(.*)<h2>Averages(.*)<h2>Yesterday(.*)</details>
UpdateRate=600
FinishAction=[!SetOption msrHeaderAlertPage URL "[*&msrHeaderAlertURL*]"][!CommandMeasure msrHeaderAlertPage "Update"]
[msrHeaderGroup]
Measure=WebParser
URL=[msrSite]
RegExp=(?siU).*property="name">(.*), <abbr title="(.*)">(.*)</abbr>.*<a href="(.*)">(?(?=<).*</div>.*class.*>)(.*)</.*href="(.*)">Past
StringIndex=1
[msrHeaderAlertURL]
Measure=WebParser
URL=[msrHeaderGroup]
StringIndex=4
RegExpSubstitute=1
Substitute="^(.*)#.*$":"\1"
(This should resolve to: "https://weather.gc.ca/warnings/report_e.html?on61")
[msrHeaderAlertPage]
Measure=WebParser
;URL=[&msrHeaderAlertURL]
UpdateRate=600
RegExp=(?siU).*<p>.*>(.*)<section
StringIndex=1
DynamicVariables=1
-
- Developer
- Posts: 22628
- Joined: April 19th, 2009, 11:02 pm
- Location: Fort Hunt, Virginia, USA
Re: WebParser value as URL for new WebParser
This:
Does NOT resolve to https://weather.gc.ca/warnings/report_e.html?on61, it resolves to /warnings/report_e.html?on61, so it is not a fully qualified URL. You can solve that though:
So as you can see, we append the first bit, https://weather.gc.ca before we set the URL on [msrHeaderAlertPage].
Code: Select all
[msrHeaderAlertURL]
Measure=WebParser
URL=[msrHeaderGroup]
StringIndex=4
RegExpSubstitute=1
Substitute="^(.*)#.*$":"\1"
(This should resolve to: "https://weather.gc.ca/warnings/report_e.html?on61")
Code: Select all
FinishAction=[!SetOption msrHeaderAlertPage URL "https://weather.gc.ca[*&msrHeaderAlertURL*]"][!CommandMeasure msrHeaderAlertPage "Update"]
You do not have the required permissions to view the files attached to this post.
-
- Posts: 182
- Joined: April 10th, 2014, 12:31 am
- Location: Canada
Re: WebParser value as URL for new WebParser
Oh, I think you've done it!
But unfortunately, I won't be able to test until tomorrow. Silly me , I think I've been blocked from the weather site! I forgot to put a UpdateRate=600 on that second parent meter in my main skin, so it was likely hitting the site every second.
But I can still access the weather site in a browser, so it seems it is only blocking Rainmeter, not everything from my IP. If I don't access it any more today, hopefully by tomorrow the block will be removed.
But unfortunately, I won't be able to test until tomorrow. Silly me , I think I've been blocked from the weather site! I forgot to put a UpdateRate=600 on that second parent meter in my main skin, so it was likely hitting the site every second.
But I can still access the weather site in a browser, so it seems it is only blocking Rainmeter, not everything from my IP. If I don't access it any more today, hopefully by tomorrow the block will be removed.