It is currently May 19th, 2021, 12:16 am

How to substitute part of url for image webparser download

General topics related to Rainmeter.
User avatar
SlowCircuit
Posts: 5
Joined: April 9th, 2021, 9:08 am

How to substitute part of url for image webparser download

Post by SlowCircuit »

I have a skin that pulls an RSS file and organizes it into titles, links and images. Works great except that one of the websites for my RSS feed has a strange link formatting. The RSS feed's image link is a tiny thumbnail. However, if I replace "/tmb/" with "/800a/" for those links, then I get a much higher res image. The problem is: I can't figure out how to do that replacement.

I know that Substitute exists, but it doesn't seem to work when it's on the parent webparser or the image download webparser. I even tried having an intermediate parser. I also tried getting the url for the image from an lua function that did the substitution, but I don't think that plays well with StringIndex.

Here is the relevant code:

Code: Select all

[Variables]
 Substitutes="/tmb/":"/800a/","<i>":"","</i>":"","<em>":"","</em>":""
 RSSLink=http://www.rssmix.com/u/12901347/rss.xml
 RegexPattern=.*<item>.*<title>(.*)<.*<link>(.*)<.*media:.*(?:url="|>)(.*)(?:\?|\")

[Feed]
 Measure=WebParser
 Url=#RSSLink#
 RegExp="(?siU)#RegexPattern#
 Substitute=#Substitutes#
 
[Item1Image]
 Measure=WebParser
 Url=[Feed]
 StringIndex=3
 Download=1
 DownloadFile=1.png
 DynamicVariables=1
 DecodeCharacterReference=1
 Substitute=#Substitutes#
Last edited by SlowCircuit on April 9th, 2021, 6:24 pm, edited 2 times in total.
User avatar
balala
Rainmeter Sage
Posts: 12670
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: How to substitute part of url for image webparser download

Post by balala »

SlowCircuit wrote: April 9th, 2021, 9:16 am Here is the relevant code:
It's extremely hard to help you, because you didn't post the needed variables, like RSSLink or RegexPattern. Please post them.
User avatar
SlowCircuit
Posts: 5
Joined: April 9th, 2021, 9:08 am

Re: How to substitute part of url for image webparser download

Post by SlowCircuit »

Okay, I edited the main post to add them.
User avatar
balala
Rainmeter Sage
Posts: 12670
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: How to substitute part of url for image webparser download

Post by balala »

SlowCircuit wrote: April 9th, 2021, 6:25 pm Okay, I edited the main post to add them.
Ok, the substitution does work, in the returned string you get the proper URL for the image. Now my question is that I suppose you'd like to get the URL of the image, then would like to download the image. Am I right?
User avatar
SlowCircuit
Posts: 5
Joined: April 9th, 2021, 9:08 am

Re: How to substitute part of url for image webparser download

Post by SlowCircuit »

To be clear, I already have it fully working EXCEPT that the URL used by the image webparser does not substitute any part of the URL as I want it to. It downloads an image, but it's downloading the image with "tmb" in it's url, which I want to be replaced with "800a".

The parent webparser parses the RSS using the regex. If I use substitute on it, no substitutes are made. From what I've read, parent webparser don't "use" the string themselves so they don't pay attention to the substitute.

The child webparser gets a specific string index of the parent webparser, which is the link to the image. If I use substitute on it, no substitutes are made. I assume this is because it is an image download and not a string. I thought maybe using an intermediate child parser would fix it, but it did not.

So, I am trying to figure out how I can make the child webparser substitute the url.
User avatar
balala
Rainmeter Sage
Posts: 12670
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: How to substitute part of url for image webparser download

Post by balala »

SlowCircuit wrote: April 9th, 2021, 7:44 pm To be clear, I already have it fully working EXCEPT that the URL used by the image webparser does not substitute any part of the URL as I want it to. It downloads an image, but it's downloading the image with "tmb" in it's url, which I want to be replaced with "800a".

The parent webparser parses the RSS using the regex. If I use substitute on it, no substitutes are made. From what I've read, parent webparser don't "use" the string themselves so they don't pay attention to the substitute.

The child webparser gets a specific string index of the parent webparser, which is the link to the image. If I use substitute on it, no substitutes are made. I assume this is because it is an image download and not a string. I thought maybe using an intermediate child parser would fix it, but it did not.

So, I am trying to figure out how I can make the child webparser substitute the url.
Substitute can be used on both, the parent and the child WebParser measure and does work on both. I just got downloaded the high resolution image. It's not the thumbnail.
I suppose the posted code is not the whole code you have. Please post the whole.
User avatar
SlowCircuit
Posts: 5
Joined: April 9th, 2021, 9:08 am

Re: How to substitute part of url for image webparser download

Post by SlowCircuit »

That's strange. My skin has a lot going on, and I don't want anyone to have to delve through the entire thing, but... I created a new skin that consists ONLY of the following:

Code: Select all

[Variables]
 Substitutes="/tmb/":"/800a/","<i>":"","</i>":"","<em>":"","</em>":""
 RSSLink=http://www.rssmix.com/u/12905273/rss.xml
 RegexPattern=.*<item>.*<title>(.*)<.*<link>(.*)<.*media:.*(?:url="|>)(.*)(?:\?|\")

[Feed]
 Measure=WebParser
 Url=#RSSLink#
 RegExp="(?siU)#RegexPattern#
 Substitute=#Substitutes#
 
[Item1Image]
 Measure=WebParser
 Url=[Feed]
 StringIndex=3
 Download=1
 DownloadFile=1.png
 DynamicVariables=1
 DecodeCharacterReference=1
 Substitute=#Substitutes#

[Item1Thumbnail]
 Meter=Image
 DynamicVariables=1
 PreserveAspectRatio=1
And it STILL only downloads the tmb version. If I remove the Download lines, and I check the log I do see that Item1Image's URL is set to the 800a one, but with the download lines, it just downloads the tmb version. If I turn on debug mode and look at the log when it downloads, it says it's downloading the tmb version.

What could possibly be different from what you're doing to cause this? I was on Rainmeter 4.4 beta, so I downloaded the final release instead but it didn't make any difference.
User avatar
balala
Rainmeter Sage
Posts: 12670
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: How to substitute part of url for image webparser download

Post by balala »

SlowCircuit wrote: April 10th, 2021, 9:40 pm And it STILL only downloads the tmb version. If I remove the Download lines, and I check the log I do see that Item1Image's URL is set to the 800a one, but with the download lines, it just downloads the tmb version. If I turn on debug mode and look at the log when it downloads, it says it's downloading the tmb version.
Not sure what is causing this, but there is a small mistake in your code. The [Item1Image] measure should return the URL of the high resolution image (due to its StringIndex=3 option and the substitution), but should also download the image (due to the Download=1 option). A WebParser measure which downloads an image has to return the path of the downloaded image. And this is a contradiction: the measure can't return both, a URL and a path of an image. That's why you have to create two distinct WebParser measures: first to get the URL of image and second to download the appropriate image.
Weird is that simply rewriting the code doesn't help: I got nothing downloaded. So I had to add a trick. See the code below:

Code: Select all

[Rainmeter]
Update=1000
DynamicWindowSize=1

[Variables]
Substitutes="/tmb/":"/800a/","<i>":"","</i>":"","<em>":"","</em>":""
RSSLink=http://www.rssmix.com/u/12905273/rss.xml
RegexPattern=.*<item>.*<title>(.*)<.*<link>(.*)<.*media:.*(?:url="|>)(.*)(?:\?|\")
URL=""

[Feed]
Measure=WebParser
Url=#RSSLink#
RegExp="(?siU)#RegexPattern#
;Substitute=#Substitutes#
FinishAction=[!SetVariable URL "[Item1Image]"][!EnableMeasure "MeasureDownloadImage"][!CommandMeasure "MeasureDownloadImage" "Update"]
 
[Item1Image]
Measure=WebParser
Url=[Feed]
StringIndex=3
;Download=1
;DownloadFile=1.png
DynamicVariables=1
DecodeCharacterReference=1
Substitute=#Substitutes#

[MeasureDownloadImage]
Measure=WebParser
Url=#URL#
Download=1
DownloadFile=1.png
UpdateRate=600
DynamicVariables=1
Disabled=1

[Item1Thumbnail]
Meter=Image
MeasureName=MeasureDownloadImage
First see that I added a URL variable to the [Variables] section. This variable is empty at the begining, but will be set to the URL of the image, when the WebParser measure gets the needed URL (in the FinishAction option of the [Feed] measure).
I removed the Download and the DownloadFile options from the [Item1Image] measure, because as said, the image will be downloaded by a new WebParser measure (namely [MeasureDownloadImage], see below). Accordingly the [Item1Image] measure gets only the URL of the image. When it does this, the first bang ([!SetVariable URL "[Item1Image]"]) of the FinishAction option of the [Feed] measure, sets the URL variable to the appropriate URL of the high resolution image.
The image will be downloaded only after this, by the newlly added [MeasureDownloadImage] measure:

Code: Select all

[MeasureDownloadImage]
Measure=WebParser
Url=#URL#
Download=1
DownloadFile=1.png
UpdateRate=600
DynamicVariables=1
Disabled=1
See thet this measure is disabled (by its Disabled=1 option). When the URL variable gets the URL (as described above), which is happening when the parent WebParser measure finishes its work, the next two bangs of the FinishAction option of the [Feed] measure, enables the downloading measure ([!EnableMeasure "MeasureDownloadImage"]) and are updating it ([!CommandMeasure "MeasureDownloadImage" "Update"]), to let it to immediately download the image.
And this is working well now, the skin downloads this way the full resolution image. Please give it a try and let me know if I am right.
Weird is that first I tried to make all this without the URL variable, by simply adding the Url=[Item1Image] option to the [MeasureDownloadImage] measure, but couldn't get it to work. Don't know why, were no error messages in the log.
SlowCircuit wrote: April 10th, 2021, 9:40 pm I was on Rainmeter 4.4 beta, so I downloaded the final release instead but it didn't make any difference.
Rarely does.
User avatar
SlowCircuit
Posts: 5
Joined: April 9th, 2021, 9:08 am

Re: How to substitute part of url for image webparser download

Post by SlowCircuit »

That definitely worked, thanks! It's a shame I have to use a workaround like this though, because there are 30 images I need to do this with, so I need 30 more measures for getting the URL, 30 variables to store the URLs, etc.
User avatar
balala
Rainmeter Sage
Posts: 12670
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: How to substitute part of url for image webparser download

Post by balala »

SlowCircuit wrote: April 18th, 2021, 9:47 pm It's a shame I have to use a workaround like this though, because there are 30 images I need to do this with, so I need 30 more measures for getting the URL, 30 variables to store the URLs, etc.
Rainmeter works this way. You'll have to create a distinct WebParser measure for each image. That's it. But don't forget that Rainmeter is not a programming language.