It is currently October 13th, 2019, 11:52 pm

Question about getting info off of web pages.

Help with creating, editing & fixing problems with skins
User avatar
Kale
Posts: 24
Joined: March 30th, 2010, 10:57 am

Question about getting info off of web pages.

Kale » April 1st, 2010, 10:51 pm

If it possible to pull text off of a website for rainmeter to display; and if so how would I go about doing that?

I'd like to make a skin to display my Warcraft characters' stats off of the WoWArmory.
[center][img]http://i23.photobucket.com/albums/b358/brandon255/TWEWYsigalt.png[/img][/center]
User avatar
jsmorley
Developer
Posts: 19543
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Question about getting info off of web pages.

jsmorley » April 1st, 2010, 10:55 pm

Kale wrote:If it possible to pull text off of a website for rainmeter to display; and if so how would I go about doing that?

I'd like to make a skin to display my Warcraft characters' stats off of the WoWArmory.
http://rainmeter.net/cms/Tips-WebParserPrimer
User avatar
Kale
Posts: 24
Joined: March 30th, 2010, 10:57 am

Re: Question about getting info off of web pages.

Kale » April 1st, 2010, 11:04 pm

Ah, right, I knew that. I was uh, just testing.

Gonna get started on that right away, I'll post it when I'm done.
[center][img]http://i23.photobucket.com/albums/b358/brandon255/TWEWYsigalt.png[/img][/center]
User avatar
jsmorley
Developer
Posts: 19543
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Question about getting info off of web pages.

jsmorley » April 1st, 2010, 11:06 pm

LOL.. Ok, 10 points for the great response... ;-)
User avatar
Kale
Posts: 24
Joined: March 30th, 2010, 10:57 am

Re: Question about getting info off of web pages.

Kale » April 1st, 2010, 11:54 pm

Pulling data just fine off a random website I used to test, but can't seem to get the data from the armory site. Know why that might be?
[center][img]http://i23.photobucket.com/albums/b358/brandon255/TWEWYsigalt.png[/img][/center]
User avatar
jsmorley
Developer
Posts: 19543
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Question about getting info off of web pages.

jsmorley » April 1st, 2010, 11:59 pm

Kale wrote:Pulling data just fine off a random website I used to test, but can't seem to get the data from the armory site. Know why that might be?
More than likely a bogus RegExp= statement.

If you get this tool, it can help a lot with building those pesky regular expressions for Rainmeter.

http://rainmeter.net/forum/viewtopic.php?f=18&t=769&hilit=rainregexp
User avatar
Kale
Posts: 24
Joined: March 30th, 2010, 10:57 am

Re: Question about getting info off of web pages.

Kale » April 2nd, 2010, 1:06 am

Thanks a bunch for the help so far. That little program is great, making it so much easier, problem I think was the the view source was giving me a different output and was making things go wonky. Now, I've run into an issue I can't seem to find a solution for.

The HTML looks like this:

Code: Select all

		function intellectObject() {
			this.base="98";
			this.effective="98";
			this.mana="1190";
			this.critHitPercent="3.92";
			this.petBonus="-1";
		
			this.diff=this.effective - this.base;
		}
		
		function spiritObject() {
			this.base="108";
			this.effective="108";
			this.healthRegen="20";
			this.manaRegen="17";
		
			this.diff=this.effective - this.base;
		}
		
		function armorObject() {
			this.base="30827";
			this.effective="31052";
			this.reductionPercent="67.09";
			this.petBonus="-1";
Now, I don't care about the first 2 items, I only want the Armor one. I tried grabbing the entire stretch from ' function ' to ' effective=" ' but with the formatting it will just give me an error on the return. I can't figure out how to manipulate it to parse without failing. And if I put in

Code: Select all

...this.effective="(.*)";.*...


It'll return ' 98 ' from the first section. So, I'm stumped. Read through the entire Primer as well as the WebParser section of the introduction and didn't find anything relating to it.
[center][img]http://i23.photobucket.com/albums/b358/brandon255/TWEWYsigalt.png[/img][/center]
User avatar
jsmorley
Developer
Posts: 19543
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Question about getting info off of web pages.

jsmorley » April 2nd, 2010, 1:36 am

Kale wrote:Thanks a bunch for the help so far. That little program is great, making it so much easier, problem I think was the the view source was giving me a different output and was making things go wonky. Now, I've run into an issue I can't seem to find a solution for.

The HTML looks like this:

Code: Select all

		function intellectObject() {
			this.base="98";
			this.effective="98";
			this.mana="1190";
			this.critHitPercent="3.92";
			this.petBonus="-1";
		
			this.diff=this.effective - this.base;
		}
		
		function spiritObject() {
			this.base="108";
			this.effective="108";
			this.healthRegen="20";
			this.manaRegen="17";
		
			this.diff=this.effective - this.base;
		}
		
		function armorObject() {
			this.base="30827";
			this.effective="31052";
			this.reductionPercent="67.09";
			this.petBonus="-1";
Now, I don't care about the first 2 items, I only want the Armor one. I tried grabbing the entire stretch from ' function ' to ' effective=" ' but with the formatting it will just give me an error on the return. I can't figure out how to manipulate it to parse without failing. And if I put in

Code: Select all

...this.effective="(.*)";.*...


It'll return ' 98 ' from the first section. So, I'm stumped. Read through the entire Primer as well as the WebParser section of the introduction and didn't find anything relating to it.
If we assume we want to get those 4 values under "function armorObject"

[MeasureArmorValues]
Measure=Plugin
Plugin=Plugins\WebParser.dll
URL=http://WhateverYourUrlIs.com
RegExp="(?siU)function armorObject().*this.base="(.*)".*this.effective="(.*)".*this.reductionPercent="(.*)".*this.petBonus="(.*)".*"
UpdateRate=600

So this measure is going to parse the entire site and return values in StringIndexes where there is (.*)

First it is looking for "function armorObject()" to be sure we start in the right place, then using .* to say skip everything until the next match, which is "this.base="" and returning everything after that by using (.*) in the first StringIndex, until it hits " at which point it uses .* to say "start skipping again, until it finds the next match which is "this.effective="" and so on.... Everything in (.*) will be returned in a StringIndex array, with StringIndex numbers and the returned values, and associated with this measure.

Now we need to use some measures to put those values in the StringIndexes into separate measures so we can display them:

[MeasureArmorBase]
Measure=Plugin
Plugin=Plugins\WebParser.dll
URL=[MeasureArmorValues]
StringIndex=1

[MeasureArmorEffective]
Measure=Plugin
Plugin=Plugins\WebParser.dll
URL=[MeasureArmorValues]
StringIndex=2

And so on....

Now we can display them:

[MeterArmorBase]
Meter=String
Measurename=MeasureArmorBase
X=0
Y=0
FontFace=Trebuchet MS
FontColor=0,255,0,255
FontSize=12
AntiAlias=1
Prefix=The base armor is: %1

[MeterArmorEffective]
Meter=String
Measurename=MeasureArmorEffective
X=0
Y=25
FontFace=Trebuchet MS
FontColor=0,255,0,255
FontSize=12
AntiAlias=1
Prefix=The effective armor is: %1

And so on...

Hope this helps.
User avatar
Alex2539
Rainmeter Sage
Posts: 648
Joined: July 19th, 2009, 5:59 am
Location: Montreal, QC, Canada

Re: Question about getting info off of web pages.

Alex2539 » April 2nd, 2010, 2:52 am

Kale wrote:Thanks a bunch for the help so far. That little program is great, making it so much easier, problem I think was the the view source was giving me a different output and was making things go wonky.
If ever you think that's the case, what you can do is add "Debug=2" to the measure. In addition to logging the strings in the same way as "Debug=1" it will create a file in the root of your drive that contains the full, exact code it's getting from the site. The file would be in "C:\Webparserdump.txt" (assuming your HDD is the C:\ drive). This comes particularly in handy for sites that use cookies or have specifically blocked Rainmeter.
ImageImageImageImage
User avatar
Kale
Posts: 24
Joined: March 30th, 2010, 10:57 am

Re: Question about getting info off of web pages.

Kale » April 2nd, 2010, 8:33 am

Spent a little more time on this 'cause I can't sleep and ended up finishing it. Then went ahead and made a second version 'cause I wanted to test out making 2 alternate skins for the same thing. Ended up with these:

Image

I guess this would be my first skin, though technically I made it using a pre-existing framework from a tutorial.
[center][img]http://i23.photobucket.com/albums/b358/brandon255/TWEWYsigalt.png[/img][/center]