How to reset a WebParser measure return value?

[Yincognito]

Hello all,

I'm having a "simple" problem with a not so "simple" answer. I'm integrating a classic external IP WebParser measure/meter

Code: Select all

---Measures WebParser---

[MS_WebParser_WebIP]
Measure=Plugin
Plugin=WebParser
Url=http://checkip.dyndns.org
RegExp="(?siU)Address: (.*)</body>"
StringIndex=1
UpdateDivider=60
UpdateRate=1
Substitute="":"No external IP"
;IfAboveValue=""
;IfAboveAction=[!Refresh]

[MT_WebParser_WebIP_Value]
Meter=STRING
MeterStyle=SSectionRightHalfValue
MeasureName=MS_WebParser_WebIP
Text="%1"

into my array of skins (which will soon arrive on DeviantArt )

and all works just fine except... the value returned by the WebParser measure (the external IP) stays the same even if I disconnect from the internet - UNLESS I manually refresh the skin (as you can see from the code and from below images, the measure updates once every minute, for testing purposes).

First, here I am - connected, everything is fine, WebParser gets its data and shows my IP :

Then, I disconnect from the internet. WebParser obviously can't connect to checkip.dyndns.org, so it throws an error and SHOULD have the value "", as it couldn't retrieve my IP. Instead, the measure which uses WebParser to get the data "remembers" the last value of my IP, and the displayed value remains the same, although I'm not connected to the internet anymore.

Now, I manually refresh the skin. WebParser throws the SAME error as before, but it gets the proper value "" (which is "substituted" with "No external IP" in the code) and as a result, my skin displays the meter and the data properly : no IP, as I am disconnected from the internet.

Finally, I reconnect to the internet, without refreshing or doing anything special. WebParser gets my IP again and passes the data to the measure and the meter BY ITSELF, as it should have done when I disconnected before. So when WebParser gets data different than "", everything works fine, but when it gets nothing it does not update the measure/meter (unless skin-refreshed).

I've tried all possible and impossible solutions to this (bangs, if conditions, if actions), none of them do the trick. It's fairly easy to see why : the "guilty" measure seems to remember the last IP value since I was online and despite the fact that WebParser gives an error of not being able to "fetch" things up, the return value of the measure stays the same (although normally WebParser returning "" as a result should trigger updating the measure to "No external IP"). I suspect the issue happens because WebParser is not executing (?) as it gives the error and therefore it isn't able to trigger the adequate response from the measure. But then... why does it work when manually refreshing the skin? I could refresh the skin in my measure, but as it is now (that is, if I uncomment those 2 lines in the code), it will just throw the measure into an infinite loop...

If anyone has some ideas on how to solve this "simple" problem, I'm all ears (and eyes)

And thank you for making Rainmeter and helping one another on this forum.

[jsmorley]

I'll say up front that this issue is a massive failure of how WebParser is coded. It is a long-standing problem, and not one that has a simple solution in today's environment.

The root of the problem is that indeed WebParser does "remember" the last string value it got from the external resource. However, that is actually as it should be, as you don't really want skins blowing up just because you have a little hiccup in your network connection, or some site is just "slow" right now. In addition, you don't want your skin to visibly "refresh" when WebParser measures are updated, you just want the old values to seamlessly turn into the new values.

The problem is exacerbated however, due to the fact that the only error trapping that WebParser has (ERRORSTRING) is based on a failure of PCRE (the regular expression engine) to "parse" the value returned. If you connect once, and get some data from the site, when you later can't connect, and WebParser "remembers" the old data, PCRE is perfectly happy with that and just keeps on parsing the old data. There is absolutely no indication possible that anything is wrong that WebParser can react to today. This is in spite of the frustrating fact that the connection error is right there in the log.

The only thing that can "reset" a WebParser measure to the starting point of having a value of "" (no string) today is a refresh of the skin. And truth be told, even that doesn't help entirely, as "" really does't mean anything. It doesn't in and of itself indicate failure, only "hang on, still working on it...".

So you will need some other method of detecting a network failure, whether that be that your computer is not able to connect to the external resource, or that the external resource isn't answering. These are two very different conditions...

Obviously you don't want to just constantly be refreshing the skin in question, who wants a weather skin that has periodic visible seizures when the WebParser measure updates? Again, the point of all this is that the old values are just seamlessly replaced with new values. At the same time, you don't want some momentary failure to connect (not at all uncommon) to set your skin to some "error" state for an extended period of time before it "tries again".

You also want to be VERY careful that you don't build some approach that ends up constantly refreshing the skin if your network or the resource is down, as that would be a huge problem. You must be careful about creating an "endless loop". This makes it problematic to have the solution be a part of the skin itself.

Now that I have laid out the challenge, give me an hour or two to chew on this and think about alternatives. I don't want to end up with some hideous Rube Goldberg solution, but this is not a particularly easy nut to crack.

[Yincognito]

I'll say up front that this issue is a massive failure of how WebParser is coded. It is a long-standing problem, and not one that has a simple solution in today's environment.

Wow, thanks for the quick response. I was expecting a "sorry, you did something wrong, here's the problem in your approach" answer, but it seems that this is indeed a problem.

I completely understand the reason behind the need of NOT interpreting "waiting for an answer from the host" as "unreachable", and I agree this "failure" does have some... benefits. But it's still something that just isn't right.

That being said...

So you will need some other method of detecting a network failure, whether that be that your computer is not able to connect to the external resource, or that the external resource isn't answering. These are two very different conditions...
..................................
Now that I have laid out the challenge, give me an hour or two to chew on this and think about alternative solutions. I don't want to end up with some hideous Rube Goldberg solution, but this is not a particularly easy nut to crack.

I know it's tough to crack, as I spent a whole day trying to "workaround" it. I DO HAVE an alternative solution though, but that only fits my particular skin setup, not the problem in general.

I could base my checking for an external IP on the Ping plugin query (you may notice that I have a latency check below the IPs line in my network skin) by "invalidating" the WebParser based IP if the latency is maximum (that is, if the timeout is reached). Of course, I should synchronize those 2 checkings so that it wouldn't become disturbing for the internet connection, and it would also involve some "waiting" for the ping thingy to reach the timeout, so that it could give the thumbs up/thumbs down verdict on the WebParser check.

I thought about this "workaround" even when debugging the WebParser flaw, but being so stubborn, I wanted to do it the "right" way, using only the WebParser check. Something that would have helped was if I could check for the error thrown by WebParser - that way I would have known that something is wrong (e.g. If the IP is the same as before && WebParser throws error no. XXXX, bang some !SetOption stuff on the meter...)

If you come up with a better solution to the problem, then I would be glad to hear/see it - thank you again for being present and confirming my suspicions on the WebParser behaviour.

.................

On another note, I noticed a strange and sometimes disturbing "feature" of the IfCondition thing, and I wanted to ask you if this is intended (or "by design") behaviour : it seems to me that the IfCondition would only accept checking against numerical values, not strings. I know that there are IfMatch and IfEqual/IfBelow/IfAbove that can check against strings, but the IfCondition is the "complete" package (able to incorporate logical AND/OR/etc.) and it also has one GREAT advantage : IfConditionMode=1, which allows my skins to "react" instantly to changes in the measured values.

For example, in this piece of code

Code: Select all

[MS_PerfMon_BatteryDischargeRate]
Measure=Plugin
Plugin=PerfMon
PerfMonObject=BatteryStatus
PerfMonInstance=ACPI\PNP0C0A\1_0
PerfMonCounter=DischargeRate
PerfMonDifference=0

[MS_PerfMon_BatteryChargeRate]
Measure=Plugin
Plugin=PerfMon
PerfMonObject=BatteryStatus
PerfMonInstance=ACPI\PNP0C0A\1_0
PerfMonCounter=ChargeRate
PerfMonDifference=0

[MS_Power_BatteryPercent]
Measure=Plugin
Plugin=PowerPlugin
PowerState=Percent
IfCondition=(MS_PerfMon_BatteryDischargeRate>0)
IfTrueAction=[!SetOption MT_Power_Battery_Title Text "Battery charge rate: -[MS_PerfMon_BatteryDischargeRate] mWh"]
IfConditionMode=1
IfCondition2=(MS_PerfMon_BatteryChargeRate>0)
IfTrueAction2=[!SetOption MT_Power_Battery_Title Text "Battery charge rate: [MS_PerfMon_BatteryChargeRate] mWh"]
IfConditionMode=1

it would have been almost imposible (or quite difficult, using "workarounds") to reach the desired result (updating the MT_Power_Battery_Title meter on the fly according to the charge/discharge rate of the battery) if it weren't for the IfConditionMode=1 setting. Now imagine the MS_PerfMon_BatteryDischargeRate / MS_PerfMon_BatteryChargeRate measures that I'm checking were strings... I already tried, it looks like IfCondition doesn't like them, whether or not I put the string measures in [] or not...

Now if this is the intended behaviour, this would clear the things up for me - otherwise I'm thinking I'm missing something, or doing something wrong

Thanks again for your reply and for Rainmeter.

P.S. Oh, and to "spit it all out at once", it seems skins are sometimes "freezing" in a dimmed transition phase between the Hide on mouse over states (hidden and showing). You may want to check that out, here it is the screenshot too (I know, a lot of CPU intensive skins, but I'm in the skin designing process, so all my desktop is a workshop for the time being, ha ha).

Notice the first skin (the upper one, with Toshiba Satellite....) is frozen in a dimmed state in the second screenshot. I have enabled Hide on mouse over on all the skins from the right side of the screen (those are my skins).

[jsmorley]

On this:

Code: Select all

[MS_PerfMon_BatteryDischargeRate]
Measure=Plugin
Plugin=PerfMon
PerfMonObject=BatteryStatus
PerfMonInstance=ACPI\PNP0C0A\1_0
PerfMonCounter=DischargeRate
PerfMonDifference=0

[MS_PerfMon_BatteryChargeRate]
Measure=Plugin
Plugin=PerfMon
PerfMonObject=BatteryStatus
PerfMonInstance=ACPI\PNP0C0A\1_0
PerfMonCounter=ChargeRate
PerfMonDifference=0

[MS_Power_BatteryPercent]
Measure=Plugin
Plugin=PowerPlugin
PowerState=Percent
IfCondition=(MS_PerfMon_BatteryDischargeRate>0)
IfTrueAction=[!SetOption MT_Power_Battery_Title Text "Battery charge rate: -[MS_PerfMon_BatteryDischargeRate] mWh"]
IfConditionMode=1
IfCondition2=(MS_PerfMon_BatteryChargeRate>0)
IfTrueAction2=[!SetOption MT_Power_Battery_Title Text "Battery charge rate: [MS_PerfMon_BatteryChargeRate] mWh"]
IfConditionMode=1

You need to look in About / Skins to see what those measures are actually returning. I'm not on a laptop and don't have any battery stuff in Perfmon, so I can't be sure. If they are returning a number, then what you have should be ok. If they are a string, then you will probably either need to use IfMatch instead of IfCondtion, or use a Substitute to turn the string into a number, then use IfCondition.

You do only need one IfConditionMode per section by the way. If one IfCondition has that set, they all do in that section.

[jsmorley]

As to this:

P.S. Oh, and to "spit it all out at once", it seems skins are sometimes "freezing" in a dimmed transition phase between the Hide on mouse over states (hidden and showing). You may want to check that out, here it is the screenshot too (I know, a lot of CPU intensive skins, but I'm in the skin designing process, so all my desktop is a workshop for the time being, ha ha).

Something that might cause that is some Lua script (or possibly some 3rd-party plugin) that is getting "hung up" or otherwise taking a long time to complete, while "blocking" Rainmeter until it is done. Start unloading skins and restarting Rainmeter til you find that villain.

Another possibility is that you have just a TON of old skins still listed in Rainmeter.ini that you are not loading currently, but have at some point in the past, even just to "try" them. While it would take a massive number of "dead" skins in Rainmeter.ini to actually slow the parsing and loading of "active" skins to the extent you are getting, it might be worth looking at. A clean Rainmeter.ini is a good thing in any case.

[jsmorley]

As to the original issue with WebParser, I've come to the conclusion that there just isn't a good one-size-fits-all answer.

Sometimes having the skin keep the old values when you disconnect from the internet is just what you want, other times, you might want to somehow react to it, but not necessarily by having the skin go wonky. You certainly don't want to start hammering it with !Refresh, and may not want to refresh at all.

The biggest logical problem I chewed on for a long time is the issue of how aggressive to be in "trying again" when the computer can't connect for whatever the reason might be. The answer is "it depends" and to be honest, I'm not sure a skin is ever going to be in a position to make the right decision at any given time.

Certainly you can use the network connectivity stuff from SysInfo to see if you are connected to the internet, and can react to that. That might be the best approach. You can disable measures / hide meters / change text / change color or any thing else you like based on that.

I just don't think a pure WebParser-only solution exists, and even if it did, I'm not sure there is any generic way to react to it that makes sense in all cases.

[Yincognito]

If they are a string, then you will probably either need to use IfMatch instead of IfCondtion, or use a Substitute to turn the string into a number, then use IfCondition.

Well, this is what I call a response from the developer himself. I didn't know you could turn a string into a number with Substitute (I always thought it would be string for string).

Either way, I looked more closely at the IfMatch thing and I succeded to do what I wanted (dynamically adjusting the GPU monitoring skin to the numbers of GPU cards, by hiding the non-existent GPU group). I was doing that by checking if the core clock of the second GPU was 0 using IfCondition, but that didn't satisfied me, so I checked after your reply with an IfMatch=^$ (empty string) on the GPU name as provided by the MSIAfterburner plugin. It worked - thank you for the guidance! Oh, and one thing I also missed : IfMatch has a similar IfMatchMode setting, so I can achieve on the fly refreshing of the measure with IfMatch too, not only IfCondition (I somehow missed the existence of an IfMatchMode before). The trick with IfMatch is knowing how to handle RegExp to properly test for some more complex patterns in strings...

You do only need one IfConditionMode per section by the way. If one IfCondition has that set, they all do in that section.

I wasn't sure of that, that's why I "cloned" the setting for the second testing. Thank you for letting me know that one setting is enough for the section.

By the way, the code regarding the battery works fine, I just provided that as an example of "what if I would be forced to check strings with IfCondition/IfConditionMode". But now, thanks to your suggestion, I know I can do that just as well with IfMatch/IfMatchMode.

[Yincognito]

Start unloading skins and restarting Rainmeter til you find that villain.

Well, I know for sure that it isn't my skins that use Lua code, so...I'm not "that villain", LOL The other skins are just for testing/inspiration, so they will be removed when my array of skins will be completed, no need to check for bad boys...

Another possibility is that you have just a TON of old skins still listed in Rainmeter.ini that you are not loading currently, but have at some point in the past, even just to "try" them. While it would take a massive number of "dead" skins in Rainmeter.ini to actually slow the parsing and loading of "active" skins to the extent you are getting, it might be worth looking at. A clean Rainmeter.ini is a good thing in any case.

That might well be the case (90% sure). I wondered how those skins should be properly "unistalled", I didn't find such an option in Rainmeter, so I simply deleted them, but I never thought their remains from Rainmeter.ini would cause problems (although I noticed "undesirable"/obsolete settings in Rainmeter.ini and wanted to clean them, but got caught up with my own skin jobs and forgot about it). Anyway, there are not a TON of them - I'm very picky about what I test/like...

Will do the Rainmeter.ini cleaning either way - thanks for letting me know the possible root of the problems.

[Yincognito]

As to the original issue with WebParser, I've come to the conclusion that there just isn't a good one-size-fits-all answer.
........
I just don't think a pure WebParser-only solution exists, and even if it did, I'm not sure there is any generic way to react to it that makes sense in all cases.

Yeah, that's about the same conclusion that I've come to, unfortunately. In the meantime, I applied my proposed solution and it works...kind of. There are several problems to this solution too :

1) sometimes checking for reaching the timeout (30000 or what the Timeout setting is) isn't enough. Strangely, sometimes even the Ping plugin returns 0 when there's no connectivity, so I need to check for BOTH timeout value AND 0 and invalidate the IP accordingly :

Code: Select all

[MS_Ping_Latency]
Measure=Plugin
Plugin=PingPlugin
DestAddress=www.google.com
UpdateDivider=60
UpdateRate=1
Timeout=10000
MinValue=0
MaxValue=10000
IfCondition=(MS_Ping_Latency=0) || (MS_Ping_Latency=10000)
IfTrueAction=[!SetOption MT_WebParser_WebIP_Value Text "Unreachable"]
IfFalseAction=[!SetOption MT_WebParser_WebIP_Value Text ""]
IfConditionMode=1

0 ms ping is safe to check and use for invalidation, as no computer in the world would have 0 ms latency when online, but it still is strange - seems like WebParser is not the only one with odd behaviour. At least this one doesn't affect the data provided (one can easily Substitute 0 with the timeout value, so that the user shouldn't be confused why having a 0 ms latency - the ideal connectivity - means that he/she isn't able to access the internet).

2) validation thing takes a lot longer than expected, even if I set the Timeout to a lower value. It takes 1-2 minutes for the validation/invalidation check to have an effect on the measures/meters. This isn't THAT bad, but for an impatient user who uses Rainmeter especially for quick access to monitoring things, that may be undesirable. And of course, it's the funny (or not that funny) thing that the validation effect seems to take the same amount of time (1-2 minutes), no matter how low I set the Timeout value. I didn't touched the UpdateRate and the UpdateDivider to set them to less than 1 minute, as it's bad for the sites (traffic), the user (ban) and the network connectivity (unnecessary load) at the same time.

................

But at least the soultion WORKS in all cases. I tested it connecting, disconnecting, connecting again and so on, and except for the "lag" in refreshing the meter values (which I somewhat expected, just not as much compared to the "user's fast forward expectations"), it gets its job done. But you're right, although I did find a workaround for the initial problem, I'm still undecided whether to apply it or not.

Hm... I just had a stroke of genius, LOL. What if I put an invisible refresh "button" on the "IP" label? This way the user could refresh the IP by himself/herself by clicking on the label (or even by hover over it).

Well, hell, I just did it and it works flawlessly, and this would be my final solution : the external IP will be updated by a skin refresh (as other measure/meter "updates" or "refreshes"/"redraws" don't really update the values like a skin refresh is tested of being able to do, especially on this WebParser monster, haha). The refresh will be triggered by the user hovering the mouse over the exact location of the external IP, so the "accidental" refreshing is minimized - the user must be quite specific on the hovering target to initiate the external IP refresh.

It seems it's harder faster Scooter after all, or how they say these days...a fast and furious solution - a bang on a specific location and that's all

Thank you again for trying to break this with me to a workable solution, but sometimes chosing the simplest route is better than trying to fix things up. It's not my style as I always try to "dominate the machine", but this time it was the best thing to do.

[jsmorley]

Yeah, I have found that you just absolutely can't "reset" a WebParser measure with anything less than a refresh of the skin. So you just can't win that fight. Basically once you have a value for a WebParser measure, you are just flat out stuck with it unless it changes to some other value on the site. If you lose connectivity, or the site goes down, or the site's web server crashes, WebParser will cheerfully keep on using that original value forever.

There are good reasons for that, but there really needs to be something, analogous to ERRORSTRING, that is triggered not by PCRE, but a failure to connect to the resource. That will optionally set the string value of the measure to whatever you define, and you can then take some action like setting your IP meter to "Not Connected" or whatever you like. What actions you might take when WebParser fails to connect might be in a wide range, starting with doing nothing... In a lot of cases, maybe most, how it works today is fine.

For now you can test for connectivity on your computer with SysInfo, and based on that, take some action, but you can't just have that refresh the skin, or you are in a hideous endless loop, at least until you again have connectivity.

So the best thing might be have the SysInfo test set some icon to red or some other indication when there is no connectivity, and green when there is. Then have a click on that icon by the user refresh the skin. That way it can be "on demand", and won't run amok.