It is currently October 24th, 2020, 6:07 am

[Idea] XML Parser measure/plugin

General topics related to Rainmeter.
User avatar
eXAKR
Posts: 31
Joined: January 11th, 2012, 12:22 pm

[Idea] XML Parser measure/plugin

Post by eXAKR »

This came up to me the other day when someone I follow on Twitter pointed out how difficult it is to craft RegExes (Regular Expressions) in programming.

While RegEx is indeed very useful and quite powerful if you know your way around it, the problem is that its syntax, to an untrained observer, is basically a slurry of random characters strung together. Even experienced RegEx users often have to fall back on using RegEx testers to make sure they didn't miss anything when crafting them. RegEx does have its use, but for programming purposes I believe we can do better.

Now, as we all know, RegEx is a big part of Rainmeter, and one of the biggest places where RegEx is used here is in the WebParser measure (formerly plugin). WebParser is often used to make pretty powerful information widgets for Rainmeter (such as weather info, RSS feeds, and the like), but crafting them to retrieve the information you need is often extremely difficult simply because of the need to use a RegEx to do so, and the syntax often have to be crafted specifically for a particular informational feed. This is all made worse by how opaque and difficult-to-understand the RegEx syntax is, as I pointed out earlier.

For the most part, WebParser is often used to retrieve information from XML feeds, and the thing is, XMLs follow a pretty-standardised format/layout that is designed to be both easily human- and machine-readable. XMLs have defined elements, attributes, tags etc.that can be easily programmatically retrieved as "objects", instead of being a text string that needs to be scraped from using a RegEx. Therefore, I'm sure a hypothetical Rainmeter plugin or measure (which I'm calling "XML Parser") can be written that can easily retrieve information from XML feeds simply by specifying which element/attribute to look for, without the need for using any RegEx.

I'm not sure if such a plugin already exist, but if it does, I believe it should be made a standard feature of Rainmeter. It would make crafting informational widgets that rely on reading XML feeds/files a whole lot easier, and make them easier to debug and test as well.
User avatar
Yincognito
Posts: 2769
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Idea] XML Parser measure/plugin

Post by Yincognito »

I agree with you that a stable XML-JSON parser (I included both, as the tasks are related) would be very useful and worthy to be included in the standard Rainmeter distribution, but I disagree that by doing that you'll be able to avoid the "scary" regex - in fact, quite the contrary, as regex would still be badly required for, say, variable / dynamic / partially known field names or values.

That being said, I don't know about XML (this format is actually quite easy to parse, even with regex, since one can effortlessly identify <sometag> and </sometag> start and end tags) but there are a couple of such plugin attempts for JSON (which is in theory harder to parse from a regex point of view, due to {, [, " delimiters not holding field identifiers, and requiring counting, which is one of the few regex weaknesses). I don't know how stable, flexible (something regex is great at), bug free and / or mature those plugins are though, and my guess is that would be a requirement for including them in a standard Rainmeter distribution.

Other than that, if one uses some tricks to "normalize" and format an XML / JSON before attempting to parse it, the job becomes easier, even with just regex.
User avatar
Mordasius
Posts: 1078
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: [Idea] XML Parser measure/plugin

Post by Mordasius »

I'm not sure this is quite what you are thinking about but maybe you might like to take a look at: [Proof of Concept] Universal Feed Reader
User avatar
jsmorley
Developer
Posts: 21424
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: [Idea] XML Parser measure/plugin

Post by jsmorley »

I'm not sure I see the important benefit to a plugin that attempts to automate the parsing of either XML or JSON formatted data. When decently formatted to begin with, both of those are not at all difficult to parse with regular expression.

Given that the way Rainmeter works at it's core, the fact that a single measure can only return a single value, means that all of the thoughts you might have of creating "arrays" of data based on the data, the kind of thing you might well do in a procedural programming language, are likely to not be terribly useful.

I'm not opposed by any means, but I think a great deal of thought would need to be given to what the real result is you are looking for, given that whatever you do would HAVE to be useful in the context of Rainmeter.

There are really only two ways today that you can attack data that can return an "array" of values in Rainmeter. The first is to use a "parent-child" approach with a plugin, where you would have a single parent measure and then multiple child measures that can return a single value based on an "index" number that reaches into the "array". The second is to use an "inline section variable" approach, where you have a single parent measure and reach into the "array" with a call to the parent that is used inline in a measure or meter. In that case you pass the "index" number (or perhaps a defining "search value") as a parameter to the section variable.

Remember that whatever you do, you can't have the data "drive the skin". You have to create the skin to have some hard-coded, defined, number of measures to return data, and meters to display it. So you have to know, or at least accept, that there is some finite number of data elements that you can deal with in the data. You can disable or hide measures and/or meters if there is "less" data than the maximum you define, but you can't "expand" the scope of the skin if there is "more" data.

In any case, I don't think that a knee-jerk desire to avoid using regular expression is enough reason. Parsing either XML or JSON is really falling off the log easy with regular expression. I'd be very interested to see a walk-through of how one would create a plugin that actually provides a much simpler interface to the data, and brings real value.
User avatar
balala
Rainmeter Sage
Posts: 11595
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: [Idea] XML Parser measure/plugin

Post by balala »

jsmorley wrote: October 12th, 2020, 2:12 pm I'm not sure I see the important benefit to a plugin that attempts to automate the parsing of either XML or JSON formatted data. When decently formatted to begin with, both of those are not at all difficult to parse with regular expression.
Even if me personally didn't try it out so far and I agree with the above, there already is a JSON parser plugin: https://forum.rainmeter.net/viewtopic.php?f=128&t=34378#p169911
Since as said I didn't try the plugin, I'm not sure if it can be used to parse XMLs, but I suppose it can't be.
User avatar
Jeff
Posts: 160
Joined: September 3rd, 2018, 11:18 am

Re: [Idea] XML Parser measure/plugin

Post by Jeff »

MikeG621
Posts: 82
Joined: March 18th, 2013, 1:59 pm

Re: [Idea] XML Parser measure/plugin

Post by MikeG621 »

jsmorley wrote: October 12th, 2020, 2:12 pmYou can disable or hide measures and/or meters if there is "less" data than the maximum you define, but you can't "expand" the scope of the skin if there is "more" data.
You can with Lua, but if you don't put limits on what you're inputting it can run away pretty quick and I've never tried to automate cleaning it up afterwards. I derived my notes skin from Enigma and at first run there's not much more than the Lua measure and the background, the script generates everything else based on the variable that defines the quantity (array size). If I increase the quantity later, it creates more measures/meters. Since I haven't automated cleanup (delete key works fine), reducing the quantity just hides what isn't being used.

That said, I don't even use that form of the skin anymore, and it's quite a bit of setup just to avoid manual copy/paste to create a repetitive skin setup. It works fine for a simple "pages" sort of skin, but I don't think that method would work well in a parser or something that would have a much more complicated output.
User avatar
Yincognito
Posts: 2769
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Idea] XML Parser measure/plugin

Post by Yincognito »

MikeG621 wrote: October 13th, 2020, 2:15 pm You can with Lua, but if you don't put limits on what you're inputting it can run away pretty quick and I've never tried to automate cleaning it up afterwards. I derived my notes skin from Enigma and at first run there's not much more than the Lua measure and the background, the script generates everything else based on the variable that defines the quantity (array size). If I increase the quantity later, it creates more measures/meters. Since I haven't automated cleanup (delete key works fine), reducing the quantity just hides what isn't being used.

That said, I don't even use that form of the skin anymore, and it's quite a bit of setup just to avoid manual copy/paste to create a repetitive skin setup. It works fine for a simple "pages" sort of skin, but I don't think that method would work well in a parser or something that would have a much more complicated output.
Lua can do that for sure, but technically it's a bit misleading, since it still needs a new "refresh cycle" to do it - meaning it can't do it dynamically, in the same refresh cycle, preserving the current values of measures, variables, etc. So, in the context of the same refresh cycle, I believe jsmorley's statement holds true.

Personally, I'm not a fan of replicating measures or meters, Lua based or not, or having a ton of them in the skin. I prefer having a stable (but limited) number of measures and meters, and then just navigating through the entire scope of values with the help of dynamic variables, setting options and / or regex quantifiers. One can easily simulate an array of N dimensions (of even irregular length) using a string and extraction of a value at the "intersection" of a number of indexes based on regex quantifiers and some defined "separators" of the string data. This of course doesn't work if you need every value in the array to be available at the same time (as opposed to only the selected values), but then alternative solutions can be found in those cases (like, for example, merging all the band bars in a visualiser into a single shape meter). In the end, screen space is limited anyway, and so is the number of meters one can place on the screen, assuming reasonable dimensions.