WebParser - GZIP web server responses

[dw-dev]

I have a skin that has been working correctly for several years, but recently quite often fails when trying to match the RegExp in the WebParser measure.

My investigations have shown that the web server at the URL specified in the WebParser measure is sometimes sending a plain text response which is fine and is correctly matched, but other times is sending a compressed (GZIP) response which cannot be matched correctly.

Can the WebParser measure handle and decode compressed (GZIP) responses?

[SilverAzide]

I don't think so either. Perhaps try adding a header "Accept: text/html" ???

[jsmorley]

WebParser is certainly not going to be able to deal with anything that isn't just UTF-8 "text".

[balala]

WebParser is certainly not going to be able to deal with anything that isn't just UTF-8 "text".

However: https://forum.rainmeter.net/viewtopic.php?f=5&t=35892&start=10#p182030

[jsmorley]

However: https://forum.rainmeter.net/viewtopic.php?f=5&t=35892&start=10#p182030

I have not seen any evidence that that will in any way work.

[balala]

I have not seen any evidence that that will in any way work.

Agree. Neither me haven't seen, but when I saw Yincognito's post have done a correlation and posted the link. Didn't try it out and now I can't either.

[Yincognito]

Can the WebParser measure handle and decode compressed (GZIP) responses?

I don't know about handling - you might want to try using some headers to see if you can force a decompressed response - but decoding, I don't think so.

However, what you can do is download the response (using WebParser's appropriate options), then use the command line version of WinZip, 7Zip or some other compression/decompression tool with RunCommand to get the decompressed output in another file (or, apparently, even to STDOUT which can be directly processed further in the RunCommand measure using a RegExp Substitute), which can be then parsed locally the normal way using your regexes. It's a bit of a convoluted process, but I believe it can be done. You can investigate if the response is compressed or not by handling the header, like this (see the comment too) or even try to parse the response and rate it compressed on parsing failure.

Other than that, some interesting links and tools regarding this, here, here and here (not sure why the latter 2 give different results for the same string, maybe it's a text vs file compression thing) - or you can search yourself other tools that do this in various languages (including PHP).

Agree. Neither me haven't seen, but when I saw Yincognito's post have done a correlation and posted the link. Didn't try it out and now I can't either.

Yeah, that doesn't have much to do with the things being discussed in this thread - that was about parsing an EXE file to get the version of Rainmeter, not about parsing a GZIP encoded file to decompress it. Sorry.

P.S. Maybe some simple link/samples of what you're trying to parse would help, just to see if we can make the whole process successful in a basic skin...