universal "feed reader" with YQL

[moshi]

Have you messed with converting a "known" standard date/time string into a TimeStamp in Rainmeter? It's easy in Lua, but rather not go there if I don't have to.

no, not really, that's way above my head. what i use is a port of the GNU date command:

Code: Select all

[MeasureTimeParse]
Measure=Plugin
Plugin=RunCommand
Program=""#@#Addons\date2\date2.exe""
Parameter="-d "[MeasureTimeParsePre]" +"%A, %H:%M""
State=Hide
OutputType=UTF8
DynamicVariables=1

[MeasureTimeParsePre] looks like 2013-12-25 16:15:00 +0700

for a (UNIX) timestamp i'd just use:

Code: Select all

Parameter="-d "[MeasureTimeParsePre]" +"%s""

also:
http://rainmeter.net/forum/viewtopic.php?f=14&t=13363&hilit=timestamp

[jsmorley]

no, not really, that's way above my head. what i use is a port of the GNU date command:

Code: Select all

[MeasureTimeParse]
Measure=Plugin
Plugin=RunCommand
Program=""#@#Addons\date2\date2.exe""
Parameter="-d "[MeasureTimeParsePre]" +"%A, %H:%M""
State=Hide
OutputType=UTF8
DynamicVariables=1

[MeasureTimeParsePre] looks like 2013-12-25 16:15:00 +0700

for a (UNIX) timestamp i'd just use:

Code: Select all

Parameter="-d "[MeasureTimeParsePre]" +"%s""

also:
http://rainmeter.net/forum/viewtopic.php?f=14&t=13363&hilit=timestamp

Ok, I'm fine with using Lua, as it has all that capability built-in and I don't need an "addon". I just need to work it so that capability is something a user can just "plug-into" the skin in a reasonably clear way, so it is only there if they want to compare a feed date/time with current.

Thanks.

[jsmorley]

Moshi, found one problem with this YQL tool.

When I get the Rainmeter feed normally with WebParser, there is a point where it has:

Tips & Tricks

Using DecodeCharacterReference=1 on the measure takes care of that:

Tips & Tricks

With the YQL version of the feed, I get:

Tips & Tricks

Which DecodeCharacterReference=1 turns into:

Tips & Tricks

Since what it does is replace the & with &, leaving... &, or right back where we started...

What I see in the raw feed from rainmeter.net is:

<title type="html"><![CDATA[Tips & Tricks • Re: universal feed reader with YQL]]></title>

YQL is turning that into:

<title type="html">Tips &amp; Tricks • Re: universal feed reader with YQL</title>

That is just stupid. It is trying to encode an already encoded entity, and even then doing a poor job of it. It is apparently just seeing the "&" and saying "Damn, I'd better encode that!!!" and ignoring the fact that it already is "&".

This is going to be a major problem with the approach in Rainmeter, unless there is some parameter to the YQL API that can correct it.

Using a Substitute like Sub="&":"&",""":"''","<":"<",">":">" or something is just not going to be good enough. There are many, many possible encoded characters in a feed. You would need the Substitute from hell to deal with all of them.

[moshi]

i can't test this weekend, but how about downloading it with DecodeCharacterReference=1 and then parsing the downloaded file with again with DecodeCharacterReference=1 ?

[jsmorley]

i can't test this weekend, but how about downloading it with DecodeCharacterReference=1 and then parsing the downloaded file with again with DecodeCharacterReference=1 ?

I dunno... Let me call Rube Goldberg and see what he thinks.

Edit:

Doesn't seem to help:

Code: Select all

[Metadata]
Name=YQLFeed
Author=JSMorley
Information=Demonstration of using the YQL syntax to obtain feeds produced in any format
Version=Dec 6, 2013
License=Creative Commons Attribution-Non-Commercial-Share Alike 3.0

[Variables]
FeedURL=http://rainmeter.net/forum/feed.php
FeedInfo=<title.*>(.*)</title>.*<link href="(.*)".*<updated>(.*)</updated>
Entry=.*<entry>(.*)</entry>
EntryItems=<title.*>(.*)</title>.*<link href="(.*)".*<updated>(.*)</updated>
Sub="&":"&",""":"''","<":"<",">":">"

[MeasureGrandparent]
Measure=Plugin
Plugin=WebParser
URL=http://query.yahooapis.com/v1/public/yql?q=select * from feednormalizer where url="#FeedURL#" and output="atom_1.0"
RegExp=(?siU)^(.*)$
DecodeCharacterReference=1

[MeasureFeed]
Measure=Plugin
Plugin=WebParser
URL=[MeasureGrandparent]
RegExp=(?siU)#FeedInfo##Entry##Entry##Entry##Entry##Entry#
DecodeCharacterReference=1

2013-12-06_115318.jpg

[moshi]

i was thinking about Download=1 and DownloadFile=something.xml, with either DecodeCharacterReference=1 or Substitute="&":"&" to have a "physical", "first-step-decoded" file to parse.

[jsmorley]

i was thinking about Download=1 and DownloadFile=something.xml, with either DecodeCharacterReference=1 or Substitute="&":"&" to have a "physical", "first-step-decoded" file to parse.

Not sure there is any difference with what I did with the "grandparent", but I'll play with that in a while.

[moshi]

if that fails, you could use FinishAction=!WriteKeyValue to a second file. then all decode and substitute options will work, of course you would have to substitute all line breaks, so everything can be overwritten on the next update.

[jsmorley]

There is only so far I am willing to go with something that is supposed to be "generic" and easy to use. Only so many bags I will hang on the side of the machine.

[moshi]

i think this is pretty much the same as i did in the example skin in the first post.
it only works for &amp; though, i don't think that'll work for things like &#39; though.

i'd love to have a !WriteFile bang btw., that would generate/overwrite a file. that would not only be cool for double-encoded feeds but also for dynamically writing scripts. i know i can do this with echo and the RunCommand plugin, but with all the capabilities Rainmeter has now, i think it should be a generic feature.