It is currently July 17th, 2019, 7:38 pm

universal "feed reader" with YQL

Tips and Tricks from the Rainmeter Community
User avatar
moshi
Posts: 1851
Joined: November 13th, 2012, 9:53 pm

universal "feed reader" with YQL

moshi » December 6th, 2013, 11:37 am

when Google Reader was closed, a lot of people spent time and energy to build universal feed readers that support RSS and ATOM feeds (and more).

at least for me that was fun, but it could have been so much easier.
Yahoo! offers YQL, the Yahoo! Query Language, an API that can do a lot of cool things, one of them is the so-called feednormalizer. it transforms any feed into a format of your choice.

we can use it with the webparser plugin. like this for example:

Code: Select all

URL="http://query.yahooapis.com/v1/public/yql?q=select * from feednormalizer where url="http://rainmeter.net/forum/feed.php" and output="atom_1.0"&env=store://datatables.org/alltableswithkeys"
for more info about the supported output formats, look here: http://www.yqlblog.net/blog/2013/03/07/yql-feednormalizer-table/
screenshot-Rainmeter-06.12.2013 , 12_28_30-001.png
for an example skin and more talk about the topic look here: http://rainmeter.net/forum/viewtopic.php?f=15&t=17195
You do not have the required permissions to view the files attached to this post.
Last edited by moshi on December 8th, 2013, 9:30 am, edited 2 times in total.
User avatar
jsmorley
Developer
Posts: 19287
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: universal feed reader with YQL

jsmorley » December 6th, 2013, 12:24 pm

Very cool. One thing, what is the point of the parm: &env=store://datatables.org/alltableswithkeys in the context of a Rainmeter skin?
User avatar
moshi
Posts: 1851
Joined: November 13th, 2012, 9:53 pm

Re: universal feed reader with YQL

moshi » December 6th, 2013, 12:37 pm

jsmorley wrote:Very cool. One thing, what is the point of the parm: &env=store://datatables.org/alltableswithkeys in the context of a Rainmeter skin?
the parameter is completely pointless for a feednormalizer query :D

it's needed for other queries like WOEID lookup or weather as far as i know. i still had it in and did not bother to remove it. doesn't hurt anyways.
User avatar
jsmorley
Developer
Posts: 19287
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: universal feed reader with YQL

jsmorley » December 6th, 2013, 12:38 pm

The other question I would have is does this normalize date formats in the feed?
User avatar
jsmorley
Developer
Posts: 19287
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: universal feed reader with YQL

jsmorley » December 6th, 2013, 12:40 pm

moshi wrote: the parameter is completely pointless for a feednormalizer query :D

it's needed for other queries like WOEID lookup or weather as far as i know. i still had it in and did not bother to remove it. doesn't hurt anyways.
Ah ok. Would you mind if I worked up a more full-blown "tips and tricks" for this? I really think this could be the answer to the missing Google Reader.
User avatar
moshi
Posts: 1851
Joined: November 13th, 2012, 9:53 pm

Re: universal feed reader with YQL

moshi » December 6th, 2013, 12:43 pm

jsmorley wrote: Ah ok. Would you mind if I worked up a more full-blown "tips and tricks" for this? I really think this could be the answer to the missing Google Reader.

not at all. this is just something i found, it's not like i would have put much coding or any effort in it.
User avatar
moshi
Posts: 1851
Joined: November 13th, 2012, 9:53 pm

Re: universal feed reader with YQL

moshi » December 6th, 2013, 12:51 pm

jsmorley wrote:The other question I would have is does this normalize date formats in the feed?
that seems to work pretty cool.

for example this:
<pubDate>Fri, 06 Dec 2013 16:45:00 +0400</pubDate>

becomes:
<updated>2013-12-06T12:45:00Z</updated>

so the ouput time is UTC (as it should be.)
User avatar
jsmorley
Developer
Posts: 19287
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: universal feed reader with YQL

jsmorley » December 6th, 2013, 12:56 pm

moshi wrote: that seems to work pretty cool.

for example this:
<pubDate>Fri, 06 Dec 2013 16:45:00 +0400</pubDate>

becomes:
<updated>2013-12-06T12:45:00Z</updated>

so the ouput time is UTC (as it should be.)
That alone is a reason to use this over almost any other approach. Having a standard way to deal with dates so you can flag "new" stuff is possible today, but very difficult and unreliable.
User avatar
moshi
Posts: 1851
Joined: November 13th, 2012, 9:53 pm

Re: universal feed reader with YQL

moshi » December 6th, 2013, 1:39 pm

here's an YQL example for merging and sorting multiple RSS feeds:

Code: Select all

URL="http://query.yahooapis.com/v1/public/yql?q=select title,link,pubDate from rss where url in ("http://rt.com/news/today/rss/","http://english.ruvr.ru/rss/export_all.xml") | sort(field="pubDate",descending="true") | truncate(count=8)"
i have not managed to "normalize" that yet, so it could be used with ATOM feeds as well.
User avatar
jsmorley
Developer
Posts: 19287
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: universal feed reader with YQL

jsmorley » December 6th, 2013, 2:09 pm

Moshi,

Not sure I see a way to turn a date string from the feed like 2013-12-06T12:45:00Z into a Time measure (Windows Timestamp) TimeStamp= that I can compare with the current time with TimeZone=0.

Have you messed with converting a "known" standard date/time string into a TimeStamp in Rainmeter? It's easy in Lua, but rather not go there if I don't have to.