It is currently May 3rd, 2024, 12:06 am

WebParser RegExp - Multiple Queries

Get help with creating, editing & fixing problems with skins
novocaine
Posts: 1
Joined: January 23rd, 2012, 9:26 pm

WebParser RegExp - Multiple Queries

Post by novocaine »

Hi,

Apologies if this has been asked before, I tried a search and couldn't find a solution.
I'm using the WebParser to retrieve data from my SickBeard installation, so far it is working well but I have run into a problem with the order in which the data is processed.

The data that is being read in:

Code: Select all

{
    "data": {
        "missed": [], 
        "soon": [
            {
                "airdate": "2012-01-26", 
                "airs": "Thursday 8:00 PM", 
                "ep_name": "The Beta Test Initiation", 
                "ep_plot": "Leonard and Penny deal with the idea of starting a relationship again and Raj finally finds a \"woman\" he can talk to without being drunk, in the form of Siri on his new iPhone. Meanwhile, Sheldon decides to start a YouTube series entitled \"Sheldon Cooper presents Fun with Flags\" with the help of Amy.", 
                "episode": 14, 
                "network": "CBS", 
                "quality": "HD", 
                "season": 5, 
                "show_name": "The Big Bang Theory", 
                "show_status": "Continuing", 
                "tvdbid": 80379, 
                "weekday": 4
            }, 
            {
                "airdate": "2012-01-27", 
                "airs": "Friday 9:00 PM", 
                "ep_name": "Forced Perspective", 
                "ep_plot": "Olivia continues to cope with the ominous warning from the Observers, while Peter and the team track a girl with the mysterious ability to predict death.", 
                "episode": 10, 
                "network": "FOX", 
                "quality": "HD", 
                "season": 4, 
                "show_name": "Fringe", 
                "show_status": "Continuing", 
                "tvdbid": 82066, 
                "weekday": 5
            }, 
            {
                "airdate": "2012-01-29", 
                "airs": "Sunday 9:30 PM", 
                "ep_name": "Stanny Tendergrass", 
                "ep_plot": "When Stan finally has enough money to afford a membership at the golf club he has worked at for the past thirty summers, his hard work and perseverance prove to be futile when the club gives a membership to Steve first. However, things are not all what they seem when Stan realizes who the club owner really is.", 
                "episode": 9, 
                "network": "FOX", 
                "quality": "SD", 
                "season": 8, 
                "show_name": "American Dad!", 
                "show_status": "Continuing", 
                "tvdbid": 73141, 
                "weekday": 7
            }, 
            {
                "airdate": "2012-01-29", 
                "airs": "Sunday 9:00 PM", 
                "ep_name": "Livin' on a Prayer", 
                "ep_plot": "When Stewie's new best friend falls ill, Lois takes him to the hospital where she finds out he has a critical, but treatable, disease. However, the boy's parents refuse treatment due to their religious beliefs, leaving Lois no choice but to take drastic measures to seek treatment for the boy herself. Meanwhile, Peter has a religious experience of his own.", 
                "episode": 12, 
                "network": "FOX", 
                "quality": "Any", 
                "season": 10, 
                "show_name": "Family Guy", 
                "show_status": "Continuing", 
                "tvdbid": 75978, 
                "weekday": 7
            }
        ], 
        "today": [
            {
                "airdate": "2012-01-23", 
                "airs": "Monday 9:00 PM", 
                "ep_name": "Kit Nelson", 
                "ep_plot": "When child killer Kit Nelson, a terrifying kidnapper who always returns his victims home - after he's killed them - reappears from the past, the team is in a race against time to catch the man before he kills again.", 
                "episode": 3, 
                "network": "FOX", 
                "quality": "HD", 
                "season": 1, 
                "show_name": "Alcatraz", 
                "show_status": "Continuing", 
                "tvdbid": 248646, 
                "weekday": 1
            }
        ]
    }, 
    "message": "", 
    "result": "success"
}
My Query:

Code: Select all

RegExp="(?siU)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)(?(?=.*airdate").*airdate": "(.*)",)(?(?=.*show_name").*show_name": "(.*)",)(?(?=.*tvdbid").*tvdbid": (.*),)"
The Debug Output:
DEBUG: (68:05:48.583) WebParser.dll: Fetching: http://192.168.0.100:8080/api/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/?cmd=future&sort=date&type=today|missed|soon
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 0) {
"data": {
"missed": [],
"soon": [
{
"airdate": "2012-01-26",
"airs": "Thursday 8:00 PM",
"ep_name": "The Beta Test Initiation",
"ep_plot": "Leonard and Pe
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 1) 2012-01-26
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 2) The Big Bang Theory
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 3) 80379
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 4) 2012-01-27
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 5) Fringe
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 6) 82066
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 7) 2012-01-29
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 8) American Dad!
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 9) 73141
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 10) 2012-01-29
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 11) Family Guy
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 12) 75978
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 13) 2012-01-23
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 14) Alcatraz
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 15) 248646
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-6-Date] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-6-Show] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-6-Link] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-7-Date] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-7-Show] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-7-Link] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-8-Date] Not enough substrings!
WARNING: (68:05:48.676) WebParser.dll: [MeasureSB1-8-Show] Not enough substrings!
WARNING: (68:05:48.692) WebParser.dll: [MeasureSB1-8-Link] Not enough substrings!
WARNING: (68:05:48.692) WebParser.dll: [MeasureSB1-9-Date] Not enough substrings!
WARNING: (68:05:48.692) WebParser.dll: [MeasureSB1-9-Show] Not enough substrings!
WARNING: (68:05:48.692) WebParser.dll: [MeasureSB1-9-Link] Not enough substrings!
The issue I have is the source is split into sections, "Missed", "Soon" and "Today". This means that the array is populated in that order "Missed then Soon then Today. I need it in the following order: Missed, Today then Soon.

Code: Select all

DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  1) 2012-01-26
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  2) The Big Bang Theory
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  3) 80379
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  4) 2012-01-27
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  5) Fringe
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  6) 82066
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  7) 2012-01-29
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  8) American Dad!
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index  9) 73141
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 10) 2012-01-29
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 11) Family Guy
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 12) 75978
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 13) 2012-01-23
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 14) Alcatraz
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 15) 248646
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 13) 2012-01-23
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 14) Alcatraz
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 15) 248646

Because the above show was listed in the "Today" section it was added at the end of the array and is displayed at the end of the list.

I really can't think of a way to process this data so the array is populated in the correct order.

Thanks in advance and please let me know if you require any further information.

Also, as a side question. Can anyone tell me why I have the following data in Index(0):
DEBUG: (68:05:48.676) WebParser.dll: [SBQuery1] (Index 0) {
"data": {
"missed": [],
"soon": [
{
"airdate": "2012-01-26",
"airs": "Thursday 8:00 PM",
"ep_name": "The Beta Test Initiation",
"ep_plot": "Leonard and Pe
Thanks.
Top
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: WebParser RegExp - Multiple Queries

Post by Mordasius »

(Original Post withdrawn because I hadn't read the question properly and can't suggest a way to solve the problem although I suspect a little LUA script searching for the patterns '.-"missed": %[(.-)%]' , '.-"soon": %[(.-)%]' and '.-"today": %[(.-)%]' might or, might not, help.)
:oops:
Last edited by Mordasius on January 24th, 2012, 11:44 am, edited 1 time in total.
Top
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: WebParser RegExp - Multiple Queries

Post by Mordasius »

One way to do it without resorting to LUA would be to set up separate "Child" measures for the "Missed", "Soon" and "Today" sections of the source. This would be very cumbersome and I'm sure it would be better done with LUA but, for what it's worth, the following sort of thing would work:

Code: Select all

[Variables]
SickbeardURL=file://#CURRENTPATH#Source.html  <-PUT CORRECT URL HERE!
GetShow=(?(?=.*"airdate").*airdate": "(.*)".*show_name": "(.*)".*tvdbid": (.*),\r)

[mSource]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=#SickbeardURL#
RegExp="(?siU)(.*)$"
UpdateRate=3600

[mSoon]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=[mSource]
RegExp="(?siU).*"soon.*\[(.*)\]"
StringIndex=1
UpdateRate=3600

[mChildSoon]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=[mSoon]
RegExp="(?siU)#GetShow##GetShow##GetShow##GetShow##GetShow##GetShow##GetShow##GetShow#"
StringIndex=1
UpdateRate=3600

;------------------------
[mSoonDate1]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=1
DecodeCharacterReference=1
UpdateRate=3600

[mSoonShow1]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=2
DecodeCharacterReference=1
UpdateRate=3600

[mSoonTVID1]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=3
DecodeCharacterReference=1
UpdateRate=3600

;------------------------
[mSoonDate2]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=4
DecodeCharacterReference=1
UpdateRate=3600

[mSoonShow2]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=5
DecodeCharacterReference=1
UpdateRate=3600

[mSoonTVID2]
Measure=Plugin
Plugin=WebParser.dll
Url=[mChildSoon]
StringIndex=6
DecodeCharacterReference=1
UpdateRate=3600

;..[mSoonTVID3] etc......

;------------------------
[BackGround]
Meter=IMAGE
W=300
H=500
SolidColor=0,0,0,50
Update=-1

[TextStyle]
StringAlign=LEFT
StringStyle=Normal
FontFace=Calibri
FontColor=ffffff
FontSize=10
;----------------------
[ShowSoon1]
Meter=STRING
MeterStyle=TextStyle
MeasureName=mSoonDate1
MeasureName2=mSoonShow1
MeasureName3=mSoonTVID1
Text="%1 :  %2  (ID: %3)"

[ShowSoon2]
Meter=STRING
MeterStyle=TextStyle
Y=R
MeasureName=mSoonDate2
MeasureName2=mSoonShow2
MeasureName3=mSoonTVID2
Text="%1 :  %2  (ID: %3)"

;..[ShowSoon3] etc......
Top
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: WebParser RegExp - Multiple Queries

Post by Mordasius »

Ok. Here is a version using a LUA script that you may be able to adapt:

WebParser2.ini file

Code: Select all

[Variables]
SickbeardURL=file://#CURRENTPATH#Source.html
; [[[[[  The above was a link to the source provided in first post ]]]]]]

[MeasureFeed]
Measure=Plugin
Plugin=Plugins\WebParser.dll
Url=#SickbeardURL#
RegExp="(?siU)(.*)$"
UpdateRate=1800
FinishAction=!EnableMeasure MeasureFeedLua #CURRENTCONFIG#

[MeasureFeedLua]
Measure=Script
ScriptFile="#CURRENTPATH#WebParser2.lua"
Disabled=1

[TextStyle]
StringAlign=LEFT
StringStyle=Normal
FontFace=Calibri
FontColor=ffffff
FontSize=10
Y=R
;_________________________________________
[BackGround]
Meter=Image
W=320
H=200
SolidColor=0,0,0,50
Update=-1

;-------------------------
[ShowMissedTitle]
Meter=STRING
MeterStyle=TextStyle
FontColor=0000ff
Y=6
Text="--- SHOWS MISSED ---"

[ShowMissed1]
Meter=STRING
MeterStyle=TextStyle
Text=None

[ShowMissed2]
Meter=STRING
MeterStyle=TextStyle

[ShowMissed3]
Meter=STRING
MeterStyle=TextStyle

[ShowMissed4]
Meter=STRING
MeterStyle=TextStyle

[ShowMissed5]
Meter=STRING
MeterStyle=TextStyle

[ShowMissed6]
Meter=STRING
MeterStyle=TextStyle
;--------------------------
[ShowTodayTitle]
Meter=STRING
MeterStyle=TextStyle
FontColor=00ff00
Text="--- SHOWING TODAY ---"

[ShowToday1]
Meter=STRING
MeterStyle=TextStyle
Text=None

[ShowToday2]
Meter=STRING
MeterStyle=TextStyle

[ShowToday3]
Meter=STRING
MeterStyle=TextStyle

[ShowToday4]
Meter=STRING
MeterStyle=TextStyle

[ShowToday5]
Meter=STRING
MeterStyle=TextStyle

[ShowToday6]
Meter=STRING
MeterStyle=TextStyle

;----------------------------
[ShowSoonTitle]
Meter=STRING
MeterStyle=TextStyle
FontColor=FFF81F
Text="--- SHOWING SOON ---"

[ShowSoon1]
Meter=STRING
MeterStyle=TextStyle
Text=None

[ShowSoon2]
Meter=STRING
MeterStyle=TextStyle

[ShowSoon3]
Meter=STRING
MeterStyle=TextStyle

[ShowSoon4]
Meter=STRING
MeterStyle=TextStyle

[ShowSoon5]
Meter=STRING
MeterStyle=TextStyle

[ShowSoon6]
Meter=STRING
MeterStyle=TextStyle
WebParser2.lua file

Code: Select all

PROPERTIES =
{

}

function Initialize()

	msWebParserMeasure = SKIN:GetMeasure("MeasureFeed")

end -- function Initialize

function Update()

		sRaw = msWebParserMeasure:GetStringValue()
		
		sMissed=string.match(sRaw,'.-"missed": %[(.-)%]' )
		sTempCounted, iNumMissed = string.gsub(sMissed, '{.-}', "%(*%)")
		
		sToday=string.match(sRaw,'.-"today": %[(.-)%]' )
		sTempCounted, iNumToday = string.gsub(sToday, '{.-}', "%(*%)")
		
		sSoon=string.match(sRaw,'.-"soon": %[(.-)%]' )
		sTempCounted, iNumSoon = string.gsub(sSoon, '{.-}', "%(*%)")

		sPatternOneShow = '.-{(.-)}'
		sPatternAirdate = '.-"airdate": "(.-)",'
		sPatternEpName = '.-"ep_name": "(.-)",'
		sPatternTVid = '.-"tvdbid":(.-),.-\n'
		
		if iNumMissed then 
			ReadFeed(sMissed, iNumMissed)
			for i = 1, iNumMissed do
				sShowDetails = ":-> "..tAirDate[i].."  -  "..tEpName[i].."   \(ID:"..tTVid[i].."\)"
				SKIN:Bang('!SetOption ShowMissed'..i..' Text \"'..sShowDetails..'\"')
			end
		end
		
		if iNumToday then 
			ReadFeed(sToday, iNumToday)
			for i = 1, iNumToday do
				sShowDetails = ":-> "..tAirDate[i].."  -  "..tEpName[i].."   \(ID:"..tTVid[i].."\)"
				SKIN:Bang('!SetOption ShowToday'..i..' Text \"'..sShowDetails..'\"')
			end
		end
		
		if iNumSoon then 
			ReadFeed(sSoon, iNumSoon)
			for i = 1, iNumSoon do
				sShowDetails = ":-> "..tAirDate[i].."  -  "..tEpName[i].."   \(ID:"..tTVid[i].."\)"
				SKIN:Bang('!SetOption ShowSoon'..i..' Text \"'..sShowDetails..'\"')
			end
		end

		SKIN:Bang('!DisableMeasure MeasureFeedLua')
		
		return "All Done"

end -- function Update

function ReadFeed(sShows, iNumShows)

	tAirDate = {}	
	tEpName = {}
	tTVid = {}

	iInit = 0
	if iNumShows then
		for i = 1, iNumShows do
			iItemStart, iItemEnd = string.find(sShows, sPatternOneShow, iInit)
			sItem = string.sub(sShows, iItemStart, iItemEnd)		
			tAirDate[i] = string.match(sItem, sPatternAirdate)
			tEpName[i] = string.match(sItem, sPatternEpName)
			tTVid[i] = string.match(sItem, sPatternTVid)					
			iInit = iItemEnd + 1
		end
	end	

end -- function ReadFeed
The neat thing about using a LUA script with SKIN:Bang('!SetOption ......') in a case like this is that it is easy to sort or format the data how you want it and it accommodates an unknown number of entries/items and can easily be expanded to accommodate more entries/items by simply adding extra blank meters in the ini file. No need to add or try keeping track of all those measures with StringIndex= statements.
Top

Return to “Help: Rainmeter Skins”