It is currently May 19th, 2019, 7:39 am

Web Parser not downloading all data (sometimes)

Help with creating, editing & fixing problems with skins
xenium
Posts: 335
Joined: January 4th, 2018, 9:52 pm

Web Parser not downloading all data (sometimes)

xenium » March 27th, 2018, 12:24 pm

Hi,
I work on a skin that displays the parameters of air quality (website source aqicn.org)
Compared with weather skin ,where data is displayed quickly,on this skin the data display is much slower (at intervals of 1.2 or 3 seconds). Sometimes when loading the skin or when changing the location, not all data is displayed, even though they exist (especially when there is more data to display), there is still 2.3 data that is not displayed.
If they refresh, all data is quickly displayed.
The skin has the update option 1000 and the UpdateRate is 600.

Is there a solution to solve the problem with no data being displayed?

Thank you
User avatar
balala
Rainmeter Sage
Posts: 8188
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Web Parser not downloading all data (sometimes)

balala » March 27th, 2018, 12:26 pm

xenium wrote:Is there a solution to solve the problem with no data being displayed?
No, without the code. Post it please, or if the skin uses some resources (included files, images and so on), pack the config and upload it.
xenium
Posts: 335
Joined: January 4th, 2018, 9:52 pm

Re: Web Parser not downloading all data (sometimes)

xenium » March 27th, 2018, 5:41 pm

I have solved the problem .
I increased the Update value from 1000 to 2000, and the data display is much faster, and there is no data that is not displayed (even with update = 1500 data is displayed faster).
That would mean an update at 15-20 minutes what is acceptable.
User avatar
balala
Rainmeter Sage
Posts: 8188
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Web Parser not downloading all data (sometimes)

balala » March 27th, 2018, 6:33 pm

xenium wrote:I increased the Update value from 1000 to 2000, and the data display is much faster, and there is no data that is not displayed (even with update = 1500 data is displayed faster).
That would mean an update at 15-20 minutes what is acceptable.
This is weird, but anyway, the Update is measured in milliseconds, NOT in seconds. So, Update=1000 means one update per second, Update=2000 means one update per two seconds.
xenium
Posts: 335
Joined: January 4th, 2018, 9:52 pm

Re: Web Parser not downloading all data (sometimes)

xenium » March 27th, 2018, 6:53 pm

I still have a question:
why ,sometimes, the data on the website is updated and in the page source the data is not updated ?
User avatar
balala
Rainmeter Sage
Posts: 8188
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Web Parser not downloading all data (sometimes)

balala » March 27th, 2018, 6:57 pm

xenium wrote:why ,sometimes, the data on the website is updated and in the page source the data is not updated ?
"In the page source" means in the skin?
xenium
Posts: 335
Joined: January 4th, 2018, 9:52 pm

Re: Web Parser not downloading all data (sometimes)

xenium » March 27th, 2018, 7:12 pm

balala wrote:"In the page source" means in the skin?
in the source page of the website, in which case the skin does not present the updated data.
User avatar
balala
Rainmeter Sage
Posts: 8188
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Web Parser not downloading all data (sometimes)

balala » March 27th, 2018, 7:31 pm

xenium wrote:in the source page of the website, in which case the skin does not present the updated data.
It will, but the skins using WebParser measure (which are used to get the online data) usually are set to a longer update rate, which means that these measures are updated less frequently. So maybe check the update rate of the parent WebParser measure. For example if the update of the skin is set to the default 1000 and the parent WebParser measure is set to UpdateRate=600, means that the WebParser measure is update once per 10 minutes (one update at each 1000ms x 600 = 600,000 ms = 600 secs = 10 minutes). So, you have to wait up to 10 minutes to the skin being updated. In the meantime you can wait patiently, or depending on the skin, you can ask for an extra update of the measure.
xenium
Posts: 335
Joined: January 4th, 2018, 9:52 pm

Re: Web Parser not downloading all data (sometimes)

xenium » March 31st, 2018, 8:52 am

Hi
The problem with partially displaying data ,sometimes,recurred.
I think the cause is how much data I'm on that web page (I guess that's why the data is displayed slower)
If they refresh, the data (including those that are not displayed) are quickly displayed all
This issue occurs (when it occurs) only when the location is loaded for the first time.
Is there a possibility that once the data is first displayed (or partially displayed), do you have a new update immediately?
Or a refrehing?

Code: Select all

[Rainmeter]
Update=1000


[Variables]
UpdateRateSeconds=600
FontFace=arial
;URL=http://aqicn.org/city/ulaanbaatar/nisekh/
URL=http://aqicn.org/city/beijing/
;URL=http://aqicn.org/city/montreal/
;URL=http://aqicn.org/city/shanghai/
;URL=http://aqicn.org/city/singapore/central/

[MeasureN]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<a href='http://aqicn.org/city/(.*)/.*'
UpdateRate=#UpdateRateSeconds#
StringIndex=1

[MeasureT]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='cur_t' class='tdcur' style='.*' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureT1]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='min_t' class='tdmin' style='.*' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureT2]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='max_t' class='tdmax' style='.*' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1

[MeasureD]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='cur_d' class='tdcur' style='.*;' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureD1]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='min_d' class='tdmin' style='.*' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureD2]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='max_d' class='tdmax' style='.*' align=center><span class='temp' format='nu' temp=.*>(.*)</span></td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1

[MeasureP]
Measure=WebParser
Url=#URL#
RegExp=(?siU)</td><td id='cur_p' class='tdcur' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureP1]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='min_p' class='tdmin' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureP2]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='max_p' class='tdmax' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1

[MeasureH]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='cur_h' class='tdcur' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureH1]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='min_h' class='tdmin' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1


[MeasureH2]
Measure=WebParser
Url=#URL#
RegExp=(?siU)<td id='max_h' class='tdmax' style='.*' align=center>(.*)</td>
UpdateRate=#UpdateRateSeconds#
StringIndex=1

=================================================================

[MeterN]
Meter=String
MeasureName=MeasureN
X=250
Y=300
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=15

[MeterT]
Meter=String
MeasureName=MeasureT
X=150
Y=360
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12



[MeterD]
Meter=String
MeasureName=MeasureD
X=150
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12


[MeterP]
Meter=String
MeasureName=MeasureP
X=150
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12


[MeterH]
Meter=String
MeasureName=MeasureH
X=150
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterT1]
Meter=String
MeasureName=MeasureT1
X=250
Y=360
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12


[MeterD1]
Meter=String
MeasureName=MeasureD1
X=250
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterP1]
Meter=String
MeasureName=MeasureP1
X=250
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterH1]
Meter=String
MeasureName=MeasureH1
X=250
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterT2]
Meter=String
MeasureName=MeasureT2
X=300
Y=360
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12



[MeterD2]
Meter=String
MeasureName=MeasureD2
X=300
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterP2]
Meter=String
MeasureName=MeasureP2
X=300
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12

[MeterH2]
Meter=String
MeasureName=MeasureH2
X=300
Y=30r
W=270
H=50
FontColor=250,250,250,250
StringStyle=Normal
StringEffect=SHADOW
FontFace=arial
StringAlign=left
FontEffectColor=0,0,0,80
AntiAlias=1
ClipString=1
FontSize=12


 
User avatar
balala
Rainmeter Sage
Posts: 8188
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Web Parser not downloading all data (sometimes)

balala » March 31st, 2018, 9:46 am

xenium wrote:The problem with partially displaying data ,sometimes,recurred.
Not sure, I tried right now a few times and it worked well each time.
However the whole structure of the WebParser measures is not the best one. Usually these measures are using a parent-child structure, which means that one of them should have to be the parent measure and the others should have to be the child measures. The parent measure should have the Url=#URL# option, while the child measures should have to be related to the parent measure, through a Url=[ParentMeasureName] option. I'm not sure how this approach would influence what is working (displaying) the data, but I think it could.
If you don't know how to modify the structure in a good way, later today I'll try to help you with this, too. Right now I have no enough time, but later today I will have. This would be the first step in my opinion.