Retrieve the last data from an XML file

[PtitChat107]

Good morning all,

For those seeking to obtain precise (and recent) data on precious metals (gold, silver, platinum and palladium), I think I have found an interesting solution that is updated every minute. It is possible to have the price, per gram, per kilo and per ounce; in euros, dollars and many other world currencies.

Only, what I haven't managed to do yet, is to recover the data in question, because the most recent is (under the name "value") always at the bottom of the page; which I did not manage to find a solution to "capture" this famous data.

Link: or.fr/api/spot-prices?metal=***&currency=***&weight_unit=***
Metal: XAU(gold)/XAG(silver)/XPD(palladium)/XPT(platinum)
Currency: USD/EUR/etc.
Unit: gr/kg/oz

I hope that this little research on my part will please some people !

[PtitChat107]

So nobody could help me ?

[mak_kawa]

Hi PtitChat107

I have accessed the API site as you wrote with some query parameters. But I could get nothing with a "Bad Request" error. What is wrong?

Anyway, I have no idea for getting "most recent data at the bottom of the page", sorry. Hope someone else can do.

[Yincognito]

Hi PtitChat107

I have accessed the API site as you wrote with some query parameters. But I could get nothing with a "Bad Request" error. What is wrong?

Anyway, I have no idea for getting "most recent data at the bottom of the page", sorry. Hope someone else can do.

Try the link posted by the OP (i.e. "value") and it will work - have no idea why.

[mak_kawa]

Hi Yincognito

Ah... ok. Possibly, &weight_unit=gr is a wrong query. Both &weight_unit=kg and &weight_unit=oz work.

[balala]

EDIT: Sorry read Yincognito's and mak_kawa's reply only after I posted mine. Sorry...

I have accessed the API site as you wrote with some query parameters. But I could get nothing with a "Bad Request" error. What is wrong?

Same for me, I also get only a "Bad request" error message. So PtitChat107, could you please check the link? There is something wrong with it.

[Yincognito]

Hi Yincognito

Ah... ok. Possibly, &weight_unit=gr is a wrong query. Both &weight_unit=kg and &weight_unit=oz work.

Yep, however it seems the data is somewhat "encrypted" if I could say so. For example, with this skin:

Code: Select all

[Variables]

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1
BackgroundMode=2
SolidColor=47,47,47,255

---Measures---

[MeasureSite]
Measure=WebParser
Url=https://or.fr/api/spot-prices?metal=XAU&currency=EUR&weight_unit=kg
;UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36
;UserAgent=Mozilla/5.0 (Android 10; Mobile; rv:68.0) Gecko/68.0 Firefox/79.0
RegExp=^(?siU).*?<spot_price>.*<date>.*\[ (.*) \].*<\/date>.*<weight_unit>.*\[ (.*) \].*<\/weight_unit>.*<ask>(.*)<\/ask>.*<mid>(.*)<\/mid>.*<bid>(.*)<\/bid>.*<value>(.*)<\/value>.*<performance>(.*)<\/performance>.*<\/spot_price>
;RegExp=^(.*)$
UpdateRate=600
FinishAction=[!UpdateMeasureGroup "SiteGroup"][!UpdateMeter *][!Redraw]
OnConnectErrorAction=[!UpdateMeasureGroup "SiteGroup"][!UpdateMeter *][!Redraw]
OnRegExpErrorAction=[!UpdateMeasureGroup "SiteGroup"][!UpdateMeter *][!Redraw]
DynamicVariables=1

[MeasureDate]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=1
UpdateDivider=-1

[MeasureUnit]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=2
UpdateDivider=-1

[MeasureAsk]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=3
UpdateDivider=-1

[MeasureMid]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=4
UpdateDivider=-1

[MeasureBid]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=5
UpdateDivider=-1

[MeasureValue]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=6
UpdateDivider=-1

[MeasurePerf]
Group=SiteGroup
;Disabled=1
Measure=WebParser
Url=[MeasureSite]
StringIndex=7
UpdateDivider=-1

---Meters---

[MeterPrecious]
Meter=String
X=0
Y=0
FontFace=Consolas
FontColor=255,255,255,255
SolidColor=47,47,47,255
Padding=5,5,5,5
FontSize=16
AntiAlias=1
MeasureName=MeasureDate
MeasureName2=MeasureUnit
MeasureName3=MeasureAsk
MeasureName4=MeasureMid
MeasureName5=MeasureBid
MeasureName6=MeasureValue
MeasureName7=MeasurePerf
Text="Date = %1#CRLF#Unit  = %2#CRLF#Ask   = %3#CRLF#Mid   = %4#CRLF#Bid   = %5#CRLF#Value = %6#CRLF#Perf  = %7"
DynamicVariables=1

the RegExp option fails, because when opening it in the browser (Google Chrome), I first get this:

Precious.jpg

and only after a couple of second (too many, IMHO) the actual XML:

Precious - XML.jpg

In that time, WebParser is probably executing its RegExp option and doesn't get anything, since it receives the data in the first screenshot.

If I set RegExp=^(.*)$ (not recommended), the data is captured, but the CPU goes to 50% (probably because of the length of the capture). If I set RegExp=^.*$ it's OK from a CPU usage point of view, but I get the data in the JSON format, instead of XML, as it can be seen in the Rainmeter Log:

Precious - JSON.jpg

It seems the site is performing some weird transformation of the data between formats at the time of parsing. Will investigate the issue further though.

[Yincognito]

So nobody could help me ?

Yep, finally got it, according to the observations in my previous post (this will get the last values, the key is the ? in the RegExp, which makes the * before it greedy, instead of ungreedy like set at the start of the RegExp option by the (?U) flag).

Code:

Code: Select all

[Variables]
CommonSubstitute='(?:^"|"$)':""

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1
BackgroundMode=2
SolidColor=47,47,47,255

---Measures---

[MeasureSite]
Measure=WebParser
Url=https://or.fr/api/spot-prices?metal=XAU&currency=EUR&weight_unit=kg
RegExp=(?siU).*?\{"date":(.*),"weight_unit":(.*),"ask":(.*),"mid":(.*),"bid":(.*),"value":(.*),"performance":(.*)\}
UpdateRate=600

[MeasureDate]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=1
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasureUnit]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=2
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasureAsk]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=3
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasureMid]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=4
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasureBid]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=5
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasureValue]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=6
RegExpSubstitute=1
Substitute=#CommonSubstitute#

[MeasurePerf]
Group=SiteGroup
Measure=WebParser
Url=[MeasureSite]
StringIndex=7
RegExpSubstitute=1
Substitute=#CommonSubstitute#

---Meters---

[MeterPrecious]
Meter=String
X=0
Y=0
FontFace=Consolas
FontColor=255,255,255,255
SolidColor=47,47,47,255
Padding=5,5,5,5
FontSize=16
AntiAlias=1
MeasureName=MeasureDate
MeasureName2=MeasureUnit
MeasureName3=MeasureAsk
MeasureName4=MeasureMid
MeasureName5=MeasureBid
MeasureName6=MeasureValue
MeasureName7=MeasurePerf
Text="Date  = %1#CRLF#Unit  = %2#CRLF#Ask   = %3#CRLF#Mid   = %4#CRLF#Bid   = %5#CRLF#Value = %6#CRLF#Perf  = %7"
DynamicVariables=1

Preview:

Precious - JSON - Parsed.jpg

The conversion between output formats of that site is quite strange though...

[mak_kawa]

Hi Yincognito

Nicely done! I was also thinking that greedy/ungreedy is the key, but my poor RegExp skill couldn't do that...

[Yincognito]

Hi Yincognito

Nicely done! I was also thinking that greedy/ungreedy is the key, but my poor RegExp skill couldn't do that...

Why not, it was just a question mark. RegExp is not that complicated, after all it just iterates character by character the string, and there are only a couple of commonly used "notations" in it. Things get a bit more complicated in the case of lookarounds or conditionals, but this was not the case.