It is currently March 28th, 2024, 10:11 pm

Is there a way to get info from the code returned by Inspect Element rather than view source?

Get help with creating, editing & fixing problems with skins
User avatar
Harkeerat16
Posts: 15
Joined: June 10th, 2017, 10:54 am
Location: India

Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by Harkeerat16 »

So, basically, I've been using this skin called Hubble Viewer 0.2 (https://www.deviantart.com/commiesloth/art/Hubble-Viewer-0-2-739602907) with no issues for the past two years. What it does is get the picture and name of whatever Hubble is looking at right now from the site https://spacetelescopelive.org/ and display it.
(It used the URL https://spacetelescopelive.org/images, but that's down now so I've been trying to use https://spacetelescopelive.org/ ))

It stopped working when the Hubble telescope was down recently but it's back online now and the website is showing the name and picture, but the skin isn't. I checked out the code and apparently, the source code has all changed from what it used to be. If you check out the source code from an earlier time of Wayback Machine (https://web.archive.org/web/20210213230531/https://spacetelescopelive.org/) it seems quite easy to extract the data from that HTML code. But the code on the current [url]view-source:https://spacetelescopelive.org/[/url] isn't making much sense to me.
Except when you use Inspect Element, it starts to make sense again and looks extractable. So is there a way to extract the code from what is being returned by Inspect Element? Thanks!
User avatar
Yincognito
Rainmeter Sage
Posts: 7029
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by Yincognito »

Harkeerat16 wrote: July 28th, 2021, 10:52 pm So, basically, I've been using this skin called Hubble Viewer 0.2 (https://www.deviantart.com/commiesloth/art/Hubble-Viewer-0-2-739602907) with no issues for the past two years. What it does is get the picture and name of whatever Hubble is looking at right now from the site https://spacetelescopelive.org/ and display it.
(It used the URL https://spacetelescopelive.org/images, but that's down now so I've been trying to use https://spacetelescopelive.org/ ))

It stopped working when the Hubble telescope was down recently but it's back online now and the website is showing the name and picture, but the skin isn't. I checked out the code and apparently, the source code has all changed from what it used to be. If you check out the source code from an earlier time of Wayback Machine (https://web.archive.org/web/20210213230531/https://spacetelescopelive.org/) it seems quite easy to extract the data from that HTML code. But the code on the current [url]view-source:https://spacetelescopelive.org/[/url] isn't making much sense to me.
Except when you use Inspect Element, it starts to make sense again and looks extractable. So is there a way to extract the code from what is being returned by Inspect Element? Thanks!
For me, https://spacetelescopelive.org/ currently displays:
NA.jpg
so it's clear that it doesn't have (yet) any image or text to show up, and why your skin doesn't produce any significant output as well. Unless they hide their data behind Javascript or such (and even then, you'd stil have the HTML to get what it's displayed), View Page Source (I didn't use Inspect Element for this, but I guess it's more or less the same thing) should be able to provide you with a source text to base your regex patterns on.

EDIT: Just checked it out again after Hubble got "online" and my suspicion was correct: the HTML now gets the displayed values through some Javascript functions, meaning that both the image and the text aren't transparently available in the HTML page source. The image link is, for example, something like:

Code: Select all

https://archive.stsci.edu/cgi-bin/dss_search?v=poss2ukstu_red&r=311.29&d=31.002&e=J2000&h=10&w=10&f=gif&c=none&fov=NONE&v3=
yielding:
Image
so you'd need to interrogate / query the API at https://archive.stsci.edu/ to get the same image. If the API is free you could do that, but you'd have to know all those query parameters (which I have the feeling that the JS at https://spacetelescopelive.org/ is computing). Bottom line, you can check the DSS search form at https://archive.stsci.edu/cgi-bin/dss_form/ because I think this is where the images or coordinates come from. Unfortunately I'm not an expert in figuring out what those fields mean, but you can try for yourself and see what you get.
You do not have the required permissions to view the files attached to this post.
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
death.crafter
Rainmeter Sage
Posts: 1399
Joined: April 24th, 2021, 8:13 pm

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by death.crafter »

The site you should refer to is: https://api.spacetelescopelive.org/observation_timelines/latest

So the description for image and text would be something like:

Code: Select all

[MeasureSite]
Measure=WebParser
URL=https://api.spacetelescopelive.org/observation_timelines/latest
RegExp=(?siU)"what_am_i_looking_at":"(.*)"
UpdateRate=3600

[MeasureSiteForPicture]
Measure=WebParser
URL=https://api.spacetelescopelive.org/observation_timelines/latest
RegExp=(?siU)"reference_image_url":"(.*)"
UpdateRate=3600

[MeasurePicture]
Measure=WebParser
URL=[MeasureSiteForPicture]
StringIndex=1
Download=1
UpdateRate=1500

[MeasureDescription]
Measure=WebParser
URL=[MeasureSite]
StringIndex=1
That's it for now. The json is returning {"what_am_i_looking_at":"Hubble is acquiring a new target"} for the time being. I was only able to see the image pattern before that.

You can check for rest of the data when you are lucky and Hubble is on a target.

P.S.: Skin is kinda old. May be try to remake it. Would be nice.

If it comes back online in meantime, I may provide the full working code.

Regards,
death.crafter
from the Realm of Death
User avatar
Yincognito
Rainmeter Sage
Posts: 7029
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by Yincognito »

death.crafter wrote: July 29th, 2021, 8:13 pmThe site you should refer to is: https://api.spacetelescopelive.org/observation_timelines/latest
Nice find - didn't realize they had an API at spacetelescopelive.org as well (though still referencing archive.stsci.edu from what I see). :thumbup:
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
death.crafter
Rainmeter Sage
Posts: 1399
Joined: April 24th, 2021, 8:13 pm

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by death.crafter »

Here is a full working code. I changed it a lot to match the current implementation:

Code: Select all

;Hubble Viewer 0.2 by CommieSloth
[Rainmeter]
Update=1000

[Metadata]
Name=Hubble Viewer
Author=CommieSloth
Information=In Version 0.2 the buttons, coordinates and apparent magnitude aren't yet functional. The image and details come from http://spacetelescopelive.org/images/, and update every hour. If you have any problems or suggestions please contact me at https://commiesloth.deviantart.com/
License=
Version=0.2

[Variables]
Enlargement=1.0

proposalID = ""proposal_id":"(.*)""
beginAt = ""begin_at":"(.*)""
category = ""category":"(.*)""
endAt = "end_at":"(.*)"
investigatorName = ""principal_investigator_full_name":"(.*)""
target = ""target_name":"(.*)""
scienceInstrument = ""science_instrument":"(.*)""
scienceInstrumentShort = ""science_instrument_acronym":"(.*)""
state = ""state":"(.*)""
whatAmILookingAt = ""what_am_i_looking_at":"(.*)""

[MeasureSite]
Measure=WebParser
URL=https://api.spacetelescopelive.org/observation_timelines/latest
;add or remove info frombelow, using the variables above | visit the site above to know about the info provided by the API
RegExp=(?siU).*#investigatorName#.*#target#.*#scienceInstrumentShort#
UpdateRate=3600

[MeasureSiteForPicture]
Measure=WebParser
URL=https://api.spacetelescopelive.org/observation_timelines/latest
RegExp=(?siU)"reference_image_url":"(.*)"
UpdateRate=3600

[MeasurePicture]
Measure=WebParser
URL=[MeasureSiteForPicture]
DecodeCodePoints=1
Download=1
StringIndex=1
DownloadFile=hubbleimage.gif

[MeasureName]
Measure=WebParser
URL=[MeasureSite]
StringIndex=1

[MeasureCelestialBody]
Measure=WebParser
URL=[MeasureSite]
StringIndex=2

[MeasureCamera]
Measure=WebParser
URL=[MeasureSite]
StringIndex=3

;[MeasureDescription]
;Measure=WebParser
;URL=[MeasureSite]
;StringIndex=1

[MeterBackground]
Meter=IMAGE
X=(11*#Enlargement#)
Y=(57*#Enlargement#)
H=(406*#Enlargement#)
W=(406*#Enlargement#)
ImageName=glitch.jpg

[MeterHubble]
Meter=IMAGE
MeasureName=MeasurePicture
X=(11*#Enlargement#)
Y=(57*#Enlargement#)
H=(406*#Enlargement#)
W=(406*#Enlargement#)

[MeterWindow]
Meter=IMAGE
X=0
Y=0
H=(561*#Enlargement#)
W=(426*#Enlargement#)
ImageName=hubble.png

[MeterCelestialBody]
Meter=String
MeasureName=MeasureCelestialBody
X=(199*#Enlargement#)
Y=(499*#Enlargement#)
FontSize=(8*#Enlargement#)
FontWeight=1
FontColor=0,0,0,255
StringAlign=Left
AntiAlias=1

[MeterName]
Meter=String
MeasureName=MeasureName
X=r
Y=R
FontSize=(7*#Enlargement#)
FontWeight=1
FontColor=0,0,0,255
StringAlign=Left
Text=by %1
AntiAlias=1

[MeterCamera]
Meter=String
MeasureName=MeasureCamera
X=(176*#Enlargement#)
Y=(525*#Enlargement#)
FontSize=(8*#Enlargement#)
FontWeight=1
FontColor=0,0,0,255
StringAlign=Left
AntiAlias=1
Screenshot 2021-07-30 030533.png
You can do extra modifications for error handling and stuff.

Regards,
death.crafter
You do not have the required permissions to view the files attached to this post.
from the Realm of Death
User avatar
death.crafter
Rainmeter Sage
Posts: 1399
Joined: April 24th, 2021, 8:13 pm

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by death.crafter »

Yincognito wrote: July 29th, 2021, 8:30 pm Nice find - didn't realize they had an API at spacetelescopelive.org as well (though still referencing archive.stsci.edu from what I see). :thumbup:
Screenshot 2021-07-30 031037.png
MrBeanWinkWinkGIF.gif
You do not have the required permissions to view the files attached to this post.
from the Realm of Death
User avatar
Yincognito
Rainmeter Sage
Posts: 7029
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Is there a way to get info from the code returned by Inspect Element rather than view source?

Post by Yincognito »

death.crafter wrote: July 29th, 2021, 9:43 pm Screenshot 2021-07-30 031037.png

MrBeanWinkWinkGIF.gif
Image
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth