emp00 wrote: ↑January 28th, 2024, 11:41 am
Got it - actually, I would be very willing to use your MyPortfolio skin, but I got scared due to our
discussions around the Market Prices meter last week. You and two other users mentioned repeatedly:
"I've had my IP banned in the past when repeatedly making quick changes and then refreshing the skin. To avoid getting blocked I used a dummy skin ...". That's why I made up my mind to rather use another webservice which already does the portfolio calculations server-side so that I don't need to poll every single stock value one-by-one from cnbc.com. My idea was to use one Webparser call on the calculated portfolio table from wallstreet-online.de, but it requires login/authentification which Rainmeter seems to struggling with, see above.
Facts are: I already have the Market Prices skin running with
6 stocks (polled every 10mins) and would like to use MyPortfolio with another 15 stocks. That
totals 21 stocks to be polled from cnbc.com every 10 mins.
Those 21 calls are executed one-after-another as per design and due to your warnings I was 100% sure this would get my IP banned right away... I mean for real-life parallel usage of your two skins both with
UpdateRate=600, not for frequent refreshing/testing.
Some quick general observations (I'm on my phone right now, so can't test yet):
- if the manual says that HTTP Authentication is supported by Rainmeter, then most likely it's the site that doesn't support it
- if cnbc.com can display more than one stock at a time, then most likely you can grab multiple stocks at a time from it with Rainmeter
- sometimes investigating the site page with Developer Tools / Inspect / Network tab can reveal the true address the site itself takes the data from, and if it uses an URL query / API in the background there might be a way to select the desired stocks to grab directly in one go, just by manipulating the query's parameters; similarly, looking for other attempts on GitHub can reveal the same, e.g. (that address, once concatenated, does produce results in the specified format, test it in the browser and you'll see):
https://github.com/madnight/cnbc-market/blob/master/index.js
- if the data updating interval on cnbc.com is large enough, you could wait for a bit before querying the next item in the skin, via UpdateDivider and other techniques (assuming the site allows the amount of polling per day that you need, of course), thus avoiding high frequency polling that might get you blocked or banned; alternatively, you could display and query just one stock at a time on demand, via some click or scroll triggering, if you don't absolutely need all 21 stocks displayed at once in the skin
- there might be other sites that can provide what you're looking for in a still free and feasible fashion, so if you find another 20 of them, problem solved (I know, a bit far fetched, but just saying)
There might be other possible considerations to be mentioned here, but although a bit difficult in this area since typically money information costs money, you might find a reasonable way to get the desired data, preferably in one go, or in separate attempts at either reasonable intervals or from different sites. Obviously, care should be taken in the testing phase to not abuse polling in the skin - because otherwise, testing things directly in the browser shouldn't produce any drawbacks (you can save the page source from the browser too, then use the resulted local file to polish your regex or whatever other details without worrying about being banned).
