It is currently December 12th, 2018, 4:07 am

[Solved] Pulling multiple values from a string

Discuss the use of Lua in Script measures.
User avatar
minusInfinite
Posts: 5
Joined: June 21st, 2018, 5:02 am
Location: Land Down Under

[Solved] Pulling multiple values from a string

minusInfinite » July 12th, 2018, 1:54 am

Hi All

I repurposed a skin to track a new ISP data usage (yeah capped data plans still exist) which currently uses a VBscript for a lot of the math work and I've been looking at ways to move it to a lua script.

Give it's my first time even looking at lua outside of some tinkering with WoW addons I'm looking for some suggestions on how to get the data I need to out of the ISPs API call.

The current version of the skin is here - https://github.com/minusInfinite/NuSkopeUsage-Rainmeter

I've attached the current rewrite and have been able to get some basic functions working in just Rainmeter alone.

I'm looking for some guidance on how to do the following.

When the skin is first installed I will need to prompt the user to give their API Key
From some testing I've done with the InputText plugin I should be able to call a skin to capture the API key, change a variable that the api key has been provided and switch the loaded skin file on update?

Change the date format
The API provided is in JSON format. While I have been able to get the Regexp to work and get the data I need I'm not sure how to convert the LastReset entry from YYYY-MM-DD to DD-MM-YYYY or event separate the string to a day and month value to I can use it get like days to next reset, average usage per day and so forth

Table Daily Usage into Monthly Usage
The API also keeps a record of about two months of usage history I've been attempting to read through the documentation and find some way (likely via Lua) of extracting all the those UploadsGB and DownloadGB fields and somehow make a monthly usage bar graph.

I know it's a lot to ask. I'm not looking for someone to build the scripts/skin for me (I'm having a lot of fun doing this rewrite) but suggestions of how to get a few of the addition options would be wonderful.
You do not have the required permissions to view the files attached to this post.
Last edited by minusInfinite on August 1st, 2018, 8:07 am, edited 1 time in total.
User avatar
balala
Rainmeter Sage
Posts: 7160
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Pulling multiple values from a string

balala » July 12th, 2018, 5:16 pm

minusInfinite wrote:When the skin is first installed I will need to prompt the user to give their API Key
From some testing I've done with the InputText plugin I should be able to call a skin to capture the API key, change a variable that the api key has been provided and switch the loaded skin file on update?
Your code has no InputText plugin measure, which would be needed to enter that API Key. This should have to be added. It could be, but what I'm not sure, how you would like to store the inputted data (or which variable you would like to use for this).
minusInfinite wrote:Change the date format
The API provided is in JSON format. While I have been able to get the Regexp to work and get the data I need I'm not sure how to convert the LastReset entry from YYYY-MM-DD to DD-MM-YYYY or event separate the string to a day and month value to I can use it get like days to next reset, average usage per day and so forth
Add the following options to the [MeasureResetDate] measure (this measure returns the date which you'd like to see into another format, right?):

Code: Select all

[MeasureResetDate]
...
RegExpSubstitute=1
Substitute='^"(\d{4})-(\d{2})-(\d{2})"$':"\3-\2-\1"
minusInfinite wrote:Table Daily Usage into Monthly Usage
The API also keeps a record of about two months of usage history I've been attempting to read through the documentation and find some way (likely via Lua) of extracting all the those UploadsGB and DownloadGB fields and somehow make a monthly usage bar graph.
Did you succeed extracting those data? Where are they?
User avatar
minusInfinite
Posts: 5
Joined: June 21st, 2018, 5:02 am
Location: Land Down Under

Re: Pulling multiple values from a string

minusInfinite » July 13th, 2018, 2:04 am

balala wrote:Your code has no InputText plugin measure, which would be needed to enter that API Key. This should have to be added. It could be, but what I'm not sure, how you would like to store the inputted data (or which variable you would like to use for this).
It was more an suggestion for how to operate it either in the same config or in separate config. I have tested some of the functions such as setting the options via bangs but not loading and unloading configs. I haven't completed setting it up yet. The end result should be below. I might end up making a setting file to include and update as needed.

Code: Select all

[Variables]
defaultInput = ' '
haveKey = 1
userKey = '**APIKEY**'

[MeasureAPI]
Measure = WebParser
URL = https://api.nuskope.com.au/usage/?Token=#userKey#
balala wrote: Add the following options to the [MeasureResetDate] measure (this measure returns the date which you'd like to see into another format, right?):

Code: Select all

[MeasureResetDate]
...
RegExpSubstitute=1
Substitute='^"(\d{4})-(\d{2})-(\d{2})"$':"\3-\2-\1"
Yeah - That works as in terms or format. Will have to figure out hour to use it for any formulas or break it down to a day,month,year stings.
balala wrote: Did you succeed extracting those data? Where are they?
I was able to get out the data for the most recent day of use via using RegExp which is only the json block below

Code: Select all

{
    "2018-07-11": {
        "UploadsGB": "0.1558",
        "DownloadsGB": "2.4264",
        "Date": "2018-07-11"
    },
But similar blocks to this is repeated about 90 times spanning about 2 months of daily date use. I could look into using RegExp lookahead assertion but all I need is the Value of the UploadsGB and DownloadGB summed up over a month period somehow. I'm not sure how to separate the values I need and how to relate them to the months their applicable.
User avatar
balala
Rainmeter Sage
Posts: 7160
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Pulling multiple values from a string

balala » July 13th, 2018, 7:51 am

Let's go step by step. Let's go step by step. For the time being, just the first two questions:
minusInfinite wrote:It was more an suggestion for how to operate it either in the same config or in separate config. I have tested some of the functions such as setting the options via bangs but not loading and unloading configs. I haven't completed setting it up yet. The end result should be below. I might end up making a setting file to include and update as needed.
Here is a first solution. Maybe not the most elegant, but I think it works and can be refined, if you want. I added a String meter ([MeterInput]), a String measure ([MeasureAPIKeyString]) and InputText plugin measure ([MeasureAPIKey]).
The plugin measure is used to input the variable. The [MeterInput] String meter prompts you where to input the data, with a click. This input possibility is available only if the key isn't set yet. The check is done by the [MeasureAPIKeyString] String measure, using an IfMatch. If the UserKey variable is empty, this measure shows the [MeterInput] meter, which can be clicked to enable the InputText plugin (to can add the variable), otherwise the [MeterInput] meter is hidden. This means that you can enter the variable only once. After this the code uses the previously added variable.
One single problem is that if you later want to change the variable, will have to do this manually: have to remove from the code the previously added variable, refresh the skin and input the value.
minusInfinite wrote:Yeah - That works as in terms or format. Will have to figure out hour to use it for any formulas or break it down to a day,month,year stings.
For example the following measure returns only the year:

Code: Select all

[MeasureYear]
Measure=String
String=[MeasureResetDate]
RegExpSubstitute=1
Substitute='^"(\d{4})-(\d{2})-(\d{2})"$':"\1"
DynamicVariables=1
The Substitute option will work only if you previously removed the Substitute='^"(\d{4})-(\d{2})-(\d{2})"$':"\3-\2-\1" (along with RegExpSubstitute=1) option from the [MeasureResetDate] measure. If you keep those options too, you have to reorder the substitutions. Let me know if you can't figure out how.
User avatar
minusInfinite
Posts: 5
Joined: June 21st, 2018, 5:02 am
Location: Land Down Under

Re: Pulling multiple values from a string

minusInfinite » July 23rd, 2018, 11:39 am

Thanks for the assistance balala

I've managed to get the skin working the way I was hoping to.

If you would like to take a look and have any ideas on improvements.

I've attached an APItext dump as you will likely not have an API key to use :P
You do not have the required permissions to view the files attached to this post.
User avatar
kyriakos876
Posts: 871
Joined: January 30th, 2017, 2:01 am
Location: Greece

Re: Pulling multiple values from a string

kyriakos876 » July 23rd, 2018, 12:26 pm

*just leaving a comment so I can't find this later*
User avatar
balala
Rainmeter Sage
Posts: 7160
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Pulling multiple values from a string

balala » July 23rd, 2018, 7:09 pm

I'm glad you got it working.
minusInfinite wrote:If you would like to take a look and have any ideas on improvements.
I'm sorry, right now I won't, because I just spend my holiday, but if in the meantime no one takes a look, next week I promise I will.
User avatar
minusInfinite
Posts: 5
Joined: June 21st, 2018, 5:02 am
Location: Land Down Under

Re: Pulling multiple values from a string

minusInfinite » July 24th, 2018, 2:35 am

balala wrote:I'm glad you got it working.

I'm sorry, right now I won't, because I just spend my holiday, but if in the meantime no one takes a look, next week I promise I will.
No Rush

Now that it's functional all I'm likely to do now is visual tweeks on the size and display of the meters and getting the skins to switch out correctly if the API key has not been provided.
User avatar
balala
Rainmeter Sage
Posts: 7160
Joined: October 11th, 2010, 6:27 pm
Location: Gheorgheni, Romania

Re: Pulling multiple values from a string

balala » July 31st, 2018, 5:54 pm

minusInfinite wrote:If you would like to take a look and have any ideas on improvements.
As I promised, here are my advices. Using the posted apidump.txt, I noticed the followings:
  • The calc.lua script gives an error message in the log, because the tData variable isn't initialized. To initialize it, add the following three line bellow the tData = {} line (Update() function):

    Code: Select all

    	tData = {}
    	for i = math.min(d0Sum*3,d1Sum*3,d2Sum*3,d3Sum*3,d4Sum*3),math.max(d0Sum*3,d1Sum*3,d2Sum*3,d3Sum*3,d4Sum*3) do
    		tData[i] = 0
    	end
    I'm not sure this is entirely correct, it need to be tested furtherly on the online content, but I hope it is. Please come back if I'm mistaken.
  • Two measures of the skin's code give two Calc: Division by 0 error message on refresh. This is happening because when a skin is refreshed, the values of all measures are 0 and when a measure is calculated as division by another measure, because you can't divide with zero, the error occurs. There is an extremely simple workaround, to fix this: you simply have to add a very small, but non-zero number to the divider. This way the error message is avoided, but the result isn't affected (just have to make sure the added value is small enough to not affect the result. That's why I modified the Formula options of the [QuotaRemainingPerDay] and [TotalUsagePct] measures, as it follows:

    Code: Select all

    [QuotaRemainingPerDay]
    ...
    Formula=Round(QuotaRemaining/(0.0001+DaysRemaining)+1,0)
    
    ...
    
    [TotalUsagePct]
    ...
    Formula=Round((TotalUsageCalc/(0.0001+Round(MeasurePlanQuota,0))) * 100,0)
    The added value (0.0001) is small enough to not modify the result, but large enough to avoid the error message.
  • There is one more warning message if the apiKey\APIKeyInput.ini skin is loaded when you're refreshing the skin. This message is given by the IfTrueAction option of the [APICheck] measure. To avoid this message, you should have to add a measure which can check if the skin is loaded and act in consequence. If you're interested how to deal with this, please let me know.
I think this is all.
User avatar
minusInfinite
Posts: 5
Joined: June 21st, 2018, 5:02 am
Location: Land Down Under

Re: Pulling multiple values from a string

minusInfinite » August 1st, 2018, 2:42 am

balala wrote:As I promised, here are my advices. Using the posted apidump.txt, I noticed the followings:
Thanks for sparing the time. I have made a number of small changes that I've posted to the project GitHub that have improved it. The major thing I had issues with visually is being able to see the Webparser measure update throughout a day. I've only recently understood that while it will connect and scan the data source it won't call other commands unless it has detected differences from what was already downloaded. Unless ForceReload=1 that is. As well how it uses UpdateRate.

Still not sure if the FinishAction option only completes one or on every UpdateRate? Or if I would need to have a timer to run the script manually again.
balala wrote:
  • The calc.lua script gives an error message in the log, because the tData variable isn't initialized. To initialize it, add the following three line bellow the tData = {} line (Update() function):

    Code: Select all

    	tData = {}
    	for i = math.min(d0Sum*3,d1Sum*3,d2Sum*3,d3Sum*3,d4Sum*3),math.max(d0Sum*3,d1Sum*3,d2Sum*3,d3Sum*3,d4Sum*3) do
    		tData[i] = 0
    	end
    I'm not sure this is entirely correct, it need to be tested furtherly on the online content, but I hope it is. Please come back if I'm mistaken.
I'll have to make sure on what I uploaded as this shouldn't be the case. The JSON objects should be as follows (This is formated. The API provides it unformatted)

Code: Select all

{
    "PlanName": "NuSkope Residential Fixed Wireless 800GB", //Index1
    "QuotaMetering": "Download And Upload", //Index2
    "SplitData": false,
    "LastReset": "2018-07-27", //Index3 Split into three separate measures
    "PlanQuotaGB": 800, //Index4
    "UploadsGB": "0.3523",  //Index5
    "DownloadsGB": "5.0123", //Index6
    "DailyUsage": { 
        "2018-07-28": { //Index7 
            "UploadsGB": "0.0047", //Index7//Index1
            "DownloadsGB": "0.0568", //Index7//Index1
            "Date": "2018-07-28"
        },
        "2018-07-27": {
            "UploadsGB": "0.3475",
            "DownloadsGB": "4.9556",
            "Date": "2018-07-27"
        },... (goes for another 300+ lines)
I've added some notation of that the Webparser RegExp should be collecting. I've used the StringIndex2 Option to get the most recent Daily Usage. In Lua, jData is the JSON object data block of Index 7 and as I only need the Values of UploadsGB, Download GB and Date I had to edit the string into something distinguishable. This method is below.

Code: Select all

    jData = mhData:GetStringValue()
    tData = {}

    editStr = string.gsub(jData, '%"%:%"', " %\=% " )

    for k,v in string.gmatch(editStr, "(%w+)%s*=%s*(%d*.%d*)") do
        table.insert(tData,v)
    end

This should initialise the table as an indexed array. With a index of 1 with a data pair of UploadsGB, DownloadsGB and a Date in YYYY-MM format. As I know every date value will be 3 key steps in a table the d*Sum variables multiplied by 3 allow me to collate the Downloads and Uploads used per a month cycle. Using the below code. I'm just surprised it lines up as well as it does from the data set I have

Code: Select all

    for i  = d1Sum*3,d2Sum*3, 3 do
        local uploads = 0
        local downloads = 0 
        uploads = tonumber(uploads) + tonumber(tData[i-2])
        downloads = tonumber(downloads) + tonumber(tData[i-1])
       p1Usage = p1Usage + (uploads + downloads)
    end
    
    SKIN:Bang('!SetOption', 'p1Usage', 'String', p1Usage)

    for i  = d2Sum*3,d3Sum*3, 3 do
        local uploads = 0
        local downloads = 0 
        uploads = tonumber(uploads) + tonumber(tData[i-2])
        downloads = tonumber(downloads) + tonumber(tData[i-1])
        p2Usage = p2Usage + (uploads + downloads)
    end
balala wrote: [*]Two measures of the skin's code give two Calc: Division by 0 error message on refresh. This is happening because when a skin is refreshed, the values of all measures are 0 and when a measure is calculated as division by another measure, because you can't divide with zero, the error occurs. There is an extremely simple workaround, to fix this: you simply have to add a very small, but non-zero number to the divider. This way the error message is avoided, but the result isn't affected (just have to make sure the added value is small enough to not affect the result. That's why I modified the Formula options of the [QuotaRemainingPerDay] and [TotalUsagePct] measures, as it follows:

Code: Select all

[QuotaRemainingPerDay]
...
Formula=Round(QuotaRemaining/(0.0001+DaysRemaining)+1,0)

...

[TotalUsagePct]
...
Formula=Round((TotalUsageCalc/(0.0001+Round(MeasurePlanQuota,0))) * 100,0)
The added value (0.0001) is small enough to not modify the result, but large enough to avoid the error message.
I've found that disabling all the calc measures and using the FinishAction option on the Webparser measure to enable them has eliminated these errors in the log until the script does it's thing and provides values to calculate. Saw this method in another skin I use and found it nifty.
balala wrote: [*]There is one more warning message if the apiKey\APIKeyInput.ini skin is loaded when you're refreshing the skin. This message is given by the IfTrueAction option of the [APICheck] measure. To avoid this message, you should have to add a measure which can check if the skin is loaded and act in consequence. If you're interested how to deal with this, please let me know.[/list]
I think this is all.
At this time I'm using a the haveKey variable but if there is a better way I'm keen to know :)