It is currently April 16th, 2024, 2:19 pm

Read a text file into a meter?

Get help with creating, editing & fixing problems with skins
User avatar
Yincognito
Rainmeter Sage
Posts: 7118
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Read a text file into a meter?

Post by Yincognito »

CMDR_Evolution wrote: October 30th, 2020, 6:28 pm Hello and thank you for the great tool.

I've used it to create myself a log file reader but when the lines exceed the room available in the window, they can no longer be seen.

Is there a way to make the screen show the bottom of the log, rather than the top please?

Many thanks
Just use a Container meter to hide the "out of window" text, and some mouse action (e.g. scroll) to, well, scroll the text up and down to your liking by decreasing or increasing its Y coordinate, in LuaTextFile.ini or your INI file equivalent:

Code: Select all

[Rainmeter]
Author=Jeffrey Morley
Update=1000
DynamicWindowSize=1

[MeasureLuaScript]
Measure=Script
ScriptFile="#CURRENTPATH#LuaTextFile.lua"
FileToRead=#CURRENTPATH#Test.txt

[MeterContainerVisible]
Meter=Image
W=300
H=160
SolidColor=0,0,0,64

[MeterContainer]
Meter=Image
W=300
H=160
SolidColor=0,0,0,255
MouseScrollUpAction=[!SetOption MeterDisplay Y (Clamp([MeterDisplay:Y]+20,([MeterContainer:H]<[MeterDisplay:H]?[MeterContainer:H]-[MeterDisplay:H]:0),0))][!UpdateMeter MeterDisplay][!Redraw]
MouseScrollDownAction=[!SetOption MeterDisplay Y (Clamp([MeterDisplay:Y]-20,([MeterContainer:H]<[MeterDisplay:H]?[MeterContainer:H]-[MeterDisplay:H]:0),0))][!UpdateMeter MeterDisplay][!Redraw]

[MeterDisplay]
Container=MeterContainer
Meter=String
MeasureName=MeasureLuaScript
W=300
FontFace=Segoe UI
FontSize=11
FontColor=255,255,255,255
SolidColor=0,0,0,1
AntiAlias=1
ClipString=1
I intentionally made the container smaller in height than the text and added a visible replica of the former, so you can see what's going on. Also, added some limits on the Y value of the text (i.e. the Clamp() function in the formulas and some conditional for exception cases), so it will "stop" when it makes no sense to continue scrolling. Feel free to adapt this to your environment (e.g. change the size of the container and its visible replica, remove the limits if you deem it appropriate, or change the amount by which the text's Y changes from 20 to whatever you need).

This seems to work fine without dynamic variables, but you can add a DynamicVariables=1 line to [MeterContainer] if for some reason the Y values don't seem to change as expected.
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
jsmorley
Developer
Posts: 22629
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Read a text file into a meter?

Post by jsmorley »

An alternative to the Container approach is to "tail" the file in Lua. This is the kind of thing you will pretty much always want to do with something like a log file, that is appended sequentially from top to bottom, but you want to display in reverse chronological order from bottom to top. That's really how you always want to handle log files. They are always created by appending lines in chronological order, but you are always going to want to display them in reverse order. Newest to oldest...

I created a text file with 450,000 lines in it. It's something like 16 megabytes in size. So yeah, it's a GIANT log file...

Then:

Skin:

Code: Select all

[Rainmeter]
Update=1000
DynamicWindowSize=1
AccurateText=1
MouseScrollDownAction=[!Refresh]

[Metadata]
Name=TailFile
Author=JSMorley & Brian
Information=Example of using Lua to "tail" a number of lines from a text file like a log or to-do list.||Simply set [Variables] to control which file is read, the number of lines, the width of the skin, and the interval in seconds.
License=Creative Commons Attribution-Non-Commercial-Share Alike 3.0
Version=July 4, 2016

[Variables]
FileToRead=#@#TailFile.log
LinesToTail=5
SkinWidth=350
SecondsBetweenRead=5
ChildPrefix=MeasureLine

[MeasureTail]
Measure=Script
ScriptFile=TailFile.lua
UpdateDivider=#SecondsBetweenRead#

[MeasureLine1]
Measure=String
UpdateDivider=-1

[MeasureLine2]
Measure=String
UpdateDivider=-1

[MeasureLine3]
Measure=String
UpdateDivider=-1

[MeasureLine4]
Measure=String
UpdateDivider=-1

[MeasureLine5]
Measure=String
UpdateDivider=-1

[StyleText]
X=5
Y=5R
W=(#SkinWidth# - 20)
H=16
FontSize=11
Padding=5,5,5,5
ClipString=1
AntiAlias=1

[StyleOdd]
FontColor=255,255,255,255
SolidColor=40,50,60,255

[StyleEven]
FontColor=230,230,230,255
SolidColor=60,70,80,255

[MeterBack]
Meter=Image
W=(#SkinWidth#)
H=([MeterLine#LinesToTail#:Y]+[MeterLine#LinesToTail#:H]+5)
SolidColor=10,20,30,255
DynamicVariables=1
LeftMouseUpAction=#FileToRead#

[MeterHeaderLeft]
Meter=String
X=5
Y=5
FontSize=11
FontColor=255,255,255,255
StringStyle=Bold
AntiAlias=1
Text=TailFile
ToolTipWidth=#SkinWidth#
ToolTipTitle=TailFile
ToolTipText=#FileToRead#

[MeterHeaderRight]
Meter=String
X=(#SkinWidth# - 10)
Y=0r
FontSize=11
FontColor=255,255,255,255
StringAlign=Right
AntiAlias=1
Text=Latest #LinesToTail# lines

[MeterLine1]
Meter=String
MeasureName=MeasureLine1
MeterStyle=StyleText | StyleOdd
Y=10R

[MeterLine2]
Meter=String
MeasureName=MeasureLine2
MeterStyle=StyleText | StyleEven

[MeterLine3]
Meter=String
MeasureName=MeasureLine3
MeterStyle=StyleText | StyleOdd

[MeterLine4]
Meter=String
MeasureName=MeasureLine4
MeterStyle=StyleText | StyleEven

[MeterLine5]
Meter=String
MeasureName=MeasureLine5
MeterStyle=StyleText | StyleOdd
Lua:

Code: Select all

function Initialize()
      
	fileToRead = SKIN:GetVariable('FileToRead')
	linesToTail = tonumber(SKIN:GetVariable('LinesToTail'))
	childPrefix = SKIN:GetVariable('ChildPrefix')

end

function Update()
   
	local inputFile = assert(io.open(fileToRead, 'r'))
	local singleChar, lineText
	local bytePosition = -1
	local i = 1

	repeat
		inputFile:seek('end', bytePosition - 1)
		singleChar = inputFile:read(1)
		if singleChar == '\n' then
			lineText = inputFile:read(-bytePosition)
			if lineText then
				lineText = string.gsub(lineText, '\r', '')
				SKIN:Bang('!SetOption', childPrefix..i, 'String', string.match(lineText, '^(.-)\n'))
				SKIN:Bang('!UpdateMeasure', childPrefix..i)
				i = i + 1
				bytePosition = bytePosition - 1
			end
		end
		bytePosition = bytePosition - 1
	until (i > linesToTail)

	io.close(inputFile)

	SKIN:Bang('!UpdateMeter', '*')
	SKIN:Bang('!Redraw')

	return 1

end
1.jpg

TailFile_1.0.rmskin


This is insanely fast, as it uses a binary "seek" to find the end of the file, then starts reading backwards from there based on linefeeds. This means it never has to read the boatloads of text starting from the top of the file. This is the approach I would always use to "tail" a file, and is particularly useful if you are reading something like a log file that might get really large.

This reads and builds the text strings using "bytes" and not "characters", so it won't be flummoxed by Unicode in Lua.

Reading the entire 450,000 lines of text in and putting the entirety of them in a meter that you constrain with a Container would just be hideously slow, memory wasteful, and inefficient in my view. Nothing particularly wrong with that approach for a small or moderately sized log file, but I really think that in all cases this makes more sense.

Note that this uses an approach where I populate individual String measures with the value, allowing you to use Substitute on them if desired to tweak things, and individual String meters to display them, allowing you to format the strings as desired and also use Inline Setting if you want to highlight key words like "error" and such if you need to. Nothing says you can't just append the strings in Lua however, and display the entire thing in one meter if that is what you want.
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Rainmeter Sage
Posts: 7118
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Read a text file into a meter?

Post by Yincognito »

jsmorley wrote: October 31st, 2020, 11:43 am Reading the entire 450,000 lines of text in and putting the entirety of them in a meter that you constrain with a Container would just be hideously slow, memory wasteful, and inefficient in my view. Nothing particularly wrong with that approach for a small or moderately sized log file, but I really think that in all cases this makes more sense.
Sorry, jsmorley, but I believe it was you who placed the whole text in a meter in the first place - I only built based on your code this time, in order to answer a question on position (and not size)...

So, while I agree that the approach is inefficient for huge files, that was not a concern in the original questions. Also, if you want to talk about efficiency and all that, PowerShell's Get-Content will do the same as your Lua code... in a single line, if using the -tail parameter, not to mention it will work on files being currently written. Sure, launching PowerShell is a bit slow the first time it is run, but that's a different topic.

Seriously now, who will read a 450000 lines log file? Not even the Rainmeter log will get that big, but of course, this was useful to prove the point, so yeah...
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
jsmorley
Developer
Posts: 22629
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Read a text file into a meter?

Post by jsmorley »

Sure. My response was based on the fact that we are now talking about a "log" file. While 450,000 lines was a extreme example indeed, done to demonstrate the crazy fast and efficient nature of "seeking" to the end of the file and reading backwards, I think that in all cases when reading and displaying a log file, you want to reverse the order that it is at least displayed in. That is just the nature of a log file.

I have no problem at all with the PowerShell approach, although my thinking was to open up more flexibility in how the data is displayed. Having it all be in one meter is ok I guess, but does sorta limit the formatting and mouse action opportunities.

In any case, my post was just an "alternative". It's up to the user to decide what works best for them based on the circumstances.
User avatar
Yincognito
Rainmeter Sage
Posts: 7118
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Read a text file into a meter?

Post by Yincognito »

Indeed. By the way, wouldn't writing lines at the beginning of the log instead of at the end be possible, or help in parsing the resulting file? Just saying...
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
jsmorley
Developer
Posts: 22629
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Read a text file into a meter?

Post by jsmorley »

Yincognito wrote: October 31st, 2020, 1:41 pm Indeed. By the way, wouldn't writing lines at the beginning of the log instead of at the end be possible, or help in parsing the resulting file? Just saying...
I don't know of any log that is ever "written" that way. It's so much faster and more efficient to just "append" a line of text to the end of a text file than it is to read the entire thing in, write a line to the top, and save it all. Every log file I have ever seen writes by appending, and just depends on whatever tool it is that "reads" it to organize things as desired. It's sorta why the entire concept of "tail" was invented, presumably in Unix.
User avatar
Yincognito
Rainmeter Sage
Posts: 7118
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Read a text file into a meter?

Post by Yincognito »

One other thing, regarding your code: would you be able to add a "toggle" variable, so you can instruct the code to read either from start to end or from end to start? Just in case the "normal" reading direction is needed once in a while.
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
jsmorley
Developer
Posts: 22629
Joined: April 19th, 2009, 11:02 pm
Location: Fort Hunt, Virginia, USA

Re: Read a text file into a meter?

Post by jsmorley »

Yincognito wrote: October 31st, 2020, 1:48 pm One other thing, regarding your code: would you be able to add a "toggle" variable, so you can instruct the code to read either from start to end or from end to start? Just in case the "normal" reading direction is needed once in a while.
I'm not sure you would want to use the same approach to just read some amount of a file starting from the top. That would likely be a much simpler thing to do, in Lua, and perhaps just using WebParser or something. I mean, sure, that code could be modified to have a parameter passed to it and branch from "end" to "top" as desired. It just probably wouldn't make sense to use the binary "seek" and read in bytes instead of characters for that. Reading in XX number of "lines" from a text file in Lua is trivial.
User avatar
Yincognito
Rainmeter Sage
Posts: 7118
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Read a text file into a meter?

Post by Yincognito »

jsmorley wrote: October 31st, 2020, 1:52 pm I'm not sure you would want to use the same approach to just read some amount of a file starting from the top. That would likely be a much simpler thing to do, in Lua, and perhaps just using WebParser or something. I mean, sure, that code could be modified to have a parameter passed to it and branch from "end" to "top" as desired. It just probably wouldn't make sense to use the binary "seek" and read in bytes instead of characters for that.
Ah, I see - you're right. ;-)
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
CMDR_Evolution
Posts: 16
Joined: October 30th, 2020, 6:25 pm

Re: Read a text file into a meter?

Post by CMDR_Evolution »

Wow! You guys have been hot on the case. Thank you so, so much!

After seeing the first reply when I woke up, I went and did more digging and found the same thread you mentioned jsmorley, then have been adjusting that today :).
https://forum.rainmeter.net/viewtopic.php?p=108002#p108002

Sorry if I hadn't explained properly but that was exactly what I was after. Something to show the bottom of the file all the time. Having it at the top isn't a problem at all, thank you. The purpose is for a TTS readout (shown on a tablet acting as a monitor) from an application, which itself uses log files. With the new rainmeter file, it gives me a text version of what I hear through the speakers for any times when I might miss something.

I'm not sure if its relevant to ask the following question in this thread but if not, I'd be happy to move it over and if it's not super easy for you, please don't waste any time as I feel guilty for all your help already!
When the length of a line of code exceeds the window, is there a way to force that text onto a lower line please?

Here's what I have now. P.S sorry for how novice I am / sloppy coding. I'm no way near as clever as you guys.

Code: Select all

[Rainmeter]
Update=500
DynamicWindowSize=1
AccurateText=1
MouseScrollDownAction=[!Refresh]

[Variables]
FileToRead="C:\Users\Alan_\AppData\Roaming\EDDI\speechresponder.out"
LinesToTail=22
SkinWidth=1024
SecondsBetweenRead=5
ChildPrefix=MeasureLine

FontColor=000,175,255,240
SolidColor=000,000,000,255
SolidColor2=000,124,180,255
................................................................................................

[MeasureTail]
Measure=Script
ScriptFile=TailFile.lua
UpdateDivider=#SecondsBetweenRead#

[MeasureLine1]
Measure=String
UpdateDivider=-1
ClipString=1

[MeasureLine2]
Measure=String
UpdateDivider=-1
ClipString=1

[MeasureLine3]
Measure=String
UpdateDivider=-1

[MeasureLine4]
Measure=String
UpdateDivider=-1

[MeasureLine5]
Measure=String
UpdateDivider=-1

[MeasureLine6]
Measure=String
UpdateDivider=-1

[MeasureLine7]
Measure=String
UpdateDivider=-1

[MeasureLine8]
Measure=String
UpdateDivider=-1

[MeasureLine9]
Measure=String
UpdateDivider=-1

[MeasureLine10]
Measure=String
UpdateDivider=-1

[MeasureLine11]
Measure=String
UpdateDivider=-1

[MeasureLine12]
Measure=String
UpdateDivider=-1

[MeasureLine13]
Measure=String
UpdateDivider=-1

[MeasureLine14]
Measure=String
UpdateDivider=-1

[MeasureLine15]
Measure=String
UpdateDivider=-1

[MeasureLine16]
Measure=String
UpdateDivider=-1

[MeasureLine17]
Measure=String
UpdateDivider=-1

[MeasureLine18]
Measure=String
UpdateDivider=-1

[MeasureLine19]
Measure=String
UpdateDivider=-1

[MeasureLine20]
Measure=String
UpdateDivider=-1

[MeasureLine21]
Measure=String
UpdateDivider=-1

[MeasureLine22]
Measure=String
UpdateDivider=-1

................................................................................................

[MeterBarBackground]
Meter=Image
X=0
Y=r

;1 Hide Title Bar
[Bar Background1]
Meter=Image
X=([MeterBarBackground:X])
Y=([MeterBarBackground:Y])
H=600
W=1024
SolidColor=#SolidColor#

................................................................................................

;Border_Top
[Border_Top]
Meter=Image
X=([MeterBarBackground:X])
Y=([MeterBarBackground:Y]+38)
H=2
W=1024
SolidColor=#SolidColor2#

;Border_Left
[Border_Left]
Meter=Image
X=([MeterBarBackground:X])
Y=([MeterBarBackground:Y]+40)
H=558
W=2
SolidColor=#SolidColor2#

;Border_Right
[Border_Right]
Meter=Image
X=([MeterBarBackground:X]+1022)
Y=([MeterBarBackground:Y]+40)
H=558
W=2
SolidColor=#SolidColor2#

;Border_Bottom
[Border_Bottom]
Meter=Image
X=([MeterBarBackground:X])
Y=([MeterBarBackground:Y]+598)
H=2
W=1024
SolidColor=#SolidColor2#

................................................................................................

[StyleText]
X=8
Y=0R
W=1014
H=25
FontSize=13
ClipString=1
AntiAlias=1

[Style]
FontColor=#FontColor#
SolidColor=#SolidColor#
FontFace=#FontName#

[meterHeader]
Meter=String
X=([MeterBarBackground:X]+512)
Y=([MeterBarBackground:Y]+4)
W=1024
H=25
Text="Ships Computer Output"
FontFace=#FontName#
FontColor=#FontColor#
FontSize=18
StringStyle=Bold
StringAlign=Center
AntiAlias=1
LeftMouseUpAction=#FileToRead#

................................................................................................

[MeterLine1]
Meter=String
MeasureName=MeasureLine1
MeterStyle=StyleText | style
Y=43

[MeterLine2]
Meter=String
MeasureName=MeasureLine2
MeterStyle=StyleText | style

[MeterLine3]
Meter=String
MeasureName=MeasureLine3
MeterStyle=StyleText | style

[MeterLine4]
Meter=String
MeasureName=MeasureLine4
MeterStyle=StyleText | style

[MeterLine5]
Meter=String
MeasureName=MeasureLine5
MeterStyle=StyleText | style

[MeterLine6]
Meter=String
MeasureName=MeasureLine6
MeterStyle=StyleText | style

[MeterLine7]
Meter=String
MeasureName=MeasureLine7
MeterStyle=StyleText | style

[MeterLine8]
Meter=String
MeasureName=MeasureLine8
MeterStyle=StyleText | style

[MeterLine9]
Meter=String
MeasureName=MeasureLine9
MeterStyle=StyleText | style

[MeterLine10]
Meter=String
MeasureName=MeasureLine10
MeterStyle=StyleText | style

[MeterLine11]
Meter=String
MeasureName=MeasureLine11
MeterStyle=StyleText | style

[MeterLine12]
Meter=String
MeasureName=MeasureLine12
MeterStyle=StyleText | style

[MeterLine13]
Meter=String
MeasureName=MeasureLine13
MeterStyle=StyleText | style

[MeterLine14]
Meter=String
MeasureName=MeasureLine14
MeterStyle=StyleText | style

[MeterLine15]
Meter=String
MeasureName=MeasureLine15
MeterStyle=StyleText | style

[MeterLine16]
Meter=String
MeasureName=MeasureLine16
MeterStyle=StyleText | style

[MeterLine17]
Meter=String
MeasureName=MeasureLine17
MeterStyle=StyleText | style

[MeterLine18]
Meter=String
MeasureName=MeasureLine18
MeterStyle=StyleText | style

[MeterLine19]
Meter=String
MeasureName=MeasureLine19
MeterStyle=StyleText | style

[MeterLine20]
Meter=String
MeasureName=MeasureLine20
MeterStyle=StyleText | style

[MeterLine21]
Meter=String
MeasureName=MeasureLine21
MeterStyle=StyleText | style

[MeterLine22]
Meter=String
MeasureName=MeasureLine22
MeterStyle=StyleText | style