It is currently March 9th, 2021, 9:30 am

[Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Report bugs with the Rainmeter application and suggest features.
Copteur
Posts: 1
Joined: January 31st, 2021, 2:47 am

[Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Copteur »

I'm writing a skin to monitor each CPU core on a dual AMD EPYC 7742 64-Core Processor. This system has 256 logical cores. I have found that as soon as I attempt to measure a CPU above 64, the usual CPU Measure throws an error in the log "CPU: Processor=65 is not valid" and while UsageMonitor doesn't throw any errors I can see, it simply does not work for Name=>64. OS here is WinServer 2019. I've tried Rain 4.3 and 4.4 r3412.

I assume CPU Measure is a known limitation given the error. Is the issue with the UsageMonitor also a known bug/limitation? Can this be fixed soon? Is there a workaround without using say the HWInfo plugin (haven't tried it yet so don't know if works but would prefer not have to install this tool)?

While the above processor is a server-class device, you can already buy AMD Threadripper Processors with 64 logical cores. Aside from existing servers, I'm pretty sure there will be PC/Workstation systems in the wild at some point soon with >64 logical cores.

Here's a simple skin that shows the issue using both measure methods:

Code: Select all

[Rainmeter]
Update=1000
BackgroundMode=2
SolidColor=0,0,0,255

[MeasureCPU]
Measure=CPU
Processor=65

[MeasureCPUUsageMonitor]
Measure=Plugin
Plugin=UsageMonitor
Category=Processor
Counter=% Processor Time
Name=65
MinValue=0
MaxValue=100

[MeterText]
Meter=String
MeasureName=MeasureCPU
X=5
Y=5
W=180
H=35
FontColor=255,255,255,255
NumOfDecimals=1
Text="Core CPUMeasure %1%#CRLF#Core UsageMonitor [MeasureCPUUsageMonitor] : [MeasureCPUUsageMonitor:1]%"
DynamicVariables=1
Bekarfel
Posts: 198
Joined: May 16th, 2012, 5:38 am

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Bekarfel »

Sounds like a candidate for the zero one infinity rule
moshi wrote:there are many Rainmeter skins that aren't really useful, so let's add another one.
jsmorley wrote:I have good news and bad news.
First the bad news. [...] We would be happy to have this happen and would love to work with anyone who is feeling ambitious.
Now the good news.
I lied, there isn't any good news...
User avatar
Brian
Developer
Posts: 2089
Joined: November 24th, 2011, 1:42 am
Location: Utah

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Brian »

It's hard to tell where the problems lies. Rainmeter does not explicitly restrict the number of logical CPU's that can be referenced (both in UsageMonitor or the CPU measure). In both cases, Rainmeter asks Windows (via the winapi) for that information. There is nothing in the MS docs that suggest there is any limitation. The MS docs do mention logical processor "relation groups", however it is unclear if this has anything to do the problem at hand.

Ultimately, this is going to be difficult to solve unless one of the team members has access to a machine like this.

-Brian
User avatar
Yincognito
Posts: 3017
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Yincognito »

Copteur wrote: January 31st, 2021, 3:31 am I'm writing a skin to monitor each CPU core on a dual AMD EPYC 7742 64-Core Processor. This system has 256 logical cores. I have found that as soon as I attempt to measure a CPU above 64, the usual CPU Measure throws an error in the log "CPU: Processor=65 is not valid" and while UsageMonitor doesn't throw any errors I can see, it simply does not work for Name=>64. OS here is WinServer 2019. I've tried Rain 4.3 and 4.4 r3412.

I assume CPU Measure is a known limitation given the error. Is the issue with the UsageMonitor also a known bug/limitation? Can this be fixed soon? Is there a workaround without using say the HWInfo plugin (haven't tried it yet so don't know if works but would prefer not have to install this tool)?
Brian wrote: February 7th, 2021, 9:05 am It's hard to tell where the problems lies. Rainmeter does not explicitly restrict the number of logical CPU's that can be referenced (both in UsageMonitor or the CPU measure). In both cases, Rainmeter asks Windows (via the winapi) for that information. There is nothing in the MS docs that suggest there is any limitation. The MS docs do mention logical processor "relation groups", however it is unclear if this has anything to do the problem at hand.

Ultimately, this is going to be difficult to solve unless one of the team members has access to a machine like this.

-Brian
I have what might be an explanation as to why this happens - take a look here, especially the 2nd answer. Apparently the "old" Perfmon counters under "Processor" are limited to a maximum of 64 logical CPUs, while the "new" ones under "Processor Information" should work over that limit. At least this is what appears to be the reasoning in the 1st link from the 2nd answer posted there. In other words, it isn't a Rainmeter thing, at least if you use UsageMonitor to get a specific counter (not sure about the CPU measure though).
User avatar
SilverAzide
Rainmeter Sage
Posts: 1143
Joined: March 23rd, 2015, 5:26 pm

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by SilverAzide »

Copteur wrote: January 31st, 2021, 3:31 am I'm writing a skin to monitor each CPU core on a dual AMD EPYC 7742 64-Core Processor. This system has 256 logical cores. I have found that as soon as I attempt to measure a CPU above 64, the usual CPU Measure throws an error in the log "CPU: Processor=65 is not valid" and while UsageMonitor doesn't throw any errors I can see, it simply does not work for Name=>64. OS here is WinServer 2019. I've tried Rain 4.3 and 4.4 r3412.
Copteur, if you are still following this thread, it might be helpful for us if you could try Yincognito's advice. I modified your sample skin slightly:

Code: Select all

[Rainmeter]
Update=1000
BackgroundMode=2
SolidColor=0,0,0,255

; the following should be the % utilization across all 256 logical cores
[MeasureCPUTotal]
Measure=Plugin
Plugin=UsageMonitor
Category="Processor Information"
Counter="% Processor Utility"
Name="_Total"
MinValue=0
MaxValue=100

; the following should be the % utilization of physical CPU 0 across all its 128 logical cores
[MeasurePhysicalCPU0]
Measure=Plugin
Plugin=UsageMonitor
Category="Processor Information"
Counter="% Processor Utility"
Name="0,_Total"
MinValue=0
MaxValue=100

; the following should be the % utilization of physical CPU 1 across all its 128 logical cores
[MeasurePhysicalCPU1]
Measure=Plugin
Plugin=UsageMonitor
Category="Processor Information"
Counter="% Processor Utility"
Name="1,_Total"
MinValue=0
MaxValue=100

; the following should be the % utilization of logical core #65 on physical CPU 0
[MeasureCPU65]
Measure=Plugin
Plugin=UsageMonitor
Category="Processor Information"
Counter="% Processor Utility"
Name="0,64"
MinValue=0
MaxValue=100

; the following should be the % utilization of logical core #65 on physical CPU 1 (or logical core #193)
[MeasureCPU193]
Measure=Plugin
Plugin=UsageMonitor
Category="Processor Information"
Counter="% Processor Utility"
Name="1,64"
MinValue=0
MaxValue=100

[MeterText]
Meter=String
MeasureName=MeasureCPUTotal
MeasureName2=MeasurePhysicalCPU0
MeasureName3=MeasurePhysicalCPU1
MeasureName4=MeasureCPU65
MeasureName5=MeasureCPU193
X=5
Y=5
W=180
H=35
FontColor=255,255,255,255
NumOfDecimals=1
Text="CPU Total = %1%#CRLF#Physical Core 0 Total = %2%#CRLF#Physical Core 1 Total = %3%#CRLF#Logical Core 65 = %4%#CRLF#Logical Core 193 = %5%"
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 3017
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Yincognito »

SilverAzide wrote: February 23rd, 2021, 1:52 am Copteur, if you are still following this thread, it might be helpful for us if you could try Yincognito's advice. I modified your sample skin slightly:
Just curious, is there any particular reason why you chose to use the "% Processor Utility" counter over "% Processor Time" in the code? I'm aware that counters and "categories"/objects might differ from machine to machine (hopefully the "Processor Information" one isn't one of those volatile counters), but for me, the "% Processor Time" counter from the "Processor Information" "category"/object yields the same values for all instances as the values from "Processor" - "% Processor Time":
PerfMon Counters.jpg
What am I missing here? Maybe you found out that the "% Processor Utility" values are more accurate (e.g. maybe closer to the Task Manager stuff) than "% Processor Time" ones? Or is there something else I'm not aware of? :confused:

P.S. Good idea to put things in the code, by the way. I skipped that part as I don't have over 64 logical CPUs and, unless you have machines having such properties, we kind of depend on whether Copteur tests produce satisfactory results or not.
You do not have the required permissions to view the files attached to this post.
User avatar
SilverAzide
Rainmeter Sage
Posts: 1143
Joined: March 23rd, 2015, 5:26 pm

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by SilverAzide »

Yincognito wrote: February 23rd, 2021, 2:38 am What am I missing here? Maybe you found out that the "% Processor Utility" values are more accurate (e.g. maybe closer to the Task Manager stuff) than "% Processor Time" ones? Or is there something else I'm not aware of? :confused:
Actually there is something... I found a few mentions on various info sources (not sure how trustworthy they are) that "% Processor Utility" is actually what Task Manager uses. Comparing the output of "% Processor Utility" vs. "% Processor Time" vs. Task Manager seems to confirm this; "% Processor Time" always (for me) reads low compared to the other two. And so does the standard Rainmeter CPU measure. "% Processor Utility" also seems to match Resource Monitor output.

I still would be a little skeptical, but then Microsoft docs say this (seen if you check the "Show description" checkbox in the Add Counters dialog of Perf Mon) [emphasis mine]:
% Processor Time is the percentage of elapsed time that the processor spends to execute a non-Idle thread. It is calculated by measuring the percentage of time that the processor spends executing the idle thread and then subtracting that value from 100%. [...] This counter is the primary indicator of processor activity, and displays the average percentage of busy time observed during the sample interval. It should be noted that the accounting calculation of whether the processor is idle is performed at an internal sampling interval of the system clock tick. On todays fast processors, % Processor Time can therefore underestimate the processor utilization as the processor may be spending a lot of time servicing threads between the system clock sampling interval. Workload based timer applications are one example of applications which are more likely to be measured inaccurately as timers are signaled just after the sample is taken.
One argument for NOT using "% Processor Utility" is this [emphasis mine]:
Processor Utility is the amount of work a processor is completing, as a percentage of the amount of work the processor could complete if it were running at its nominal performance and never idle. On some processors, Processor Utility may exceed 100%.
"% Processor Time" will never exceed 100% since it is calculated by subtracting from 100. So, I'm not sure which is better in the long run... :confused: Perhaps "% Processor Utility" doesn't account for overclocking, so if you are running at 125% of the base clock, the numbers are 25% higher?
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 3017
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Yincognito »

SilverAzide wrote: February 23rd, 2021, 12:36 pm Actually there is something... I found a few mentions on various info sources (not sure how trustworthy they are) that "% Processor Utility" is actually what Task Manager uses. Comparing the output of "% Processor Utility" vs. "% Processor Time" vs. Task Manager seems to confirm this; "% Processor Time" always (for me) reads low compared to the other two. And so does the standard Rainmeter CPU measure. "% Processor Utility" also seems to match Resource Monitor output.

I still would be a little skeptical, but then Microsoft docs say this (seen if you check the "Show description" checkbox in the Add Counters dialog of Perf Mon) [emphasis mine]:

One argument for NOT using "% Processor Utility" is this [emphasis mine]:

"% Processor Time" will never exceed 100% since it is calculated by subtracting from 100. So, I'm not sure which is better in the long run... :confused: Perhaps "% Processor Utility" doesn't account for overclocking, so if you are running at 125% of the base clock, the numbers are 25% higher?
Hmm... it seems you're absolutely right on this one, as stated here (I imagine the statements are true for Win 10 as well, even though they're marked as for Win 8). Since it's a Microsoft link, I suppose it can be rated as trustworthy - after all, these are their own products. And yes, in my view the "nominal performance" part in your quote means exactly the CPU's original frequency (i.e. not under- or over- clocked).

Personally, having the CPU usage over 100% wouldn't bother me a bit, as long as I display somewhere in the skin the nominal/original CPU frequency. Currently I do have the CPU frequency displayed for precisely that reason, but it does react to underclocking, since it's a PowerPlugin based measure (this can easily be changed though).

I'm actually much more concerned at having the other CPU measurements (e.g. the CPU measures, the Alias based UsageMonitor measures, the various "external" - like HWiNFO or MSI Afterburner - plugin based measures) following the same "methodology", so that all the values add up nicely and relate to one another. I have no idea whether they (all) "internally" use "Processor"/"% Processor Time" or "% Processor Utility" based values, but to me having the same "base" of measurement is absolutely critical if I want my skins to be accurate. I mean, if I get program A use 35% CPU and program B use 25% CPU (with the rest of the running programs at 0% CPU for the sake of argument) and the total CPU usage is measured at 85%, it would make no sense as 35 + 25 yields nowhere near 85 (I'm assuming the same moment in time when all these are measured - an impossible thing, but you get the idea). For example, in the case of processes, only "% Processor Time" is available for measurement in the PerfMon counters, and not "% Processor Utility". Sure, there is a "% Privileged Time" (or even "% User Time") available there, but I'm not sure if that is the "% Processor Utility" equivalent or even enough to get a "% Processor Utility" based value. Sources clearly state that there is a connection/similarity in the measurement methodology between the two, but whether they are the same it's an entirely different story, and I get the feeling that the answer to that is not exactly satisfactory... :???:

Bottom line, it's a bit of a mess, and currently the only common denominator of all these is still "% Processor Time", I suppose. As inaccurate as it is (at least compared to the Task Manager, if one rates the Task Manager values as being "the right ones" - yet another question to answer, LOL), it is found in most of these counters. I have no clue what the external programs like HWiNFO or MSI AB use to measure stuff like that (if they use one of these methods and not some other system entirely, that is - again, another dilemma), and that would also play a role in a "same standards" decision.
User avatar
SilverAzide
Rainmeter Sage
Posts: 1143
Joined: March 23rd, 2015, 5:26 pm

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by SilverAzide »

I don't have a Windows 7 machine anymore to test with, but I wonder if the "% Processor Utility" counter even exists in Windows 7. Seems it was introduced in Windows 8. Great find on that link, BTW! :thumbup:

Edit: The "% Processor Utility" counter does not exist on Windows 7.
Gadgets Wiki GitHub More Gadgets...
User avatar
Yincognito
Posts: 3017
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: [Bug?] CPU Core Counts >=64 Measure=CPU & UsageMonitor Wrong

Post by Yincognito »

SilverAzide wrote: February 23rd, 2021, 4:27 pmI don't have a Windows 7 machine anymore to test with, but I wonder if the "% Processor Utility" counter even exists in Windows 7. Seems it was introduced in Windows 8.
No, it doesn't. It was introduced in Win 8.
SilverAzide wrote: February 23rd, 2021, 4:27 pmGreat find on that link, BTW! :thumbup:
It's all about writing the right keywords and identify the "essence" of the question when googling, really. For the 1st link regarding the OP's question and Brian's answer I used "% Processor Time perfmon over 64 logical cpu" when googling (I was certain it was a PerfMon issue and not a programming one as Brian alluded to, as my first thought was to ask the OP whether the issue happens in PerfMon as well, or just in Rainmeter), and for the 2nd link regarding MS confirmation of what you said about the Processor Utility counters I used a simple "processor utility vs processor time" when googling. In both cases, the answer was the first Google result, and it actually took more time to write my replies than finding the answers I was looking for. :confused:

It's not rocket science, but it's true that not formulating the right search text results in not finding the answer to a question.