Windows Performance Metrics That Actually Matter in a Crisis

Many, many years ago, around the Windows Server 2008 era, I attended a Microsoft course called Vital Signs at Thames Valley Park in Reading. It was awesome, I learnt so much about Windows performance, how to diagnose problems, and which counters to watch when a system decides to run like a dog, not a whippet, maybe a fat lab. “So, just Windows being Windows then?”

One of the key lessons was that a relatively small selection of performance counters can help diagnose most common performance issues. Windows exposes thousands of counters, but understanding a core set covering processor, memory, disk and network activity gives you a practical starting point. The value comes from understanding what those counters tell you and how they relate to each other.

This article provides the information learnt from the Vital Signs course for the modern versions of Windows and 64-bit operating systems, expanding beyond the paging and memory limitations associated with 32-bit Windows. The same core troubleshooting principles apply, using a small, carefully selected set of performance counters to identify the source of most common performance issues.

The bit I can’t do for you is interpret your results. That’s the slow grind, watching for unusual trends, counters that keep climbing, and working out which changes line up with the slowdown.

Where to Start

For general Windows troubleshooting, the objective is usually to answer four questions:

1. Is the CPU overloaded or is a particular thread, driver or processor causing a bottleneck?

2. Is the system genuinely short of memory or is something leaking?

3. Is storage taking too long to service I/O?

4. Is the network saturated, dropping traffic or causing TCP retransmissions?

   
   

The counters below are intended for current 64-bit Windows Server and Windows client systems.

The numbers are a guide, not commandments. A CPU hitting 100% for five seconds is Windows doing some work. A disk briefly reaching 50 ms latency during a flush operation is not necessarily a disaster either.

What matters is sustained behaviour at the same time the actual problem is occurring.

Using Performance Monitor

Performance Monitor remains one of the most useful built-in Windows troubleshooting tools.

Run:

perfmon

Then open:

Monitoring Tools
    Performance Monitor

Click the green + button.

You can then select:

Performance object
Instance
Counter

For example:

Processor Information
    _Total
        % Processor Time

or:

LogicalDisk
    C:
        Avg. Disk sec/Read

Add the counters you want and Perfmon begins graphing them in real time.

Configure Perfmon for best results

Within the Performance Monitor graph, right-click and select Properties. On the General tab, under Graph elements, set Sample every to 3–5 seconds and Duration to 1,000 seconds.

This reduces the collection frequency, helping to limit monitoring overhead on an already struggling system. It also provides a longer window of activity on the graph, making sustained bottlenecks and recurring patterns easier to identify. Bear in mind that longer sampling intervals can hide brief spikes.

Next, open the Scale tab and use Selected counters to adjust the scale of individual counters so they can be compared on the same graph. You are changing how the values are displayed, the underlying measurements remain unchanged.

For example, PhysicalDisk\Disk Transfers/sec might show 1,000 operations per second, while PhysicalDisk\Avg. Disk Queue Length shows 10 outstanding requests for the same disk. At the same scale, disk activity dominates the graph and makes changes in the queue difficult to see. Set the scale for Disk Transfers/sec to 0.01 and leave Avg. Disk Queue Length at 1.0, bringing both lines into a comparable visual range.

You can now see whether increases in disk activity coincide with a growing queue. If Avg. Disk sec/Read or Avg. Disk sec/Write also rises, this provides stronger evidence that storage is struggling to keep up. A queue length of 10 alone does not establish a bottleneck, it needs to be considered alongside latency and the storage’s capabilities.

Where possible, run Perfmon from another server or workstation. Click +, select Another computer, enter the target server name and add the required counters. If the target becomes unresponsive or BSoD itself into ground, again, the remote Perfmon window retains the recent graph history collected before the failure.

In practice, monitoring locally is often more convenient, just remember that a reboot will lose the live graph history.

Core Performance Counter Set

For a general look at Windows performance, these are the counters I start with:

Processor Information(_Total)\% Processor Time
Processor Information(_Total)\% User Time
Processor Information(_Total)\% Privileged Time
Processor Information(_Total)\% DPC Time
Processor Information(_Total)\% Interrupt Time

System\Processor Queue Length
System\Context Switches/sec

Memory\Available MBytes
Memory\% Committed Bytes In Use
Memory\Pages Input/sec
Memory\Pages Output/sec
Memory\Pool Paged Bytes
Memory\Pool Nonpaged Bytes

LogicalDisk(*)\Avg. Disk sec/Read
LogicalDisk(*)\Avg. Disk sec/Write
LogicalDisk(*)\Disk Reads/sec
LogicalDisk(*)\Disk Writes/sec
LogicalDisk(*)\Avg. Disk Bytes/Transfer

Network Interface(*)\Bytes Total/sec
Network Interface(*)\Output Queue Length
Network Interface(*)\Packets Received Errors
Network Interface(*)\Packets Received Discarded
Network Interface(*)\Packets Outbound Errors
Network Interface(*)\Packets Outbound Discarded

TCPv4\Segments Retransmitted/sec

Process(*)\% Processor Time
Process(*)\Private Bytes
Process(*)\Working Set
Process(*)\Handle Count
Process(*)\Thread Count
Process(*)\IO Read Bytes/sec
Process(*)\IO Write Bytes/sec

That is enough to identify most common Windows resource bottlenecks without collecting every counter Microsoft has invented since Windows NT.

Now let’s break down what these counters tell us and what to look for when things start slowing down.

Storage

Storage latency remains one of the quickest ways to make an otherwise healthy Windows server feel completely broken. Virtualisation adds another layer of complexity, as the delay may originate within the guest, the hypervisor, or the underlying storage.

If your hypervisor runs on Linux, clearly not VMWare, any longer, the following doesn't apply to the host.

A slow application inside a VM can be a symptom of storage contention elsewhere, even when the guest’s processor and memory usage appear normal.

The primary counters are:

Counter	Healthy	Warning	Critical
\LogicalDisk(*)\Avg. Disk sec/Read	<15 ms	>25 ms	>50 ms
\LogicalDisk(*)\Avg. Disk sec/Write	<15 ms	>25 ms	>50 ms
\PhysicalDisk(*)\Avg. Disk sec/Read	<15 ms	>25 ms	>50 ms
\PhysicalDisk(*)\Avg. Disk sec/Write	<15 ms	>25 ms	>50 ms

Investigate sustained periods of high latency rather than short peaks. Anything lasting a minute or longer becomes a point of interest.

NVMe and SSD

Those thresholds are generic Windows guidance, they are not performance targets for NVMe's.

A modern local NVMe device would normally operate substantially below 15 ms. If a device normally responds in fractions of a millisecond and suddenly spends several minutes at 5–10 ms, something has changed even though it technically remains inside the generic Windows threshold.

With flash storage, compare performance against the device's normal baseline as well as the absolute Windows limits.

An enterprise SAN, Storage Spaces array, virtual disk and local PCIe NVMe device can all present as flash storage while behaving very differently.

Disk Queue Length

Useful counters that I use are:

\PhysicalDisk(*)\Avg. Disk Queue Length
\LogicalDisk(*)\Avg. Disk Queue Length

Do not use queue length as a standalone diagnosis.

Modern SSD, NVMe, RAID and SAN controllers are designed to maintain multiple outstanding I/O requests.

The useful correlation is:

High queue + Low latency

The storage system is probably processing requests efficiently.

Whereas:

High queue + Rising latency

means the storage subsystem may be saturated.

A queue is evidence of outstanding work.

Latency tells you whether the storage is coping with it.

Additional Storage Counters

I normally add:

\LogicalDisk(*)\Disk Transfers/sec
\LogicalDisk(*)\Disk Reads/sec
\LogicalDisk(*)\Disk Writes/sec

\LogicalDisk(*)\Disk Read Bytes/sec
\LogicalDisk(*)\Disk Write Bytes/sec

\LogicalDisk(*)\Avg. Disk Bytes/Transfer

Disk Transfers/sec gives total IOPS.

Avg. Disk Bytes/Transfer gives some indication of the workload type. Lots of small transfers may indicate transactional or random I/O, while large transfers are more typical of backups, file copies and sequential workloads.

To find what is generating the I/O:

\Process(*)\IO Read Operations/sec
\Process(*)\IO Write Operations/sec
\Process(*)\IO Read Bytes/sec
\Process(*)\IO Write Bytes/sec

It is usually worth finding the process responsible before declaring the SAN broken.

If your in the situation where Perfmon is in active use, good luck resolving the issue.

I’ve certainly used it in anger, from whole systems falling over, as the Storage monkeys had monthly full backups running during the working day, then denied all knowledge or responsibility until Perfmon pointed straight at them.

Memory

High memory utilisation is not automatically bad.

Unused RAM achieves very little, so Windows makes extensive use of physical memory for applications, the filesystem cache and the Standby List.

The question is whether Windows is experiencing actual memory pressure.

Counter	Healthy	Warning	Critical
\Memory\Available MBytes	>10% RAM or ≥4 GB	<10%	<1% or <500 MB
\Memory\% Committed Bytes In Use	0–60%	60–80%	>80%
Paged/Nonpaged Pool	Normal baseline	Significant growth	Sustained abnormal consumption

Available Memory

\Memory\Available MBytes

Treat these thresholds as guides, rather than fixed rules. Less than 10% available memory does not automatically mean a server is struggling, particularly on systems with large amounts of RAM. A server with 256 GB of RAM and 8% available still has approximately 20 GB available.

Look for sustained memory pressure and its effect on performance before treating a percentage alone as a problem.

Committed Memory

Monitor:

\Memory\% Committed Bytes In Use
\Memory\Committed Bytes
\Memory\Commit Limit

Committed memory is virtual memory for which Windows has guaranteed backing storage through physical RAM and the page file.

If commit approaches the system Commit Limit, Windows has a much more serious problem than merely having very little free physical RAM.

Applications may begin failing allocations or generating out-of-memory errors.

Paging

Useful counters include:

\Memory\Page Faults/sec
\Memory\Pages/sec
\Memory\Pages Input/sec
\Memory\Pages Output/sec

These counters are easily misunderstood.

Page Faults/sec includes both soft and hard page faults.

Pages/sec represents pages read from or written to disk while resolving hard page faults, but that does not automatically mean page-file thrashing.

Hard faults can involve executable files, DLLs and memory-mapped files as well as the page file.

For genuine memory pressure I want to see several counters agreeing:

Available MBytes falling
+
Commit increasing
+
Pages Input/sec or Pages Output/sec sustained
+
Storage activity increasing

A single counter showing a high value is not enough to confirm a performance problem.

Kernel Memory

These are monitors I add to every Perfmon:

\Memory\Pool Paged Bytes
\Memory\Pool Nonpaged Bytes

Paged Pool contains kernel allocations that can potentially be paged out.

Nonpaged Pool must remain resident in physical memory.

On modern 64-bit Windows, absolute pool sizes can be substantial, so a fixed MB threshold is not particularly useful.

What matters is unexpected growth over time.

For example:

08:00    800 MB
12:00    1.4 GB
18:00    2.7 GB
02:00    5.1 GB

If the workload has returned to normal but Nonpaged Pool has not, I would start looking for a

kernel leak.

Pool Paged Bytes and Pool Nonpaged Bytes track kernel memory used by Windows and drivers. Paged pool can be paged out when appropriate.

Nonpaged pool must remain in physical RAM, so sustained abnormal growth can progressively reduce memory available elsewhere.

Watch for a steady climb or repeated steps upwards that never settle, even after the workload drops. That deserves investigation, particularly if available memory falls alongside it.

Common suspects include:

• network and storage drivers

• antivirus products

• filesystem filters

• endpoint security agents

• backup software

• other kernel-mode filter drivers

  
  

Again, Perfmon to the rescue with another oldie, an HP firmware/driver issue that caused paged pool memory to jump every hour or so until the server eventually ate itself. It’s the sort of problem I rarely come across these days with 64-bit Windows.

Tools such as PoolMon or Windows Performance Recorder are then better suited to finding the responsible pool tag or driver.

Processor

Overall CPU utilisation is useful, but on a modern multi-core system _Total can hide a surprisingly large amount of detail.

Counter	Healthy	Warning	Critical
% Processor Time	<50%	50–80%	>80–85% sustained
% User Time	<50%	50–80%	>80%
% Privileged Time	<30%	30–50%	>50%
% DPC Time	Low	Rising	>15% with CPU >85%
% Interrupt Time	Low	>30%	>50%

Investigate continuously high CPU around 85% and above. DPC activity above 15% becomes significant when overall CPU is also above 85%, while Interrupt Time above 30% is considered high and above 50% very high.

Use:

\Processor Information(*)\% Processor Time

rather than relying solely on _Total.

One logical processor sitting permanently at 100% while the other 31 processors are largely idle can be a very real single-threaded bottleneck that looks insignificant in the overall CPU figure.

User Time

\Processor Information(*)\% User Time

High User Time normally means application code is consuming the processor.

Use:

\Process(*)\% Processor Time

to identify which process is responsible.

One slightly confusing detail is that Perfmon process CPU is not normalised the way Task Manager normally displays CPU.

A process fully consuming one logical processor can report approximately:

100%

Two processors:

200%

Four:

400%

Privileged Time

\Processor Information(*)\% Privileged Time

This is kernel-mode activity.

Sustained high Privileged Time can indicate:

• heavy storage activity

• networking

• excessive system calls

• filesystem filters

• security software

• driver problems

  
  

If the server has high CPU but most of it is Privileged Time, throwing more processors at the application may achieve remarkably little.

DPC and Interrupt Time

Monitor:

\Processor Information(*)\% DPC Time
\Processor Information(*)\% Interrupt Time

These become particularly useful when diagnosing:

NIC drivers
Storage controllers
HBAs
Virtualisation
USB devices
Hardware
Kernel drivers

High CPU combined with high Privileged, DPC or Interrupt Time points away from ordinary application processing and towards kernel or hardware activity.

Processor Queue Length

\System\Processor Queue Length

This represents runnable threads waiting for CPU time. A queue by itself does not prove CPU starvation.

A much better pattern is:

Processor utilisation >85%
+
Processor Queue Length sustained
+
Queue approaching or exceeding roughly
2 threads per logical processor

Correlate Processor Queue Length with high processor utilisation, context switching, DPC and interrupt activity rather than diagnosing it independently.

Also monitor:

\System\Context Switches/sec

Very high context-switch rates can indicate excessive thread concurrency, locking or poorly behaved software.

Numbers above roughly 15,000 context switches/sec per logical processor can warrant investigation when accompanied by high CPU, but workload baseline matters enormously here.

Network

The primary counters are:

\Network Interface(*)\Bytes Total/sec
\Network Interface(*)\Bytes Sent/sec
\Network Interface(*)\Bytes Received/sec

General guidance is:

Utilisation	Status
<50%	Healthy
50–80%	Monitor
>80%	Investigate

The percentage is relative to the negotiated interface speed.

The calculation is:

Utilisation % =
((Bytes Total/sec × 8) / Current Bandwidth) × 100

For reference:

1 Gbit/s  ≈ 125 MB/sec
10 Gbit/s ≈ 1.25 GB/sec
25 Gbit/s ≈ 3.125 GB/sec

A server shifting 900 Mbit/sec over a 1 Gbit interface is obviously rather busy. The same workload on a 10 Gbit NIC is napping and should be worked harder.

Network Queues, Errors and Drops

Monitor:

\Network Interface(*)\Output Queue Length

\Network Interface(*)\Packets Received Errors
\Network Interface(*)\Packets Outbound Errors

\Network Interface(*)\Packets Received Discarded
\Network Interface(*)\Packets Outbound Discarded

For Output Queue Length:

Value	Interpretation
0	Normal
1–2	Monitor
>2	Investigate

An interface operating at 80% while cleanly processing traffic might simply be busy.

An interface operating at 30% while dropping packets is considerably more suspicious.

TCP Performance

Interface utilisation only tells part of the story.

Also monitor:

\TCPv4\Segments Retransmitted/sec
\TCPv6\Segments Retransmitted/sec

Increasing retransmissions indicate TCP is having to resend data. Possible causes include:

Packet loss
Congestion
Firewall problems
MTU problems
Wireless links
Faulty infrastructure
Remote system problems

It does not tell you which of those is responsible. It tells you TCP isn't happy.

Also useful:

\TCPv4\Connection Failures

An increasing value shows failed TCP connection establishment.

Correlate that with firewall logs, application logs, port utilisation and packet captures rather than immediately deciding somebody is SYN flooding the server.

Process-Level Diagnostics

Once a system-level resource looks unhealthy, identify the process responsible.

The useful counters are:

\Process(*)\% Processor Time
\Process(*)\Private Bytes
\Process(*)\Working Set
\Process(*)\Handle Count
\Process(*)\Thread Count
\Process(*)\IO Read Bytes/sec
\Process(*)\IO Write Bytes/sec

Private Bytes

\Process(*)\Private Bytes

This represents private committed virtual memory belonging to the process. There is no sensible universal maximum.

SQL consuming 20 GB may be completely normal.

A small monitoring service growing from 200 MB to 20 GB over several days probably isn't.

Look for trends.

Working Set

\Process(*)\Working Set

Working Set represents the process's currently resident physical memory.

Correlate it with:

\Memory\Available MBytes

to find which process is consuming physical RAM. Correlate when investigating memory pressure.

Handles

\Process(*)\Handle Count

A high number is not automatically a leak.

Continuous growth is more useful:

08:00     2,500
12:00     5,200
18:00    11,000
02:00    24,000

If the workload disappears but the handles never return, start investigating.

Threads

\Process(*)\Thread Count

Again, hundreds of threads can be perfectly normal. A process continuously creating threads and never releasing them is not.

Performance troubleshooting works much better when looking for behaviour rather than treating arbitrary numbers as sacred.

Using Data Collector Sets

For the most, I go straight to Perfmon, as covered above. However, watching a graph is less practical when an issue occurs overnight, unless there's overtime, appears intermittently, or takes several hours to develop.

Data Collector Sets record selected performance counters to a log for later analysis. They can run on a schedule and capture activity over longer periods, allowing you to review what happened before, during and after a slowdown without sitting in front of the server.

They are also useful for establishing a baseline, comparing normal operation with a reported problem, and checking whether a configuration change has improved performance. Configure a sensible sampling interval and log retention to keep collection overhead and disk usage under control.

Open:

perfmon

Then:

Data Collector Sets
    User Defined

Right-click User Defined and select:

New
    Data Collector Set

Give it a useful name, for example:

Tenaka OS Performance

Select:

Create manually (Advanced)

Then:

Create data logs
    Performance counter

Add the counters required.

For active troubleshooting I normally use a 5-second sample interval.

For a long-running general baseline, 15 seconds usually provides more than enough resolution while keeping the resulting BLG files manageable.

Choose somewhere sensible to store the logs, for example:

D:\PerfLogs

Otherwise, you’ll fill the OS drive, adding the servers woes, because someone, probably you, forgot Perfmon was still running. It never happened, no evidence... honest.

Complete the wizard and choose:

Save and close

You can then right-click the Data Collector Set and select:

Start

and later:

Stop

Scheduling a Data Collector Set

Open the properties of the Data Collector Set. The Schedule tab allows it to start automatically.

The Stop Condition tab allows you to specify:

Overall duration
Maximum size
Other stop conditions

This is useful for overnight captures or servers where a fault only appears after several hours.

For example:

Start: 18:00
Duration: 14 hours
Interval: 15 seconds

You then have a complete overnight performance record instead of asking Task Manager what happened six hours ago.

Task Manager remains strangely unhelpful about events it wasn't there to witness.

Reviewing a Perfmon BLG File

Perfmon normally stores performance logs as .blg files.

Open:

perfmon

Select:

Performance Monitor

Open Properties and select:

Source

Change the data source from:

Current activity

to:

Log files

Add the .blg file.

You can now add individual counters from the recorded data and inspect exactly what was happening at the problem time.

I normally start with:

CPU
Memory
Disk latency
Network

Find the resource that moved first, then add process counters.

It is considerably easier than dumping every available counter on the graph and producing what looks like an ECG from somebody being attacked by a bear.

Creating the Collector with LOGMAN

Perfmon's GUI is useful, but for repeatable deployment logman.exe is better.

Logman is built into current Windows Server and Windows client versions and can create, start, stop, update, import and export Data Collector Sets.

Create a text file containing the counters:

C:\PerfLogs\Tenaka-Counters.txt

For example:

\Processor Information(_Total)\% Processor Time
\Processor Information(_Total)\% User Time
\Processor Information(_Total)\% Privileged Time
\Processor Information(_Total)\% DPC Time
\Processor Information(_Total)\% Interrupt Time

\System\Processor Queue Length
\System\Context Switches/sec

\Memory\Available MBytes
\Memory\% Committed Bytes In Use
\Memory\Pages Input/sec
\Memory\Pages Output/sec
\Memory\Pool Paged Bytes
\Memory\Pool Nonpaged Bytes

\LogicalDisk(*)\Avg. Disk sec/Read
\LogicalDisk(*)\Avg. Disk sec/Write
\LogicalDisk(*)\Disk Reads/sec
\LogicalDisk(*)\Disk Writes/sec
\LogicalDisk(*)\Avg. Disk Bytes/Transfer

\Network Interface(*)\Bytes Total/sec
\Network Interface(*)\Output Queue Length
\Network Interface(*)\Packets Received Errors
\Network Interface(*)\Packets Received Discarded
\Network Interface(*)\Packets Outbound Errors
\Network Interface(*)\Packets Outbound Discarded

\TCPv4\Segments Retransmitted/sec

\Process(*)\% Processor Time
\Process(*)\Private Bytes
\Process(*)\Working Set
\Process(*)\Handle Count
\Process(*)\Thread Count
\Process(*)\IO Read Bytes/sec
\Process(*)\IO Write Bytes/sec

Create the collector:

logman create counter Tenaka-OS-Performance ^
-cf "C:\PerfLogs\Tenaka-Counters.txt" ^
-si 00:00:05 ^
-f bincirc ^
-max 1024 ^
-o "C:\PerfLogs\Tenaka-OS-Performance"

Start it:

logman start Tenaka-OS-Performance

Check it:

logman query Tenaka-OS-Performance

Stop it:

logman stop Tenaka-OS-Performance

Using:

-f bincirc

creates a binary circular log.

With:

-max 1024

the collector can continue running without endlessly consuming disk space.

Logman is the cmd equivalent for creating and controlling Perfmon Data Collector Sets.

Diagnostic Cheat Sheet

Problem	Primary evidence	Corroborating counters
Storage bottleneck	Disk latency sustained >25 ms	Queue increasing + process I/O
Physical memory pressure	Available MBytes low	Commit high + paging + disk activity
Application memory leak	Commit increasing	Process Private Bytes continuously growing
Kernel memory leak	Paged/Nonpaged Pool growing	No equivalent process memory increase
CPU saturation	CPU >80–85% sustained	Processor Queue Length increasing
Single-thread bottleneck	Individual CPU at 100%	Process CPU around 100% for one active thread
Driver/kernel problem	Privileged Time high	DPC/Interrupt Time high
Network saturation	Interface >80%	Output queue + drops/retransmissions
Network fault	Drops/errors increasing	TCP retransmissions while bandwidth remains low

Final Thoughts

Hopefully, this gives you somewhere to start when systems are crashing or grinding to a halt and you’re wondering what the F### is going on. Stay calm, review the counters and be patient, searching for the needle in the haystack may take a while

That said, I once spent an entire day investigating a server while my colleagues, more rivals than friends at this moment in time, we were competing to see who's right, and they offered the usual wisdom: “It’s Windows, just reboot it.” Yeah, cheers, very helpful. After getting nowhere with Perfmon, Event Viewer and everything else. I was junior so maybe not that much "else", I gave up, towel thrown in and rebooted it. Problem resolved....

There’s a lesson in there. Capture what you can, but don’t lose an entire day trying to explain a one-off wobble. Sometimes it really is just Windows having a moment. If it keeps happening, that’s when “just reboot it” stops being a particularly useful answer. And.... why schedule a nightly reboot, we've all seen it.