When NTLM Hardening Breaks SID Lookup: Diagnosing RPC Endpoint Mapper and LSA Failures

Tenaka
6 days ago
6 min read

NTLM hardening can expose dependencies that are not obvious during normal Windows administration.

In other words, the lab was working perfectly well until I decided it clearly wasn't secure enough. I’d successfully hardened the lab to the point where even Windows itself was being treated as a security threat, a very solid effort on my part... Idiot

I was trying to configure Windows Event Collector server with a source-initiated WEC subscription. When an Active Directory security group was added to the subscription, Event Viewer failed to resolve the group:

The specified domain either does not exist or could not be contacted.

The account CONTOSO.LOC\RG_WEC_Subscription
cannot be translated to a security identifier.

This resulted in a slight detour from WEC and WEF, whilst I fixed the lab.

Testing the SID translation directly produced an even more misleading error:

([System.Security.Principal.NTAccount]'CONTOSO\RG_WEC_Subscription').Translate(
    [System.Security.Principal.SecurityIdentifier]
)

Result:

Exception calling "Translate" with "1" argument(s):
"The trust relationship between this workstation and the primary domain failed."

The obvious conclusion would be a failed computer account trust, It wasn't.

The machine trust was healthy, DNS worked, Kerberos worked, LDAP worked and SYSVOL was accessible.

Microsoft documents that enabling the 'Enable RPC Endpoint Mapper Client Authentication' cannot be used with 'Network security: Restrict NTLM: Incoming NTLM traffic – Deny all accounts'.

Microsoft recommends retaining the NTLM restriction rather than enabling authenticated RPC endpoint resolution where the two conflict.

The Configuration That Caused the Problem

The member server had the following Group Policy enabled:

Computer Configuration
  Policies
    Administrative Templates
      System
        Remote Procedure Call
          Enable RPC Endpoint Mapper Client Authentication

Configured as:

Enabled

The corresponding registry setting is:

HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Rpc

EnableAuthEpResolution

When enabled, RPC clients authenticate to the Endpoint Mapper for RPC calls containing authentication information.

The domain controllers were also hardened with:

Computer Configuration
  Policies
    Windows Settings
      Security Settings
        Local Policies
          Security Options
            Network security: Restrict NTLM:
            Incoming NTLM traffic

Configured as:

Deny all accounts

The effective registry value on the domain controllers was:

reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0" /v RestrictReceivingNTLMTraffic

Result:

RestrictReceivingNTLMTraffic    REG_DWORD    0x2

Deny all accounts prevents incoming NTLM authentication from both domain and local accounts.

Microsoft records audit and block information for this policy under the NTLM Operational event log.

The member server's outgoing NTLM policy was only configured for auditing:

reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0" /v RestrictSendingNTLMTraffic

Result:

RestrictSendingNTLMTraffic    REG_DWORD    0x1

In this configuration 1 represents Audit all, rather than an outbound NTLM block. Microsoft recommends using Audit all before moving to Deny all so that remaining NTLM dependencies can be identified.

The important combination was therefore:

System	Setting	Configuration
Member server	RPC Endpoint Mapper Client Authentication	Enabled
Member server	Outgoing NTLM	Audit all
Domain controllers	Incoming NTLM	Deny all accounts

The result was a failure in Windows SID/name resolution while most other domain functionality continued to work.

Why the Error Was Misleading

The error returned by Windows was:

The trust relationship between this workstation and the primary domain failed.

A broken secure channel can certainly produce that error, so the first step was to test it:

Test-ComputerSecureChannel -Verbose

Result:

VERBOSE: Performing the operation "Test-ComputerSecureChannel"
on target "WEC01".

True

VERBOSE: The secure channel between the local computer
and the domain CONTOSO.LOC is in good condition.

This was then tested against each domain controller individually:

Test-ComputerSecureChannel -Server DC01 -Verbose
Test-ComputerSecureChannel -Server DC02 -Verbose
Test-ComputerSecureChannel -Server DC03 -Verbose

All returned:

True

At that point, resetting the computer account or removing and rejoining the server to the domain would have been the wrong action.

Check Domain Controller Discovery

The next checks confirmed that the server could find and communicate with Active Directory:

whoami
$env:LOGONSERVER

nltest /dsgetdc:contoso.loc
nltest /sc_query:contoso.loc

A healthy nltest /dsgetdc result should identify a domain controller and report capabilities similar to:

GC
DS
LDAP
KDC
TIMESERV
WRITABLE
DNS_DC
DNS_DOMAIN
DNS_FOREST

nltest /sc_query should return:

Trusted DC Connection Status Status = 0 0x0 NERR_Success

Again, this confirmed that Netlogon and the machine secure channel were functioning.

Test the SID Lookup Directly

The Windows account translation could be reproduced independently of WEC.

For a domain group:

([System.Security.Principal.NTAccount]'CONTOSO\RG_WEC_Subscription').Translate(
    [System.Security.Principal.SecurityIdentifier]
)

This failed.

Testing several objects made the behaviour more obvious:

$Accounts = @(
    'CONTOSO\AdminUser',
    'CONTOSO\Domain Computers',
    'CONTOSO\RG_WEC_Subscription'
)

foreach ($Account in $Accounts) {

    try {

        $SID = ([System.Security.Principal.NTAccount]$Account).Translate(
            [System.Security.Principal.SecurityIdentifier]
        )

        "$Account = $SID"
    }

    catch {

        "$Account = FAILED - $($_.Exception.Message)"
    }
}

The result looked like:

CONTOSO\AdminUser = S-1-5-21-xxxxxxxx-xxxx

CONTOSO\Domain Computers = FAILED -
The trust relationship between this workstation
and the primary domain failed.

CONTOSO\RG_WEC_Subscription = FAILED -
The trust relationship between this workstation
and the primary domain failed.

This was an important diagnostic result.

A currently logged-on user could be resolved, while domain security groups could not.

The WEC problem was therefore not specific to Event Viewer. Windows account-to-SID translation itself was failing.

Windows uses LSA account lookup mechanisms to translate account names and SIDs.

Query the Group SID Directly Through LDAP

LDAP provided another useful test because it allowed the group SID to be read directly from each domain controller without using the failing account-name translation mechanism.

$DCs = 'DC01','DC02','DC03'

foreach ($DC in $DCs) {

    $Root = [ADSI]"LDAP://$DC/RootDSE"

    $Base = [ADSI](
        "LDAP://$DC/" + $Root.defaultNamingContext
    )

    $Search = New-Object `
        System.DirectoryServices.DirectorySearcher($Base)

    $Search.Filter = `
        '(&(objectClass=group)(sAMAccountName=RG_WEC_Subscription))'

    $Result = $Search.FindOne()

    if ($Result) {

        $SID = New-Object `
            System.Security.Principal.SecurityIdentifier(
                $Result.Properties['objectsid'][0],
                0
            )

        "$DC : $SID"
    }
    else {

        "$DC : Group not found"
    }
}

All three domain controllers returned the same SID:

DC01 : S-1-5-21-xxxxxxxx-xxxxxxxx-xxxxxxxx-12117
DC02 : S-1-5-21-xxxxxxxx-xxxxxxxx-xxxxxxxx-12117
DC03 : S-1-5-21-xxxxxxxx-xxxxxxxx-xxxxxxxx-12117

This ruled out:

AD replication
Missing group
Different group objects
LDAP connectivity
Incorrect SID

Check RPC and SMB Connectivity

Because SID/name lookup can involve RPC, TCP 135 and SMB were checked next:

Test-NetConnection DC01 -Port 135
Test-NetConnection DC01 -Port 445

Expected:

TcpTestSucceeded : True

The same tests were performed against each domain controller.

SYSVOL access was also tested:

Get-ChildItem \\DC01.contoso.loc\SYSVOL

This succeeded.

Normal SMB communication to Active Directory was therefore available.

Where firewalls exist between domain members and domain controllers, remember that TCP 135 is only the RPC Endpoint Mapper. RPC services may subsequently use dynamic high ports, so allowing TCP 135 alone does not prove that every RPC operation will succeed.

Verify Kerberos

Kerberos was also checked explicitly:

klist get cifs/DC01.contoso.loc

A successful result should show a ticket for:

cifs/DC01.contoso.loc

The ticket in this case used AES and had been issued by a domain controller.

LDAP Kerberos tickets were also already present:

klist

with entries similar to:

ldap/DC01.CONTOSO.LOC
ldap/DC02.CONTOSO.LOC
GC/DC02.CONTOSO.LOC/CONTOSO.LOC

At this stage the basic AD stack was working:

DNS             Working
Kerberos        Working
LDAP            Working
SMB             Working
SYSVOL          Working
Netlogon        Working
Secure Channel  Working
SID Lookup      Failing

Check the NTLM Policies

The member server was checked for outgoing NTLM restrictions:

reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0" /v RestrictSendingNTLMTraffic

Result:

RestrictSendingNTLMTraffic    REG_DWORD    0x1

This was Audit all.

The domain controllers were checked for incoming NTLM restrictions:

reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0" /v RestrictReceivingNTLMTraffic

Result:

RestrictReceivingNTLMTraffic    REG_DWORD    0x2

The DCs were denying incoming NTLM.

The RPC Endpoint Mapper Client Authentication policy on the member server was simultaneously requiring authenticated endpoint resolution.

NTLM Event Logs

NTLM auditing and blocking events are recorded under:

Event Viewer
  Applications and Services Logs
    Microsoft
      Windows
        NTLM
          Operational

Event IDs in the 8001 to 8004 range are associated with NTLM auditing and restriction activity.

The recent events can be queried with PowerShell:

Get-WinEvent `
    -FilterHashtable @{
        LogName = 'Microsoft-Windows-NTLM/Operational'
        Id      = 8001,8002,8003,8004
    } |
    Select-Object TimeCreated,Id,Message |
    Format-List

The Security log can provide supporting evidence as well. For network logons, Event ID 4624 can be inspected for:

Logon Type:             3
Authentication Package: NTLM

The Fix

The NTLM restriction on the domain controllers was intentional security hardening and did not need to be removed.

The conflicting RPC policy was overridden on the affected member server:

Computer Configuration
  Policies
    Administrative Templates
      System
        Remote Procedure Call
          Enable RPC Endpoint Mapper Client Authentication

Set:

Disabled

An OU-specific GPO can be used where a wider security baseline enables the setting.

Apply policy:

gpupdate /force

The RPC Endpoint Mapper Client Authentication setting requires a reboot before the change takes effect.

After restarting the server, the SID translation was tested again:

([System.Security.Principal.NTAccount]'CONTOSO\Domain Computers').Translate(
    [System.Security.Principal.SecurityIdentifier]
)

and:

([System.Security.Principal.NTAccount]'CONTOSO\RG_WEC_Subscription').Translate(
    [System.Security.Principal.SecurityIdentifier]
)

Both now returned valid SIDs.

Final Configuration

The resulting hardening configuration kept the important NTLM restriction:

Domain Controllers

Network security:
Restrict NTLM:
Incoming NTLM traffic

Deny all accounts

The conflicting member-server policy was disabled:

Member Server

System
  Remote Procedure Call
    Enable RPC Endpoint Mapper Client Authentication

Disabled

The problem was not that NTLM had been disabled. The problem was that one RPC policy still expected to use an authentication mechanism that another policy had deliberately removed.

When NTLM Hardening Breaks SID Lookup: Diagnosing RPC Endpoint Mapper and LSA Failures

The Configuration That Caused the Problem

Why the Error Was Misleading

Check Domain Controller Discovery

Test the SID Lookup Directly

Query the Group SID Directly Through LDAP

Check RPC and SMB Connectivity

Verify Kerberos

Check the NTLM Policies

NTLM Event Logs

The Fix

Final Configuration

Recent Posts

Comments