Remote Desktop is the front door to almost every Windows VPS, and when it stops working the whole server feels offline. The connection dialog shows a generic failure, or worse, it hangs and then drops. This guide is a diagnostic playbook: it maps the most common RDP errors on Windows Server 2019/2022/2025 to the exact registry value, firewall rule, group policy setting or certificate store entry that fixes them. For a broader orientation on what the platform offers before you dig into troubleshooting, see our Windows VPS guide.
Step 0: Confirm the failure layer before changing anything
Three things fail independently: the network path (port 3389 reachable), the Remote Desktop listener, and authentication. Test them in order so you do not harden the wrong layer.
# Reachability from your workstation
Test-NetConnection -ComputerName 203.0.113.20 -Port 3389
# Listener state on the server (run from console access)
Get-NetTCPConnection -LocalPort 3389 -State Listen
Get-Service TermService | Select-Object Status, StartType
# NLA and security layer configuration
Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" |
Select-Object UserAuthentication, SecurityLayer, MinEncryptionLevel, PortNumberUserAuthentication = 1 means Network Level Authentication (NLA) is required. SecurityLayer values are 0 (RDP Security Layer), 1 (Negotiate) and 2 (TLS). The default and recommended configuration is 1 for both.
RDP error codes and what they actually mean
| Code | Message | Root cause and fix |
|---|---|---|
| 0x3 | “The remote computer can’t be found” | DNS or wrong hostname/IP. Test with the literal IP. Check that the record is not stale. |
| 0x204 | “An internal error has occurred” | Usually CredSSP / encryption oracle mismatch. Patch both ends, or set AllowEncryptionOracle = 2 as a temporary workaround. |
| 0x516 | “No Remote Desktop License Servers available” | Grace period expired on an RDS session host. Point the host at a licensing server or in a VPS context, remove the RDS role if you only need two admin sessions. |
| 0x904 | “Remote Desktop can’t connect…” | Listener not bound, port moved, or TCP-level firewall drop. Verify PortNumber and the inbound rule. |
| 0xC000006D | “The logon attempt failed” | Wrong password, expired password, or the account is locked out. Check the Security event log for 4625. |
| 0x1104 | “Because of a security error, the client could not connect” | Certificate mismatch after a hostname change. Rebind the RDP certificate. |
| 0x1101 | “This computer can’t connect to the remote computer” | RDP-Tcp listener disabled or fDenyTSConnections back at 1. |
Fix 1: The connection is refused entirely (0x904, 0x1101)
If Test-NetConnection fails on port 3389, the listener or the firewall is the problem. fDenyTSConnections is the master switch — some hardening scripts flip it back to 1.
# Re-enable Remote Desktop at the registry level
Set-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server" -Name fDenyTSConnections -Value 0
# Ensure the firewall allows RDP on all three profiles
Set-NetFirewallRule -DisplayGroup "Remote Desktop" -Enabled True -Profile Any
Get-NetFirewallRule -DisplayGroup "Remote Desktop" | Select-Object Name, Enabled, Profile
# Verify the listener is bound to all addresses
Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" | Select-Object PortNumberIf you changed the port away from 3389, remember that the client must use host:port, and any host-level firewall outside Windows (provider security group) must be updated too. Renaming the default listener breaks tools that assume RDP-Tcp; keep the name and change only the port.
Fix 2: NLA and credential errors (0xC000006D, 0x204)
NLA authenticates you before the session is created, which is why an NLA failure looks like a credential error even when the password is correct. Three causes dominate:
- CredSSP encryption oracle remediation. An unpatched client against a patched server throws “Authentication error occurred — the requested function is not supported.” Patch the client, or relax the server temporarily:
New-Item -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\CredSSP\Parameters" -Force | Out-Null
Set-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\CredSSP\Parameters" -Name AllowEncryptionOracle -Value 2
# 0 = Force Updated Clients (default, safest), 2 = Vulnerable (temporary only)- The user is not a member of Remote Desktop Users. Only Administrators are allowed by default, and on hardened builds even that is restricted by policy. Add the account and confirm the right:
Add-LocalGroupMember -Group "Remote Desktop Users" -Member "CONTOSO\jdoe"
# The policy that actually grants the logon right
secedit /export /cfg C:\temp\secpol.cfg
Select-String -Path C:\temp\secpol.cfg -Pattern "SeRemoteInteractiveLogonRight"- Account lockout or expired password. A VPS that has been idle for 60+ days may have a serviceable but expired local password, or your source IP has been blocked after failed attempts. Check:
Get-WinEvent -LogName Security -MaxEvents 200 |
Where-Object { $_.Id -in 4625, 4740 } |
Select-Object TimeCreated, Id, Message -First 10Event 4740 is an explicit lockout; 4625 tells you whether the failure was a bad password, an unknown user or a logon-type restriction (“logon type 10” is RemoteInteractive). If the sub-status is 0xC0000234, the account is locked — wait or unlock it via the console.
Fix 3: Certificate and identity warnings (0x1104)
“The identity of the remote computer cannot be verified” is not fatal — click through it and the session works. But it usually means the RDP listener is presenting a self-signed certificate whose subject does not match the name you typed. After you rename a server or attach a public DNS record, rebind the certificate:
# Find the RDP certificate thumbprint
Get-ChildItem Cert:\LocalMachine\My |
Where-Object { $_.EnhancedKeyUsageList.ObjectId -contains "1.3.6.1.5.5.7.3.1" } |
Select-Object Subject, Thumbprint, NotAfter
# Bind it to the RDP listener
$thumb = "A1B2C3D4E5F6..."
$path = "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp"
Set-ItemProperty $path -Name SSLCertificateSHA1Hash -Value ([byte[]] -split ($thumb -replace "(..)", "0x`$1 "))For a standalone VPS the simplest correct setup is an internal CA or a self-signed certificate with the FQDN you actually use, combined with NLA enabled. If you manage several servers, a proper PKI is worth the effort — it removes the warning for every user at once.
Fix 4: Connects then drops, black screen, or immediate disconnect
- Black screen for ~30 seconds then 0x904: often a GPU/display driver or the session is being redirected to a console session. Force a new session with
mstsc /adminand check for a hungLogonUI. - Disconnects every few minutes: idle/keep-alive mismatch. On the server set
MaxIdleTimeandMaxDisconnectionTimeunder the connection’sWinStationskey, and on the client set keep-alive:
# Server: kill dead sessions after 30 min, but never auto-disconnect an active one
Set-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" -Name MaxIdleTime -Value 1800000
Set-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" -Name MaxDisconnectionTime -Value 0
# Client keep-alive (on your workstation), .rdp file
# keepalive:i:1
# (also set HKLM\SOFTWARE\Microsoft\Terminal Server Client\KeepAliveInterval=1)- “You have exceeded the number of connections” / only two sessions: Windows Server in non-RDS mode grants two administrative sessions. If they are wedged, log in with
mstsc /adminor a provider KVM console and clear them:
quser
qwinsta
# Then log off the stale session
logoff 3Preventing the next outage
Most recurring RDP incidents come from three self-inflicted changes: a hardening script that disables NLA compatibility, an IP-based firewall rule that locks out the administrator after an ISP address change, and an expired RDP certificate. Before you make firewall or policy changes, confirm you have console access through the provider panel. If you are still choosing a provider, compare Windows VPS options with out-of-band console access built in — it turns a lockout into a five-minute recovery instead of a support ticket.
Once RDP is stable, move on to the rest of the stack: securing RDP with Network Level Authentication and reviewing Windows Firewall profiles and rules are the natural next steps.
