Troubleshooting RDP Connection Failures: Error Codes and Fixes for Windows VPS

A Windows VPS that suddenly refuses RDP connections is rarely “the server is down”. In most ticket queues the hostname resolves, ICMP answers, port 3389 is open, and the client still fails with something like 0x904, 0x516, or a bare “An internal error has occurred”. Those codes come from three different layers — the client Win32/WinINet stack, the CredSSP authentication handshake, and the server-side session manager — and each one points at a different fix. Reading the code before touching a firewall rule saves hours.

This guide works through the codes you will actually see on a Windows Server 2016-2022 VPS, what each layer means, and the registry or PowerShell change that resolves it. The defaults cited are for a standard Windows Server 2022 image with RDP enabled by the provider, which is what most plans ship with — see the Windows VPS plan comparison for what your image includes.

The three layers that produce RDP error codes

Layer one is the mstsc client and its underlying WinINet connection, which fails with 0x104, 0x904, or 0x11001 before authentication even starts. Layer two is CredSSP/NLA, which fails with 0x4 or 0x500 when certificate or encryption policy mismatches. Layer three is the session manager on the server, which fails with 0x516 or 0x5 when you have exhausted sessions or the Remote Desktop Services service is unhealthy. If you can see the login screen and then get disconnected, you are at layer three.

Check which layer you are at by connecting with NLA disabled from the client: mstsc /v:203.0.113.10:3389 /public bypasses some gateway behaviour, and clearing the “Always ask for credentials” box removes saved-credential noise. If the connection now reaches the security prompt, the failure was client-side credential caching and not server policy.

0x104 and 0x904: the client never reaches the listener

0x104 means TCP connect failed. Either the listener is not bound, the provider firewall is closed, or the Windows firewall dropped the packet. Test from outside with Test-NetConnection -ComputerName 203.0.113.10 -Port 3389; a False on TcpTestSucceeded means nothing is listening. On the box (via provider console) run Get-NetTCPConnection -LocalPort 3389 -State Listen and Get-Service TermService. A stopped TermService is the classic cause after a Windows Update reboot interrupted by a snapshot rollback.

0x904 is a timeout after TCP succeeded but the TLS/session handshake stalled — usually an MTU problem on tunneled links or a security appliance in the path resetting long-lived connections. Lower the client interface MTU to 1400 and retry. If the connection is via a VPN, confirm the VPN is not blocking outbound TCP 3389 to the VPS public IP; several corporate VPN profiles do exactly that.

0x516: session limit or disconnected-session exhaustion

0x516 means “your session was disconnected because there are no Remote Desktop License Servers available” or, more often on unlicensed VPS, that the two interactive sessions are already taken by stale disconnected sessions. Windows keeps a disconnected session in Disc state for hours by default; a third login refuses. List them with query session /server:localhost and clear with logoff <id>.

Cut the retention windows so this stops recurring. Under HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services set MaxDisconnectionTime to 900000 (15 minutes, DWORD in ms) and MaxIdleTime to 3600000. Both require fDenyTSConnections=0 and a gpupdate /force or service restart. For admin-only access on a VPS this is the single highest-value change after NLA.

0x4 and 0x500: CredSSP and NLA handshake failures

0x4 is SEC_E_INTERNAL_ERROR raised inside CredSSP; 0x500 is an authentication failure surfaced by the same stack. Both commonly appear after a patch changes NLA enforcement and the client is older than the server’s policy. On the server, HKLM\SYSTEM\CurrentControlSet\Control\Lsa value LimitBlankPasswordUse=1 blocks accounts with blank passwords, and UserAuthentication controls NLA requirement (1 = required).

Do not “fix” 0x4 by setting AllowEncryptionOracle=2 on the client — that is the patched CredSSP downgrade that BlueKeep-era advisories specifically told you to remove. Instead confirm the account is not locked and that its password is not expired: net user myxiechengxuan shows the expiry. An expired cloud-image admin password produces 0x500 on every attempt while the console login still works.

Quick diagnosis table by code

Code	Layer	Most common cause	First command to run
0x104	TCP	Listener down / firewall	Test-NetConnection -Port 3389
0x904	TLS handshake	MTU, VPN, appliance reset	ping -f -l 1400 host
0x11001	Client credential	Saved credential rejected	cmdkey /list
0x516	Session manager	Stale disconnected sessions	query session
0x4	CredSSP	NLA/encryption mismatch	Event 36871 in System log
0x500	CredSSP	Password expired / locked	net user <name>
0x5	Session manager	RDS service hung	Restart-Service TermService

Reading the evidence in the event log

Every failed logon attempt writes Security event 4625 with a Status and SubStatus pair. 0xC000006D / 0xC000006A is wrong password, 0xC0000234 is account locked, 0xC0000072 is disabled, and 0xC000015B is logon-type-not-granted — which is what you get when the account is not in Remote Desktop Users and the server still let the TCP connection through.

Pull them in one line: Get-WinEvent -FilterHashtable @{LogName='Security';Id=4625;StartTime=(Get-Date).AddHours(-1)} | Select-Object TimeCreated,@{n='Sub';e={$_.Properties[6].Value}}. If there are no 4625 events at all despite the client failing, the attempt never reached LSA and the problem is network or listener, not credentials. That single distinction eliminates roughly half the guesswork.

When to stop debugging and change the listener

If the server sits behind a provider NAT and the console shows a 0.0.0.0:3389 listener but external tests fail, the mapping has drifted after a rebuild. Either re-apply the port mapping in the provider panel or move RDP to a non-standard port and open only that port: Set-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -Name PortNumber -Value 3391, then restart TermService.

For anything exposed to the public internet, put RDP behind a jump host or a WireGuard tunnel and leave 3389 closed. A code-by-code fix is only useful while you still control who can reach the listener at all. Before you spend another evening on 0x516, check whether the plan itself is the bottleneck — the current image options and included resources are listed on the provider comparison table.