byStarTW says to YSITD
The error message means that the WinVNC server you are connected to is not an UltraVNC one.  You probably have a previous VNC distro installed on the server (running in service mode) that is run instead of the UltraVNC server.