I ran into a really strange problem tonight on a server (domain controller) where I was attempting to upgrade WSUS 2.0 running with a SQL Express backend to WSUS 3.0.
The setup program said "success" but after seeing nothing new in the Start menu, I checked the wsussetup.log and I saw that it failed. I tried uninstalling WSUS 2.0 from Add/Remove Programs, but that faild too (invalid parameter). I then tried re-installing over, no go. So I manually deleted every folder and uninstalled SQL completely, then tried again. No good! I was stuck inbetween a broken WSUS 2.0 and broken WSUS 3.0 and it wouldn't uninstall or install over, even though everything was manually deleted. In the wsussetup.log it kept saying...
2007-08-19 22:41:24 Success MWUSSetup Validating pre-requisites...
2007-08-19 22:41:24 Error MWUSSetup Failed to determine if an higher version of WSUS is installed. Assuming it is not... (Error 0x80070002: The system cannot find the file specified.)
2007-08-19 22:41:24 Error MWUSSetup IsWSUSInstalledOnPort80: cannot found WSUS installed on any website (Error 0x8000FFFF: Catastrophic failure)
2007-08-19 22:41:44 Error MWUSSetup CSqlConnection::Open: Failed to open SQL connection to instance MACAW (Error 0x80004005: Unspecified error)
2007-08-19 22:41:44 Error MWUSSetup CUpgradeDriver::PerformPreSetupActions: Failed to open connection to SQL instance %computername% (Error 0x80004005: Unspecified error)
2007-08-19 22:41:44 Error MWUSSetup CSetupDriver::LaunchSetup: Failed to perform pre-setup actions (Error 0x80004005: Unspecified error)
2007-08-19 22:41:44 Error MWUSSetup CProgressManager::EnableCancel: GetDlgItem returning error (Error 0x80070578: Invalid window handle.)
2007-08-19 22:41:54 Error MWUSSetup DoInstall: Wsus setup failed (Error 0x80004005: Unspecified error)
So what can I do? I ran good ol' regmon from SysInternals (I still haven't gotten used to procmon) and looked through all the entries from the installer MSI trying to find out how this thing is detecting if WSUS is installed... And I found it. Even if it doesn't show up in Add/Remove Programs, there are still entries in the installer areas of the registry that trigger an upgrade option, even if it shouldn't. If it finds the GUID {6CD64D0A0598A1549809358CE67166E6} in any installer location of the registry (such as HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Products OR user-specific portions of the registry), it will assume WSUS 2.0 is installed, even if it isn't. The solution? Delete all occurances of that GUID in the installer areas of the registry. Then it will magically install WSUS 3.0 from scratch.