Building standalone Qt application on windows
In any case, you need to build a static version of qt as described here : http://doc.trolltech.com/qq/qq10-windows-deployment.html
To provide an icon for the application, see http://doc.trolltech.com/4.4/appicon.html
You will still need to bundle the file c:\qt\4.4.0\bin\mingwm10.DLL , though it's said that it is possible to get a static version of this file.
Using MS Visual C++
You need to install MSVC express from http://www.microsoft.com/express/download You'll have the toolchain but not the interface, but we don't care here.
Binaries compiled with MSVC++ also needs a runtime dll, but it is shipped with windows so you don't have to ship it with your binary.