Getting rid of the .exe file more than likely will not for sure get rid of the virus.
A few years ago that might have worked if you were lucky, but as of now, viruses and malware are allot more advanced, and will usually install helper files (more than one, and sometimes two or three) that...