I usually end up detailing past midnight at least once per week. Week before last, I was in a crunch, had more work than I could handle, deadlines that had to be met, and detailed all the way through to 6 in the morning. That was a bad idea. I realized that I reach a certain point of exhaustion, and my productivity goes waaaay down. I get so tired that it takes me 5 hrs to do what should've taken 2, and my quality of work suffers.
As a general rule, though, I love detailing late at night. I'm nocturnal by nature (seriously). If you're inside the garage with the door down, it shouldn't bother the neighbors. You've GOT to keep your garage clean. If you don't have plenty of room, you'll get slowed down and your productivity will drop. You've also got to have REALLY good lighting. Everything looks clean when it's not well lit.
Just my thoughts.