There are always people out to complain about something.
IMO I wouldn't answer the phone calls. Id let them go to voicemail and if he is dumb enough to leave you some nastygrams then you have something on him. If it continues and the calling is non stop then you have harassment by communications.
The thing about having a recorder going while you are on the phone with the person is that its against the wire tap act and could not be used in a court setting. Though a good idea in some instances it wouldn't go far in the court setting.
I would say let it go, forget about it, and move on. Should things escalate to something more serious like threatening phone calls, continuing harassment, or anything of that nature save and or print out a log from the phone company for your records and for the police to see so that it can be noted in a report and for a judge should you have to go that far.
I hope that nothing goes further and the guy stops. Once he sees you are not bothered by his remarks hopefully he will give up, move on, and you can go about your everyday.
Good luck keep us posted!!