Test drive both, it made me choose a challenger. Once you get over the habit of stabbing the pedal off every line, here's what you may want to consider:
Challenger is just a bigger car.
People can actually sit in the back seat if necessary.
You as the driver have plenty of room and that's great for extended drives and daily commutes.
You have a useable trunk.
Field of vision in Challenger is better than mustang, and 10x better than a camaro.
It can actually suck up the bumps in the road.
Tons of room under the hood if you're a fix-it-yourself kind of guy.
It actually looks like a muscle car, instead of a muscle-Euro hybrid, or Transformer.
The interior is nicely laid out, and feels a bit more high quality than the competition.
If you have the coin and "the need", you can have the fastest certified production car of the big three (Hellcat). Heck of a base to build on, and when you get going that fast, every manufacturer costs big coin to go faster.
The Challenger gets bagged on for cornering ability, but it's handled any street I've driven down just fine. If you don't road course your car, you'll never know it's limitations in that regard.
Nice, big, flat body panels to buff out. Finally, a reason for those 6.5" pads on my FLEX.
There are other reasons, and some negatives too, but that's the major points that interested me when I was looking in 2012. I've gotten a little more realistic in my personal expectations of my mid life crisis mobile than brand loyalty or fanboism of a particular car.