In my opinion, I think it's Deathstroke. I know they're only faced each other a handful of times, but each time they fight, Batman can barely hold his own against him. Not to mention that Deathstroke has beaten Batman more than vice versa.
As far as his family and girlfriend knows, Terry works for Bruce as a job, so he's gotta have money coming in, right? Wouldn't it be a bit suspicious if he's going to see Bruce all this time and doesn't make a single dollar?
Let's say the Punisher is going after the Joker and he uses the same tricks he does on Batman, traps and all. Do you think the Punisher would be able to survive through it all and capture and kill the Joker, even when they meet face to face?