They are by far three of my favourite characters in American TV shows, if Dexter were to battle against the two meth cooks would he succeed in eliminating them both assuming that Dexter already knows the identities of Walt & Jesse and that the duo are aware of who he is as well?
Although Dex and Walt are both extremely intelligent, Walt has less experience in finding and killing a target which Dex by far has an abundance of and that's what gives him his advantage. Dex is also better at preps and is more intelligent than Walt in many ways (except for chemistry of course), ultimately I think that Dex would approach and eliminate Heisenberg before he could see it coming.
As for Jesse Pinkman, seeing how he's less intelligent, less thoughtful, less prepared, less prudent, less precautious, more reckless and overall an inferior to Walt, there's no way that he could out smart Dexter, Dex would eat him for breakfast. He'd of course stand a better chance if he remains obedient to Walt but ultimately, I still reckon that Dexter would end up killing the both of them.