There was a comic event in which they fight, it was just before Han walks in on them in the movie at cloud city. Fett had a lightsaber he had taken from a jedi survivor he had killed. Vader beat Fett pretty quickly. However the lightsabers were all they used. Also the reason for the fight was to test their saber skills alone. Vader considered Fett unworthy to wield that weapon and took it from him at the end of the fight. If the really duked it out I think Fett would come out on top.