We start off with a list of my 5 favorite Avengers who weren't members of the team for very long. Obviously, I had to make a cut off for what was deemed a "brief tenure" since some could say Starfox or Quasar had brief tenures. If a character appeared in more than 6 issues as an active Avenger, they're not eligible for the list.
5. Demolition Man
Probably the biggest example of a short stint as an Avenger, Demolition Man (or more commonly D-Man) was recruited by Steve Rogers for the Avengers after they disassembled (back in the 80s, before it was "cool"). The Avengers needed to reform to battle the threats of Inferno and the Evolutionary War, and at the time, D-Man was one of Steve's closest friends and therefore one of his the first choices for the team. D-Man was an Avenger for all of an issue. And not even an issue of Avengers but instead an issue of Captain America. His sole mission saw The Avengers duo went to the Antarctic to rescue John Walker from Flag-Smasher and ULTIMATUM. Despite his silly Daredevil-meets-Wolverine costume, D-Man was inherently a nice guy and actually earned his place as an Avenger as evidenced by him sacrificing himself to save the life of a terrorist resulting in him being blown up over the Arctic sea (in a scene obviously mirroring what happened to Bucky). Robbing him of the noble death, D-Man was brought back a few years later and since helming the Marvel Universe Brian Bendis has seemingly made it his prerogative to make D-Man into even more of a laughing stock as a mentally-ill homeless villain.
Darkhawk had a solo series that lasted 50 issues, he was a member of the New Warriors and the Avengers and he has recently played prominent parts in events like War of Kings. Despite all that, he still struggles to find a proper place in both the Marvel Universe and in ours. Darkhawk joined the Avengers in a story that ran across annuals in 1992 called "Assault on Armor City". In fact, it was a really good story. In a typical comic book manner, after they fought for a while, Darkhawk teamed up with the Avengers and helped them battle the real bad guy: Professor Power. Darkhawk then appeared with the Avengers in their own book for 3 issues as they battled Doctor Demonicus and the Pacific Overlords. And that was it for Darkhawk's tenure as an active Avenger. He opted to resign to being a "reserve" member which didn't work out well for him since the Avengers abandoned that entire system a couple of months afterwards.
It's impossible to do this list without listing the most famous and memorable Avenger who wasn't with the team for long. Hulk was a founding member who quit in the second ever issue of Avengers. Hulk is a loner and a liability, so it only makes sense that he wouldn't stick around with the team for long. Although Hulk has been optioned to rejoin the team a few teams, most notably during his "Professor Hulk" days during his frequent team-ups with the Avengers during the Infinity Gauntlet, War and Crusade, he doesn't have any desire the re-enter their ranks. Simply put, Hulk and the Avengers aren't a match. In fact, he's appeared more often as an adversary to the Avengers than as an ally. Although, with the Hulk an Avenger in the movies and Marvel's pandering to the movie market, I wouldn't expect it to be long before Hulk rejoins the team fully for some banal reason.
In the early 80s, the rather odd decision was made regarding the traditional and fairly common Spider-Man and Fantastic Four enemy Sandman. After one defeat too many, Sandman decided to move towards reforming. Working mostly as an ally to Silver Sable and Spider-Man, at the height of his heroic phase he was chosen by Captain America to join the Avengers following receiving a pardon for his former crimes. Sandman wasn't an Avenger for long, in fact if you were just reading the Avengers book you wouldn't have gotten much of an explanation for his sudden disappearance! A few months after joining, over in Amazing Spider-Man, the Avengers guest starred (don't forget Spider-Man was an Avenger back then too) and Sandman left the team at the end of the issue after a misunderstanding with Captain America. In a stupid moment of 90s nonsense, Howard Mackie and John Byrne decided to have Sandman become a villain again and ignore all the interesting work done with his reform.
With Siege over and done with, the Avengers recruited some new members for their ranks. Among them was fan favorite and Cosmic darling Nova on the Secret Avengers team. Although putting him on the black-ops group did seem a bit of a bizarre choice, Nova is a character who legitimately fits the Avengers. The fact that it took them so long to get around to making him an Avenger is more of a surprise than anything else. Sadly, Nova's tenure with the team was undeservedly cut short when he perished over in Thanos Imperative. Of all the brief tenures, Nova was probably the most upsetting since he so obviously deserved more of a run with the team.