This is an often asked question and one to which there isn't an easy answer.  The terms come from the Golden Age of comics when comics had a more grandiose sense of being - they were based more in fantasy than in real life, so fictional place names were more fitting with the times.  As comics evolved somethings were seen as so much ties to the mythology surrounding the heroes, whether this be for the bad (underwear on outside of costume) or good (character backgrounds.)   Marvel got a lot of good press in the 1960s and 1970s when it rolled out its modern lineup of characters.  A large portion of these characters were based in real places (specifically New York City) and even the made up locations sounded more real.  Does this allow the reader to identify more with the comic?  I don't think so necessarily.  But why hasn't DC ever come out and just said that say Metropolis is really New York City and Gotham is really Philadelphia? (an association I always thought to make sense, at least geographically.) 
 
I think there are a couple of reasons for this.  The first is that in being vague the cities can be anywhere.  One of the long time running jokes in the Simpsons is that their hometown of Springfield never has a state assigned to it.  This allows fans to debate the location and argue where it really is (some have even made the case its in Canada).  Springfield makes sense for this joke, it is the most popular town name in the USA.  Gotham and Metropolis also benefit from it, because those names aren't taken (at least that I know.)  While everyone debates the two most famous DC cities, it is generally considered that they are on the East Coast of the US somewhere, but this doesn't hold true for all cities.  Central City might suggest Chicago, but there are a lot of potential cities.  Coast City might be San Francisco, but then so might Gateway City.  Even the one time home of the JLA, Happy Harbor is so vague as to be anywhere, though it is identified as being in Rhode Island.  The application of fictional cities to any city means that the fictional cities can represent a lot more people than any one city can.   
 
Back to only Gotham and Metropolis though, these cities almost always are associated in some way with New York City, although they have been clearly defined as three separate places.  One writer succinctly described the distinction being that Metropolis is New York in day time and Gotham is New York at night.  I think this is on the right track but it is missing a little something.  For any of you who ever watched the Oliver Stone movie "Nixon", you might want to save yourself the time and read the encyclopedia instead.  He paints Nixon as someone fascinated by JFK, which seems more like Stone himself than the ex-president.  However, there is one great line I liked in the film, though not in its original context.  Nixon is looking at a portrait of Kennedy and says "When they look at you they see who they want to be, when they look at me, they see who they are."  For me this is the real reason between Gotham and Metropolis.  Gotham is the reality of modern life, cities full of problems and internal strife.  Metropolis is the potential of modern life, still with its share of problems, but dealt with more progressively.  In short Metropolis is New York City as we dream, and Gotham is New York City as it is.