Please remember as you read this that I'm not an expert on cameras. I like pictures first and foremost. The gearhead gene in me is pretty strong, but it's not the main current. I have deliberately stayed away from collecting, since I lack the money, the display space, and the passion to do it well.
There are hundreds if not thousands of sub-specialties in camera collecting. You can collect stereo cameras, consumer snapshot cameras, Kodaks, Japanese cameras, monorails, pre-WWI cameras, Leica copies, you name it—and you can spend a lifetime happily amassing a collection without ever straying outside the bounds of the specialty you choose.
You can collect oddities, rarities, or "association" items—pieces that were owned or used by well-known personalities. I think I've mentioned before that I once owned a Leica M6 that belonged to the actor Jack Lemmon. Ralph Gibson, in New York City, still uses the old Leitz Focomat that Robert Frank used to print the original pictures for The Americans, which is a piece that I hope ends up in the Smithsonian some day.
And, of course, research is the key to any good collection (of anything). The collection itself is just the physical manifestation of everything you'll learn along the way; the knowledge is really the fun part.
It would probably help to know before you start whether you want to concentrate on cameras you can use, or cameras for display only. Using classic cameras is often a challenge, but can be a lot of fun, too.
If you're just starting out, and assuming you want to gather a nice general overview of photography's history, this is my idea of a few things you might want in order to make a good start:
A late 19th–early 20th century folding or box wooden view camera such as this E. & H. T. Anthony Fairy Novel with mahogany body, cherry base, and nickel hardware, manufactured from 1884 to 1898 in New York. You can find such cameras (although possibly not this one) on Ebay surprisingly easily, once you know what to look for, often for not very much money too. The site I mentioned the other day is a good resource for American cameras. Obviously, for a collectible, original condition is the highest priority.
Eventually you would want a 20th-century Deardorff in your collection too, but those are often still in use and are much in demand by collectors (especially in Japan), and can be pricey.
A Kodak box camera. I'd choose a No. 2A or No. 3 to start with. Hold out for a good one; they're common and cheap. These were descended from the original "You Press the Button; We Do the Rest" Kodaks.
A Graflex Speed Graphic. The so-called "press cameras" that were in wide use from the '20s to the '50s. There are many different varities of press cameras, but the Speed or Crown Graphics are probably the most iconic. See Graflex.org to start your research. Again, these are still very common and not expensive, but they come in every conceivable sort of condition; do your research first.
A good folder. The Zeiss Ikonta (of which there were innumerable varieties) is probably the pinnacle of this sort of camera, but there was a multitude of different types; early or late, beautiful or cheap, take your pick. Jurgen Kreckel's certo6 website specializes in repair and restoration for use, but is also a good place to learn about brands and models. Of course many of these cameras are still fully usable, as you know if you follow TOP. This nice shot was made by Michael Ging using a 1936 Zeiss Ikonta B.
A screwmount Leica with a 50mm collapsible Elmar. The so-called "Barnack cameras" were the "miniature" camera that helped establish 35mm as the eventual standard. Sooner or later any self-respecting collection must also contain a Leica M3, preferably with a 7-element 50mm Summicron, but start with a screwmount. There's a huge literature about antiquarian Leicas, and vast amounts of information on the web; here is a nice brief introductory page by Andrew Yue. Screwmount Leicas require a little more care in use than more modern cameras, but are still very usable; I have some nice shots made with them.
A Rollei TLR. No camera collection would be complete without a few well-chosen TLRs, and the ones made by Germany's Franke & Heidecke took pride of place for many decades. These can be expensive but are also easy to collect—and you have a slight advantage if you intend to display your find, because the later models more sought-after by users are perhaps not as appropriate as collector pieces as earlier models, which users stay away from.
The Pentax Spotmatic. The canonical lens is the Super-Takumar 50mm ƒ/1.4. Naturally, your collection will eventually have to have a fine Nikon F to make any claim to completeness, but F's are expensive—especially nice ones. More popular in the 1960s was "The Pentax." Spotmatics were so well made that sellers often think average ones are exceptionally good! For old cameras, Spotties can be very clean, so keep your standards high.
Sadly, the best book on the subject, Gerjan van Oosten's The Ultimate Asahi Pentax Screw Mount Guide 1952—1977, is now unobtanium.
The Polaroid SX-70. There's a quixotic effort underway to make film for this camera again, which would of course affect its usability, but either way it's highly collectible and accessible in cost. Instant-film cameras in general would make yet another specialty collection, but any general collection needs a few high points.
The Canon AE-1. Another camera that's still very inexpensive and all over Ebay. The AE-1 was significant because, as promoted by Aussie tennis star John Newcombe, it was the best-selling SLR of its day, and really raised Canon above its Japanese competitors and put it on the map as an eventual rival to Nikon. It was also the first camera to use plastic extensively to replace metal; the top plate—previously the most expensive single part of SLRs—was ABS plastic, cleverly chrome-plated to look like metal. The innovation allowed Canon to undercut competitors' pricing while still making more profit. Wikipedia has a good article on the AE-1.
Other Canons of roughly the same era are eminently collectible now too. The A-1 was the most automated camera of its day, and, in my view, marks the end of the "classic SLR" era of MMM (mechanical, metal, manual) SLRs, and the T-90 was the foundation for the EOS camera line, a design heritage still evident in today's Canon DLSRs.
An early digital camera. Don't forget that the most valuable collections are of things that people figure out to collect before everyone else knows they're valuable. Years from now, people will be amazed that we were awash in rare early-period digital cameras and we didn't even care. I hope there are at least a few intrepid souls out there collecting early digitals!
Check out DigiCamHistory.com for more.