I’m sitting here in the airport in Milwaukee*, off to the beautiful Southern Tier of upstate New York to visit my girlfriend. I spent all last night trying to reconfigure the computers to the new regime, in which I’ll be using the new laptop** docked at home and detached on the road. It’s not easy going from a 2TB onboard drive to a 512GB one.
Here’s the read on disk space for me: the old iMac with its 2TB drive had 800GB free (I’m rounding the numbers a bit). Once all the music and video files were transferred to a 2TB WD Passport drive (and a backup), the main computer said there was 1TB free. After copying the main folder containing the lion’s share of my pictures, the iMac said 1.75TB free. Swinging my formidable arithmetical skills into play, I calculate that I have 200GB of music and 750GB of pictures.
And if you’ve been reading right along, you’ll recall a salient fact that I brought up recently: I’m a light shooter. And another pertinent post was the one in which I asked people about their heaviest shooting days. Which gives rise to a small question in my mind: what the heck do heavy shooters do with all that shooting? How do you archive 10, 50, or 100X the paltry amount of shooting that I do?
I’m curious as to whether people actually keep it. Do you edit heavy shoots and discard everything that’s clearly unpromising? I don’t, or at least I haven’t. I just dump everything in a folder and there it sits. Or do you tend to maintain large numbers of redundant external hard drives?
I think if I do my digital OC/OL/OY project, I’m going to use the opportunity to revamp my habits and bring them up to something like modernistic 2007 standards. I just don’t pay as much attention as I should. (It hurts me when I’m searching for images too. You won’t believe this, but I file everything according to what camera I used to take it. Might not make much sense, but it just happens to be what my brain remembers—I can picture the photograph in my mind, and I’ll remember what lens it was taken with. Then I can deduce that camera and that’s how I go look for the picture. I don’t suppose many other people have such an idiosyncratic filing method. Nor am I recommending it, I hasten to add!)
Speaking of the digital OC/OL/OY, I think you can use any lens you want to—just as I think you should modify the exercise to fit your needs, your capabilities, and your life in whatever way seems to make the most sense to you. We’re havin’ fun here, after all. I could do the exercise with any of the 35mm or 40mm (or equivalent) lenses I have, for example. But my ambition is to do it with the A7 and Zeiss 55mm—that’s far enough away from my comfort zone that it would challenge me. And maybe lead to new kinds of work. As I said before, I really don’t know if I can do this exercise myself. But I’d like to.
This is a working vacation, so you’ll be hearing from me regularly this week.
Mike
*Posted later. Speaking of 2007, the Milwaukee airport doesn't have free Wi-Fi.
**It’s a 13-inch MacBook Air with 8GB RAM and a 512GB solid-state drive. Good enuff.
Original contents copyright 2014 by Michael C. Johnston and/or the bylined author. All Rights Reserved. Links in this post may be to our affiliates; sales through affiliate links may benefit this site.
(To see all the comments, click on the "Comments" link below.)
Featured Comments from:
Don Davis: "Over a thirty-nine year newspaper career, I always regarded myself as a 'light shooter,' as well. The joke on the staff was: 'You can shoot the end of the world on two rolls of Tri-X.' When pressed, editing endless frames means you can often overlook the right one. Just my two cents."
Tommy Williams: "We are probably on opposite ends of the computer technology spectrum (I have been working in software for nearly 25 years). And yet here is an odd commonality: I still group my photos in my filesystem based on which camera I took them with, too. Even though I have 80K+ photos cataloged in Lightroom going back to my first digital camera in 1997, I still often start by thinking of which camera I used to take the picture rather than using other keywords, or dates, or locations."
Gene Forsythe: "When I am shooting (primarily wildlife and canine sports) I am what you would consider a heavy shooter, so I might address your question on storage and archiving.
"I just returned from 10 days in South Africa, with some 25,000+ images. That will reduce to no more than 200 over the next week or so (first pass cut over 15,000 alone). What happens is this—the lion is in super light, and just waking up, and we are perfectly positioned (for a change) with no obstructions. He looks like he is going to yawn and show his teeth. I start shooting at 8fps, and continue till he is finished, about 45 frames later. Now I am back home and there are 39 so-so frames, three OK frames, two nice shots, and one worth printing. I keep the last three and dump the rest. And so it goes. My master backup is a pair of 3TB portable drives which are currently sitting in the safe. Once everything is edited down to the final set and a backup made of just those shots, I will pull the two portable drives and erase them so I can use them on my next trip."
Doug Plummer: "I'm a professional photographer, and it's not uncommon for me to shoot 2,000+ a day on an assignment. On a day that I'm shooting for myself it can be as few as a dozen or several hundred. I'm very committed to shooting daily (I've had a daily photo blog for more than nine years now), whether I'm working or not. I fill up a lot of hard drives—I have 24TB in a JBOD box connected to the desktop via eSata (I shoot video too, which swamps storage issues as far as still goes), and I backup on my old hard drives as I upgrade."
Ben: "Lightroom and keyword tagging. Try it and you'll never go back."
Adrian Malloch: "I find that deleting photos is the antithesis of creating photos. It completely discombobulates me to choose photos for annihilation. Since I'm a professional photographer shooting 500 to 5000 frames a day that is a lot of raw files to accumulate, but storage is cheap compared to the time taken to expunge the unworthy. After an exhausting day of photography, an exhausting evening of selections, processing and uploading to proof galleries for clients, I am in no fit mental or emotional state to eradicate. That precious time I use to find the gems, the moments, the photographs that cannot be ignored. Lightroom enables me to quickly narrow my selection without distraction of the 'maybes' and the 'nearly but not quite.' and then focus my attention on the frames that I want the client to see.
"After 10 years of freelance work I have 22 TB of photos on external hard drives and over 50 TB of backups. My approach is regularly vindicated by the continuing value of previously rejected photographs. For example, I recently reviewed some older assignments and put together a large collection into a book. Nearly a third of the photographs that are in the book are images that didn't make the cut when first edited."
I shoot a lot, thousands of images a week at minimum. They are heavily edited, and I toss more than half, at least. The raw files that I keep are stored on a server in our office, with an onsite backup. We used to have an offsite backup, but we switched tech support and now they tell me they're working on that :). There are about 7TB of raw files on that server.
My little Macbook Air has only 256GB internal, so I have to be especially careful to keep up with editing and processing. I have run out of space when downloading cards after a couple of long days of shooting.
Posted by: Ken Bennett | Friday, 14 November 2014 at 08:02 PM
Mike, I told you the most I've ever shot in a day was between 400 and 500 images at an auto show. And that's true. But I don't shoot that much every day. Nor even most days. Not even half the time. I don't even shoot most days - two days a week on average. And I'm careful about what I shoot. I typically come away from an outing or event with one or two dozen exposures. That's how I've been able to save pretty much all that I've shot on digital.
Posted by: Steve Biro | Friday, 14 November 2014 at 08:44 PM
>> ... to visit my girlfriend.
Yay!
Good for you.
Posted by: D B | Friday, 14 November 2014 at 08:47 PM
This is what I do. Most of the data I backup is music or photos. I store the photos by year and roughly by date.
http://mutable-states.com/backups.html
I suppose if I shot enough that I needed a 10TB archive or something I'd need a couple of those large disk arrays. But I'd probably keep the flow the same.
Most recent shooting on the laptop. Main archive on the desktop with the large pile of external disks.
Posted by: psu | Friday, 14 November 2014 at 09:10 PM
Re Storage:
Get yourself one of these: http://www.wdc.com/en/products/products.aspx?id=1240
it's a 2 TB Buss powered Thunderbolt RAID (0 or 1)
Re bigger systems:
Most folks who run studio systems advocate keeping Data separate from the OS anyway so by keeping the OS and applications on a fast SSD, and Data on the RAID you could use the same data on both machines. You need to be conscientious about backup, but that's a requirement anyway.
A modification of this would be to just keep your pictures folder on an external.
Many studios use large raid arrays, with video it's a must, but for stills, modern 3-4TB USB3 drives are fast and reliable and cheap.
They are also great for rotating offsite backup, which is also really important.
Posted by: Michael Perini | Friday, 14 November 2014 at 11:11 PM
I don't waste much time editing and discarding stuff - got to put a value on your own time.
1Tb of storage costs about $40 for a bare drive.
I have an old PC (worth about $100) on the network which can hold 4 drives, each of 4TB.
Posted by: Hugh | Friday, 14 November 2014 at 11:31 PM
You asked many questions, I have many answers.
First, the question of portable storage for you.
Since you have a macbook air, I recommend a bigger drive. Next time, get the 4T raid thunderbolt with integral connection which is also powered via the connection. I just bought this for my workflow through your Amazon link.
http://www.amazon.com/gp/aw/d/B00ITI0514/ref=mp_s_a_1_1?qid=1416029600&sr=8-1&pi=AC_SY200_QL40#
You asked what heavy shooters use, and Leica S files are huge and I shoot a lot. This is the answer for me.
This is heavy in weight but tiny in size. My entire photo file is just over 3T so it fits on this tiny device. 83,000 photos. Of course when you get to home base you want to back it up to a desk drive which your iMac can access and I don't know if your iMac is recent enough for thunderbolt? If not get an external drive that has both thunderbolt and another connector that will work with your iMac.
Second, you asked if you keep all your photos. Yes I do so far. As long as storage costs keep falling, I can afford to keep them all. But your concept of filing by camera is OK only if you also add some keywords in Lightroom.
I skipped keywords for years and it was a mistake. Try the discipline of many keywords.
Third, you now gave us permission to use any lens one wanted for the OC/OL/OY. I did what your recommended accidentally in 2010 when I got my Leica S2 with a 35mm lens. I was waiting for them to issue the 120mm for an entire year, so I worked entirely with the 35mm for that first year. That was the combination I had in New England when I shot the June TOP print offer, but by 2011 I had added the 120mm. Now I have added the 24mm and a 30-90 zoom, but that year with the 35mm taught me plenty.
Fourth, when in a place with no WiFi use that fancy new iPhone as a personal hotspot to your Mac Air. Essential for your business.
Posted by: Jack | Saturday, 15 November 2014 at 01:12 AM
In Aperture you can make "smart" folders that contain the photos of individual cameras no matter what filing system you use. I presume the same applies to Lightroom etc.
Posted by: Richard Parkin | Saturday, 15 November 2014 at 02:43 AM
Speaking of the digital OC/OL/OY, I think you can use any lens you want to—just as I think you should modify the exercise to fit your needs, your capabilities, and your life in whatever way seems to make the most sense to you.
Great!
I have narrowed my lens choices to two:
Voigtlander Color-Skopar C 35mm F/2.5 LTM, or
Zeiss Planar 2/50 T* ZM
Most likely, the Planar because I'm new to it.
Here's how I plan to modify TOP/OC/OL/OY to suit my circumstances:
I'm using a week (1/52) rather than a day (1/365) as the numeraire for shooting/printing during my OY. I'll set aside at least 35 minutes (5 min X 7) for a week's shooting.
I'll print the top 5 keepers of the week at home in postcard-sized prints on standard paper. And the top-of-the week in letter-sized photo paper. I'll also have the top-of-the-week photos printed professionally (commercially, that is) each month (as "control" for in-case my entry-level photo-printer goes screwy on me).
I'll post my keepers in a weekly private blog with my notes.
My OY will start during the Holidays (Christmas lights and traditional lanterns, among others) and I'll start printing early next year. Hopefully, I can get my printer at a hefty discount during the post-season sale.
So, there. I'm committed!
Thanks, Mike!
Posted by: Sarge | Saturday, 15 November 2014 at 03:22 AM
I - more or less frequently - keyword my photos in Lightroom, but I often find myself filtering for cameras/lenses when searching a certain photo. Would not be surprised if many people's minds work like that.
Posted by: Andreas | Saturday, 15 November 2014 at 04:38 AM
Yes, I noticed the word "girlfriend" in there. Big smile.
Posted by: Peter Klein | Saturday, 15 November 2014 at 05:26 AM
Sounds to me you should be filing by lens, not by camera... And while that sure wouldn't make sense for me and my own needs, I don't think of it as being crazy either. Horses for courses, right?
Posted by: Thomas Paris | Saturday, 15 November 2014 at 07:09 AM
Something I discovered recently (and wish I'd discovered 10 years ago) is the power of using "keywords" in PS Lightroom. I now import all photos through Lightroom directly from the memory card, and "keyword" them all at once with multiple words depicting whatever happens to be appropriate for those pictures, which can include general and specified categories, location, date, type of picture, etc., (the camera make, lens, ISO, and such, are automatically recorded by Lightroom with the metadata so there is no need to "keyword" these items), whatever you want. Doing this has made my life a whole lot easier.
It takes a little (only a little) discipline at first to remember and do the "keywording," but the time saved finding photographs later on is very comforting. I can narrow a years worth of photos down to about 25-50 with a few button clicks, then easily find the one I want. The process is made even quicker using Lightroom's flagging, rating, and color coding system for individual photos.
cfw
Posted by: cfw | Saturday, 15 November 2014 at 07:28 AM
Hello Mike.
I was a heavy shooter some years ago, often using burst mode and bracketing. Spending too much time editing and too much money on storage were two of the reasons I now shoot much less photos. Another reason is just that I'm a more experienced photographer now.
Since I'm currently on a process of rebuilding my website and remastering my photos, I'm also cleaning a lot of storage space discarding unninteresting and repeated photos. The RAW files that I keep I convert them to DNG. This operation can save a lot of storage space, but I do keep the original RAW files of the absolute best photos.
Posted by: Ricardo Silva Cordeiro | Saturday, 15 November 2014 at 08:49 AM
"Or do you tend to maintain large numbers of redundant external hard drives?"
Lets see, I've 5 3TB drives, 5 2TB drives another 10 assorted drives including a couple little passport drives. Windows tends to get weird with more than 24 drives and the card reader takes 4 drive letters. There are about 8 more drives that aren't plugged in.
Does that count?
The worst part is that there are 22 drives with 22 data cables 3 usb hubs with yet more data cables 25 power supplies some of which have a line cord a box and a low voltage line to the drive and some are wall-warts plugged into about 8 power strips. That's about 70 or 80 wires
Current project is to take everything but the 5 3TB drives off line and store them at a different address.
A lot of that is non photography related , but lightroom says that I have about 400,000 images.
It's not so much that I'm no good at editing but what are the "keepers" when I take the picture usually are not what I value a couple years later. I think my current workflow is take and catalogue a lot of photos, wait 5 years, then go through them as though someone else took them and I found them at a flea market.
If you find yourself in my situation, whatever you do, do not ever use hardware raid. It seems like a good idea, but boy is ever a bad idea for the individual user. Also give each of your drives a name and paint it on the drive.
Posted by: hugh crawford | Saturday, 15 November 2014 at 10:45 AM
I've been reading your blog daily for years, ever since I found your site by reading your Luminous Landscape article on the Pentax FA Ltd. lenses, yet I don't seem to recall you ever mentioning using any particular software for cataloguing your photos. If you already use, or are open to using Adobe's Lightroom, then I cannot recommend Peter Krogh's multimedia book "Organizing Your Photos with Lightroom 5" highly enough to deal with the issues you've described above. Lightroom is a fabulous program for cataloguing your photos, but doesn't really come with any instructions on how best to use it. Peter teaches you this and it's so easy and intuitive (after you've learned it) you'll wonder how you didn't come up with this technique on your own. His method reminds me of the head-slapping simplicity and "why didn't I think of that" moment of the Post-It note. The book is well-written with clear instructions and is accompanied with 7 hours of videos showing you exactly how to do what you've learned in the book. See: http://thedambook.com/organizing-your-photos-with-lightroom-5/
For instance you'll learn:
- You can store all your photos on a drive at home, but leave the Lightroom catalogue on your MacBooks hard drive. When you go on a road trip you can leave the external drive at home and still be able to work with all your photos. It even works well if your collection of photos is large enough to span multiple drives.
- How do you cull your photos in the digital age and what do you do with the ones that aren't keepers?
- The best way to back up all those photos and your Lightroom catalogue.
- You'll learn how to find that photo of Zander taken in Maine with the Minolta 7D, or any other photo you can think of within 15 seconds or less.
I have no connection to Peter Krogh or his book, I'm simply a reader and Lightroom user who loves what I've learned from the above mentioned multimedia book/videos, and appreciates how it has revolutionized how I deal with my digital images.
As for the music collection, may I suggest you leave it on your external drive for at home, but copy it to a 256GB SDXC memory card (this one: http://www.bhphotovideo.com/c/product/1016078-REG/pny_technologies_p_sdx256u1h_ge_256gb_elite_sdxc_uhs_1_card.html can be had for $110) and leave it in your SD card slot for playing music while you travel. This frees up the maximum amount of HD space on your computer for downloading photos from your memory cards while on the road. You could also use the memory card at home, but I'd prefer to leave that slot free so I didn't have to keep popping out my music card every time I want to insert a card to download photos as I'd prefer to not interrupt the music when downloading my photos.
Posted by: Bruce Johnson | Saturday, 15 November 2014 at 11:21 AM
Girlfriend?
Posted by: Steve Snyder | Saturday, 15 November 2014 at 11:22 AM
As a great admirer of David Vestal, I adopted his very simple filing method back when I shot film and transferred it to digital when I moved there. He suggested you start each year afresh and just count up. Thus I started this year at 14001. I always add the camera immediately afterward and then when I print I add the file number. This goes on every print I make. So a recent picture would be 14090-123.
I am not suggesting you change of course, your method works wonderfully for you. But I love David Vestal's simple method.
Posted by: James Weekes | Saturday, 15 November 2014 at 11:24 AM
I hear ya.
I'm not even a photographer, I'm just a guy who likes taking pictures, and I've got 18,256 RAW files because I was always shooting RAW+jpeg (and "superfine" jpegs at that!).
I've got a 750GB Samsung SSD connected to a Thunderbolt port on my 13" Macbook Pro Retina (shoulda gone with the MBP Retina!), and I just bought a 1TB 7200rpm USB 3 WD external HD to offload the RAW files from the SSD. There's still 59,000 jpegs on the SSD.
I'm a digital hoarder.
I recently wrote a blog post about it: http://nice-marmot.net/November_2014.html#note_230
I've set all my cameras to "normal" for jpeg quality (I seldom print, and never larger than A3). Turn on RAW when it's something special and the environment is challenging.
I shoot every day with a handful of cameras (All Olympus, at least I understand the menu system! Well, plus the iPhone.) I share a few here and there on Facebook, Twitter, or the blog, and I've been posting more to SmugMug. I shoot just to record my surroundings, when they feel remarkable to me in some way. The moon on a crystal clear night? Forget it. Seems impossible to resist.
But I'm trying to delete more after every import. And share more of the ones I keep. And I need to make prints to give to family, because I'm also going through my late father's old WW II snapshots. Paper is probably the way to go for longevity.
I probably need a 12-step program.
Posted by: dave rogers | Saturday, 15 November 2014 at 11:39 AM
Filing images based on the camera used is certainly one of the more creative approaches I've ever heard of. ;-) FWIW, It would have never occured to me to do it that way. Horses for courses...
I use a venue/date format. Mundane, but effective, especially as dates are by necessity unique signifiers.
For example, for the motocycle Grand Prix race at Laguna Seca in July 2009, I would create a folder entitled "MotoGP MRLS July 2009". The folder contained a Lightroom catalog just for that event, and sub-folders for each session, e.g. Sat AM practice, Sat PM qual(ifying), Sun PM Race. The complete take for each session would go in the designated folders. Once I got back home, all I had to do was copy the race folder over to my desktop for further editing and archiving (backups are important!). Using a unique LR catalog inside the race folder meant that all my edits made at the track also migrated along with the 3000-4000 images I had shot that weekend (I typically shot 20,000-30,000 images over the course of the racing season).
For my general-purpose (anything that is not racing) photography, I create a folder and LR catalog for each month of the year, which is stored on an external 1TB Thunderbolt drive. I also copy that folder and catalog to my backup drive. Turns out I shoot about a terabyte of images a year, so for each year, I have a two 1 TB drives, the master and the backup.
For your trip, I would create a folder (and LR catalog) entitled "Southern Tier 11-14", for example.
Cheers,
Stephen
Posted by: Stephen Scharf | Saturday, 15 November 2014 at 11:51 AM
Mike,
Good filing systems are what the other guy has when he finds something faster than you could with your system.
There is no really good system. Mine is somewhat like yours.
Sort is by YEAR, then ORIGINALS BY CAMERA then processed files by subject. There are several COLLECTIONs folders as well.
Main finder is by year, then camera, then subject.
Most important is memory of or have it written down somewhere.
Posted by: Bill | Saturday, 15 November 2014 at 11:53 AM
I started filing by sequential roll numbers back in the 1960s. That somewhat broke down in the 1990s. I started shooting digitally in 2000, and originally filed those by camera (by date within that). But I fairly shortly switched to the current system where I simply file by date.
I don't delete enough, I suppose. But deletion is forever, so it's a big deal. And in groups of people, sometimes I find decades later that secondary people have become of interest, for example, so deleting the less good group shot would cost me the best shot of that one person.
Thumbs Plus reports 177K thumbnails; that includes multiples for the images I've made web jpegs or other derivatives from the original file for. I seem to have 112,700 digital original files in the server (so that doesn't include *any* of the scanned film images).
I'm running a home fileserver with 3.63T of usable disk, 2.04T in use. The backups are to 3TB external drives (three of them in rotation). This wouldn't stand up to shooting full-time for all that long, but it would be easy today to build a server with more space; say a 5-bay unit with 8TB usable (or even 16TB usable, but that means any single disk lost means the whole system has no redundancy during the entire repair period; also there's no way to upgrade it without taking it out of service). (The current unit has 12 hot-swap bays.)
I wouldn't feel the need to keep client shots forever the way I tend to with my own shots.
Posted by: David Dyer-Bennet | Saturday, 15 November 2014 at 12:30 PM
I'm amused by Don's two rolls of Tri-X story. When I was shooting TV newsfilm (yes, you read that right, news FILM. I had the station logo on the side of my dinosaur.) we had a saying, anything newsworthy gets :45 seconds, the second coming gets a minute and a half sound-on.
Posted by: Bill Pearce | Saturday, 15 November 2014 at 12:48 PM
"visit my girlfriend"
what? Where did she come from? You can't drop that line here and offer no further explanation. After all, we are family.
Enjoy!
[Can't jinx it. [g] --Mike]
Posted by: Joseph Reeves | Saturday, 15 November 2014 at 01:49 PM
Here is what I do, and it's been working well for the last 7-8 years;
- I rename and convert to DNG when I bring in my RAW files inside Lightroom
- The naming: YYYYMMDD-originalfilename.dng (I'll always know when the image was taken, regardless of where in ends up in the hard drive - it also sorts well)
- I also put them in different Sub-directory; for example today's images will go under: 2014 >> 2014-10 >> 2014-10-15 >>
- The last part could be: 2014-10-15-delmarbeach
Posted by: Shadzee | Saturday, 15 November 2014 at 02:15 PM
I take about 10,000 shots a year, and I keep everything. Instead of deleting, I just use Lightroom's "star" tags to keep track of what I think of a photo and where I am in editing it. (0 = uninteresting, 1 = interesting, 2 = edited and worked up, 3 = tagged and titled, 4 = published to Flickr, 5 = printed) My thought 5 years ago when I first started taking pictures more seriously, I figured that storage tech would improve quickly enough that I'd always be able to find sufficient storage for everything, so I chose to never delete anything. So far, hasn't been a problem...
Posted by: Nicholas Condon | Saturday, 15 November 2014 at 02:22 PM
I'm a light shooter too. And I didn't start shooting all digital until 2008. And I ruthlessly edit everything, first in camera and again when I'm converting Raw to JPG. I was a ruthless editor when I shot film, throwing out tons of slides.
I transfer photos from my Mac Mini to multiple Transcend 1TB portable hard drives and, from time to time, I delete the original files from the Mini. I went to portable hard drives when the two WD hard drives I was using for both photo back up and as my Time Machine crashed about the same time.
Thus far, I've only used something over 500GBs of space on the 1TB drives. Again, light shooter and ruthless editor.
Posted by: Dogman | Saturday, 15 November 2014 at 04:10 PM
Hi Mike
I keep my photo archive on a dedicated (read: obsolete) Windows XP computer that isn't connected to the internet, and thus never needs to be updated.
The image files are set in camera with a unique prefix, so I can trace them back to the camera they came from. I use Olympus' Transfer program to import them into folders named by date. I sweep each year's photos into a folder named "LASTNAME 2014". The folders from other family members are similarly named, e.g. MOM 2014, FATHER-IN-LAW 2003, FILM SCANS 1974, etc. These folders live in the root directory of the pair of storage drives. The archive is divided by time into 2011 and earlier (about 1.8 terabytes) and 2012 and later (about 1.2 terabytes). I have a pair of external drives that I use Microsoft's SyncToy to copy changes to. I browse the folders with Picasa, which presents a timeline view that matches how I think of my photos.
The overall scheme looks like this:
I:\2011 and earlier\LASTNAME 2011\1-1-2011\E52-00001.orf
J:\2012 and later\LASTNAME 2012\1-1-2012\PL1-0001.orf
The archive grows in both directions, as I scan my family's negatives on one end, and take new photos at the other end. As it grows, I buy new drives and copy and verify the archive on to them. On average the price of storage has dropped faster than I've accumulated files (though I got ahead in 2010, which was concerning). The replaced drives are put in a shoebox. I've always replaced these drives due to lack of space before I've had to replace them due to failure.*
How do I work with this? I make all of my selects on the archive machine, and save my working copies to a memory card. I then do all my edits on a contemporary laptop, with an up to date copy of Lightroom. Once I am done, I transfer the edits and sidecar files back to the archive machine in the appropriate YEAR folder, in a subfolder called EDITS. This makes it easy to identify the smaller number of files that were valuable enough to work with.
The system is...slow to work with. Waiting to turn on the archive machine, copying files to and from memory cards. But to me, it is worth it, because the only data losses I've ever encountered have been entirely my fault for deleting things I thought I had backed up. I do not need to delete anything to make this system work. I only ever have to keep track of four current drives, and I only plug in the backup drives when I am making backups.
*don't ask about the failed experiment in 2003 with a cheap RAID 0 array.....
Posted by: Trecento | Saturday, 15 November 2014 at 04:18 PM
I like your thinking ;-)
I've got a similar, hybrid naming convention. For film stuff, it's camera/YY-MM-DD, then a subdirectory under there for the processed files. Like you, I seem to remember which camera I took the shot with. Plus, I started this scheme before the days of cataloguing tools, and it mirrors my negative sleeves' naming...
For digital, I import the files into iPhoto/Aperture and use project names in conjunction with keywords or tags.
Posted by: Paul H | Saturday, 15 November 2014 at 04:22 PM
Is this "girlfriend" a new car??
[No, a real woman...who needs a new car, actually. :-) --Mike]
Posted by: Eric Rose | Saturday, 15 November 2014 at 04:26 PM
I have an older version of the Netgear ReadyNAS 316 - a 6 drive bay RAID box. I currently have 6TB of drives, and am using about 2TB for music and photo files. http://www.downloads.netgear.com/files/GDC/datasheet/en/RN300-RN500-RN700.pdf
This solution is a bit on the expensive side but the quality of the Linux implementation is outstanding, as has been my experience of the customer service from Netgear. I have about 50,000 photos including high-resolution scans of medium format chromes (~660 MB each!) and over 3,000 albums of music. Photos are cataloged in Lightroom by place, then year, and key worded. I leave the camera and lens info in the exif sidecar files.
Posted by: Michael Bade | Saturday, 15 November 2014 at 04:54 PM
Hey, happy for you re GF.
I used the camera name method up until a few years ago, when I realized that I was always searching around a certain >-date-< for an image, and by that time, had gone through too many cameras. Now I use a simple hierarchy of year (eg, 2014) folders, and each upload from the card goes into a sub-folder that follows this pattern: YYMMDD_initials_camera_keywords - e.g.,
141115_MR_GX7_FirstSnowToday-atHome.
My initials there because it could be my wife's shoot instead. I still want to know the camera, especially when using more than one. Then a few high-level keywords. No spaces, to keep filename short as possible.
I don't format and re-use cards. I treat them like film. When they're full, I write the camera, and date put away, and put them in a small metal box. Sort of a last-ditch archive as well.
Backup runs automatically each evening, but also, from time to time I do a copy to a pair of external drives, one of which I keep at home in a media safe, the other in a bank's safe deposit box.
As I learn to use Lightroom, I intend to keyword images at import time, but so far, haven't made much headway in that direction.
Posted by: MikeR | Saturday, 15 November 2014 at 06:26 PM
Since we've been on the topic, yesterday I took about 150 pictures, of 31 subjects. Some of these subjects were of a great number of people- variables- that I had no control over; I took many shots trying to get the picture that I wanted. This raised the average number of shots per subject.
By the way, do you think that Kirk Tuck would benefit from this One Camera/One Lens/One Year idea? (Great big grin!)
Posted by: Roger Bradbury | Saturday, 15 November 2014 at 06:53 PM
I organize by camera and date. Since storage is (almost) free, I delete nothing. About 1.3 TB. What ever happened to MB?
Final images are copied to the cloud -- Microsoft OneDrive (unlimited storage with an Office 365 subscription).
Images in OneDrive are in folders with appropriate names in chronological order. Files are still named by camera and date.
Locally, images (and other data files) are automagically backed up to an external drive using File History -- part of Windows 8. I use a second external drive to monthly back up everything for storage off site -- in case the house burns down.
Posted by: Speed | Saturday, 15 November 2014 at 07:25 PM
For several years now I've only been shooting one sheet of 4x5 Portra of each set-up, even portraits. Unless I know that I made an obvious mistake or my subject blinked, I stick to the single sheet discipline. Of course a big reason I can get away with this is thanks to Kodak quality, Edgar Praus's processing, and Photoshop.
With my aging 12mp digital D300 cameras I shoot less and less than I used to. For a portrait maybe 30-40 of each set-up, which I edit down to 2-3 selects from each. For a landscape I will delete as I go, saving only the best and a safety.
Editing film I throw away every piece that is mediocre and will often spend an evening tossing old film out of my archival notebooks. Of course larger film and slides are easier to edit than 35mm negatives, which are pretty much impossible to get through. Sometimes though you can toss entire sheets/rolls since you know the shoot wasn't any good... lots of 20-year old commercial stuff has hit the trash bin.
As for digital, I keep one folder that is the "core" best of the best, then organized more folders into broad topics. Naming conventions are set at "year-month-day_subject_sequence_camera or film size.tif" using .tif as a standard over .psd in case a true Photoshop killer ever materializes ;-p But I am brutal and rather toss than keep a file. Especially older scans or files from earlier cameras that technically aren't up to modern standards. Some of my scans from the 90s were good but many weren't. Surprisingly most of the old Kodak Photo CDs from 20 years ago are still excellent scans.
In the end, it pays to be brutal and to throw away anything doubtful as you go along. It's Darwinian thinking and it works, live with the fact that you should have recognized the significant images when you were working on them and if you haven't gotten back to edit further after years it is pointless. Even in a feeble disabled state, it would be better to be busy creating new work than dwelling on the past. Forget the old images and move forward creating fresh work. The old images were valuable -- for learning... but otherwise their usefulness is done.
I've shot professionally since 1993 and my life's work fits onto a one TB drive and four archival notebooks. I have two boxes of tearsheets and a couple of portfolio books. Everything else is gone, probably 30-40x in quantity but abysmal in quality.
Back ups are easy and I have drives distributed to family homes around the area. I even have a set of priority files on a keychain thumb drive. And having some larger jpgs online isn't such a bad back up either, in a pinch you could re-create a decent portfolio from them.
Oh and I'll be tarred and feathered for saying so but I don't want to be tied to a Lightroom database for perpetuity. I edit, catalog, and back-up "manually".
The one thing I can't edit are my older, dead relative's film. I can't toss it because it would be too hard. The answer for me has been to send most of it to Scan Cafe and then to box it up and stick it in the recesses of a closet forever. The vintage family photos go onto a Facebook album with the privacy commands set to "family only". Works well.
Thanks for the opportunity to rant about this!
Posted by: Frank P | Saturday, 15 November 2014 at 09:13 PM
I am a light shooter, for my "normal" stuff which is fairly static I usually take one or two setup shots then the final, done! If I am snapping people that changes exponentially and usually runs in the dozens to get "expressions".
I have 3 internal 2TB drives, 1 external 2tb and one network 2tb drive for remote access. with 10 years of digital under my belt I have about the same storage as you used up. about 17k in images, mostly duds but space is cheap. I copy my cards onto two drives on upload, one is for work files, the other is a straight BU of the "negatives" that remain untouched. the work files are backed up to the third internal drive when they are print ready. then once a month or so they make it to the external drives.
Posted by: dale | Saturday, 15 November 2014 at 10:18 PM
Those WD My Passport drives hit the sweet spot for size, capacity and cost. It's relatively cheap to keep three of them in the rotation for backups.
I bought my first digicam a couple of years ago, and Leica were kind enough to toss in a Lightroom license. I've been able to keep all my pictures fairly organized with it. You can set up "smart collections" to sort by camera or lens (or practically any other criteria), it can pull that info out of the EXIF file. Keywording is important for searching; LR has a default "smart collection" of pictures without keywords, and when I'm really bored I'll sit down and work on some of the backlog.
Posted by: Chuck Albertson | Sunday, 16 November 2014 at 12:43 AM
I currently shoot maybe 5000 frames a year.
I have digital images going back to late 1990's. But storage grew faster than I could shoot then, so I saved everything. Organized by camera because I never had more than one that was working. With an Olympus E-1 in the early noughts, I got into the habit of doing my selection at the same time as rendering the raw files (in Capture One), backing up all the usable raw files (maybe 90% of them) and keeping the jpgs that resulted organized by month, all cameras. I still do that. My backups are on USB disks, currently 2 TB each, one at home and one in the office. Each time the cost of the new, say 2TB, equals the amount I paid for the old 1 TB, I copy the old stuff to a new disk and back up into the remaining space, leaving the older archive on the shelf as the backup's backup. Only the most recent 2-3 months lives in uncompressed raw form on the laptop, but I can keep two years of jpeg selects available. Of course, there is still 15 years of carefully labelled strips of film in glassine envelopes sitting in a friend's attic in a dry place in CA, that I have to deal with someday...
scott
Posted by: scott kirkpatrick | Sunday, 16 November 2014 at 02:08 AM
I use a Macbook Air when I need to be portable, and an iMac when I'm in the studio. Both use SSDs, the Air exclusively and the iMac for OS and apps and a few other things.
I have used computers so long that I recall a time when hard drive space was precious. It isn't any more. Drives are cheap and capacious today, so I would virtually never delete files simply to reduce the amount of data I store. In fact, that strategy typically doesn't even work that well — you might extend the time before you need to upgrade a bit, but not that much.
I have quite a few drives, partly because I have many image (and other) files and partly because I'm a bit fanatical about backups, especially after experiencing a hard drive failure many years ago at a very bad and critical time. I have a highly redundant system, such that I have a minimum of four complete copies of everything on my primary drives, including one copy that is kept off-site. (I like to say that if your backup procedures don't make your friends thing you are paranoid, you aren't being careful enough.)
When I travel I carry one (or occasionally more) of the very small 2TB external drives. I carry copies of all of the first line Photoshop files in case anyone needs an image from me while I'm traveling. This also means that if The Great Quake comes while I'm on the road, I'll still have my best files. ;-)
Repeat after me: Drives are cheap. Lost data is a horror.
Posted by: G Dan Mitchell | Sunday, 16 November 2014 at 02:59 AM
Just looked at my current Lightroom Library. This one started in June 2013 and contains all of my images. The count is 354,442 images. I'll start a new library on January 1. The three most recent months remain on my IMac hard drive and then everything migrates over to a couple Drobo's that total about 24TB at the moment. A long-term nightmare.
Posted by: John Gillooly | Sunday, 16 November 2014 at 10:01 AM
As a wildlife photographer I usually take around 500 shots each time I am out. Before I import those RAW files in Lightroom, I do a 100% preview (in the import window) for each photo and uncheck for import every photo not sharp, bad light, bad compo, etc. Usually I keep about 25% and if I am lucky I have 5 to 10 nice shots worthwile posting. I am also very happy if there is one picture that is worth printing large, but does not happen so often....All in all, I keep about 12000 pictures per year, stored on multiple drives and backups, all organized by date in a named folder which makes it pretty easy for me to find something back.
Posted by: Frank Baldé | Sunday, 16 November 2014 at 11:03 AM
I have always been a light shooter. During my professional career (now on indefinite health leave), I kept my shooting down to a minimum. One of my niches for years centered on dog photography. I rarely took more than thirty frames during a session. My percentage of keepers to duds (10%) has remained consistent throughout the last forty years.
Posted by: Bob Rosinsky | Sunday, 16 November 2014 at 11:16 AM
I'm with Adrian M - deleting images takes far too much effort mentally to be worth it - I do try to get rid of the obvious dead images, but beyond that, I'm haunted by the packrat credo of 'I might need that!'
Hard rives are cheap. Online storage, not so much, and bandwidth isn't unlimited, but a Flickr Pro account, Amazon Prime image storage and redundant drives make it less scary than a filing cabinet full of negatives.
I also sort by camera and date - something possible in digital that I could only do in film by film size - metadata is a disorganized person's friend!
Posted by: Rob L. | Sunday, 16 November 2014 at 12:14 PM
I use a fairly simple system of folders. Each subject gets a folder named yymm-subject. Each year gets a folder with subfolders for 'people' and 'other.' It works for me so long as I can resist naming folders "miscellaneous" or "to sort."
Right now working files for the year are on a 1T external drive with another as backup and yet another across town in storage. For several years now I've just about filled a $100 hard drive with photos - files get bigger but drives get cheaper. With 2 backups that's about $300 per year - not much money as photography goes. Or much less than one latte a day.
As to cameras, the other day I was looking at a photo of a friend. It is dated 2010 but I know it has to be older because I remember it was done with Olympus and I switched to Panasonic in 2008. I got to thinking about how long I had known her. We first met when she modeled for me, and that session was done with Nikon, but not too long before I switched to Olympus in '05. So the friendship is around 10 years.
So maybe I should try your filing system. :-)
Posted by: Gato | Sunday, 16 November 2014 at 12:50 PM
The storage habits of the infrequent, non-prolific shooter are just as interesting. One method I observed, whose simplicity I greatly admired: Keep everything on the original media (SD cards). Label and physically organize, and purchase additional cards as needed. As my pixel count has plateaued and flash storage prices have fallen, I sometimes consider how I might safely make a transition to this method.
Posted by: Michael Barker | Monday, 17 November 2014 at 10:05 AM
Adrian Malloch's point hit home with me, one of those things I've felt but couldn't put my finger on. From now forward I will not feel guilty about leaving images on the drive, even the obviously crappy ones. It kind of feels like I just deleted an unneeded concern from my mind!
Posted by: John Krumm | Monday, 17 November 2014 at 10:24 AM
Ugh.. I've been around the block in solutions to this. I swing between being a very light shooter (50-70 shots per week or two) to being quite heavy (500-1400/day/camera) when I'm either in the mood or shooting dance or events.
I started out with named folders,
then moved to lightroom,
then capture-one,
then back to lightroom,
then back to named folders with the move to bibble for a few years.
And occasionally I was forced to go back and find those images... :/
So I'm currently doing named folders and I doubt that I will change. The named folders are tagged with a date and a short reminder of whats in them, I download the entire camera into them with a complete file structure for homogeneity.
Then I use whatever raw developer I'm using at the moment to build previews and only develop the selects that I like and export them into a sub folder. This allows me to do a recursive slideshow with a fast viewer such as geeqie and find a .jpg which I can use to figure out the filename for the raw file.
As far as running out of space... I keep buying bigger and bigger hard-drives and doing the onion thing.
I keep threatening to make a symlinked database structure (but not a real database as those tie me down more than a filesystem already does) but I haven't gotten there yet. A few times over the years I've moved things onto external drives for deep storage, however I usually get fed up with the hassle and buy bigger drives. The next step is likely a NAS or two as the 12TB versions have gotten cheap enough to make some sense.
Posted by: SoulNibbler | Monday, 17 November 2014 at 10:53 AM
Plus One for Trecento...I do the majority of image related 'moves' on an 'air-gapped' Dell lap-top running Windows XP, service pak 2. Never a hiccup, never connected to the "world-wide-virus-generator", never updated beyond my available drivers for periphrials. Photo Jobs? 32 gig, class 10 SD cards are less than 20 bucks on sale at Office Max, after job delivery, I slide the lock and throw them in the physical job folder, with a 'contact print' (we need a new name) of the best images.
Posted by: Crabby Umbo | Monday, 17 November 2014 at 02:15 PM
BTW, I wonder what's going to happen with all those amateur photographers collections that are saving millions of images on multiple terrabyte drives? Heck, there's people on this site with shot counts higher than the film I've saved on paying jobs (and other valuable photo assignments) since 1975, and I'm a pro! I can see someone coming upon your glass plate neg collection, or 8X10 sheet film after you've died in your bed, and thinking it's something valuable; but you're little black hard-drives? Tossed!
Posted by: Crabby Umbo | Monday, 17 November 2014 at 02:23 PM
I only store unprocessed photos on system drive. As soon as my edits are done and final images exported, I move the raw files onto an external. The externals will eventually be labeled by decades even if a 1TB portable goes a long way. I rely on Time Machine for backups. Once an external is updated with new files, I leave it plugged in and trigger a Time Machine backup.
Posted by: dav1dz | Monday, 17 November 2014 at 07:09 PM
Its only a backup if you have two distinct copies, preferably in two different locations.
Ask yourself "what would happen if my backup drive got destroyed?". If it meant you would lose stuff then you do not have a backup.
Posted by: Ed | Tuesday, 18 November 2014 at 05:32 AM
I have frequently pulled images out of photo sets many decades old and not previously printed. The most frequent cause is someone's death; but it happens for other reasons too. So the idea of deleting anything I'm not immediately using is not just repugnant (emotional reaction), but shown by history to be a very bad idea for me.
Arguably the cause is that I wasn't adequately picking the photos to print in the first place. At least sometimes that is true; there have been periods when I was doing better at taking photos than at any other part of the process. But some of the cases are a photo that would not have made the cut normally that becomes of interest because of changes over time -- the premature death, or the success, of one of the people in the picture makes it of interest long later.
Posted by: David Dyer-Bennet | Tuesday, 18 November 2014 at 05:39 PM
I've always thought date was the best primary key for photos; even my original roll number scheme tracks with date (since I assigned the numbers as I loaded the film into the camera).
However, I make a point of adding IPTC info, including date for scanned photos (it's already in the digital ones of course), and location information, and keywords and such. I don't use the filesystem and naming convention instead of metadata, I use it in addition to metadata.
Posted by: David Dyer-Bennet | Tuesday, 18 November 2014 at 05:45 PM
External drives are dirt cheap. Unlike girlfriends. *wink*
Posted by: Edie Howe | Tuesday, 18 November 2014 at 08:20 PM