26Dec Managing your film project’s data needs
It’s never a very sexy topic, but data management for your film project is very important. In a single day of shooting with two cameras you can easily fill up 5 or more 32GB cards and that’s not even counting the behind the scenes shots and stills. If for some reason you have to get everything done in a single day and end up shooting tell 1 a.m. (like i did last weekend) things could get even worse.
Lets use last weekends shoot as an example. Here is what we ended up with after a single 14 hour day of shooting on two cameras. The 153 GB of data doesn’t include another 16 GB worth of behind the scenes footage (shot on a HF S100) or the 8GB worth of production stills that were also captured over the same weekend. In total the shoot filled up 3 32GB cards CF cards and one 64GB SDXC card for what will eventually become a 10 min short film.
In general it takes about twice as much time to fill up a card as it does to copy it’s contents over to a hard drive. Even so, don’t procrastinate about this. The second a card fills up, label it with the camera it came from and any scene info you can remember (a sticky note works well for this), get it to the computer and start copying files over. Note the word “copy”, if at all possible try to keep your footage in at least two places. The longer that footage stays on a memory card without being backed up, the more chances you have of misplacing, damaging, or otherwise loosing your footage.
It’s also a very good idea to make sure your footage is backed up properly. Sure we all use copy and paste commands, but that doesn’t always guarantee that the file on your memory card made it to your hard drive without errors. For peace of mind I use TeraCopy for all of my important file transfers. This gives you a CRC checksum value for every file that is being copied, then it verifies that the final copied file matches the source file. This guarantees that you wont end up with a file that wont open in post.
If you have enough usb ports on your laptop, why not use two usb card readers. That way you can copy footage from both cameras at the same time. When one camera’s card fills up on a two camera shoot, chances are the second camera isn’t far behind. Go ahead and change out both cards and maybe even both batteries at the same time. It’s better to change out batteries and memory cards all at once, then to change out a single camera only to have the second camera need attention in another 15 min.
Label and date each new folder you copy your footage into and use a single folder for each card. If you have time create a text file in that folder with the information you wrote on your label earlier. Try to include the times you started using the card and when you removed it from the camera. In post you can look at your shooting schedule and get a general idea of what clips are in which folder simply by comparing the times and dates with the schedule If you were only able to shoot in that restaurant from 9 to 11 a.m. for example, you’ll know where to start looking for the restaurant footage. Every little bit helps in post.
Before I go to bed after a long day of shooting, I start coping the files from the back up drive over to a second storage drive. Be sure to turn off power management settings on your laptop so that the computer stays on all night and copies your files. If you have enough memory cards to finish the job without clearing the data, it’s a good idea to leave them a lone. If not, don’t clear off your cards until you have the files copied over to a second drive. Once you’re done backing up, send one drive home with the director and take the other home with you, that way your files are in two different physical locations.
Above is a Seagate 1tb usb powered drive ($79), in a $9 hard drive case. The Case is big enough to hold a single card reader, two usb cables, and a hard drive in a single protective case. It makes for a nice clean way to have everything you need to manage your files in a single place and it makes life easier when you are packing for a shoot.
Once you’re home, copy all of your files to your NAS and you’re ready to start editing.
December 26th, 2012 at 9:35 pm
Are you aware of a Mac equiv. to TeraCopy?
December 27th, 2012 at 5:41 am
There are two that I know of but I have not used them: md5deep and arRsync
I’m pretty sure there is also a way to do this on a mac via Terminal, but i’m not a mac user. I think the command might be “sum”.
December 27th, 2012 at 12:55 am
Do.you shoot AVCHD? Isn’t it unsafe to directly copy AVCHD files from a memory card? In the case of a Canon camera, shouldn’t you use the file transfer utility that comes with the camera to transfer AVCHD files? Thanks for the awesome blog!
December 27th, 2012 at 5:52 am
The Old Canon HF S100 (behind the scenes camera) uses a pure AVCHD format, the Canon 5d mark III and my other cameras use h.264 which AVCHD is based on. I’ve never used the Canon utility to copy files and I have yet to run into any data problems. The card losses I’ve come across have been caused by physical damage, i.e. dropped in a toilet, stepped on, run over, and so on.
December 27th, 2012 at 11:36 am
Thanks for the great info. I downloaded TeraCopy and it seems to work pretty well. BTW Best to download from the software company original site as opposed to Download.com which will add a bunch of adware. But I did notice that some people were talking about issues they had with 64 bit Windows 7 systems. Did you ever have any issues?
December 27th, 2012 at 3:17 pm
I haven’t run into any problems on windows 7 and I’ve been using TeraCopy for a few years now. Seems to be very solid and takes the place of the native copy and past functions in windows. If I run into any issues I’ll post something.
December 27th, 2012 at 4:46 pm
Thanks Deejay. I trust you far more than a bunch of anon comments. Do you think it’s worth it to buy the pro version?
December 27th, 2012 at 6:33 pm
I paid for it, but I don’t think you “need” to if the those extra features aren’t anything you want.