Tag: Tips

Controlling JPEG file sizes in Lightroom 3

Lightroom JPEG export options

Summary: file sizes of low-resolution JPEGs that you export using Lightroom can be excessive if you don’t take steps to control the embedded metadata. The large file sizes can affect your site’s loading speed and that in turn can now affect your Google search rank.

re-photo.co.uk
Earlier this week, I was catching up on some well written and informative articles by London photographer and writer Peter Marshall when I came across this one mentioning the release of Lightroom 3.2RC; in the piece, Peter mentioned that he’d found Lightroom 3 to be generating relatively low quality JPEG files for a given file size, at the 600-pixel dimensions that he uses for his site.

Peter had noticed two significant things: first, that he was getting better JPEGs at any given file size when he created them using Lightroom’s web module (which is designed to export a complete web site) instead of the usual JPEG export method; second, that the problem was worse when he was exporting JPEGs of images for which he’d made use of Lightroom’s local adjustments, meaning brushes and graduated filters.

It turned out (see his follow-up post here, and the comments that follow it) that the metadata associated with the file was causing the bump in file size. Clicking “Minimize Embedded Metadata” when exporting JPEGs helps; installing a copy of Jeffrey Friedl’s Metadata Wrangler plugin for Lightroom 3 fixes the problem completely and has added benefits: you can set up presets that get saved with Lightroom’s own export presets. That means you can build a one-click Lightroom export preset that generates the right picture size for your site with all but the unimportant metadata removed, and just the important stuff retained. (Presets are the key to working quickly within Lightroom and are probably its most overlooked feature.)

Above: the checkbox used during export to minimise metadata. Not as effective as Jeffrey’s Metadata Wrangler.

Explanation: all local brush and graduated filter adjustments that you apply in Lightroom become part of an images’s metadata and are included on export, bumping up the size of the final JPEG file—particularly noticeable for small JPEGs, because this metadata size is a constant and can easily double the size of a file. Not a huge problem if you’re hosting one or two images on a page but if you’re putting up many, the extra file size is significant. For blogs that have the usual rolling front page, hosting all images from the last ten or twenty posts, this sort of thing can make a big difference for your visitors… and for Google.

Important for your photography site’s Google rank

These days, your site’s Google rank is partly dependent on the speed at which your site loads—see this important article from Google on the subject. It’s well worth doing what Peter is doing, optimising carefully and minimising JPEG image size. (On which note, if you’re using WordPress software and your own hosting account to manage your site’s content, you should make every attempt to install and enable WP Super Cache to speed up your site’s response under load. This isn’t the appropriate place to discuss the technical aspects of that plugin but it does its job very well. Obviously, make complete site backups first.)


Above: JPEG file sizes before and after reducing metadata in different ways. No affect on image quality.

Photo forum

As Peter mentions, we met at a monthly London event called Photo Forum where photographers (mostly photojournalists) show and discuss their work. The two of us here at Shoot Raw have been three or four times and always enjoyed it. It’s a busy event but a good way to see work that might be new to you and to meet other photographers, established and upcoming. (At the time of writing, the next event is on 9th September 2010.) It’s a credit to Jacobs Professional Services that they host the event every month.

Friedl on JPEG quality versus size in Lightroom

Jeffrey’s definitive article on Lightroom 3’s JPEG quality made the rounds a while back. If you haven’t read it and you generate JPEGs from Lightroom, pay a visit. It’s the last word on Lightroom 3’s JPEG quality versus file size and even those of us who thought we knew exactly what Lightroom was doing in this area learned a few things.

The metadata

You can subscribe to these articles for free by e-mail (managed by Google) or by RSS/Atom. We never pass on your details to anyone. If you’re interested in technical training or help, please consider using our contact form to get in touch. You can also support this site by shopping at amazon.co.uk using this link or amazon.com using this link. (You pay the same low Amazon price but we get a bit from Amazon’s profit.) Thanks to everyone who uses this link… we don’t get to see your names in the Amazon reports so we don’t know who you are but we’re very grateful for your support.

Lightroom 3 point-curve tutorial, online tips and Shoot Raw updates

A great piece on Lightroom 3’s point-curve editor

I was some way into the writing of a long article about Lightroom 3’s excellent new point curve editor (available in the beta 2 release) when I came across Gene McCullagh’s comprehensive piece on the topic over at lightroomsecrets.com. It’s very well written and worth reading carefully—it left my own half-finished attempt seem mostly superfluous. Beginners who wish to try creating custom tone curves after reading Gene’s post should start with the Linear curve, which contains only two control points—it just makes things a little easier to begin with. A linear curve makes it easier to follow the advice on adding a control point and holding down the option (alt) key while adjusting the position of that point.

Lenswork Technology Bog—highly recommended

One day, we’ll add a long-overdue list of resources that are useful to your raw workflow but since we’re on the topic of the new point curve editor in Lightroom 3, now seems like a good time to mention one very useful site that dealt with tone curves recently. Take a look at Brooks Jensen’s use of a custom tone curve in Lightroom to control highlights in print.

Brooks’s post is based on Lightroom 2, which offers a parametric tone curve but no ability to control the end points, preventing him from using the curve editor within Lightroom 2 itself to finely control the appearance of highlights in print. His workaround was to create a custom tone curve using Adobe Camera Raw, export that curve and then use it in Lightroom. It should now be possible to use the curve editing within Lightroom 3 to allow at least a similar level of control.

The Lenswork Technology Blog is excellent reading for photographers, particularly those of us who produce our own prints; the same is true of the Lenswork podcasts and the Ask Brooks blog. If you’re in the UK, you can get the Lenswork podcast via the UK iTunes Store (for free) here.

Welcome to our new subscribers

Lots of you have subscribed recently to the blog—thanks and welcome.

We went through a bit of a busy patch recently, as you might have gathered from the absence of blog posts but we now have a few days before the next scheduled training session so we’re firing off a few updates and articles. We’re hoping to add a little more information to the site and reorganise things a little, too. Meanwhile, if you’re interested in our main offering—one-to-one workflow courses in the UK based on Lightroom—we’re now taking bookings for May and June. Call 0333 577 5703 if you’d like to chat, or drop us a line. We’re doing a bit more travelling in the UK now and have worked out some ways of making our Lightroom courses more affordable—we’ll put details up on the training page soon and add a post here when it’s done.

We now use Lightroom 3 for all training

We recently switched to using Lightroom 3 beta 2 for all our training—it didn’t really make much sense to base tuition on Lightroom 2, given the quality of the current beta, the improved rendering of raw images and the new features that beta 2 offers. Reaction from photographers continues to be very positive—we’re very, very pleased with the direction in which Adobe is taking Lightroom 3.

Can you help?

We always appreciate new readers and subscribers (articles and subscriptions are free). If you find anything on our site useful, we’d love it if you could tell other photographers about us by forwarding our URL or by putting a link to us on your site, Facebook page or blog. Thank you!

To subscribe (for free)

If you’re not currently a subscriber, you can choose e-mail via Google or RSS/Atom for your feed reader. (For the e-mail option, you get a single e-mail message in the morning whenever there’s a new article on the site but nothing otherwise.) The blog is a collection of tips, news and information for photographers, mostly covering workflow, image quality and technical subjects.

Lightroom 3, Adobe Camera Raw and Photoshop CS4

Some good news and updates recently for users of Lightroom and Photoshop CS4. Along with the good news is continued confusion in various online forums about Adobe Camera Raw and its relationship with Lightroom so here’s an attempt to explain some of what the two have in common.

What is Adobe Camera Raw?

Adobe Camera Raw (ACR) is a software plugin used by Photoshop to decode (or “demosaic”) raw files. A version is supplied with every new copy of Photoshop (and Photoshop Elements) and the plugin is regularly updated as new cameras are released because new cameras mean new types of files for the ACR plugin to understand. Any new major release of ACR (like ACR 6.0, 7.0, etc.) is usually available only for the latest versions of Photoshop and Photoshop Elements. The current version for Adobe Photoshop CS4 is ACR 5.7, released last week and available from Adobe as a free download. It works with Photoshop CS4 and Photoshop Elements 8.

Links to the (free) Adobe Camera Raw 5.7 downloads:

Does Lightroom use Adobe Camera Raw?

No, not directly. Lightroom’s decoding (or demosaicing) of raw files doesn’t rely on the ACR plugin itself so installing a new version of ACR won’t change Lightroom’s rendering or allow it to read new types of raw files. However, Lightroom and ACR share functionality and code—it’s just that Lightroom’s implementation is contained within the application itself. A new release of the ACR plugin for Photoshop usually means a corresponding release of Lightroom. The two are independent but usually updated together.

Adobe Camera Raw 5.7 and why it’s important for Lightroom 3 beta 2 users

ACR 5.7, like Lightroom 2.7, offered support for raw files from newly released cameras but that’s not the reason for this article. Version 5.7 also included a nice surprise for users of Lightroom 3 beta 2. It turned out to include the code to allow it to decode raw files using the new raw decoding engine contained in the Lightroom 3 betas—specifically, Lightroom 3 beta 2. This code is what gives Lightroom 3 its much improved colour noise reduction (see previous articles here and here) and finer detail. Till now, if you were working in a Lightroom 3 beta and chose “Edit in Photoshop” (command-E on the Mac, ctrl-E on Windows), you’d see an error message in Photoshop CS4 (if you were using LR3 beta 2) or just wonky results (in LR3 beta 1). When you install ACR 5.7, that problem is gone, allowing you to work on a raw image in Lightroom 3 beta 2, then open the raw file (including your Lightroom edits) in Photoshop CS4 and continue to do pixel-level work. Very nice.

A few weeks back, we wrote about the new and old process versions for Lightroom 3; what ACR 5.7 offers is really a way for Photoshop to work on raw images that you edited in Lightroom using the 2010 process. It doesn’t allow you access to the new noise-reduction controls directly from Photoshop but it does appear to respect the settings that you used within Lightroom 3 beta 2.

Sidenote: when you work on a raw image in Lightroom and choose “Edit in Photoshop”, Lightroom doesn’t immediately create a TIFF and send it across to Photoshop—the ACR plugin within Photoshop reads the original raw file and the list of changes you’ve made in Lightroom and applies those changes itself. The TIFF gets created only when you save the file in Photoshop. Now that version 5.7 contains a raw decoding engine that’s compatible with Lightroom 3’s, this process works again for users of CS4 and Lightroom 3.

What all this seems to suggest is that there will be a version of ACR for Photoshop CS4 that will remain compatible with the final release of Lightroom 3, meaning that you don’t have to update to Photoshop CS5 immediately to keep tight integration between Lightroom and Photoshop. If that’s true, it’s a very welcome gesture.

The obligatory gasp about Photoshop CS5’s content-aware fill

Photoshop CS5 looks like a very strong release for a certain type of photographer. If you haven’t already seen the content-aware fill demonstration, you’ve been missing out so take five minutes to watch it now. It’ll make your jaw drop.

More Lightroom 3 features revealed: correction for lens distortion

The final version of Lightroom 3 (and ACR 6.1) will allow us access to lens-correction features that have long been lurking.

Distortion correction has been around for a while in Lightroom

For some compact cameras that shoot raw and for some Micro Four-Thirds camera-and-lens combinations, both Lightroom and ACR have been providing behind-the-scenes corrections of lens distortion. Users of the Canon S90, the Panasonic Lumix LX-3 and some wide-angle Micro Four-Thirds lenses (such as the excellent 20mm Panasonic f/1.7) have seen automatic correction of very significant barrel distortion but Lightroom 3 (and ACR 6.1) will extend that benefit, in some form, to the rest of us.

Control over the degree of correction

Many users of the cameras and lenses mentioned above probably didn’t even know that their images were being corrected, sometimes for an eye-popping level of geometric distortion. The feature just worked, unbidden, behind the scenes. Adobe’s engineers generally seem to have matched the correction that the camera manufacturers applied to JPEGs generated by the camera and/or the results produced by the raw converters shipped with the cameras, meaning that by design, images from cameras like the S90 and LX-3 show some residual level of barrel distortion after automatic correction within Lightroom—correction over which the user has had no control, till now. With Lightroom 3, we’ll be able to fade the degree of correction for things like vignetting and distortion.

The best bit: we’ll be able to profile our own lenses

Out of the box, the lens-correction feature will support some lenses from Canon, Nikon and Sigma (who even issued a press release about it) but potentially the strongest aspect of Adobe’s implementation is that we will get a mechanism to allow us to profile our own lenses for optical defects. There are other solutions to the lens-correction problem (DXO Optics Pro, for example, or PTLens) but Adobe’s looks like it might be the strongest so far for a couple of reasons: firstly, the existence of an easy way to profile your own lenses (the proof of the pudding will be in the tasting, of course) and secondly, the apparent concern of the Adobe Lightroom/ACR team to get local corrections working well with this new feature. This is harder than it sounds: say you’ve removed a spot of sensor dust from an image or you’ve added saturation and sharpness to a an area of a photograph: quite how should Lightroom react when you later switch on the automatic correction for lens for distortion? Should it even let you switch it on if you’ve applied local corrections?

Head over to Tom Hogarty’s blog post, where he shows how it’s all going to work. Congratulations to the Lightroom team on what looks to be an excellent implementation.

Switching between old process and new process in Lightroom 3

Lightroom 3 is built around a new, improved demosaicing engine (the raw converter code that makes a full-resolution, full-colour image out of your raw file). It also includes the older conversion engine used in Lightroom 2 and earlier so to see the full power of Lightroom 3, you need to make sure you’re using what Adobe calls the 2010 process (that’s the new raw conversion engine) on each of your raw files. The old engine is referred to as the 2003 process. (In Lightroom 3 public beta 1, they were called process version 1 and process version 2—the new names are a definite improvement.) This is a per-image setting so you can choose which images use which process and can mix old and new in one catalogue.

How do you tell which process you’re using?

As of Lightroom 3 public beta 2, the main sign that you’re using the old (2003) process is the unmissable presence of an exclamation mark in the lower right of your image when you’re in the Develop module. The symbol appears next to any image that’s using the old engine and looks like this:

To switch to the 2010 process, just click that warning symbol. You’ll get the chance to review the before-and-after changes and to apply the changes to the whole filmstrip. Comparing the the changes side by side at 100% can be a useful way to understand the differences between old engine and new.


Above; the box you see when you click the exclamation mark in the Develop module

A warning about noise reduction and sharpening

In Lightroom 3 public beta 2, released today, local sharpening controls (brushes, gradients) and luminance noise reduction are both much more powerful than they were in Lightroom 2. You might find some settings for sharpening, negative sharpening and luminance noise reduction that you’d used previously to be way too high for the new versions so carefully review as you update your work.

To switch back to the 2003 process

To remind yourself of how a picture looked using the old demosaicing engine (aka process version 1 or the 2003 process), you can always switch back. In Lightroom’s Settings menu, go to Process, where you’ll see a choice between the 2003 version and the 2010 version. (The old process is labelled 2003 because Adobe Camera Raw—or ACR—dates back to that year. Even though Lightroom was only released in 2006 as a public beta, it shares code with ACR, which means that at some point, there will be a version of the ACR plugin that has offers this new raw conversion engine, too.)


Above: switching between processes using the Settings menu

Alternatively, you can now choose your process version from a new menu item in the Camera Calibration section of the Develop module (lower right).


Above: the new Process menu inside the Develop module’s Camera Calibration section

Use virtual copies and the compare function

When you begin to use Lightroom 3, it can be useful to make a virtual copy of a picture and use the old engine (2003 process) and the new side by side on the same image to get a feel for the difference between the two raw converters. For the full effect, choose a high-ISO image, with local sharpening applied. The second public beta now includes an option to view images side by side when you convert from the old process to the new but it can be useful to do it manually, particularly for images in which you’ve used many brushes. Compare results at 100% using the compare function (hit C in the grid, with both versions of the picture selected) and you should see a significant—and sometimes dramatic—difference in quality.

Why do some or all files use the old process version in Lightroom 3?

If Lightroom 3 can tell that you’ve done some work on a raw file in Lightroom 2 or a pre-v6 release of Adobe Camera Raw, it will keep using the 2003 process for that image so that the picture continues to look just as it did in Lightroom 2. That’ll happen if it sees an XMP sidecar file next to the raw file and can tell that the XMP file was created by Lightroom 2 or ACR 5.x or earlier.

What are the main differences between the 2003 process and the 2010 process?

The new 2010 process uses the new demosaicing engine in Lightroom 3, offering finer detail and better rendering of high-ISO work, among other things. Dramatically improved noise reduction, too, and much more powerful local sharpening controls. (Brushes, gradients.)

For an early and subtle example of the difference in quality between version one and version two with default settings, even at low ISO, see an earlier post on protecting fine colour detail in Lightroom 2. Towards the end, it contains some Lightroom 3 screen shots of the example image—there’s a clear difference in the way that colour noise reduction works.

Now that Lightroom 3 beta 2 is available and contains working luminance noise reduction, we’ll post more example files soon. Initial testing suggests very impressive NR results but it will take more time to be absolutely sure.

Which camera settings matter when you shoot raw?

Almost all DSLRs come with plenty of options governing aspects of image quality. They almost all affect JPEG output but which of those settings and options matter when you shoot raw? We’ll look at this briefly in this article and in more detail later.

Exposure settings matter, whether you shoot raw or JPEG

The exposure of a given scene will be determined by these settings:

  • Shutter speed
  • Aperture
  • ISO setting

(The ISO setting in digital photography is not entirely straightforward and will perhaps get its own article. For now, we’re keeping things simple.)

As you’d expect, aperture, shutter speed and ISO setting all make a difference to the images that your camera records, whether you shoot raw or JPEG.

Secondary camera settings: do they affect raw files?

A DSLR’s menu system will offer control over other aspects of your image, whether you’ve told it you’re shooting raw or JPEG:

  • Sharpness
  • Contrast
  • Saturation
  • Colour mode (Adobe RGB, sRGB)
  • White balance

Sometimes you’ll see picture controls that change many secondary settings together. For example, your camera might offer an option like “Vivid”, which sets saturation higher and might also affect contrast and/or colour mode. These controls might also change the way colour appears, including small shifts in hue, sometimes emulating certain types of film.

The questions

Which of these secondary settings, if any, affect your raw files? Which of them should you be concerned about when shooting raw? And, again assuming you shoot raw, do they affect any other aspect of your photography.

Before answering them and explaining the answers, let’s categorise everything inside a raw file as being one of two things.

A note about raw files

Simplifying somewhat, everything inside a raw file is either a measurement of the light falling on a part of the sensor or it’s a note about camera settings. You could say that it’s either data (the values for light on the sensor) or metadata.

Let’s say that you take a shot on a current 24-megapixel DSLR. Inside the resulting raw file will be roughly 24 million values representing the light that fell on the sensor during the exposure, one value for each sensor element, or sensel. We can consider all this stuff to be the data that’s eventually turned into an image in your raw converter.

The metadata is a record of things like the shutter speed you used, the aperture, the type of camera, the type of lens you used, the white balance and sharpness settings, which saturation you’d specified and so on. Lots and lots of stuff, only some of it documented publicly by the camera manufacturer.

Some answers

So which of the secondary settings listed above directly affect the data in your raw file?

The answer is: not a single one. Think of there being a note in the raw file that says: this photograph was shot with sharpness set to high and contrast to minimum, noise reduction set to zero and colour space set to Adobe RGB 1998; colour temperature was 5500K and the green-magenta bias was 30% magenta. Most raw converters, including Lightroom and Aperture, read the note and display large parts of your metadata but the only thing they use to influence their initial versions of your images is the white balance information. Everything else—including all the other secondary settings listed above—is ignored. Even the white balance setting is used only for the initial rendering.

Despite the presence of the metadata “note” inside your raw file, none of the 24 million values recorded by your 24-megapixel camera to represent the scene itself were directly affected by those secondary settings.

When you shoot JPEG, things are very different; each and every secondary setting above has a chance to affect each and every pixel of the JPEG. In the case of white balance or contrast, it is almost guaranteed to affect every pixel, often in a way that might limit the changes you can cleanly make later. That doesn’t mean JPEGs are bad or unusable, particularly if you’re in complete control of your light and white balance—it just means that the raw converter inside your camera is making decisions that it might be best to leave for later.

To be continued…

Although the secondary settings have no direct effect on the raw file, they do affect the histograms you see on the back of the camera. As a photographer, you might use those histograms to change the exposure settings (shutter speed, aperture, ISO) that you choose, perhaps as a result of exposure compensation that you dial in. The story isn’t over. We’ll revisit this topic soon.

And speaking of data and metadata…

The metadata

You can get all future articles by e-mail or RSS (both are free) and you can call us on 0333 577 5703 to book your own Lightroom course or to book a support visit. Visit the main blog page here. Have fun shooting!

Migrating your library from Aperture to Lightroom or Lightroom to Aperture

Following the release of Aperture 3 last month, the Shoot Raw site logs show that we’re getting quite a few people visiting from Google, looking for ways to migrate a photo library from one application to the other. (In both directions, interestingly.) Based on the search terms people are using, it’s clear that these visitors are looking for one thing above all others. Here’s the bad news:

There is currently no way to move or migrate your raw files from Lightroom to Aperture or from Aperture to Lightroom without losing your edits.

That’s what most of you have been searching for but it’s not possible today. If Apple, Adobe or a third-party publisher provides this functionality, we’ll post something on it straight away and we’ll edit this page to point to the new article. (Subscribe for free via e-mail or RSS to get all our articles when they’re posted.)

Possible in theory

It is theoretically possible for someone to write software to make the move easier. Both Lightroom and Aperture store all your edits as metadata. Any third-party conversion software would read and interpret the metadata in the database used by Lightroom 2 or Lightroom 3, or inside the XMP sidecar files; it would then translate that information, which describes all the edits you’ve made in Lightroom, and add it to the Aperture 3 database or vice versa if you’re travelling in the opposite direction. Your original raw files would remain untouched and in their current locations or would be relocated from an Aperture package.

One problem is that there’d have to be a lot of subjective decisions about interpretation, given the differences in camera profiles between the two applications, their different approaches to noise reduction, sharpening, highlight recovery and so on. It’s a difficult task and the market for such an application is quite small so the product is likely to be expensive. I hope one appears but would consider it unlikely.

Moving JPEG or TIFF renderings of your raw files

If you decide to export JPEGs or TIFFs, you probably need no further help—you can bake your changes into the files, export them and include the IPTC metadata. You can do this as well moving your raw files, using your JPEGs as guidance when reimplementing edits.

If you’ve decided to move the raw files

If you’ve decided to redo your all your edits as and when they’re needed, here’s what works and what doesn’t.

Migrating raw files from Aperture to Lightroom and vice versa

  • Your keywords will travel between the applications if you choose to write XMP sidecar files or if, in Aperture, you select the pictures you’d like to export, choose File=> Export=> Master and then include the IPTC metadata, either as a sidecar file (strongly recommended) or written to the original raw file (definitely not recommended). See below.
  • Ratings (star ratings, from zero to five stars) will also travel in XMP files if you’re moving to Lightroom but not if you’re moving from Lightroom to Aperture. (Apple’s somewhat unconvincing explanation of this is here. ) One workaround if you’re migrating to Aperture: in Lightroom, select all photos with a one star rating, add a keyword like “onestar”; select all photos with a two-star rating and add a keyword like “twostar”; repeat for all star ratings before saving XMP files. (See below for the XMP procedure.) After importing into Aperture, reverse the process: find all the images with the “onestar” keyword, assign them a star and so on. (C’mon, Apple, you can do better than this!) [Update: Apple’s notes for the Aperture 3.0.3 update suggest that it should now be able to read ratings from XMP sidecar files. We haven’t gone back to test this.]
  • Edits… no. As above, you currently need to bake your changes into TIFF files or JPEGs if you’re desperate to keep your edits intact. Of course, you can always do both—move your raw files and export TIFF or JPEGs of certain images to sit alongside.
  • Mapping of IPTC metadata can be tricky. Apple has an article on the subject here.

With these limitations in mind, here are some suggestions for moving your library from Aperture and Lightroom or Lightroom to Aperture with the appropriate options in place. Again, this is for moving raw files without edits but with keywords and other IPTC metadata.

Including metadata with raw files from Aperture 3

This screenshot from Aperture 3 shows the pop-up menu that includes the two relevant options. To get to this stage, choose File=> Export=> Master.

We’d recommend getting Aperture 3 to write the XMP sidecars, as shown in the screenshot  above. In general, try to avoid any alterations to the raw files themselves whenever you can. (In the past, there have been examples of bugs in raw-conversion software that have caused difficulties with modified raw files; these difficulties didn’t show up for a long time. Whenever possible, leave the raw files untouched and write sidecars.)

Including metadata with raw files from Lightroom 2 or Lightroom 3

Exporting metadata from Lightroom is as simple as selecting your pictures and hitting command-S (ctrl-S on Windows) to write an XMP sidecar file for each image. Alternatively, when you’re viewing your grid (to get there, hit G), just select all your pictures and go to the Metadata menu. Choose:

Metadata=> Save Metadata to FIle

When this operation is complete, each raw file will have an XMP sidecar file next to it and Aperture will read these sidecars when you import the pictures. You can then import them into Aperture the way you’d import any new raw files and your keywords will be included. (Remember: Aperture will currently ignore your photo’s star ratings—see above for a workaround.)

One-on-one Lightroom workflow training in the UK

If you’re migrating from Aperture to Lightroom, we have a really great training offer on this month for our Lightroom courses in London and the South East. You get to save lots of time and trouble and master your workflow quickly, with expert tuition. See this page.

Ten backup tips for Adobe Lightroom users

Here’s an article about backing up your work when you’re using Lightroom. It offers ten tips for backup, addressing common mistakes and misconceptions about what photographers should back up and how. It’s a long article but you can skim it very quickly and see if any of the headings tell you something new. If you have no inclination to read it but you think you should do something about making backups of your work, you can always call us.

1. Your Lightroom catalogue is critically important. You need to back it up.

Lightroom maintains a collection of information called a catalogue. The catalogue tracks the names and locations of all the images you import into Lightroom together with the changes you make to those images. Brushes, gradients, keywords, ratings, crops, colour and contrast changes—everything you’ve done to every image in Lightroom is in that catalogue. The Lightroom catalogue has a filename that ends with .lrcat. Find the catalogue and make sure it’s being backed up. See screen shots below to learn how to identify your catalogue file.

Shoot_Raw_Catalog

Above: the Lightroom catalogue icon as it appears in Mac OS X. This particular catalogue is called (fairly obviously) Shoot Raw.

Below: how to find the location and filename of your Lightroom catalogue after choosing Catalog Settings inside the Lightroom application. This example shows a brand new, empty Lightroom catalogue.
Test Lightroom catalogue

2. Back up your image files—they are NOT stored in your Lightroom catalogue

Lightroom doesn’t usually alter your original image files at all, whether you shoot raw, JPEG or TIFF. It uses the original files together with the changes recorded in the Lightroom catalogue to generate all its previews, prints and exports, including web galleries. So: back up both your catalogue and the image files that Lightroom sees.

3. Run your backups when Lightroom is not running

This is an important point that many photographers miss when using Lightroom. If your backup software (something like Time Machine, if you’re on a Mac) backs up your data while you’re working in Lightroom, there’s a good chance that even if it manages to grab a version of that all-important Lightroom catalogue, what it gets might be useless when you need to restore it and you’re in a major, major panic. So make sure you run your backup software when Lightroom itself is not running.

4. RAID isn’t backup. Neither is a Drobo.

If you keep your pictures and Lightroom catalogue on a Drobo or on a suitably configured RAID and you experience the failure of a hard drive, you can usually continue working without data loss. That’s very useful but on its own, it’s not a substitute for a proper backup plan. A backup plan is more likely to protect you against human error and software glitches.

Here are some examples. Let’s say you accidentally ask Lightroom to remove and delete some original raw files instead of the low-res JPEG versions that you once generated for client approval. Or imagine that your Lightroom database becomes damaged and refuses to open. Neither a RAID nor a Drobo will save you. You can’t go back in time and pull back an older version—if your Lightroom catalogue became corrupt yesterday, it’s the corrupt version that’s now safely duplicated across the RAID. There’s no guarantee that you can undelete the raw files you deleted, especially if you don’t discover your mistake for a fortnight. If you one day make a change to a complex, layered file in Photoshop and accidentally saved only the flattened version—how do you get last week’s version back to remove one of the changes?

RAID systems and Drobos help eliminate downtime by guarding against loss of data due to hard drive failure but they do not perform the same job as a backup plan. One is not a substitute for the other.

5. Save space: you don’t have to back up Lightroom’s preview files

If you want to save a chunk of space on your backup drives and get faster backups, you can safely ask your backup software to ignore Lightroom previews and Adobe Camera Raw previews. (For my main Lightroom catalogue, the Lightroom previews are fairly modest in size and quantity but they still amount to tens of gigabytes—that’s a lot of backup space for a folder that changes daily and can be recreated at any time by Lightroom.)

Just find your Lightroom catalogue (see screen shots in step 1) and you’ll find the Lightroom previews sat right next to it. See the screen shot below for the icon. Its filename ends with .lrdata. When you’ve found it, tell your backup software to skip this item when backing up.

Shoot_Raw_Previews

Above: the icon for Lightroom Previews. This file is created automatically by Lightroom, taking its name from the catalogue name that you choose. In this case, it takes the name Shoot Raw Previews.

6. Don’t rely solely on Lightroom’s own catalogue backup function.

Lightroom can back up its own catalogue file every so often. (Scroll back up to the picture of the Catalog Settings window and you’ll see how and where to activate this function.) The way it works is this: you launch Lightroom and it asks you whether you want to back up the catalogue. You look at the message and you say: no, I’m going to have to skip the backup this one time. Don’t you? Be honest—that’s what you do. You do it because you have work to do in Lightroom right now (that’s why you’re launching the application) and you can remember how long it took last time. You tell yourself that you’ll let it do the backup next time. We all do it.

Lightroom_catalogue_backup_prompt
Above: the all-too-familiar prompt and the impossibly tempting Skip Now button

That’s not the only problem. Are you even backing up the catalogue to a separate volume? And when you’ve finished your marathon Lightroom session, the backup copy that you made before you began is already out of date. Lightroom 3 is going to address a couple of these issues by prompting you to back up when you quit the application instead of when you launch it but there’s more.

The catalogue doesn’t contain your images (see point 2) and is no good without them so you need to back those up separately. If you do that, you have to get the (possibly out-of-date) catalogue and the backup of your images in sync when it’s time to restore. And then there are the Lightroom presets you’ve created and any Lightroom plug-ins you’ve installed: are they getting backed up?

The solution is this: get your backup software to run regularly and automatically when Lightroom itself isn’t running. Get it to back up all your stuff, including your images and your Lightroom catalogue. Test it from time to time. Job done. Lightroom’s own backup is great when you’re working in the field with a laptop and some extra drives but it’s not intended to be a replacement for a backup plan.

7. Don’t rely solely on XMP sidecar files

If you don’t know what XMP sidecar files are, you’re not relying on them at all so feel free to scroll down to point 8.

You can ask Lightroom to automatically create and update files called XMP sidecars. The application will create an XMP file for each image that you have in your catalogue. Each XMP file lives right next to the image that it represents.

Each of these sidecars contains any ratings and labels you assigned the image, some metadata from the camera, the keywords you’ve entered and details of all your Lightroom work on that picture. You can send someone an original raw or JPEG file (not an export) together with its XMP sidecar from Lightroom; the lucky recipient will then be able to import the two things together and use Lightroom to see and modify all your adjustments—even the brushes and gradients. That’s useful. But if you rely on backing up these sidecars instead of backing up your Lightroom catalogue, you’ll find that when you start over in a new catalogue, you’ll have lost all of your Lightroom collections and collection sets.

There’s also a performance hit if you tell Lightroom to automatically generate and update these XMP sidecars all the time. Every time you make a change to a file, Lightroom needs to update its catalogue and the sidecars. Twice the work.

Screenshot of XMP files next to NEF (raw) files
Above: image files in a folder with their XMP sidecar files next to them, one per image. These XMP files were generated on demand in Lightroom, not automatically. (Select the images in Lightroom, hit Command-S. Lightroom will write XMP sidecars for them.) That’s our recommendation—just generate them when you need them.

8. Restore your data as a test

Many photographers will first test their backup schemes only when disaster has struck. That’s not the time to find out that your best work hasn’t been backed up at all or that your Lightroom catalogue can’t be restored. Check everything when the going’s good: restore your data to a spare volume and test that it’s valid and complete.

Personal note: I have seen automatic daily backup schemes that were missing all the important stuff on a drive—backups that were not getting a single thing of use in the event of a hard drive failure. (I am not making this up.) The only thing worse than not backing up at all is thinking that you are when in fact you’re not. Backup software today is much easier to use than it once was but don’t leave it to chance—check your backed up data by restoring it to a blank drive and using it. (After a successful test, delete the restored data!)

9. Start with enough space for two to three times your current data size

If you have a terabyte of data, start with at least 2TB of space on your backup drives. Leave plenty of room for expansion and archive. (See the note about the Drobo, below.)

10. Keep off-site copies of your data

A good single backup volume or RAID set next to your main machine will cover you in the event of hard drive failure but probably not fire, flood or theft, all of which might affect your backups as well. Keep a copy of your data off-site and update it every so often.

If you made it this far, well done. If you’re already doing all the stuff in the article, that’s even better. The biggest problem we see is that photographers either aren’t backing up at all or are not backing up what’s needed. Are you one of them? Don’ t be.

Useful links to do with storage, backups and Lightroom

  • XMP sidecar files. If you’d like to find out more, there’s some background in the Wikipedia article here.
  • Time Machine—backup software that’s supplied with Mac OS X. More info on Apple’s site here.
  • Notes on storage devices. This is from Apple and the notes are pretty basic but will be useful for many—the article is here.
  • Mac Pro RAID. A note on Apple’s site about the software and hardware RAID options you get with a Mac Pro here.
  • Drobo. A Drobo (from “data robot”) is a decent choice for a photographer to use as a backup storage device because it’s so easy to expand its volume as the device fills up. (That’s not a recommendation to keep your work on a Drobo—see point 4. The suggestion is that you keep your work where it is and back it up to a Drobo.)

    Shoot_Raw_Drobo_S_Lightroom_backup
    Above: a five-bay Drobo S. This version has space for five drives and is quicker than its four-bay predecessor. The bottleneck was the Drobo’s internal CPU and design, which has been upgraded.

    You can add hard drives to the Drobo and it will give you extra backup space without any reconfiguring and without needing to copy your old data across. You can keep working even if a hard drive fails (in the Drobo S, you can have two hard drives fail) or you can take one drive out and put a bigger drive in its place. You still lose no data and you don’t need to manually copy stuff across to the new drive—the Drobo sorts it all out. To calculate how much real, useable space you get when you add drives to the Drobo, use the appropriate capacity calculator on the Data Robotics site here.

    If you’re thinking of buying a Drobo or two, read this first: data on the disks in a Drobo can only be read in a Drobo of that type. So if yours fails—if its power supply goes, for example—you can’t slide a drive out of it, put it into your Mac Pro and read useful data from it. For a backup device, that limitation is an acceptable price to pay for the convenience and scalability that you get in return but it’s one of the reasons we don’t recommend using a Drobo as primary, online storage unless you have another one as a backup. (Drobos are not unreliable—quite the opposite—but it’s important to be aware of that aspect of a Drobo’s design.)

  • Snow Leopard. Time Machine is supplied with Leopard (Max OS X 10.5) and Snow Leopard (Mac OS X 10.6). You can read about Snow Leopard on Apple’s site. It runs Lightroom faster than Leopard but you must remember to switch Lightroom to 64-bit mode to benefit from the speed boost. (Find Lightroom on your hard drive, select it, choose “File=> Get Info” and make sure the box that says “Run in 32-bit mode” is not ticked.) Snow Leopard requires an Intel Mac.
  • Carbon Copy Cloner. Great software (donationware) from Mike Bombich for making a clone of a single drive (for example, your boot drive) to supplement (not to replace) your data backup plan. Download it from Mike’s site.
  • SuperDuper! Alternative to CCC. Like Carbon Copy Cloner, it can be used to make clones of your boot drive, to supplement (not to replace) your data backup plan. Get it from Shirt Pocket.
  • Online backup services. There are several, with Backblaze and Mozy among the more popular. We haven’t used them or set up clients on them but they’re generally well regarded. They usually cost a few dollars a month and the idea is that your stuff trickles, encrypted, up to a central storage pool using your broadband connection. Probably not much good for your entire catalogue (depends how much you shoot) but potentially good for office documents, paperwork and perhaps a collection of your best work. Some services will send you a DVD or drive of your stuff in the event that you need to restore huge amounts of data.

We can help

If you’re a little stuck when it comes to this sort of stuff and you’re located in the UK, we can help. Call us on 020 3092 2907 or mail develop@shootraw.co.uk.

New: the Shoot Raw Store!

A new thing for us: a virtual store, powered by Amazon UK. We’ve put up a dedicated page about it. You can get to the store by clicking here. We’ve sold a few things already, much to our surprise. Thank you for your support—it’s very encouraging and we really appreciate it!