non-destructive editing

Comments

15 comments

  • Avatar
    David Tillett

    John, I don’t think that Lightroom stores its step by step history in its XMP sidecar files. I looked inside one of mine and it just seemed to be a recording of the state of all the sliders etc at the time the editing ended, and history came up as one of the items not included in XMP sidecar files when I did a Google search. I suspect that any saving if step by step history is in the Lightroom catalogue itself, along with other things like version information that is not held in XMP files. Hence the small size of Lightroom XMP files.

    If a Luminar image is saved with its history then it will contain a number of sets of state information, one for each editing step. As things are done at the moment I think that this is something of an overkill as the size can mount up with a full dump of current state for each change, including masking brush strokes. I can understand why Skylum might have chosen this approach as it makes it easy to go back several steps just by resetting everything to their values in the saved state, rather than having to go back step by step. However, I do hope that they take a more space efficient approach in the DAM, otherwise the storage requirements to hold histories for lots of edits will go through the roof, regardless of whether this information is held in some form of database or as sidecar files.

    When Luminar was a Mac only product the way that .lmnr files were designed makes perfect sense to me. They use a standard Mac function to package up different media types in a way that the system can treat as one file. There is a jpg thumbnail in there, though for some reason it doesn’t get displayed unless Neptune is installed. I suspect this is down to a bug in the way that .lmnr files are registered in the system rather than an issue with Luminar or .lmnr files in general. By allowing original files to be included in this package there is just one file that needs to be copied if sending it to someone else with Luminar, no remembering to send both original and XMP files. And it allows history to be recorded. Other applications that are mentioned as being alternatives to Luminar such as ON1 and DXO have no history functionality at all.

    With any non-destructive editing program there will always be lock-in. All the various sidecar files hold details of settings that are specific to that program. Even if sliders have the same names differences in their processing may mean that an image will appear different when sliders are set to same value. There may be attempts to mimic, say, Lightroom settings so that program can make an approximation based in values in Lightroom XMP file, but this falls down as it might well have done today when Adobe make big changes such as their new profiles and presets.

    2
    Comment actions Permalink
  • Avatar
    John Lilley

    Robert and David, thanks for the replies. I do understand all that. But the fact is that the sidecar files that Lightroom generates are very tiny, and contain the history. I can create virtual copies that take practically no space. If I do similar edits in Luminar, the files are very huge. I was hoping that the Skylum DAM would also have that ability to save raw edits in a sidecar file.

    1
    Comment actions Permalink
  • Avatar
    Phil Hurd

    Robert, that was a really good description of how things work, well put. I have turned off XMP sidecar files for Lightroom as I don't like my hard drive being junked up with them. I assume that Lightroom holds the edits inside its database. 

    A number of issues I have found with using Apple Photos as a Library . . . .

    1. Even when referencing files (i.e. not storing images inside the Photos database) the library file takes up a lot of disk space. I have around 46,000 RAW files taking up 1.13TB on an external drive. Cataloguing these in Photos produces a library file that is 100GB, much larger than the Lightroom library file.

    2. You can only import images into Photos in batches, try and do your whole collection and mcOS tells you there is not enough disk space, even though there is. I think that it assumes you are fully importing into the Photos library even when referencing the files.

    3. You cannot export more than a single image to Aurora HDR from Apple Photos, so basically useless if you do a lot of HDR photography.

     

    1
    Comment actions Permalink
  • Avatar
    Andrew Hewitt

    Just to add a thought to that, during a period when I was deciding on whether to use Photos or Lightroom, I did discover, by accidental stupidity, and resulting in a corrupt library file, that the XMP files were an absolute saviour.

    A single library file, such as is used by Photos, Aperture or Lightroom (and others too), is subject to being lost due to user error, file corruptions or drive failures. Due to its nature, it can be highly likely to lose the entire library, with all organisation and all edits lost.

    I found that by using the XMP sidecar files, I was able to reimport the entire collection, and retain all the edits I had applied, as well as all keywords and ratings etc. and was able to rebuild a new library from the organised folder structure I was using as a master storage system.

    I do now have a much better organised redundant copy and backup setup, but nonetheless, although I can recover my library file if needed (although having a backup copy is no guarantee that you can recover anything), I think it helps to have another level that can be recovered from.

    The library file size will depend on what you've done to your images, they will likely contain sets of thumbnails and previews of each image, and these may depend on whether you've applied edits or not. In Lightroom you can set different sizes and qualities for the previews, in Photos you cannot, but I'd suspect you could surpass the Photos library size if you set Lightroom to save 1:1 previews at high quality for all images.

    Photos is actually quite good at what it's meant for, and not too shabby for more complex tasks - to a point, but be wary of the library file.

    By personal experience, I'd use sidecar files if at all possible - they'e only 8-9KB each anyway, a small fraction of a Raw file (they're not even used for non-raw files).

     

    1
    Comment actions Permalink
  • Avatar
    Phil Hurd

    Which probably gives Skylum an impossible task i.e. pleasing all of the people all of the time!

    1
    Comment actions Permalink
  • Avatar
    John Lilley

    David, good point! The Lightroom catalog is really big so I suspect that you are correct about the edits being there. Still hoping for a good DAM for Mac with hierarchical keywords and the ability to search on multiple criteria like Lightroom. I used to use IDImager but that went away (I didn't like its replacement). I also used iMatch - extremely powerful and flexible, but, alas, Windows only.

    Thanks to all who replied with your input - all good stuff and I really appreciate it!

    1
    Comment actions Permalink
  • Avatar
    Angela Andrieux

    Hi John, I can understand your frustration with the file sizes an appreciate your feedback! We'll be releasing a big update mid-month with better performance so I invite you to try again after that release. When the DAM is launched later this year the storage issue should also improve.

    0
    Comment actions Permalink
  • Avatar
    David Tillett

    When saving a native Luminar file there are check boxes on the save dialogue. If you clear both history and save original resources then you get a small side car file. If you don’t move original file then opening Luminar native file will allow you to make further changes. 

    It is history that seems to cause file size to shoot up, particularly if using masks. 

    0
    Comment actions Permalink
  • Avatar
    Robert Garnett

    Hi John, NDE seems to spawn a number of myths and mysteries, but let me try to suggest how you can contain file sizes.

    #1 using Photos, Lightroom or any other photo-editor as the host and Luminar as a plug-in will result in an additional file to your original RAW. 

    #2 Luminar does not corrupt or modify your original RAW file but stores its own edits in one of two ways. The most common is that the host software creates a TIFF or JPEG file to represent the edited image returned from the Luminar plug-in. TIFF files are usually the reason for file size increase - try converting an unprocessed RAW to TIFF and you will see what I mean.

    #3 Luminar is designed to run either as stand-alone or plug-in. More versatile than my previous favourite Nik Collection. However, at this stage I know of no DAM that can display LMNR files, other than Luminar itself, though on once an image is opened. You could blame Apple, as that deficiency in reading thumbnails could be attributed to them, but I guess they won't care. Wait as Angela says for "later this year". The image you see in Lightroom is of the TIFF file created by "Stamping" the Luminar edits in to the file sent to the plug-in.

    #4 If you can live with identifying the images you want to edit in one App and then using folder/file name to open directly in Luminar there is an easy solution to your problem. Incidentally, for those who save images in a folder structure, you should set preferences in Photos to not copy them in to the Photos catalog. By default Photos adds the original image to its catalog thus duplicating files and increasing storage sizes. In Lightroom you can also go {File/Export with preset/Luminar2018 - Open source files) to go direct to Luminar. Do not include Lightroom adjustments as that will create the TIFF file you want to avoid. The downside is that you then have two memory hungry apps open at the same time. Not advisable for an 8GB MacBook Air!

    #5 From stand-alone Luminar,when you have made your edits, go {File/Save} to open the dialog box to choose where and how you want to save your edited image. You may want to use a separate folder for Luminar files, or you can save them alongside the RAW originals. You can also rename if you want. The key is to uncheck all boxes that would otherwise save Original Resources, History or Windows Compatibility. (Skylum recommends saving History but I am unconvinced of the benefit). If you opened the file from Lightroom, do not click "Apply" as that will create a TIFF file in your catalog and folders. Close the window after you have saved the LMNR file.

    #6 The LMNR file you have saved can be reopened by Luminar and the details of all your presets/filters/layers are preserved for you to modify. Something you won't be able to do with edits made as a plug-in. If you check the file-size in Finder you should see that it is only a couple of MBs depending on the extent of your editing. If you like, you can refer to this minimal file as a "sidecar" as it adds its processes to the RAW original when you open the LMNR file.

    Hope that helps but please ask if anything is unclear.

    0
    Comment actions Permalink
  • Avatar
    Angela Andrieux

    Hi John - We'll release specifics on how the DAM will handle edits/sidecar files/virtual copies/etc. when we get closer to the launch.  We do appreciate the feedback and are taking all of those factors into consideration for the design/functionality of the DAM.

    0
    Comment actions Permalink
  • Avatar
    John Lilley

    Thanks, Angela. I'll be looking forward to the DAM release. Hopefully it will allow me to leave the Adobe world!

     

    0
    Comment actions Permalink
  • Avatar
    Andrew Hewitt

    I'd like to add a nod to that too. The .lmnr file is somewhat limiting. Using sidecars is much more of a standard way to work, and, in my experience, is certainly more reliable than a proprietary database.

    I've been trialling darktable for a couple of weeks, and got to like it quite a lot. My suggestion would be to have a look at that, it uses XMP files for all its edits and history, but also handles versions and stacking as well - it just creates incrementally numbered XMP files for each version (much better than Lightroom IMHO).

    Of course your developers might be coming up with all their own fantastic ideas on how to do things better, but sometimes things that ain't broke, don't need fixing.

    One of the things that's driven people away from Lightroom, and similar solutions, is the idea of a closed in system, and of course has created an awful lot of discussion, with many users being very wary of their next choices. Personally I don't need you to tie me in with a database that locks me in to your software, I'll keep using it as long as it does its job for a reasonable cost, but I'd still want to be able to access my photos outside of Luminar.

    Cheers.

    0
    Comment actions Permalink
  • Avatar
    Phil Hurd

    I think its all a matter of personal choice in the end Andrew. Lightroom provides the facility to switch-off XMP sidecar files, my preference. The files in Lightroom are referenced only, the originals still exist on the hard-disk and as I now only edit in Luminar or Aurora (sometimes CK Tonality) and the Skylum Apps create a separate file. This leaves the original untouched, so I can't lose my edits. Finally I backup my Lightroom catalogue regularly (to a separate disk), back up all my images (also to a separate disk) and I  copy the originals to a BluRay disk after converting them to Adobe .DNG files.

    0
    Comment actions Permalink
  • Avatar
    Andrew Hewitt

    Hi Phil, yes I agree, it is absolutely a matter of preference. I was merely offering some food for thought on it.

    0
    Comment actions Permalink
  • Avatar
    Todd Prior

    Just as an aside to all the comments above. 1. I agree all the extra files are a pain but as noted they can be extremely helpful especially if you have spent a really long time on an image and you don't want to rely on your database 100%. 2. Also nice for external media to take the edits with you....I only wish you could embed a small low res preview in them. Currently I would have 3 copies of my files...originals, sidecars and exported JPEGs of the edits. There is an open source software called Lightzone that embeds a JPEG preview in their sidecar files so that you don't need all 3 files and best of all you can view your edited files in your file system, other apps or on your phone......to me this would be a great way to be able to work....if anyone at skylum is listening...ha ha

    0
    Comment actions Permalink

Please sign in to leave a comment.