UKH

I'm on the Adobe photography plan and am looking to get off it. I find that I'm not using Lightroom sufficiently often to justify the cost - and I basically don't use any of the other bundled stuff. But I still want something with Lightroom-like features for more occasional use, so I'm looking at Darktable.

My understanding is that the best migration path is to generate XML sidecars using Lightroom, and then inhale these into Darktable. However it seems that LR only generates sidecars for raw files, and for jpegs etc it writes IPCT metadata directly to the source image file. I'm not keen on this as most of my photos are non-raw and modifying them breaks the whole non-destructive editing premise of Lightroom - even if it is to migrate off it.

I appreciate that editing and filtering settings are unlikely to carry across to Darktable with anything like full fidelity, and I'm ok with that. My main concern is manually-entered captions and geolocation metadata - of which I have quite a lot. Has anyone been down this route, and/or have any thoughts about options for successfully migrating?

Being a software dev, and knowing that the Lightroom catalogue is just a Sqlite database, one option is my writing some code to generate xmp sidecars for my jpegs using data in the database. From a bit of reading, Adobe basically store a fully-formed xmp data element in the database, but its a binary blob and compressed in a non-standard way. I'm pretty sure I could extract it, but it's work I'd rather avoid if tools are already available for this. Does anyone know of anything that already does this?

Other options/thoughts?

In reply to Andy Johnson:

Your question is way above my pay-grade, however I have moved away from my ageing Lightroom 5.7 to PhotoLabs DxO and it seems to have kept the tags that I added to my images.

I never kept a functioning catalogue so I never had to rely on it or panic if the catalogue became corrupt. Just a thought if you don't want to use DarkTables.

In reply to Andy Johnson:

Darktable supports also IPTC data. So your keywording will transfer over If you have pushed them to the jpg already. Naturally the ”development” settings in LR you will loose (BTW. Sidecar and RAW will also loose them IIRC).

I’m in the process of doing the same. Albeit from and older LR (5.7 IIRC)… or rather started that project some years ago and haven’t really done much during the last few years… also majority of my pics were RAWs, so did the sidecar thing. But some were jpg and I do recall the keywords went in just fine…

In reply to HeMa:

Just had a look, and it seems that there is an option to ’save metadata to file’ which should push all the IPCT data into the jpg. But not your pic edits.

In reply to HeMa:

Thats what I don't want. I want the original jpegs to remain unchanged, because that's the way that LR otherwise behaves (non-destructive). Ideally I'd like LR to create xmp sidecars for my jpegs, but at the moment I'm fairly sure that isn't possible.

In reply to The Lemming:

I looked at DxO and was very impressed. I think it's worth the £200 they charge, but personally I can't really justify the cost for something that I'd use relatively infrequently.

In reply to Andy Johnson:

I refuse to justify the cost of paying monthly to rent software, however I am happy to pay for quality software that gives me pleasure and enjoyment, even if its a hobby rather than a job.

I'm also happy knowing that I paid a one-off fee and don't feel as though my creative work is taken hostage and that I have to continually pay a company, Adobe, money just so that I can tinker with my images every now and then or being unable to view my changes years down the line simply because I stopped paying that company.

Horses for coarses.

In reply to The Lemming:

I completely agree with all of that. Thats why I'm getting off the Adobe subscription too. I guess its just where each individual draws the value-for-money line.

What I like about Darktable is that it seems to be decent quality, it seems to have some momentum behind it, and because it is open source I can legally possess the source code. So no-one can take me hostage, and if it goes wrong then I can (probably) fix it myself. But I recognise that this isn't for everyone.

In reply to Andy Johnson:

IPCT data lives in adifferent datablock than the pixels. So saving the ”metadata” is still non destructive for the actual photo side of things. Heck, you can even add keywords and other metadata directly from your OS.

In reply to The Lemming:

For clarity, AFAIK if you stop paying the Adobe subscription you can continue to do basic edits and view images. 

In reply to HeMa:

Thanks. Yes, I know. I appreciate that IPCT metadata goes in app13, which is separate from the image data. Same with EXIF and embedded XMP. I've worked on image handling software and am familiar with JFIF and JPEG etc.

When I used the phrase "non destructive" I should have probably used something like "non-modifying" instead. I don't want my original images to be modified: I want the image and metadata edits to be stored separately. Thats how Lightroom works in all respects (as far as I know) except for the save metadata / ctrl-s command. Its also how earlier generations of apps like Picassa worked.

This might seem a bit perfectionist, but the separation of image and metadata in a jpeg only works if editing software follows the rules of the file format. In the past I've occasionally had photos be visually corrupted by badly written software that stomped on the pixels, or when app crashed while doing something to the image file. In the end a jpeg file is just a sting of bits, and any separation is kind of an illusion. Thats why I don't like editors that modify the image file.

So all that I want to achieve is to extract the image XMP data that exists inside the Lightroom catalogue into sidecar files so that I can import it all into Darktable, and to do that without modifying the image file.

In reply to Andy Johnson:

Yeah. I recall LR only offers the sidecar option for RAW files. So unless you find a pluging to generate the sidecar for your jpgs, then you either allow that needed metadata to be stored on the files directly (and thus be available in Darktable)… or you start tweaking with the db to create the sidecars your self…

oh, but before you spend too much time this problem. Verify, that Darktable actually consumes sidecars for jpg files… there is a good reason Why LR doesn’t support it (because the EXIF, IPCT and such data is directly supported by the file format… unlike a variety of RAW formats).

In reply to HeMa:

It does. You have to include the image file type in sidecar filename - e.g. the sidecar for "image1.jpg" is named "image1.jpg.xmp"

As I suspected, it turns out to be fairly straightforward to extract the xmp data from the lightroom catalogue file. I built a little windows gui app which I'll try to post to github in case its useful to anyone else.