[Feature] JPEG XL HDR support

[Qhilm]

The feature

I export my developed (read: tuned/modified) DSLR pictures in JPEG XL HDR P3 from Adobe Lightroom and they appear very dark in Immich, both when using Firefox on macOS and Chrome on macOS. I have attached two samples:

• JPEG_XL_HDR.jxl <= this appears very dark
• JPEG_XL_SDR.jxl <= this appears fine.

Github does not support jxl files, hence I put both in a zip file.

Disclaimer 1: I know "does google photos have it?" is a criteria for Immich Feature Requests and JPEG XL support (let alone JPEG XL HDR P3) is not yet available on google photos: Google has a heated and complicated relation to JPEG XL, but it is being adopted by many big players such as Apple, Adobe, etc., hence I am confident it will be added in time to google photos as well.

Disclaimer 2: this is not a very "backup" centric feature and I understand Immich main purpose is backing up mobile photos. I do wish Apple and Google default camera apps would start to use JPEG XL HDR tomorrow, as it's awesome and it's long overdue to have large support for HDR in photos (not tone mapping) – I never understood why video had it for so long while photos where lagging. I guess there's less money to be made with photos ;)

JPEG XL HDR Test.zip

Platform

• Server
• Web
• Mobile

[mertalev]

Thanks for providing example images! HDR images appearing darker is interesting. I can take a look at it later to see if there's anything we can do there.

As for general JPEG XL support, the question isn't so much about whether Google Photos has it or if it's backup-related (immich is more than a photo backup solution at this point), but client support. Chrome (infamously) doesn't support it and I don't think the Flutter mobile app does either, although the latter can be remedied.

I've been laying the groundwork for allowing generated images to be in other formats besides JPEG and WebP. The primary motivation has been for HDR AVIF, but it might be viable to add JPEG XL as a target too if we can get it working on the mobile app.

[Starbix]

I'm not sure if this should be in issue instead, if so, I'll post it there as well.
I also use Adobe Lightroom to develop (in my case convert from negative) my pictures and then export them as AVIF HDR (P3). I use AVIF because my MacBook doesn't support the newest macOS and thus doesn't support JPEG XL natively (I really hope I can switch soon though).

These AVIFs look great natively, in Safari, and in the Photos app. After uploading them to Immich, the conversion to thumbnails renders the images very flat and with a wrong black point. Exporting the AVIF to JPG using macOS's Preview looks good.

I've attached a few files to showcase the differences. The AVIF file is the original file as exported by Lightroom, the preview file is the AVIF exported to JPG by Preview, and immich-thumbnail is directly downloaded from Immich. Note that the original color profile is Display P3; SMPTE ST 2084 PQ, while the the color profile for the thumbnail is sP3C (which I've never heard of).

Using ffmpeg to convert the AVIF to JPG without any options yields a very similar result to Immich's thumbnail. Using the options -vf "zscale=transfer=linear,tonemap=hable,zscale=transfer=bt709" yields a better result. Not as good as the export done by Preview though (definitely partly because of the preservation of the color space).
avif.zip

A correct color conversion for HDR pictures in AVIF (or JPEG XL) is crucial as they aren't simple SDR pictures with gain maps.

[AngelaDMerkel]

A small update, but Google and Adobe seem to have plans to unify their gain map strategy per this talk by Eric Chan who is the lead developer on all of Adobe's HDR features. Apple is currently working with this ISO for their HDR, but it seems in the future they might try to homogenise their format with Adobe and Google.

EDIT: apparently Adobe software can (or will be able to) produce AVIFs/JXL with gain maps, but it's still experimental and not exposed by default:

In the talk they discuss how some unique box code referred to as hrgm could be added to JXL, but that doesn't seem to be the standard way that the devs expected HDR to be included in JXL images. Devs commenting on an imagemagick issue have said they plan to include support for AVIF and HEIC in libultrahdr so there is honestly probably nothing that needs to be done on the part of vips for this to work considering magick will get it virtually out of the box. There is an open PR to include the new support.

I asked one of the devs for JXL Jon Sneyers if he can provide some additional context and if he responds I'll update.

[AngelaDMerkel]

Update: it appears that JXL uses this draft spec to encode HDR base images without the use of any gain map. JXL also supports gain maps, but my understanding is that HDR is handled in JXL primarily by encoding this way.

[jcupitt]

Ah nice. That's a much more sensible standard than those dumb gain maps!

[AngelaDMerkel]

I agree, and my experience is that rendering is much better using this technique than gain maps.

[Starbix]

As mentioned here, Photomator is able to convert AVIF (with only NCLX) to AVIF (with ICC) so that libvips can create nice looking preview JPGs. Not really practical for my workflow (especially since Photomator isn't free), but probably quite useful for testing.