HDR videos thumbnails not tone-mapped correctly [web]

[Thinkscape]

The bug

HDR videos from iPhone 13-16 uploaded to Immich have incorrectly tone-mapped thumbnails.

The OS that Immich Server is running on

Debian

Version of Immich Server

v1.123

Version of Immich Mobile App

1.123.0 build.186

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

services:
  immich:
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - /var/data/immich/upload:/usr/src/app/upload
      - /dev/dri:/dev/dri
      - /etc/localtime:/etc/localtime:ro
    env_file: /var/data/config/immich/immich.env
    depends_on:
      - redis
      - database
    ports: 
      - 2283:2283
    networks:
      - internal
      - traefik_public
    deploy:
      mode: global
        
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - /var/data/immich/cache:/cache
      - /dev/dri:/dev/dri
    env_file: /var/data/config/immich/immich.env
    networks:
      - internal
    deploy:
      mode: global
      labels:
        - traefik.enable=false

  redis:
    image: redis:6.2
    networks:
      - internal
    deploy:
      labels:
        - traefik.enable=false

  db:
    # image: postgres:14
    image: tensorchord/pgvecto-rs:pg14-v0.2.0
    env_file: /var/data/config/immich/immich.env
    volumes:
      - /var/data/runtime/immich/database:/var/lib/postgresql/data
    networks:
      - internal
    deploy:
      labels:
        - traefik.enable=false
  
  pgbackups:
    image: prodrigestivill/postgres-backup-local
    restart: always
    env_file: /var/data/config/immich/immich-db-backup.env
    volumes:
      - /var/data/immich/database-dump:/backups
    networks:
      - internal
    depends_on:
      - db
      
networks:
  traefik_public:
    external: true
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.8.0/24

Your .env content

IMMICH_VERSION=release
PUID=1000
PGID=1000
TZ=Australia/Sydney

MACHINE_LEARNING_GPU_ACCELERATION= #optional
MACHINE_LEARNING_WORKERS=1 #optional
MACHINE_LEARNING_WORKER_TIMEOUT=120 #optional

###################################################################################
# Database
###################################################################################

# These are for the Immich components
DB_HOSTNAME=db
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# These are specific to how the postgres image likes to receive its ENV vars
POSTGRES_PASSWORD=postgres
#POSTGRES_USER=postgres
POSTGRES_DB=immich

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=


###################################################################################
# JWT SECRET
###################################################################################

JWT_SECRET=...
###################################################################################
# MAPBOX
####################################################################################

# ENABLE_MAPBOX is either true of false -> if true, you have to provide MAPBOX_KEY
ENABLE_MAPBOX=false
MAPBOX_KEY=

###################################################################################
# WEB - Required
###################################################################################

# This is the URL of your vm/server where you host Immich, so that the web frontend
# know where can it make the request to.
# For example: If your server IP address is 10.1.11.50, the environment variable will
# be IMMICH_API_URL_EXTERNAL=http://10.1.11.50:2283/api
# !CAUTION! THERE IS NO FORWARD SLASH AT THE END

IMMICH_API_URL_EXTERNAL=https://immich.example.com/api


####################################################################################
# WEB - Optional
####################################################################################

# Custom message on the login page, should be written in HTML form.
# For example VITE_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>[email protected]</i><br>Password: <i>demo</i>"

VITE_LOGIN_PAGE_MESSAGE=

NODE_ENV=production

Reproduction steps

1. Have a HDR video from iPhone 13 - 16 (default settings)
2. Disable tone mapping in video transcoding settings in Administration (b/c the new player can play HDR videos)
3. Upload video to Immich

Expected

The thumbnail matches a frame from the video

Actual

The thumbnail looks pale, washed-out, even though the video plays well on hover and in full-screen

Relevant log output

No response

Additional information

No response

[mertalev]

Disabling tone-mapping doesn't mean the output is in HDR. It just means the colors are clipped instead of mapped. You should keep the setting enabled unless you have a specific reason (generally hardware compatibility issues or troubleshooting). The setting applies to thumbnails too.

[Thinkscape]

This is further enabled by #12104 released in https://github.com/immich-app/immich/releases/tag/v1.122.0

[mertalev]

This seems like a regression. Did you change any other transcoding settings?

[Thinkscape]

How is having HDR support a regression exactly?

Just did a test with a fresh file on v1.123.0, getting neat HDR transcoded videos, just the thumbnails are broken as per my original issue.

  "ffmpeg": {
    "accel": "nvenc",
    "accelDecode": true,
    "acceptedAudioCodecs": [
      "aac"
    ],
    "acceptedContainers": [],
    "acceptedVideoCodecs": [
      "h264"
    ],
    "bframes": -1,
    "cqMode": "auto",
    "crf": 20,
    "gopSize": 0,
    "maxBitrate": "17000k",
    "preferredHwDevice": "auto",
    "preset": "medium",
    "refs": 0,
    "targetAudioCodec": "aac",
    "targetResolution": "original",
    "targetVideoCodec": "h264",
    "temporalAQ": true,
    "threads": 0,
    "tonemap": "disabled",
    "transcode": "optimal",
    "twoPass": false
  },
  "image": {
    "colorspace": "p3",
    "extractEmbedded": false,
    "preview": {
      "format": "webp",
      "quality": 90,
      "size": 1440
    },
    "thumbnail": {
      "format": "webp",
      "quality": 90,
      "size": 1080
    }
  },

[mertalev]

It's a regression insofar as it's an unintentional change in behavior that hasn't been tested. The output also seems strange - it seems like 8-bit HDR? "True" HDR is supposed to be at least 10 bits, and many GPUs don't support 10-bit output. We'll need to do more testing around the output, hardware compatibility and the impact on browser compatibility.

[Thinkscape]

Nothing can be done with nvenc/qs etc. lacking 10bit support, but from my testing ffmpeg just falls back to software encoder as per usual, so no dramas there. Indeed it's yuv420p b/c it's being requested by immich - the result looks better than any of the tone mapped videos, and plays well. I can't tell much difference from native (source) file played on my iPhone, compared to immich web playing that non-mapped transcoded bt2020 file in Chromium 🤷