Constant FFMPEG crashes on .15 beta's on both FFMPEG 5.0 and 7.0

[sluflyer06]

Describe the problem you are having

FFMPEG dies constantly (can be few minutes or 1.5 hours) since moving to .15, this is on both beta4 and beta5, i've also tried using FFMPEG 5.0 on beta5 with same results, I've tried different GPU's, different Nvidia drivers, i've tried putting every camera through go2rtc and none of them, Frigate is basically a paperweight right now, the streams are always dead and die soon after a restart. Frigate was rock solid stable for a very long time before this with zero issues.

Version
0.15-beta5-tensortt

Frigate config file
config.yml.txt

Frigate Config

mqtt: host: 10.10.10.60 port: 1883 # client_id: frigate user: asonstein password: Subarupower3 topic_prefix: frigate # stats_interval: 60 auth: failed_login_rate_limit: 1/second;5/minute;20/hour trusted_proxies: - 172.18.0.0/16# <---- this is the subnet for the internal docker compose network telemetry: network_interfaces: - enX0 stats: network_bandwidth: true ffmpeg: hwaccel_args: preset-nvidia

go2rtc:
streams:
garage_main:
- ffmpeg:rtsp://admin:Subarupower3%[email protected]:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
objects:
track:
- person
- car
- dog
- cat
- face
- bicycle
- license_plate
- amazon
- usps
- ups
- fedex
- dhl
- bird
- raccoon
- fox
- squirrel
- rabbit
- package

filters:
person:
# Optional: minimum width*height of the bounding box for the detected object (default: 0)
min_area: 1000
car:
min_area: 5000

#detectors:

tensorrt:

type: tensorrt

device: 0 #This is the default, select the first GPU

detectors:
coral:
type: edgetpu
device: usb

model:
path: plus://0d2c7eacc565cd065d8eb62855d19b94

cameras:
frontdoor:
zones:
yard:
coordinates: 0.029,0.964,0.994,0.981,0.983,0.752,0.017,0.554
inertia: 3
loitering_time: 0
objects:
- person
- car
- dog
- cat
- face
- bicycle
- license_plate
- amazon
- usps
- ups
- fedex
- dhl
- bird
- raccoon
- fox
- squirrel
- rabbit
- package
porch:
coordinates: 720,576,565,576,588,440,208,447,0,429,0,576
ffmpeg:
output_args:
record: preset-record-generic-audio-copy
inputs:
- path:
rtsp://admin:Subarupower3#@10.10.10.200:554/cam/realmonitor?channel=1&subtype=1&authbasic=64
hwaccel_args: preset-nvidia
roles:
- detect
- path:
rtsp://admin:Subarupower3#@10.10.10.200:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
hwaccel_args: preset-nvidia
roles:
- record
- audio
ui:
order: 1
detect:
width: 720
height: 576
fps: 5
snapshots:
enabled: true
required_zones:
- yard
quality: 90
record:
enabled: true
retain:
days: 7
mode: all
events:
retain:
default: 30
motion:
mask:
- 339,264,720,294,720,0,0,0,0,218,65,258
review:
alerts:
required_zones: yard
detections:
required_zones:
- yard
- porch
garage:
ffmpeg:
output_args:
record: preset-record-generic-audio-aac
inputs:
- path: rtsp://127.0.0.1:8554/garage_main
input_args: preset-rtsp-restream
roles:
- record
- audio
- path:
rtsp://admin:Subarupower3#@10.10.10.174:554/cam/realmonitor?channel=1&subtype=2&authbasic=64
hwaccel_args: preset-nvidia
roles:
- detect
ui:
order: 0
detect:
width: 1280
height: 720
fps: 5
snapshots:
enabled: true
required_zones:
- GarageZone
quality: 90
record:
enabled: true
retain:
days: 7
mode: all
events:
retain:
default: 30

zones:
  GarageZone:
    coordinates: 0.005,0.372,0.556,0.215,0.949,0.391,0.895,0.983,0.012,0.991
    loitering_time: 3
    inertia: 4
review:
  alerts:
    required_zones: GarageZone
  detections:
    required_zones: GarageZone
motion:
  mask: 
    0.002,0.003,0.003,0.283,0.132,0.296,0.23,0.287,0.466,0.214,0.517,0.173,0.52,0.003
objects:
  filters:
    car:
      mask:
        - 0.307,0.303,0.383,0.282,0.355,0.169,0.237,0.202,0.24,0.31
        - 0.718,0.407,0.788,0.221,0.963,0.25,1,0.476,0.942,0.452

backyardmain:
ffmpeg:
output_args:
record: preset-record-generic-audio-aac
inputs:
- path:
rtsp://admin:Subarupower3#@10.10.10.153:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
roles:
- record
- audio
- path:
rtsp://admin:Subarupower3#@10.10.10.153:554/cam/realmonitor?channel=1&subtype=2&authbasic=64
hwaccel_args: preset-nvidia
roles:
- detect
onvif:
host: 10.10.10.153
port: 80
user: admin
password: Subarupower3#
ui:
order: 2
detect:
width: 1280
height: 720
fps: 5
snapshots:
enabled: true
required_zones:
- Back-backyard
- EastEdgePatio
- Garden
quality: 90
record:
enabled: true
retain:
days: 7
mode: all
events:
retain:
default: 30
zones:
Back-backyard:
coordinates: 1,0.376,0.186,0.286,0.256,0.16,0.328,0.114,0.507,0.071,0.932,0.124
loitering_time: 0
inertia: 2
objects:
- person
- dog
- cat
- face
- bicycle
- bird
- raccoon
- fox
- squirrel
- rabbit
EastEdgePatio:
coordinates: 0.728,1,0.79,0.345,0.662,0.334,0.289,1
loitering_time: 0
objects:
- person
- dog
- cat
- face
- bicycle
- bird
- raccoon
- fox
- squirrel
- rabbit
inertia: 2
Garden:
coordinates: 0.011,0.403,0.206,0.245,0.196,0.195,0.003,0.331
loitering_time: 0
objects:
- person
- dog
- cat
- face
- bicycle
- bird
- raccoon
- fox
- squirrel
- rabbit
review:
alerts:
required_zones:
- Back-backyard
- EastEdgePatio
- Garden
motion:
mask:
0.235,0.101,0.595,0.037,0.713,0.045,0.809,0.057,0.863,0.059,0.918,0.087,0.943,0.114,0.961,0.138,0.998,0.165,0.999,0.004,0.226,0
objects:
filters:
person:
mask:
0,0.669,0.127,0.722,0.166,0.762,0.194,0.81,0.216,0.855,0.235,0.901,0.237,0.946,0.227,0.968,0.154,0.996,0.003,0.994
car:
mask: 1,1,0.98,0.059,0,0.09,0.039,1
YardEast:
ffmpeg:
output_args:
record: preset-record-generic-audio-aac
inputs:
- path:
rtsp://admin:Subarupower3#@10.10.10.125:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
roles:
- record
- audio
- path:
rtsp://admin:Subarupower3#@10.10.10.125:554/cam/realmonitor?channel=1&subtype=2&authbasic=64
hwaccel_args: preset-nvidia
roles:
- detect
ui:
order: 3
detect:
width: 1280
height: 720
fps: 5
snapshots:
enabled: true
required_zones:
- EastYard
quality: 90
record:
enabled: true
retain:
days: 7
mode: all
events:
retain:
default: 30
zones:
EastYard:
coordinates: 0.458,0.981,0.005,0.862,0.315,0.03,0.551,0.02
loitering_time: 0
objects:
- person
- dog
- cat
- face
- bicycle
- bird
- raccoon
- fox
- squirrel
- rabbit
objects:
filters:
car:
mask: 0.089,0.597,1,0.556,0.999,0.999,0.035,1
review:
alerts:
required_zones: EastYard
version: 0.14
camera_groups:
Cameras:
order: 1
icon: LuAirplay
cameras:
- garage
- backyardmain
- frontdoor
- YardEast

Docker Compose
dockercomposefrigate.txt

Docker Compose

version: "3.7"

services:
mosquitto:
image: eclipse-mosquitto
hostname: mosquitto
container_name: mosquitto
restart: unless-stopped
ports:
- "1883:1883"
- "9001:9001"
volumes:
- /home/anthony/docker/configs/mosquitto:/etc/mosquitto
- /home/anthony/docker/configs/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf

frigate:
container_name: frigate
privileged: true # this may not be necessary for all setups
runtime: nvidia
restart: unless-stopped
image: ghcr.io/blakeblackshear/frigate:0.15.0-beta5-tensorrt
network_mode: host
devices:
- /dev/bus/usb:/dev/bus/usb
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
shm_size: "512mb" # update for your cameras based on calculation above
volumes:
- /etc/localtime:/etc/localtime:ro

  - /home/anthony/docker/configs/frigate/config:/config
  - /media/Frigate:/media/frigate
  - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
    target: /tmp/cache
    tmpfs:
      size: 2000000000
ports:
  - "5000:5000"
  - "1935:1935" # RTMP feeds
  - "8971:8971"
  - "1984:1984" #go2RTC web interface
  - "8554:8554" #rtsp restream
environment:
  FRIGATE_RTSP_PASSWORD: *     
  PLUS_API_KEY: ****

Logs [frigate-logs.txt](https://github.com/user-attachments/files/18399162/frigate-logs.txt) [frigate_logs.txt](https://github.com/user-attachments/files/18399164/frigate_logs.txt)

Systems Stats

Storage Page

Host specs
Supermicro H13SRA-TF motherboard
Threadripper 7960x
128GB RDIMM ECC 6000mhz
XCP-NG VM host
Frigate run on Ubuntu 24.x
GPU's: Quadro P2000, RTX3090 Turbo

[NickM-27]

The errors in the logs have nothing to do with nvidia, they are showing issues with /tmp/cache being full

[sluflyer06]

they are, here you see when it stopped recording and then i just restarted frigate and now you see new clips

[NickM-27]

Okay, then for some reason the removal is not working for these files in /tmp/cache as the file descriptor is being left open indefinitely. I would probably suggest starting with a super basic single camera config and see if that still happens.

[sluflyer06]

is there things that changed in .15 you think could be at play? I've been using Frigate a long time and recently your plus subscription without any issues, all this started happening trying to use .15. I could easily turn off 3 of the POE cameras and leave just the front door running.

[NickM-27]

No, if this was something directly frigate related we would expect many reports from users of issues, this is likely to be something very specific.

And just for clarity sake, I don't work on Frigate+, that is Blake

[sluflyer06]

I want to provide an update, I couldn't get frigate to run for typically more than 20-45 minutes ever since going to .15, it's now been close to 25 hours up and tmp/cache is closing out teh files and writing them, not building up a bunch, the single change I made was to make backups of the 3 database files and then remove them and let frigate build a new database. I read a response you made to someone else in a discussion sometime in past and you had mentioned to them that their files weren't closing out because of database corruption so I thought that might be worth a try here.

This was the discussion I found that led me to want to try wiping out the frigate database, I can continue to monitor that this issue stays away and report back, I saved the database in case you wanted it to see what went wrong and see if there is an issue to be fixed or maybe its just random bad luck. #7581 (comment)