[Config Support]: /tmp/cache fills up intermittently #11402

submac · 2024-05-16T20:43:15Z

submac
May 16, 2024

Describe the problem you are having

Hi,
My /tmp/cache fills up for no obvious reason. My Frigate instance sometimes works for a week without issues, but more frequently it works for a few hours, or even only for 10 minutes before the cache fills up. When Frigate is working as expected, my cache uses ~50MB. When it spikes to 4GB, it does that within minutes and causes streams to crash. Logs don't give me any hints.
I'm running a Proxmox VM with Debian, Docker and Portainer.

Any ideas on how to troubleshoot it?

Thanks,
Maciek

Version

0.13.2-6476F8A

Frigate config file

# yaml-language-server: $schema=http://<host>:5000/api/config/schema.json

mqtt:
  enabled: True
  host: 10.20.0.2
  port: 1883
  topic_prefix: frigate
  user: "{FRIGATE_MQTT_USER}"
  password: "{FRIGATE_MQTT_PASSWORD}"
database:
  path: /config/frigate.db
ffmpeg:
  hwaccel_args: preset-vaapi
detectors:
  coral:
    type: edgetpu
    device: usb
objects:
  track:
    - bicycle
    - bus
    - car
    - cat
    - dog
    - motorcycle
    - person
detect:
  width: 1980
  height: 1080
  fps: 5
record:
  enabled: True
  sync_recordings: True
  retain:
    days: 7
    mode: motion
  events:
    retain:
      default: 7
      mode: active_objects
snapshots:
  enabled: True
  timestamp: True
  bounding_box: True
  retain:
    default: 14
birdseye:
  enabled: True
  mode: continuous
  restream: True
  width: 2560
  height: 1440
go2rtc:
  streams:
    sc01:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc01.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc01#audio=opus"
    sc01_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc01.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    # sc01_live:
    #   - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc01.{FRIGATE_DOMAIN}:554/live
    sc02:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc02.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc02#audio=opus"
    sc02_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc02.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc03:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc03.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc03#audio=opus"
    sc03_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc03.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc04:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc04.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc04#audio=opus"
    sc04_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc04.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc05:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc05.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc05#audio=opus"
    sc05_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc05.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc06:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc06.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc06#audio=opus"
    sc06_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc06.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc07:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc07.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc07#audio=opus"
    sc07_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc07.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2
    sc08:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc08.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:sc08#audio=opus"
    sc08_detect:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@sc08.{FRIGATE_DOMAIN}:554/cam/realmonitor?channel=1&subtype=2

cameras:
  sc01:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc01_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc01
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 2
    zones:
      sc01_podjazd:
        coordinates: 0,920,147,1016,296,1080,442,1080,713,1080,882,1080,1573,1080,1835,1080,1980,1080,1980,721,1933,550,1881,428,1835,331,1776,222,1533,201,1508,268,1414,256,1364,333,691,332,642,258,482,258,346,285,145,396,0,479
      sc01_droga_podjazd:
        coordinates: 147,399,351,293,638,265,671,315,1370,309,1420,250,1781,263,1881,425,1936,551,1980,710,1980,327,1980,0,1425,0,996,0,549,0,0,0,0,499
  sc02:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc02_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc02
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 1
    zones:
      sc02_ogrod_front:
        coordinates: 0,1080,239,1080,390,1080,617,1080,811,1080,1231,1080,1530,1080,1616,973,1712,828,1779,702,1834,585,1817,572,1873,457,1920,322,1810,267,1702,208,1633,178,1633,158,1615,146,1576,151,1566,189,1546,196,1544,222,1385,267,1319,247,1117,308,1020,456,863,675,506,825,171,982,0,700
      sc02_droga_front:
        coordinates: 66,824,175,986,517,840,873,666,1026,447,1123,315,1316,249,1368,265,1550,221,1546,193,1603,157,1684,198,1771,242,1855,290,1910,325,1936,235,1959,116,1980,0,1709,0,1385,0,947,0,498,0,0,0,0,381,0,679
  sc03:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc03_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc03
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 4
    zones:
      sc03_ogrod_taras:
        coordinates: 0,1080,232,1080,527,1080,827,1080,1173,1080,1498,1080,1791,1080,1980,1080,1980,496,1877,374,1816,293,1752,229,1648,210,1517,193,1444,192,1269,182,1067,183,849,190,639,207,446,229,281,261,147,292,0,317
      sc03_polana:
        coordinates: 150,290,282,257,446,228,639,205,850,187,1063,177,1266,182,1441,187,1508,194,1643,208,1749,224,1797,271,1844,329,1895,388,1947,461,1980,514,1980,367,1980,239,1980,117,1980,0,1794,0,1590,0,1356,0,1136,0,941,0,716,0,476,0,237,0,0,0,0,162,0,314
  sc04:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc04_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc04
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 7
    zones:
      sc04_ogrod_taras:
        coordinates: 0,1080,278,1080,717,1080,1034,1080,1597,1080,1980,1080,1980,830,1980,597,1980,289,1980,267,1867,246,1769,228,1677,214,1556,201,1405,189,1213,185,982,190,724,206,485,214,288,231,142,251,44,379,0,442
      sc04_polana:
        coordinates: 70,342,150,243,292,228,488,209,726,204,979,186,1210,183,1401,190,1554,198,1672,212,1763,226,1832,237,1859,241,1913,252,1951,263,1980,294,1980,136,1980,0,1887,0,1760,0,1607,0,1442,0,1239,0,994,0,719,0,474,0,265,0,0,0,0,73,0,252,0,437
  sc05:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc05_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc05
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 8
    zones:
      sc05_ogrod_tyl:
        coordinates: 93,509,142,595,204,682,314,825,443,966,543,1080,738,1080,1194,1080,1651,1080,1980,1080,1980,731,1980,453,1917,432,1678,354,1403,290,1148,242,932,210,764,190,632,176,547,171,478,170,420,170,360,170,208,282,107,360,53,404
      sc05_droga_tyl:
        coordinates: 107,359,206,279,297,219,369,162,472,166,547,171,634,177,765,188,931,206,1145,237,1401,285,1673,349,1914,428,1980,455,1980,260,1980,122,1980,0,1859,0,1698,0,1486,0,1261,0,1070,0,832,0,619,0,462,0,219,0,0,0,0,135,0,289,54,406
  sc06:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc06_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc06
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 9
    zones:
      sc06_ogrod_tyl:
        coordinates: 0,1080,186,1080,396,1080,629,1080,857,1080,1210,1080,1544,1080,1665,977,1765,874,1872,736,1980,582,1980,363,1927,315,1845,254,1770,203,1738,200,1698,203,1642,204,1576,203,1494,210,1381,218,1235,231,1045,254,800,290,561,339,310,402,107,470,0,509
      sc06_droga_tyl:
        coordinates: 111,466,312,393,561,336,804,286,1044,248,1236,226,1381,210,1489,201,1577,199,1639,199,1693,195,1734,197,1770,199,1844,250,1921,315,1948,337,1980,323,1980,142,1980,0,1888,0,1775,0,1697,0,1628,0,1554,0,1433,0,1284,0,1084,0,821,0,549,0,303,0,0,0,0,183,0,370,0,514
  sc07:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc07_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc07
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 6
    zones:
      sc07_przejscie:
        coordinates: 0,1080,292,1080,628,1080,1070,1080,1470,1080,1931,1080,1920,984,1865,756,1769,543,1680,360,1531,343,1294,324,924,303,503,302,153,322,0,338
      sc07_za_ogrodzeniem_przy_przejsciu:
        coordinates: 155,322,502,296,927,298,1294,316,1529,338,1676,356,1730,441,1778,531,1823,635,1870,744,1898,853,1922,982,1935,1080,1980,1080,1980,953,1980,776,1980,564,1980,388,1980,190,1980,0,1819,0,1647,0,1401,0,985,0,487,0,83,0,0,0,0,172,0,337
  sc08:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/sc08_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/sc08
          input_args: preset-rtsp-restream
          roles:
            - record
    birdseye:
      order: 3
    zones:
      sc08_przejscie:
        coordinates: 1616,1080,1678,981,1741,880,1797,774,1844,671,1884,570,1922,443,1955,307,1980,197,1980,133,1816,61,1785,86,1748,79,1699,108,1658,136,1592,176,1499,238,1367,327,1171,446,892,620,546,828,235,1014,111,1080
      sc08_za_ogrodzeniem_przy_przejsciu:
        coordinates: 107,1080,235,1009,550,824,894,617,1173,440,1367,322,1497,234,1587,176,1653,136,1722,91,1757,100,1811,59,1880,88,1980,129,1980,0,1832,0,1675,0,1493,0,1250,0,968,0,741,0,496,0,260,0,0,0,0,199,0,476,0,780,0,1080



# Stack config
version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable

    shm_size: 256mb # update for your cameras based on calculation
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/dri/card0

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /media/frigate/clips/:/media/frigate/clips
      - /media/frigate/recordings/:/media/frigate/recordings
      - /root/frigate/:/config/
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 4000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_MQTT_USER: ${FRIGATE_MQTT_USER}
      FRIGATE_MQTT_PASSWORD: ${FRIGATE_MQTT_PASSWORD}
      FRIGATE_RTSP_USER: ${FRIGATE_RTSP_USER}
      FRIGATE_RTSP_PASSWORD: ${FRIGATE_RTSP_PASSWORD}
      FRIGATE_DOMAIN: ${FRIGATE_DOMAIN}
      FRIGATE_ANNKE_USER: ${FRIGATE_ANNKE_USER}
      FRIGATE_ANNKE_PASSWORD: ${FRIGATE_ANNKE_PASSWORD}

Relevant log output

2024-05-16 19:51:30.480980561  [INFO] Preparing Frigate...
2024-05-16 19:51:30.491568427  [INFO] Starting Frigate...
2024-05-16 19:51:31.803451044  [2024-05-16 19:51:31] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-05-16 19:51:31.803455613  [2024-05-16 19:51:31] frigate.app                    INFO    : Creating directory: /media/frigate/exports
2024-05-16 19:51:31.898009675  [2024-05-16 19:51:31] peewee_migrate.logs            INFO    : Starting migrations
2024-05-16 19:51:31.901516440  [2024-05-16 19:51:31] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-05-16 19:51:31.921030262  [2024-05-16 19:51:31] frigate.app                    INFO    : Recording process started: 701
2024-05-16 19:51:31.922683603  [2024-05-16 19:51:31] frigate.app                    INFO    : go2rtc process pid: 89
2024-05-16 19:51:31.940775236  [2024-05-16 19:51:31] detector.coral                 INFO    : Starting detection process: 710
2024-05-16 19:51:34.595681025  [2024-05-16 19:51:31] frigate.app                    INFO    : Output process started: 715
2024-05-16 19:51:34.595686082  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc01: 748
2024-05-16 19:51:34.595687785  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc02: 750
2024-05-16 19:51:34.595693263  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc03: 751
2024-05-16 19:51:34.595715924  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc04: 754
2024-05-16 19:51:34.595718229  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc05: 755
2024-05-16 19:51:34.595742975  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc06: 757
2024-05-16 19:51:34.595801376  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc07: 760
2024-05-16 19:51:34.596015201  [2024-05-16 19:51:32] frigate.app                    INFO    : Camera processor started for sc08: 761
2024-05-16 19:51:34.596020993  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc01: 763
2024-05-16 19:51:34.596022624  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc02: 767
2024-05-16 19:51:34.596024028  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc03: 770
2024-05-16 19:51:34.596092034  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc04: 773
2024-05-16 19:51:34.596141019  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc05: 777
2024-05-16 19:51:34.596186052  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc06: 784
2024-05-16 19:51:34.596218217  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc07: 792
2024-05-16 19:51:34.596272318  [2024-05-16 19:51:32] frigate.app                    INFO    : Capture process started for sc08: 796
2024-05-16 19:51:34.601002941  [2024-05-16 19:51:31] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-05-16 19:51:34.601200589  [2024-05-16 19:51:34] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-05-16 19:52:47.428533794  [2024-05-16 19:52:47] frigate.record.util            INFO    : Deleting 7 recordings files with missing DB entries
2024-05-16 20:03:02.550523529  [2024-05-16 20:03:02] ffmpeg.sc02.record             ERROR   : [segment @ 0x5604eb071880] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.558246365  [2024-05-16 20:03:02] ffmpeg.sc02.record             ERROR   : [segment @ 0x5604eb071880] Failure occurred when ending segment '/tmp/cache/sc02@20240516200243+0000.mp4'
2024-05-16 20:03:02.558253770  [2024-05-16 20:03:02] ffmpeg.sc02.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.558255446  [2024-05-16 20:03:02] ffmpeg.sc02.record             ERROR   : Error writing trailer of /tmp/cache/sc02@%Y%m%d%H%M%S%z.mp4: Invalid argument
2024-05-16 20:03:02.558257026  [2024-05-16 20:03:02] watchdog.sc02                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.558258390  [2024-05-16 20:03:02] watchdog.sc02                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.601739749  [2024-05-16 20:03:02] ffmpeg.sc03.record             ERROR   : [segment @ 0x55dc78408a80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.601826363  [2024-05-16 20:03:02] ffmpeg.sc03.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.601878524  [2024-05-16 20:03:02] ffmpeg.sc03.record             ERROR   : [segment @ 0x55dc78408a80] Failure occurred when ending segment '/tmp/cache/sc03@20240516200244+0000.mp4'
2024-05-16 20:03:02.601946869  [2024-05-16 20:03:02] ffmpeg.sc03.record             ERROR   : Error writing trailer of /tmp/cache/sc03@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.602046223  [2024-05-16 20:03:02] watchdog.sc03                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.602201850  [2024-05-16 20:03:02] watchdog.sc03                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.609487283  [2024-05-16 20:03:02] ffmpeg.sc04.record             ERROR   : [segment @ 0x5645adf7d680] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.609708927  [2024-05-16 20:03:02] ffmpeg.sc04.record             ERROR   : [segment @ 0x5645adf7d680] Failure occurred when ending segment '/tmp/cache/sc04@20240516200243+0000.mp4'
2024-05-16 20:03:02.609867704  [2024-05-16 20:03:02] ffmpeg.sc04.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.616428893  [2024-05-16 20:03:02] ffmpeg.sc04.record             ERROR   : Error writing trailer of /tmp/cache/sc04@%Y%m%d%H%M%S%z.mp4: Invalid argument
2024-05-16 20:03:02.616613749  [2024-05-16 20:03:02] watchdog.sc04                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.616736689  [2024-05-16 20:03:02] watchdog.sc04                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.628200076  [2024-05-16 20:03:02] ffmpeg.sc07.record             ERROR   : [segment @ 0x558c6ff3f680] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.628504025  [2024-05-16 20:03:02] ffmpeg.sc07.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.641578537  [2024-05-16 20:03:02] ffmpeg.sc07.record             ERROR   : [segment @ 0x558c6ff3f680] Failure occurred when ending segment '/tmp/cache/sc07@20240516200253+0000.mp4'
2024-05-16 20:03:02.642061586  [2024-05-16 20:03:02] ffmpeg.sc07.record             ERROR   : Error writing trailer of /tmp/cache/sc07@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.642660974  [2024-05-16 20:03:02] ffmpeg.sc08.record             ERROR   : [segment @ 0x563438f1ce80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.643140366  [2024-05-16 20:03:02] ffmpeg.sc08.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.643144378  [2024-05-16 20:03:02] ffmpeg.sc08.record             ERROR   : [segment @ 0x563438f1ce80] Failure occurred when ending segment '/tmp/cache/sc08@20240516200253+0000.mp4'
2024-05-16 20:03:02.643146378  [2024-05-16 20:03:02] ffmpeg.sc08.record             ERROR   : Error writing trailer of /tmp/cache/sc08@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.643147854  [2024-05-16 20:03:02] watchdog.sc08                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.643287320  [2024-05-16 20:03:02] watchdog.sc08                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.643312459  [2024-05-16 20:03:02] watchdog.sc07                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.643383305  [2024-05-16 20:03:02] watchdog.sc07                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.713604804  [2024-05-16 20:03:02] ffmpeg.sc06.record             ERROR   : [segment @ 0x55f17ae20cc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.713800634  [2024-05-16 20:03:02] ffmpeg.sc06.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.713862886  [2024-05-16 20:03:02] ffmpeg.sc06.record             ERROR   : [segment @ 0x55f17ae20cc0] Failure occurred when ending segment '/tmp/cache/sc06@20240516200253+0000.mp4'
2024-05-16 20:03:02.713908504  [2024-05-16 20:03:02] ffmpeg.sc06.record             ERROR   : Error writing trailer of /tmp/cache/sc06@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.713959789  [2024-05-16 20:03:02] watchdog.sc06                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.713998402  [2024-05-16 20:03:02] watchdog.sc06                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.784724665  [2024-05-16 20:03:02] ffmpeg.sc05.record             ERROR   : [segment @ 0x558ee03b8680] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.792329239  [2024-05-16 20:03:02] ffmpeg.sc05.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.793930248  [2024-05-16 20:03:02] ffmpeg.sc05.record             ERROR   : [segment @ 0x558ee03b8680] Failure occurred when ending segment '/tmp/cache/sc05@20240516200253+0000.mp4'
2024-05-16 20:03:02.794008406  [2024-05-16 20:03:02] ffmpeg.sc05.record             ERROR   : Error writing trailer of /tmp/cache/sc05@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.794148658  [2024-05-16 20:03:02] watchdog.sc05                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.794153364  [2024-05-16 20:03:02] watchdog.sc05                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:02.804292040  [2024-05-16 20:03:02] ffmpeg.sc01.record             ERROR   : [segment @ 0x55ada2aa3340] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-05-16 20:03:02.804531177  [2024-05-16 20:03:02] ffmpeg.sc01.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-05-16 20:03:02.804621405  [2024-05-16 20:03:02] ffmpeg.sc01.record             ERROR   : [segment @ 0x55ada2aa3340] Failure occurred when ending segment '/tmp/cache/sc01@20240516200253+0000.mp4'
2024-05-16 20:03:02.804774658  [2024-05-16 20:03:02] ffmpeg.sc01.record             ERROR   : Error writing trailer of /tmp/cache/sc01@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-05-16 20:03:02.804835780  [2024-05-16 20:03:02] watchdog.sc01                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:02.804889688  [2024-05-16 20:03:02] watchdog.sc01                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.576978338  [2024-05-16 20:03:12] ffmpeg.sc02.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.577239555  [2024-05-16 20:03:12] ffmpeg.sc02.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.577249200  [2024-05-16 20:03:12] ffmpeg.sc02.record             ERROR   : 
2024-05-16 20:03:12.577331683  [2024-05-16 20:03:12] watchdog.sc02                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.577376090  [2024-05-16 20:03:12] watchdog.sc02                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.621845728  [2024-05-16 20:03:12] ffmpeg.sc03.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.622004628  [2024-05-16 20:03:12] ffmpeg.sc03.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.622017637  [2024-05-16 20:03:12] ffmpeg.sc03.record             ERROR   : 
2024-05-16 20:03:12.622059152  [2024-05-16 20:03:12] watchdog.sc03                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.622086697  [2024-05-16 20:03:12] watchdog.sc03                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.625840755  [2024-05-16 20:03:12] ffmpeg.sc04.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.631657055  [2024-05-16 20:03:12] ffmpeg.sc04.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.631850828  [2024-05-16 20:03:12] ffmpeg.sc04.record             ERROR   : 
2024-05-16 20:03:12.631928538  [2024-05-16 20:03:12] watchdog.sc04                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.631987216  [2024-05-16 20:03:12] watchdog.sc04                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.660734185  [2024-05-16 20:03:12] ffmpeg.sc07.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.660740052  [2024-05-16 20:03:12] ffmpeg.sc07.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.660741329  [2024-05-16 20:03:12] ffmpeg.sc07.record             ERROR   : 
2024-05-16 20:03:12.660742851  [2024-05-16 20:03:12] watchdog.sc07                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.660744310  [2024-05-16 20:03:12] watchdog.sc07                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.666919085  [2024-05-16 20:03:12] ffmpeg.sc08.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.666925790  [2024-05-16 20:03:12] ffmpeg.sc08.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.666927669  [2024-05-16 20:03:12] ffmpeg.sc08.record             ERROR   : 
2024-05-16 20:03:12.666993916  [2024-05-16 20:03:12] watchdog.sc08                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.667015033  [2024-05-16 20:03:12] watchdog.sc08                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.730621485  [2024-05-16 20:03:12] ffmpeg.sc06.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.730709893  [2024-05-16 20:03:12] ffmpeg.sc06.record             ERROR   : Error initializing output stream 0:0 -- 
2024-05-16 20:03:12.738050666  [2024-05-16 20:03:12] ffmpeg.sc06.record             ERROR   : 
2024-05-16 20:03:12.738186894  [2024-05-16 20:03:12] watchdog.sc06                  INFO    : Terminating the existing ffmpeg process...
2024-05-16 20:03:12.738188490  [2024-05-16 20:03:12] watchdog.sc06                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-05-16 20:03:12.805438514  [2024-05-16 20:03:12] ffmpeg.sc05.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-05-16 20:03:12.805490714  [2024-05-16 20:03:12] ffmpeg.sc05.record             ERROR   : Error initializing output stream 0:0 --

Frigate stats

{"cameras":{"sc01":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":765,"detection_enabled":1,"detection_fps":6.6,"ffmpeg_pid":794,"pid":750,"process_fps":4.9,"skipped_fps":0.0},"sc02":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":769,"detection_enabled":1,"detection_fps":4.3,"ffmpeg_pid":773,"pid":752,"process_fps":5.0,"skipped_fps":0.0},"sc03":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":772,"detection_enabled":1,"detection_fps":5.0,"ffmpeg_pid":776,"pid":754,"process_fps":4.8,"skipped_fps":0.0},"sc04":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":777,"detection_enabled":1,"detection_fps":18.0,"ffmpeg_pid":787,"pid":756,"process_fps":4.8,"skipped_fps":0.0},"sc05":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":782,"detection_enabled":1,"detection_fps":9.0,"ffmpeg_pid":790,"pid":757,"process_fps":4.8,"skipped_fps":0.0},"sc06":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":788,"detection_enabled":1,"detection_fps":6.7,"ffmpeg_pid":800,"pid":760,"process_fps":4.8,"skipped_fps":0.0},"sc07":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":793,"detection_enabled":1,"detection_fps":10.3,"ffmpeg_pid":803,"pid":762,"process_fps":4.8,"skipped_fps":0.0},"sc08":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":801,"detection_enabled":1,"detection_fps":3.9,"ffmpeg_pid":815,"pid":763,"process_fps":4.7,"skipped_fps":0.0}},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"16":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"17":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"25":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"26":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"28":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"30":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"31":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"32":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"33":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"42":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"43":{"cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"81":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"82":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"83":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"92":{"cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"11.2","cpu_average":"17","mem":"0.1"},"109":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"110":{"cmdline":"python3 -u -m frigate","cpu":"3.3","cpu_average":"9","mem":"0.9"},"120":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"147":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"148":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"149":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"150":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"158":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"177":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"198":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"212":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"229":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"240":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"261":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"280":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"297":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"309":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"329":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"358":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"378":{"cmdline":"nginx: cache manager process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"701":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.1"},"703":{"cmdline":"frigate.recording_manager","cpu":"0.0","cpu_average":"0","mem":"0.1"},"711":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(69)","cpu":"0.3","cpu_average":"0","mem":"0.0"},"712":{"cmdline":"frigate.detector.coral","cpu":"26.2","cpu_average":"27","mem":"0.1"},"714":{"cmdline":"frigate.output       ","cpu":"0.2","cpu_average":"6","mem":"0.2"},"720":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"722":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"723":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"724":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"725":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"726":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"727":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"728":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1980x1080 -i pipe: -f mpegts -s 1320x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"729":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 2560x1440 -i pipe: -f mpegts -s 2560x1440 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"2","mem":"0.1"},"750":{"cmdline":"frigate.process:sc01 ","cpu":"4.0","cpu_average":"4","mem":"0.3"},"752":{"cmdline":"frigate.process:sc02 ","cpu":"2.9","cpu_average":"3","mem":"0.3"},"754":{"cmdline":"frigate.process:sc03 ","cpu":"3.6","cpu_average":"3","mem":"0.3"},"756":{"cmdline":"frigate.process:sc04 ","cpu":"11.5","cpu_average":"10","mem":"0.3"},"757":{"cmdline":"frigate.process:sc05 ","cpu":"4.3","cpu_average":"5","mem":"0.3"},"760":{"cmdline":"frigate.process:sc06 ","cpu":"4.8","cpu_average":"5","mem":"0.3"},"762":{"cmdline":"frigate.process:sc07 ","cpu":"6.5","cpu_average":"7","mem":"0.3"},"763":{"cmdline":"frigate.process:sc08 ","cpu":"2.9","cpu_average":"3","mem":"0.3"},"765":{"cmdline":"frigate.capture:sc01 ","cpu":"2.5","cpu_average":"3","mem":"0.3"},"769":{"cmdline":"frigate.capture:sc02 ","cpu":"2.3","cpu_average":"3","mem":"0.3"},"772":{"cmdline":"frigate.capture:sc03 ","cpu":"2.6","cpu_average":"3","mem":"0.3"},"773":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc02_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"3.7","cpu_average":"4","mem":"0.1"},"776":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc03_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"4.5","cpu_average":"5","mem":"0.1"},"777":{"cmdline":"frigate.capture:sc04 ","cpu":"2.2","cpu_average":"2","mem":"0.3"},"782":{"cmdline":"frigate.capture:sc05 ","cpu":"2.6","cpu_average":"3","mem":"0.3"},"787":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc04_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"3.3","cpu_average":"4","mem":"0.1"},"788":{"cmdline":"frigate.capture:sc06 ","cpu":"2.8","cpu_average":"3","mem":"0.3"},"790":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc05_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"5.0","cpu_average":"5","mem":"0.1"},"793":{"cmdline":"frigate.capture:sc07 ","cpu":"2.5","cpu_average":"2","mem":"0.3"},"794":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc01_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"4.2","cpu_average":"4","mem":"0.1"},"800":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc06_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"4.3","cpu_average":"4","mem":"0.1"},"801":{"cmdline":"frigate.capture:sc08 ","cpu":"2.2","cpu_average":"3","mem":"0.3"},"803":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc07_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"4.0","cpu_average":"4","mem":"0.1"},"815":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/sc08_detect -r 5 -vf fps=5,scale_vaapi=w=1980:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"3.6","cpu_average":"4","mem":"0.1"},"1164":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc08@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc08/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1172":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc06@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc06/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1180":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc01@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc01/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1188":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc03@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc03/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1196":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc04@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc04/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1204":{"cmdline":"ffmpeg -hide_banner -y -i /tmp/cache/sc02@20240516201123+0000.mp4 -c copy -movflags +faststart /media/frigate/recordings/2024-05-16/20/sc02/11.23.mp4","cpu":"0.0","cpu_average":"0","mem":"0.0"},"2873":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"}},"detection_fps":63.8,"detectors":{"coral":{"detection_start":1715891507.466011,"inference_speed":8.02,"pid":712}},"gpu_usages":{"intel-vaapi":{"gpu":"2.08%","mem":"-%"}},"processes":{"go2rtc":{"pid":92},"logger":{"pid":701},"recording":{"pid":703}},"service":{"last_updated":1715891508,"latest_version":"0.13.2","storage":{"/dev/shm":{"free":169.2,"mount_type":"tmpfs","total":256.0,"used":86.8},"/media/frigate/clips":{"free":1813277.2,"mount_type":"ext4","total":5078521.7,"used":3009228.5},"/media/frigate/recordings":{"free":1813277.2,"mount_type":"ext4","total":5078521.7,"used":3009228.5},"/tmp/cache":{"free":0.0,"mount_type":"tmpfs","total":3814.7,"used":3814.7}},"temperatures":{},"uptime":1246,"version":"0.13.2-6476f8a"}}

Operating system

Proxmox

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

NickM-27 · 2024-05-16T20:44:43Z

NickM-27
May 16, 2024
Collaborator Sponsor

this is usually an issue when requesting a large clip for notifications or other purposes via the API. The only way to know for sure though is to check the contents of /tmp/cache once this has occurred and see what is taking up space

8 replies

submac May 22, 2024
Author

@NickM-27 have you had a moment to look at the above stats maybe?

NickM-27 May 22, 2024
Collaborator Sponsor

I don't see anything that would indicate a problem for the recordings. I would suggest enabling debug logs for frigate.record.maintainer and paste the logs after this problem starts happening again

submac Jun 2, 2024
Author

@NickM-27
I set the maintainer to debug mode. I think copy times were ok. There are only a few above 1s. What's weird is the log was output until 19:06 and then there was nothing until 19:17. Between those two timestamps cache was constantly filling up until it hit max (3.73GB) at 19:17 what you can also see in the logs.
Container CPU looked stable at that time at around 40%. Download on NAS was at ~8MB/s.
It looks like something hanged and started filling up the cache but I don't have enough Frigate knowledge to figure this out.

2024-06-02 19:05:20.619130437  [INFO] Preparing Frigate...
2024-06-02 19:05:20.665933414  [INFO] Starting Frigate...
2024-06-02 19:05:22.528847178  [2024-06-02 19:05:22] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-06-02 19:05:22.528929210  [2024-06-02 19:05:22] frigate.app                    INFO    : Creating directory: /media/frigate/exports
2024-06-02 19:05:22.613397230  [2024-06-02 19:05:22] peewee_migrate.logs            INFO    : Starting migrations
2024-06-02 19:05:22.620597076  [2024-06-02 19:05:22] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-06-02 19:05:22.639441056  [2024-06-02 19:05:22] frigate.app                    INFO    : Recording process started: 701
2024-06-02 19:05:22.642376155  [2024-06-02 19:05:22] frigate.app                    INFO    : go2rtc process pid: 87
2024-06-02 19:05:22.668757460  [2024-06-02 19:05:22] detector.coral                 INFO    : Starting detection process: 710
2024-06-02 19:05:25.314473468  [2024-06-02 19:05:22] frigate.app                    INFO    : Output process started: 712
2024-06-02 19:05:25.321401601  [2024-06-02 19:05:22] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-06-02 19:05:25.326024000  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc01: 750
2024-06-02 19:05:25.326183652  [2024-06-02 19:05:25] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-06-02 19:05:25.326236693  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc02: 751
2024-06-02 19:05:25.326350732  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc03: 753
2024-06-02 19:05:25.326378162  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc04: 756
2024-06-02 19:05:25.326445814  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc05: 758
2024-06-02 19:05:25.326615828  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc06: 759
2024-06-02 19:05:25.326619581  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc07: 761
2024-06-02 19:05:25.326726651  [2024-06-02 19:05:23] frigate.app                    INFO    : Camera processor started for sc08: 763
2024-06-02 19:05:25.326729225  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc01: 765
2024-06-02 19:05:25.326874442  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc02: 769
2024-06-02 19:05:25.326877117  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc03: 772
2024-06-02 19:05:25.326878580  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc04: 776
2024-06-02 19:05:25.327111562  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc05: 781
2024-06-02 19:05:25.327113357  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc06: 788
2024-06-02 19:05:25.327114506  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc07: 794
2024-06-02 19:05:25.327115536  [2024-06-02 19:05:23] frigate.app                    INFO    : Capture process started for sc08: 800
2024-06-02 19:05:39.361108245  [2024-06-02 19:05:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/05.25.mp4 in 1.1199798583984375 seconds.
2024-06-02 19:05:39.362794356  [2024-06-02 19:05:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/05.25.mp4 in 0.7342000007629395 seconds.
2024-06-02 19:05:39.659722613  [2024-06-02 19:05:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/05.25.mp4 in 0.33056116104125977 seconds.
2024-06-02 19:05:43.047398403  [2024-06-02 19:05:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/05.25.mp4 in 0.36812686920166016 seconds.
2024-06-02 19:05:43.075664175  [2024-06-02 19:05:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/05.25.mp4 in 0.32111096382141113 seconds.
2024-06-02 19:05:43.076488646  [2024-06-02 19:05:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/05.25.mp4 in 0.37767720222473145 seconds.
2024-06-02 19:05:43.128062212  [2024-06-02 19:05:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/05.25.mp4 in 0.3311450481414795 seconds.
2024-06-02 19:05:43.134239291  [2024-06-02 19:05:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/05.25.mp4 in 0.4767160415649414 seconds.
2024-06-02 19:05:47.648880927  [2024-06-02 19:05:47] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:05:49.254360863  [2024-06-02 19:05:49] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/05.34.mp4 in 0.6575210094451904 seconds.
2024-06-02 19:05:49.256050361  [2024-06-02 19:05:49] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/05.34.mp4 in 0.39417505264282227 seconds.
2024-06-02 19:05:49.423247370  [2024-06-02 19:05:49] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/05.34.mp4 in 0.18882107734680176 seconds.
2024-06-02 19:05:53.028944524  [2024-06-02 19:05:53] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/05.36.mp4 in 0.3310580253601074 seconds.
2024-06-02 19:05:53.176680132  [2024-06-02 19:05:53] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/05.34.mp4 in 0.3778350353240967 seconds.
2024-06-02 19:05:53.187685090  [2024-06-02 19:05:53] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/05.34.mp4 in 0.4435279369354248 seconds.
2024-06-02 19:05:53.201665058  [2024-06-02 19:05:53] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/05.34.mp4 in 0.3591189384460449 seconds.
2024-06-02 19:05:53.230739166  [2024-06-02 19:05:53] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/05.34.mp4 in 0.32666707038879395 seconds.
2024-06-02 19:05:57.649674626  [2024-06-02 19:05:57] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:05:59.202093939  [2024-06-02 19:05:59] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/05.44.mp4 in 0.8434398174285889 seconds.
2024-06-02 19:05:59.203283182  [2024-06-02 19:05:59] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/05.44.mp4 in 0.6593868732452393 seconds.
2024-06-02 19:05:59.204585447  [2024-06-02 19:05:59] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/05.44.mp4 in 0.21988296508789062 seconds.
2024-06-02 19:05:59.382171862  [2024-06-02 19:05:59] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/05.44.mp4 in 0.2035360336303711 seconds.
2024-06-02 19:06:02.649737138  [2024-06-02 19:06:02] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:06:02.924291729  [2024-06-02 19:06:02] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/05.46.mp4 in 0.25975799560546875 seconds.
2024-06-02 19:06:02.991301988  [2024-06-02 19:06:02] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/05.44.mp4 in 0.29850196838378906 seconds.
2024-06-02 19:06:03.035824963  [2024-06-02 19:06:03] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/05.44.mp4 in 0.3204529285430908 seconds.
2024-06-02 19:06:03.049770915  [2024-06-02 19:06:03] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/05.44.mp4 in 0.2979300022125244 seconds.
2024-06-02 19:06:07.650466219  [2024-06-02 19:06:07] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:06:09.264747965  [2024-06-02 19:06:09] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/05.54.mp4 in 1.0564451217651367 seconds.
2024-06-02 19:06:09.267030335  [2024-06-02 19:06:09] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/05.54.mp4 in 0.688960075378418 seconds.
2024-06-02 19:06:09.277287431  [2024-06-02 19:06:09] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/05.54.mp4 in 0.2184619903564453 seconds.
2024-06-02 19:06:09.414306253  [2024-06-02 19:06:09] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/05.54.mp4 in 0.18082594871520996 seconds.
2024-06-02 19:06:12.974533381  [2024-06-02 19:06:12] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/05.56.mp4 in 0.31052708625793457 seconds.
2024-06-02 19:06:13.001959136  [2024-06-02 19:06:13] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/05.54.mp4 in 0.3114030361175537 seconds.
2024-06-02 19:06:13.098599126  [2024-06-02 19:06:13] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/05.54.mp4 in 0.37207794189453125 seconds.
2024-06-02 19:06:13.103484153  [2024-06-02 19:06:13] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/05.54.mp4 in 0.3453810214996338 seconds.
2024-06-02 19:06:17.654294614  [2024-06-02 19:06:17] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:06:19.157040340  [2024-06-02 19:06:19] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/06.04.mp4 in 0.6286251544952393 seconds.
2024-06-02 19:06:19.158339162  [2024-06-02 19:06:19] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/06.04.mp4 in 0.2348041534423828 seconds.
2024-06-02 19:06:19.310603518  [2024-06-02 19:06:19] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/06.04.mp4 in 0.176131010055542 seconds.
2024-06-02 19:06:22.935845727  [2024-06-02 19:06:22] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/06.06.mp4 in 0.2567729949951172 seconds.
2024-06-02 19:06:23.044035844  [2024-06-02 19:06:23] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/06.04.mp4 in 0.3241422176361084 seconds.
2024-06-02 19:06:23.060071666  [2024-06-02 19:06:23] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/06.04.mp4 in 0.3621029853820801 seconds.
2024-06-02 19:06:23.088192116  [2024-06-02 19:06:23] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/06.04.mp4 in 0.33362698554992676 seconds.
2024-06-02 19:06:23.119419303  [2024-06-02 19:06:23] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/06.04.mp4 in 0.29232001304626465 seconds.
2024-06-02 19:06:27.650111476  [2024-06-02 19:06:27] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:06:29.235740857  [2024-06-02 19:06:29] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/06.14.mp4 in 0.9033241271972656 seconds.
2024-06-02 19:06:29.235747251  [2024-06-02 19:06:29] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/06.14.mp4 in 0.4419560432434082 seconds.
2024-06-02 19:06:29.288756506  [2024-06-02 19:06:29] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/06.14.mp4 in 0.2670009136199951 seconds.
2024-06-02 19:06:29.490403457  [2024-06-02 19:06:29] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/06.14.mp4 in 0.2871119976043701 seconds.
2024-06-02 19:06:32.653958876  [2024-06-02 19:06:32] frigate.record.maintainer      DEBUG   : Found 1 old frames.
2024-06-02 19:06:33.126350945  [2024-06-02 19:06:33] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/06.16.mp4 in 0.43386411666870117 seconds.
2024-06-02 19:06:33.286461059  [2024-06-02 19:06:33] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/06.14.mp4 in 0.5451951026916504 seconds.
2024-06-02 19:06:33.298670942  [2024-06-02 19:06:33] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/06.14.mp4 in 0.5077829360961914 seconds.
2024-06-02 19:06:33.334161738  [2024-06-02 19:06:33] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/06.14.mp4 in 0.514739990234375 seconds.
2024-06-02 19:06:39.224836172  [2024-06-02 19:06:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc01/06.24.mp4 in 1.200990915298462 seconds.
2024-06-02 19:06:39.226229692  [2024-06-02 19:06:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc05/06.24.mp4 in 1.0148069858551025 seconds.
2024-06-02 19:06:39.227512360  [2024-06-02 19:06:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc04/06.24.mp4 in 0.8120589256286621 seconds.
2024-06-02 19:06:39.360211149  [2024-06-02 19:06:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc03/06.24.mp4 in 0.5344560146331787 seconds.
2024-06-02 19:06:39.786536769  [2024-06-02 19:06:39] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc02/06.24.mp4 in 0.5944149494171143 seconds.
2024-06-02 19:06:43.508087433  [2024-06-02 19:06:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc08/06.26.mp4 in 0.8464319705963135 seconds.
2024-06-02 19:06:43.525203305  [2024-06-02 19:06:43] frigate.record.util            INFO    : Deleting 4 recordings files with missing DB entries
2024-06-02 19:06:43.791411531  [2024-06-02 19:06:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc06/06.24.mp4 in 1.0968530178070068 seconds.
2024-06-02 19:06:43.802246156  [2024-06-02 19:06:43] frigate.record.maintainer      DEBUG   : Copied /media/frigate/recordings/2024-06-02/19/sc07/06.24.mp4 in 1.035370111465454 seconds.
2024-06-02 19:17:24.015296237  [2024-06-02 19:17:24] ffmpeg.sc02.record             ERROR   : [segment @ 0x56241a4eafc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.015488344  [2024-06-02 19:17:24] ffmpeg.sc02.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.022177268  [2024-06-02 19:17:24] ffmpeg.sc02.record             ERROR   : [segment @ 0x56241a4eafc0] Failure occurred when ending segment '/tmp/cache/sc02@20240602191714+0000.mp4'
2024-06-02 19:17:24.022182820  [2024-06-02 19:17:24] ffmpeg.sc02.record             ERROR   : Error writing trailer of /tmp/cache/sc02@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.022184544  [2024-06-02 19:17:24] watchdog.sc02                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.022186082  [2024-06-02 19:17:24] watchdog.sc02                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.023027857  [2024-06-02 19:17:24] ffmpeg.sc05.record             ERROR   : [segment @ 0x55c8db038900] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.039011152  [2024-06-02 19:17:24] ffmpeg.sc06.record             ERROR   : [segment @ 0x555dfe75c680] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.039016554  [2024-06-02 19:17:24] ffmpeg.sc05.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.039018216  [2024-06-02 19:17:24] ffmpeg.sc05.record             ERROR   : [segment @ 0x55c8db038900] Failure occurred when ending segment '/tmp/cache/sc05@20240602191714+0000.mp4'
2024-06-02 19:17:24.039019459  [2024-06-02 19:17:24] ffmpeg.sc06.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.039021695  [2024-06-02 19:17:24] ffmpeg.sc05.record             ERROR   : Error writing trailer of /tmp/cache/sc05@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.039063222  [2024-06-02 19:17:24] ffmpeg.sc06.record             ERROR   : [segment @ 0x555dfe75c680] Failure occurred when ending segment '/tmp/cache/sc06@20240602191714+0000.mp4'
2024-06-02 19:17:24.039065825  [2024-06-02 19:17:24] watchdog.sc05                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.039067007  [2024-06-02 19:17:24] watchdog.sc05                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.039082809  [2024-06-02 19:17:24] ffmpeg.sc06.record             ERROR   : Error writing trailer of /tmp/cache/sc06@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.039084134  [2024-06-02 19:17:24] watchdog.sc06                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.039085427  [2024-06-02 19:17:24] watchdog.sc06                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.070675717  [2024-06-02 19:17:24] ffmpeg.sc08.record             ERROR   : [segment @ 0x5591b8566d40] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.070791990  [2024-06-02 19:17:24] ffmpeg.sc08.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.071169634  [2024-06-02 19:17:24] ffmpeg.sc08.record             ERROR   : [segment @ 0x5591b8566d40] Failure occurred when ending segment '/tmp/cache/sc08@20240602191714+0000.mp4'
2024-06-02 19:17:24.071174625  [2024-06-02 19:17:24] ffmpeg.sc08.record             ERROR   : Error writing trailer of /tmp/cache/sc08@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.071176595  [2024-06-02 19:17:24] watchdog.sc08                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.071178463  [2024-06-02 19:17:24] watchdog.sc08                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.163009404  [2024-06-02 19:17:24] ffmpeg.sc01.record             ERROR   : [segment @ 0x564b38eadfc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.163104455  [2024-06-02 19:17:24] ffmpeg.sc01.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.163200930  [2024-06-02 19:17:24] ffmpeg.sc01.record             ERROR   : [segment @ 0x564b38eadfc0] Failure occurred when ending segment '/tmp/cache/sc01@20240602191714+0000.mp4'
2024-06-02 19:17:24.163264482  [2024-06-02 19:17:24] ffmpeg.sc01.record             ERROR   : Error writing trailer of /tmp/cache/sc01@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.163330804  [2024-06-02 19:17:24] watchdog.sc01                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.163390307  [2024-06-02 19:17:24] watchdog.sc01                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.243464177  [2024-06-02 19:17:24] ffmpeg.sc07.record             ERROR   : [segment @ 0x56391b8d5040] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.243997385  [2024-06-02 19:17:24] ffmpeg.sc04.record             ERROR   : [segment @ 0x559d5022afc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.250884656  [2024-06-02 19:17:24] ffmpeg.sc07.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.251440088  [2024-06-02 19:17:24] ffmpeg.sc07.record             ERROR   : [segment @ 0x56391b8d5040] Failure occurred when ending segment '/tmp/cache/sc07@20240602191714+0000.mp4'
2024-06-02 19:17:24.251553312  [2024-06-02 19:17:24] ffmpeg.sc07.record             ERROR   : Error writing trailer of /tmp/cache/sc07@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.251619200  [2024-06-02 19:17:24] watchdog.sc07                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.251684003  [2024-06-02 19:17:24] watchdog.sc07                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.253133752  [2024-06-02 19:17:24] ffmpeg.sc04.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.253459669  [2024-06-02 19:17:24] ffmpeg.sc04.record             ERROR   : [segment @ 0x559d5022afc0] Failure occurred when ending segment '/tmp/cache/sc04@20240602191714+0000.mp4'
2024-06-02 19:17:24.253906618  [2024-06-02 19:17:24] ffmpeg.sc04.record             ERROR   : Error writing trailer of /tmp/cache/sc04@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.253910084  [2024-06-02 19:17:24] watchdog.sc04                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.253911730  [2024-06-02 19:17:24] watchdog.sc04                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:24.301678365  [2024-06-02 19:17:24] ffmpeg.sc03.record             ERROR   : [segment @ 0x560478fadfc0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-06-02 19:17:24.301991036  [2024-06-02 19:17:24] ffmpeg.sc03.record             ERROR   : av_interleaved_write_frame(): No space left on device
2024-06-02 19:17:24.301994711  [2024-06-02 19:17:24] ffmpeg.sc03.record             ERROR   : [segment @ 0x560478fadfc0] Failure occurred when ending segment '/tmp/cache/sc03@20240602191714+0000.mp4'
2024-06-02 19:17:24.301996699  [2024-06-02 19:17:24] ffmpeg.sc03.record             ERROR   : Error writing trailer of /tmp/cache/sc03@%Y%m%d%H%M%S%z.mp4: No space left on device
2024-06-02 19:17:24.302157196  [2024-06-02 19:17:24] watchdog.sc03                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:24.302160628  [2024-06-02 19:17:24] watchdog.sc03                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.038352215  [2024-06-02 19:17:34] ffmpeg.sc02.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.038449252  [2024-06-02 19:17:34] ffmpeg.sc02.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.038511910  [2024-06-02 19:17:34] ffmpeg.sc02.record             ERROR   : 
2024-06-02 19:17:34.038571397  [2024-06-02 19:17:34] watchdog.sc02                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.038629918  [2024-06-02 19:17:34] watchdog.sc02                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.059827751  [2024-06-02 19:17:34] ffmpeg.sc05.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.060033566  [2024-06-02 19:17:34] ffmpeg.sc05.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.060059766  [2024-06-02 19:17:34] ffmpeg.sc05.record             ERROR   : 
2024-06-02 19:17:34.060226458  [2024-06-02 19:17:34] watchdog.sc05                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.060230325  [2024-06-02 19:17:34] watchdog.sc05                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.060991902  [2024-06-02 19:17:34] ffmpeg.sc06.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.068708115  [2024-06-02 19:17:34] ffmpeg.sc06.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.068755605  [2024-06-02 19:17:34] ffmpeg.sc06.record             ERROR   : 
2024-06-02 19:17:34.068875310  [2024-06-02 19:17:34] watchdog.sc06                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.068877695  [2024-06-02 19:17:34] watchdog.sc06                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.091794915  [2024-06-02 19:17:34] ffmpeg.sc08.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.091880420  [2024-06-02 19:17:34] ffmpeg.sc08.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.091921169  [2024-06-02 19:17:34] ffmpeg.sc08.record             ERROR   : 
2024-06-02 19:17:34.092017850  [2024-06-02 19:17:34] watchdog.sc08                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.092020896  [2024-06-02 19:17:34] watchdog.sc08                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.181476514  [2024-06-02 19:17:34] ffmpeg.sc01.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.181728446  [2024-06-02 19:17:34] ffmpeg.sc01.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.181731259  [2024-06-02 19:17:34] ffmpeg.sc01.record             ERROR   : 
2024-06-02 19:17:34.181733011  [2024-06-02 19:17:34] watchdog.sc01                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.181807625  [2024-06-02 19:17:34] watchdog.sc01                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.262558847  [2024-06-02 19:17:34] ffmpeg.sc07.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.262660298  [2024-06-02 19:17:34] ffmpeg.sc07.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.262715574  [2024-06-02 19:17:34] ffmpeg.sc07.record             ERROR   : 
2024-06-02 19:17:34.262749314  [2024-06-02 19:17:34] watchdog.sc07                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.262786911  [2024-06-02 19:17:34] watchdog.sc07                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.266148166  [2024-06-02 19:17:34] ffmpeg.sc04.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.266377850  [2024-06-02 19:17:34] ffmpeg.sc04.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.266560708  [2024-06-02 19:17:34] ffmpeg.sc04.record             ERROR   : 
2024-06-02 19:17:34.266563865  [2024-06-02 19:17:34] watchdog.sc04                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.266565076  [2024-06-02 19:17:34] watchdog.sc04                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:34.322619483  [2024-06-02 19:17:34] ffmpeg.sc03.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:34.322730452  [2024-06-02 19:17:34] ffmpeg.sc03.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:34.322886969  [2024-06-02 19:17:34] ffmpeg.sc03.record             ERROR   : 
2024-06-02 19:17:34.322989163  [2024-06-02 19:17:34] watchdog.sc03                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:34.323071990  [2024-06-02 19:17:34] watchdog.sc03                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.059405689  [2024-06-02 19:17:44] ffmpeg.sc02.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.070091133  [2024-06-02 19:17:44] ffmpeg.sc02.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.070095490  [2024-06-02 19:17:44] ffmpeg.sc02.record             ERROR   : 
2024-06-02 19:17:44.070097258  [2024-06-02 19:17:44] watchdog.sc02                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.070118011  [2024-06-02 19:17:44] watchdog.sc02                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.090325516  [2024-06-02 19:17:44] ffmpeg.sc06.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.099824394  [2024-06-02 19:17:44] ffmpeg.sc05.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.100051659  [2024-06-02 19:17:44] ffmpeg.sc06.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.100054484  [2024-06-02 19:17:44] ffmpeg.sc05.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.100142469  [2024-06-02 19:17:44] ffmpeg.sc05.record             ERROR   : 
2024-06-02 19:17:44.100244788  [2024-06-02 19:17:44] watchdog.sc05                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.100317520  [2024-06-02 19:17:44] watchdog.sc05                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.100445335  [2024-06-02 19:17:44] ffmpeg.sc06.record             ERROR   : 
2024-06-02 19:17:44.100571385  [2024-06-02 19:17:44] watchdog.sc06                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.100575054  [2024-06-02 19:17:44] watchdog.sc06                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.128992826  [2024-06-02 19:17:44] ffmpeg.sc08.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.128999393  [2024-06-02 19:17:44] ffmpeg.sc08.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.129001408  [2024-06-02 19:17:44] ffmpeg.sc08.record             ERROR   : 
2024-06-02 19:17:44.129003273  [2024-06-02 19:17:44] watchdog.sc08                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.129005113  [2024-06-02 19:17:44] watchdog.sc08                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.198813504  [2024-06-02 19:17:44] ffmpeg.sc01.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.198904493  [2024-06-02 19:17:44] ffmpeg.sc01.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.198971742  [2024-06-02 19:17:44] ffmpeg.sc01.record             ERROR   : 
2024-06-02 19:17:44.199032866  [2024-06-02 19:17:44] watchdog.sc01                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.199090184  [2024-06-02 19:17:44] watchdog.sc01                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.283054420  [2024-06-02 19:17:44] ffmpeg.sc07.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.283204845  [2024-06-02 19:17:44] ffmpeg.sc07.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.283285789  [2024-06-02 19:17:44] ffmpeg.sc07.record             ERROR   : 
2024-06-02 19:17:44.283288115  [2024-06-02 19:17:44] watchdog.sc07                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.283324402  [2024-06-02 19:17:44] watchdog.sc07                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.288575320  [2024-06-02 19:17:44] ffmpeg.sc04.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.288584201  [2024-06-02 19:17:44] ffmpeg.sc04.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.297949253  [2024-06-02 19:17:44] ffmpeg.sc04.record             ERROR   : 
2024-06-02 19:17:44.297954666  [2024-06-02 19:17:44] watchdog.sc04                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.297955760  [2024-06-02 19:17:44] watchdog.sc04                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-06-02 19:17:44.340259342  [2024-06-02 19:17:44] ffmpeg.sc03.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
2024-06-02 19:17:44.340263986  [2024-06-02 19:17:44] ffmpeg.sc03.record             ERROR   : Error initializing output stream 0:0 -- 
2024-06-02 19:17:44.340265237  [2024-06-02 19:17:44] ffmpeg.sc03.record             ERROR   : 
2024-06-02 19:17:44.340266585  [2024-06-02 19:17:44] watchdog.sc03                  INFO    : Terminating the existing ffmpeg process...
2024-06-02 19:17:44.340267755  [2024-06-02 19:17:44] watchdog.sc03                  INFO    : Waiting for ffmpeg to exit gracefully...

NickM-27 Jun 2, 2024
Collaborator Sponsor

The problem is you have sync_recordings: True this should not be kept on continuously. It is checking for all of your files and that is causing an I/O block for the recording process

submac Jun 3, 2024
Author

Yes but I added that after the problem started occurring, just to see if it changes anything. It actually helped a bit but let me set it back to False and report back.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Config Support]: /tmp/cache fills up intermittently #11402

{{title}}

Replies: 1 comment 8 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

[Config Support]: /tmp/cache fills up intermittently #11402

submac May 16, 2024

Describe the problem you are having

Version

Frigate config file

Relevant log output

Frigate stats

Operating system

Install method

Coral version

Any other information that may be helpful

Replies: 1 comment · 8 replies

NickM-27 May 16, 2024 Collaborator Sponsor

submac May 22, 2024 Author

NickM-27 May 22, 2024 Collaborator Sponsor

submac Jun 2, 2024 Author

NickM-27 Jun 2, 2024 Collaborator Sponsor

submac Jun 3, 2024 Author

submac
May 16, 2024

Replies: 1 comment 8 replies

NickM-27
May 16, 2024
Collaborator Sponsor

submac May 22, 2024
Author

NickM-27 May 22, 2024
Collaborator Sponsor

submac Jun 2, 2024
Author

NickM-27 Jun 2, 2024
Collaborator Sponsor

submac Jun 3, 2024
Author