[Feature Request] S6 should read number of available CPUs and update the worker_processes directive in nginx config before starting nginx

[NickM-27]

Users running frigate may configure docker to a limited number of CPU cores. Nginx worker_processes auto directive creates a number of processes equal to the number of CPU cores (ignoring the cgroup limit). If the difference between available cores and configured cores is too large, nginx will spawn too many processes.

When starting nginx, the s6 process should read the number of available CPU cores from cgroups and adjust the worker_processes directive accordingly

[felipecrs]

@NickM-27 your implementation is not cgroup aware:

❯ docker run --rm --cpus 2 ubuntu nproc --all
16

If you want to take into account the cgroups limits defined by docker, you'll need to change it a little bit. ChatGPT gave me this:

if [ -f /sys/fs/cgroup/cpu/cpu.cfs_quota_us ]; then 
  echo "Number of CPUs allocated: $(awk '{q=$1} END {getline <"/sys/fs/cgroup/cpu/cpu.cfs_period_us"; printf "%.2f", q/$1}' /sys/fs/cgroup/cpu/cpu.cfs_quota_us)"
elif [ -f /sys/fs/cgroup/cpu.max ]; then 
  echo "Number of CPUs allocated: $(awk -F' ' '{q=$1; p=$2} END {printf "%.2f", q/p}' /sys/fs/cgroup/cpu.max)"
else 
  echo "CPU quota information not found."
fi

The first case is for cgroupsv1, the second case is for cgroupsv2. It worked for me in the second case because my system is cgroupsv2 as of now. I did not test the cgroupsv1 though.

Number of CPUs allocated: 2.00

I remember one of the requirements was to make it honor the docker run --cpus flag.

[NickM-27]

I tested on my unraid system and it absolutely respected the --cpus flag, so not 100% sure what the difference is in your system

[felipecrs]

Weird. Let me test on my HAOS...

➜  /config docker run --rm --cpus 2 ubuntu nproc --all
4

What OS are you running docker in your unraid?

I know LXC behaves differently than docker. In this case, nproc --all would return correctly (because of lxcfs).

[NickM-27]

not sure I understand the question. docker runs directly on unraid which is the OS

[felipecrs]

Sorry, never played with unraid myself.

There must be some kind of customization/optimization from unraid side then.

For any other case, nproc --all is not cgroup aware and therefore will not respect docker run --cpus.

This is a very known quirk for applications that ships with docker support. The same applies to --memory as well, you cannot rely on output of free -m or parsing /proc.

Here is one example: oracle/docker-images#1963

[NickM-27]

Okay, I tried your examples above and both failed. The first one says the file does not exist and the second returns 0.0. Also tried on my macbook in docker and both failed. I am concerned that this won't be reliable.

The idea changes from the original solution a little bit as the number of processes allowed for nginx is limited to 4 which should solve the issue even in cases where a user has more CPUs and the number passed in to the container is not respected

[felipecrs]

Yeah. The PR definitely improves the situation. But for example, for people with 4 CPU cores and running Frigate with --cpus 2, unless on unraid perhaps, the nginx would be configured with 4 cores.

Here is another ref: moby/moby#42356

I was unable to make it work yet on a cgroupv1 system though. I will let you know once I figure out.

[felipecrs]

@NickM-27 I put it in a repo: https://github.com/felipecrs/cgroups-scripts

❯ cat get_cpus.sh | docker run --rm -i ubuntu bash
cgroup v2 detected.
No CPU limits set.
CPUs: 16

❯ cat get_cpus.sh | docker run --rm -i --cpus 2 ubuntu bash
cgroup v2 detected.
CPUs: 2

Another system with cgroup v1:

❯ cat get_cpus.sh | docker run --rm -i ubuntu bash
cgroup v1 detected.
No CPU limits set.
CPUs: 8

❯ cat get_cpus.sh | docker run --rm -i --cpus 2 ubuntu bash
cgroup v1 detected.
CPUs: 2

Please try it again if you don't mind. It should be as easy as:

curl -fsSL https://raw.githubusercontent.com/felipecrs/cgroups-scripts/master/get_cpus.sh | docker run --rm -i --cpus 2 ubuntu bash

[felipecrs]

@uhthomas since you were the original issue reporter, maybe you can cast your opinion here too. Please read the conversation above.