Default TreadPool size to number of physical cores

[malfet]

TODO: Some benchmarks

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125963

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 230082a with merge base 538877d ():

NEW FAILURE - The following job has failed:

• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  test_cuda.py::TestCuda::test_set_per_process_memory_fraction

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh) (trunk failure)
  test_cuda_multigpu.py::TestCudaMultiGPU::test_memory_stats

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[janeyx99]

I do believe this change is for the better but I'm not an expert and so I cannot ascertain that is always better. @malfet what's the plan on the benchmarks haha

[facebook-github-bot]

@janeyx99 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[Skylion007]

Will this work properly for partial CPU allocations on SLURM clusters?

[msaroufim]

This might be a noop concern but was also curious how a change like this affects the performance of distributed jobs

[malfet]

Will this work properly for partial CPU allocations on SLURM clusters?

I can only hope for it :)

[Skylion007]

I'm just worried becaues there might be say 16 logical cores, and only 4 are available to the job meaning that the cores become over-subscribed.

[janeyx99]

@Skylion007 that is the exact type of problem we're attempting to fix, as processors usually means threads and cores means actual cores (what we're thinking). @malfet's PR will be strictly an improvement on that front as written compared to before the change.

[gajjanag]

LGTM - tested both a hyperthreaded multicore and non hyperthreaded multicore. It gives the correct thread count now.

[sanchitintel]

tested both a hyperthreaded multicore and non hyperthreaded multicore. It gives the correct thread count now.

Hi @gajjanag, can you please clarify if you meant you were getting incorrect counts earlier with torch.get_num_threads?
For instance, even with HyperThreading enabled on an Intel machine, I only see physical core count with torch.get_num_threads at my end (without this patch).

Thanks!