[Feature] "fork" start method for mutli-collectors

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2024

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 13 New Failures

As of commit dbfbf2b with merge base 4bce371 ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 74dccdd92574e01e65bb8572ab090d25c8f45564597b8dbf10e7e8aced30ccf9 /exec failed with exit code 139
• Lint / python-source-and-configs / linux-job (gh)
  torchrl/collectors/collectors.py:40:1: F401 'torchrl._utils._ProcessNoWarn' imported but unused
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_parallel_env[True-False-False]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	54.2871ms	54.0431ms	18.5038 Ops/s	17.3849 Ops/s	$\textbf{\color{#35bf28}+6.44\%}$
test_sync	29.8342ms	29.4905ms	33.9093 Ops/s	33.6748 Ops/s	$\color{#35bf28}+0.70\%$
test_async	51.6210ms	27.2807ms	36.6559 Ops/s	35.9471 Ops/s	$\color{#35bf28}+1.97\%$
test_simple	0.3865s	0.3377s	2.9614 Ops/s	2.9098 Ops/s	$\color{#35bf28}+1.77\%$
test_transformed	0.5099s	0.4685s	2.1347 Ops/s	2.1605 Ops/s	$\color{#d91a1a}-1.19\%$
test_serial	1.2270s	1.1899s	0.8404 Ops/s	0.8253 Ops/s	$\color{#35bf28}+1.83\%$
test_parallel	1.0825s	1.0499s	0.9525 Ops/s	0.9521 Ops/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-True-True-True-True]	0.1404ms	20.9731μs	47.6800 KOps/s	46.9945 KOps/s	$\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-True-True-True-False]	52.0480μs	12.7805μs	78.2440 KOps/s	77.2475 KOps/s	$\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-True-True-False-True]	43.6820μs	12.2038μs	81.9416 KOps/s	80.7522 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-True-True-False-False]	34.1040μs	7.3081μs	136.8338 KOps/s	134.4498 KOps/s	$\color{#35bf28}+1.77\%$
test_step_mdp_speed[True-True-False-True-True]	90.7300μs	22.2368μs	44.9706 KOps/s	44.0352 KOps/s	$\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-True-False-True-False]	54.9320μs	13.8372μs	72.2689 KOps/s	69.6319 KOps/s	$\color{#35bf28}+3.79\%$
test_step_mdp_speed[True-True-False-False-True]	39.2430μs	13.3977μs	74.6398 KOps/s	73.0124 KOps/s	$\color{#35bf28}+2.23\%$
test_step_mdp_speed[True-True-False-False-False]	35.4760μs	8.5069μs	117.5512 KOps/s	113.9689 KOps/s	$\color{#35bf28}+3.14\%$
test_step_mdp_speed[True-False-True-True-True]	96.4010μs	23.3889μs	42.7552 KOps/s	41.9420 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-False-True-True-False]	51.2760μs	15.0945μs	66.2492 KOps/s	63.7416 KOps/s	$\color{#35bf28}+3.93\%$
test_step_mdp_speed[True-False-True-False-True]	41.0770μs	13.4020μs	74.6159 KOps/s	73.6936 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-True-False-False]	32.3110μs	8.5000μs	117.6467 KOps/s	113.6314 KOps/s	$\color{#35bf28}+3.53\%$
test_step_mdp_speed[True-False-False-True-True]	51.9470μs	24.6147μs	40.6262 KOps/s	40.1665 KOps/s	$\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-False-True-False]	58.7500μs	16.1707μs	61.8402 KOps/s	59.8413 KOps/s	$\color{#35bf28}+3.34\%$
test_step_mdp_speed[True-False-False-False-True]	43.0300μs	14.5043μs	68.9449 KOps/s	67.3405 KOps/s	$\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-False-False-False-False]	32.4310μs	9.6160μs	103.9934 KOps/s	100.2295 KOps/s	$\color{#35bf28}+3.76\%$
test_step_mdp_speed[False-True-True-True-True]	51.1150μs	23.5328μs	42.4939 KOps/s	41.4055 KOps/s	$\color{#35bf28}+2.63\%$
test_step_mdp_speed[False-True-True-True-False]	37.7410μs	15.1326μs	66.0826 KOps/s	64.1335 KOps/s	$\color{#35bf28}+3.04\%$
test_step_mdp_speed[False-True-True-False-True]	80.1700μs	15.6607μs	63.8542 KOps/s	61.9783 KOps/s	$\color{#35bf28}+3.03\%$
test_step_mdp_speed[False-True-True-False-False]	35.8370μs	9.7182μs	102.8994 KOps/s	99.6656 KOps/s	$\color{#35bf28}+3.24\%$
test_step_mdp_speed[False-True-False-True-True]	35.4570μs	24.6628μs	40.5469 KOps/s	39.2103 KOps/s	$\color{#35bf28}+3.41\%$
test_step_mdp_speed[False-True-False-True-False]	50.2940μs	16.2939μs	61.3725 KOps/s	58.8615 KOps/s	$\color{#35bf28}+4.27\%$
test_step_mdp_speed[False-True-False-False-True]	48.2510μs	16.6478μs	60.0680 KOps/s	58.7680 KOps/s	$\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-True-False-False-False]	36.7390μs	10.8105μs	92.5027 KOps/s	89.5333 KOps/s	$\color{#35bf28}+3.32\%$
test_step_mdp_speed[False-False-True-True-True]	66.3940μs	25.6571μs	38.9756 KOps/s	38.0081 KOps/s	$\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-False-True-True-False]	51.0960μs	17.4729μs	57.2314 KOps/s	55.6140 KOps/s	$\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-False-True-False-True]	41.5380μs	16.5176μs	60.5415 KOps/s	58.1251 KOps/s	$\color{#35bf28}+4.16\%$
test_step_mdp_speed[False-False-True-False-False]	32.5910μs	10.8348μs	92.2953 KOps/s	88.9341 KOps/s	$\color{#35bf28}+3.78\%$
test_step_mdp_speed[False-False-False-True-True]	57.1370μs	26.5944μs	37.6019 KOps/s	36.2431 KOps/s	$\color{#35bf28}+3.75\%$
test_step_mdp_speed[False-False-False-True-False]	41.8880μs	18.5289μs	53.9698 KOps/s	52.3023 KOps/s	$\color{#35bf28}+3.19\%$
test_step_mdp_speed[False-False-False-False-True]	70.4120μs	17.5525μs	56.9720 KOps/s	54.6874 KOps/s	$\color{#35bf28}+4.18\%$
test_step_mdp_speed[False-False-False-False-False]	41.2870μs	11.8624μs	84.3000 KOps/s	80.7410 KOps/s	$\color{#35bf28}+4.41\%$
test_values[generalized_advantage_estimate-True-True]	9.7670ms	9.4440ms	105.8872 Ops/s	106.6291 Ops/s	$\color{#d91a1a}-0.70\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.3735ms	34.9751ms	28.5918 Ops/s	30.1779 Ops/s	$\textbf{\color{#d91a1a}-5.26\%}$
test_values[td0_return_estimate-False-False]	0.2080ms	0.1647ms	6.0717 KOps/s	6.1158 KOps/s	$\color{#d91a1a}-0.72\%$
test_values[td1_return_estimate-False-False]	23.7583ms	23.4607ms	42.6244 Ops/s	42.7404 Ops/s	$\color{#d91a1a}-0.27\%$
test_values[vec_td1_return_estimate-False-False]	37.6747ms	34.9821ms	28.5860 Ops/s	30.0533 Ops/s	$\color{#d91a1a}-4.88\%$
test_values[td_lambda_return_estimate-True-False]	34.2067ms	33.7877ms	29.5966 Ops/s	29.7624 Ops/s	$\color{#d91a1a}-0.56\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.4543ms	35.1333ms	28.4630 Ops/s	30.2693 Ops/s	$\textbf{\color{#d91a1a}-5.97\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	11.2502ms	8.3322ms	120.0165 Ops/s	122.5160 Ops/s	$\color{#d91a1a}-2.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.8774ms	2.0714ms	482.7629 Ops/s	503.1707 Ops/s	$\color{#d91a1a}-4.06\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4836ms	0.3525ms	2.8365 KOps/s	2.9167 KOps/s	$\color{#d91a1a}-2.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	48.9582ms	46.9723ms	21.2892 Ops/s	25.2964 Ops/s	$\textbf{\color{#d91a1a}-15.84\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.8105ms	3.1344ms	319.0408 Ops/s	332.0547 Ops/s	$\color{#d91a1a}-3.92\%$
test_dqn_speed	1.9118ms	1.3315ms	751.0562 Ops/s	741.4665 Ops/s	$\color{#35bf28}+1.29\%$
test_ddpg_speed	3.0242ms	2.6384ms	379.0148 Ops/s	372.5697 Ops/s	$\color{#35bf28}+1.73\%$
test_sac_speed	8.4495ms	8.1195ms	123.1597 Ops/s	122.4417 Ops/s	$\color{#35bf28}+0.59\%$
test_redq_speed	14.6560ms	12.9421ms	77.2671 Ops/s	75.4862 Ops/s	$\color{#35bf28}+2.36\%$
test_redq_deprec_speed	14.7084ms	12.9090ms	77.4656 Ops/s	76.2111 Ops/s	$\color{#35bf28}+1.65\%$
test_td3_speed	8.2923ms	8.0524ms	124.1861 Ops/s	122.2868 Ops/s	$\color{#35bf28}+1.55\%$
test_cql_speed	0.1059s	38.4014ms	26.0407 Ops/s	26.9781 Ops/s	$\color{#d91a1a}-3.47\%$
test_a2c_speed	8.9740ms	7.4016ms	135.1068 Ops/s	128.6351 Ops/s	$\textbf{\color{#35bf28}+5.03\%}$
test_ppo_speed	8.6273ms	7.6082ms	131.4374 Ops/s	127.9711 Ops/s	$\color{#35bf28}+2.71\%$
test_reinforce_speed	7.3307ms	6.5102ms	153.6046 Ops/s	148.9392 Ops/s	$\color{#35bf28}+3.13\%$
test_iql_speed	33.2124ms	32.2200ms	31.0366 Ops/s	30.0900 Ops/s	$\color{#35bf28}+3.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.2746ms	2.1222ms	471.2086 Ops/s	464.8577 Ops/s	$\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0703ms	0.4916ms	2.0343 KOps/s	2.0165 KOps/s	$\color{#35bf28}+0.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7364ms	0.4729ms	2.1144 KOps/s	2.0975 KOps/s	$\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.7980ms	2.1768ms	459.3993 Ops/s	467.7150 Ops/s	$\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7813ms	0.4875ms	2.0513 KOps/s	2.0565 KOps/s	$\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.4286ms	0.4645ms	2.1529 KOps/s	2.1547 KOps/s	$\color{#d91a1a}-0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.7760ms	1.2912ms	774.5022 Ops/s	781.3008 Ops/s	$\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	4.5746ms	1.2320ms	811.6946 Ops/s	827.1845 Ops/s	$\color{#d91a1a}-1.87\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.4548ms	2.2818ms	438.2528 Ops/s	448.4233 Ops/s	$\color{#d91a1a}-2.27\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8289ms	0.6092ms	1.6416 KOps/s	1.6458 KOps/s	$\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	90.8132ms	0.6520ms	1.5338 KOps/s	1.7135 KOps/s	$\textbf{\color{#d91a1a}-10.49\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.8746ms	2.1550ms	464.0323 Ops/s	439.3049 Ops/s	$\textbf{\color{#35bf28}+5.63\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5971ms	0.4910ms	2.0368 KOps/s	1.9936 KOps/s	$\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6963ms	0.4709ms	2.1237 KOps/s	2.0348 KOps/s	$\color{#35bf28}+4.37\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.4650ms	2.1326ms	468.9097 Ops/s	451.9874 Ops/s	$\color{#35bf28}+3.74\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	91.4082ms	0.6077ms	1.6455 KOps/s	2.0455 KOps/s	$\textbf{\color{#d91a1a}-19.55\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5983ms	0.4624ms	2.1626 KOps/s	2.1391 KOps/s	$\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.4407ms	2.2758ms	439.4043 Ops/s	436.6678 Ops/s	$\color{#35bf28}+0.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8186ms	0.6108ms	1.6372 KOps/s	1.6425 KOps/s	$\color{#d91a1a}-0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.8536ms	0.5886ms	1.6989 KOps/s	1.7141 KOps/s	$\color{#d91a1a}-0.89\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	88.1152ms	6.9105ms	144.7074 Ops/s	138.5479 Ops/s	$\color{#35bf28}+4.45\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	14.8743ms	12.0265ms	83.1497 Ops/s	83.4823 Ops/s	$\color{#d91a1a}-0.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.6668ms	1.0334ms	967.7049 Ops/s	943.7081 Ops/s	$\color{#35bf28}+2.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	87.2916ms	6.9177ms	144.5561 Ops/s	145.7229 Ops/s	$\color{#d91a1a}-0.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.8356ms	13.5305ms	73.9068 Ops/s	83.5126 Ops/s	$\textbf{\color{#d91a1a}-11.50\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.7461ms	1.0307ms	970.1833 Ops/s	960.5441 Ops/s	$\color{#35bf28}+1.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	86.7413ms	5.6543ms	176.8570 Ops/s	179.9119 Ops/s	$\color{#d91a1a}-1.70\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	92.4519ms	13.9199ms	71.8395 Ops/s	80.8160 Ops/s	$\textbf{\color{#d91a1a}-11.11\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.7311ms	1.3448ms	743.6208 Ops/s	733.9345 Ops/s	$\color{#35bf28}+1.32\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1011s	0.1005s	9.9518 Ops/s	9.4268 Ops/s	$\textbf{\color{#35bf28}+5.57\%}$
test_sync	91.2707ms	88.2453ms	11.3320 Ops/s	11.5001 Ops/s	$\color{#d91a1a}-1.46\%$
test_async	0.1759s	88.3969ms	11.3126 Ops/s	11.2342 Ops/s	$\color{#35bf28}+0.70\%$
test_single_pixels	0.1113s	0.1109s	9.0204 Ops/s	9.0006 Ops/s	$\color{#35bf28}+0.22\%$
test_sync_pixels	67.4512ms	66.4798ms	15.0422 Ops/s	15.1193 Ops/s	$\color{#d91a1a}-0.51\%$
test_async_pixels	0.1235s	55.8890ms	17.8926 Ops/s	17.6643 Ops/s	$\color{#35bf28}+1.29\%$
test_simple	0.6515s	0.6494s	1.5399 Ops/s	1.4885 Ops/s	$\color{#35bf28}+3.46\%$
test_transformed	0.8507s	0.8492s	1.1775 Ops/s	1.1314 Ops/s	$\color{#35bf28}+4.08\%$
test_serial	2.1114s	2.0481s	0.4882 Ops/s	0.4778 Ops/s	$\color{#35bf28}+2.18\%$
test_parallel	1.8473s	1.8031s	0.5546 Ops/s	0.5573 Ops/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-True-True-True]	96.4740μs	33.6200μs	29.7442 KOps/s	30.8312 KOps/s	$\color{#d91a1a}-3.53\%$
test_step_mdp_speed[True-True-True-True-False]	43.0320μs	20.1445μs	49.6413 KOps/s	50.9469 KOps/s	$\color{#d91a1a}-2.56\%$
test_step_mdp_speed[True-True-True-False-True]	33.8210μs	19.2536μs	51.9383 KOps/s	54.2376 KOps/s	$\color{#d91a1a}-4.24\%$
test_step_mdp_speed[True-True-True-False-False]	28.1510μs	11.4460μs	87.3670 KOps/s	89.1586 KOps/s	$\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-True-False-True-True]	58.3320μs	35.6403μs	28.0582 KOps/s	28.5411 KOps/s	$\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-True-False-True-False]	45.8820μs	21.9550μs	45.5477 KOps/s	46.4255 KOps/s	$\color{#d91a1a}-1.89\%$
test_step_mdp_speed[True-True-False-False-True]	36.3910μs	21.1761μs	47.2231 KOps/s	49.0012 KOps/s	$\color{#d91a1a}-3.63\%$
test_step_mdp_speed[True-True-False-False-False]	27.1010μs	13.2947μs	75.2178 KOps/s	75.7809 KOps/s	$\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-True-True]	56.0530μs	36.9427μs	27.0689 KOps/s	27.5930 KOps/s	$\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-False-True-True-False]	47.4420μs	24.2096μs	41.3059 KOps/s	42.4588 KOps/s	$\color{#d91a1a}-2.72\%$
test_step_mdp_speed[True-False-True-False-True]	46.3320μs	21.1153μs	47.3591 KOps/s	48.9579 KOps/s	$\color{#d91a1a}-3.27\%$
test_step_mdp_speed[True-False-True-False-False]	29.0220μs	13.6713μs	73.1460 KOps/s	76.8377 KOps/s	$\color{#d91a1a}-4.80\%$
test_step_mdp_speed[True-False-False-True-True]	56.4630μs	39.0528μs	25.6064 KOps/s	26.4298 KOps/s	$\color{#d91a1a}-3.12\%$
test_step_mdp_speed[True-False-False-True-False]	40.6710μs	25.6710μs	38.9545 KOps/s	40.4611 KOps/s	$\color{#d91a1a}-3.72\%$
test_step_mdp_speed[True-False-False-False-True]	43.5720μs	22.4374μs	44.5684 KOps/s	45.2835 KOps/s	$\color{#d91a1a}-1.58\%$
test_step_mdp_speed[True-False-False-False-False]	39.5820μs	15.1358μs	66.0685 KOps/s	67.7971 KOps/s	$\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-True-True-True-True]	55.1520μs	37.1365μs	26.9277 KOps/s	27.7442 KOps/s	$\color{#d91a1a}-2.94\%$
test_step_mdp_speed[False-True-True-True-False]	41.5520μs	23.3528μs	42.8214 KOps/s	42.9880 KOps/s	$\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-True-True-False-True]	40.3010μs	24.4036μs	40.9776 KOps/s	41.7600 KOps/s	$\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-True-True-False-False]	41.6810μs	14.9801μs	66.7553 KOps/s	67.0872 KOps/s	$\color{#d91a1a}-0.49\%$
test_step_mdp_speed[False-True-False-True-True]	62.7520μs	39.6808μs	25.2011 KOps/s	26.3343 KOps/s	$\color{#d91a1a}-4.30\%$
test_step_mdp_speed[False-True-False-True-False]	52.2820μs	25.7422μs	38.8467 KOps/s	40.1527 KOps/s	$\color{#d91a1a}-3.25\%$
test_step_mdp_speed[False-True-False-False-True]	51.3220μs	26.3357μs	37.9713 KOps/s	38.5418 KOps/s	$\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-True-False-False-False]	36.2320μs	17.3882μs	57.5102 KOps/s	60.0250 KOps/s	$\color{#d91a1a}-4.19\%$
test_step_mdp_speed[False-False-True-True-True]	65.9430μs	41.5196μs	24.0850 KOps/s	25.2260 KOps/s	$\color{#d91a1a}-4.52\%$
test_step_mdp_speed[False-False-True-True-False]	45.3020μs	27.8239μs	35.9403 KOps/s	36.8853 KOps/s	$\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-False-True-False-True]	48.9320μs	26.0957μs	38.3204 KOps/s	38.4037 KOps/s	$\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-True-False-False]	42.0310μs	17.2224μs	58.0639 KOps/s	60.0381 KOps/s	$\color{#d91a1a}-3.29\%$
test_step_mdp_speed[False-False-False-True-True]	65.9230μs	41.6454μs	24.0123 KOps/s	23.9906 KOps/s	$\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-False-False-True-False]	45.8520μs	28.6011μs	34.9637 KOps/s	34.8313 KOps/s	$\color{#35bf28}+0.38\%$
test_step_mdp_speed[False-False-False-False-True]	75.1530μs	27.3101μs	36.6165 KOps/s	36.4014 KOps/s	$\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-False-False-False-False]	39.4210μs	18.6417μs	53.6432 KOps/s	54.4570 KOps/s	$\color{#d91a1a}-1.49\%$
test_values[generalized_advantage_estimate-True-True]	25.7653ms	25.1624ms	39.7418 Ops/s	41.2570 Ops/s	$\color{#d91a1a}-3.67\%$
test_values[vec_generalized_advantage_estimate-True-True]	79.9698ms	3.1552ms	316.9403 Ops/s	313.5795 Ops/s	$\color{#35bf28}+1.07\%$
test_values[td0_return_estimate-False-False]	97.6440μs	67.3299μs	14.8522 KOps/s	15.6251 KOps/s	$\color{#d91a1a}-4.95\%$
test_values[td1_return_estimate-False-False]	54.9314ms	53.7123ms	18.6177 Ops/s	19.0819 Ops/s	$\color{#d91a1a}-2.43\%$
test_values[vec_td1_return_estimate-False-False]	2.0753ms	1.7664ms	566.1246 Ops/s	571.3228 Ops/s	$\color{#d91a1a}-0.91\%$
test_values[td_lambda_return_estimate-True-False]	87.2170ms	85.3652ms	11.7144 Ops/s	11.9500 Ops/s	$\color{#d91a1a}-1.97\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.1115ms	1.7513ms	570.9953 Ops/s	571.7244 Ops/s	$\color{#d91a1a}-0.13\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.4180ms	23.6987ms	42.1965 Ops/s	42.4167 Ops/s	$\color{#d91a1a}-0.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8745ms	0.6930ms	1.4431 KOps/s	1.4448 KOps/s	$\color{#d91a1a}-0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7070ms	0.6515ms	1.5348 KOps/s	1.5659 KOps/s	$\color{#d91a1a}-1.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5022ms	1.4506ms	689.3813 Ops/s	692.2464 Ops/s	$\color{#d91a1a}-0.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9600ms	0.6587ms	1.5182 KOps/s	1.5047 KOps/s	$\color{#35bf28}+0.89\%$
test_dqn_speed	8.2562ms	1.4369ms	695.9187 Ops/s	676.8008 Ops/s	$\color{#35bf28}+2.82\%$
test_ddpg_speed	2.9647ms	2.6860ms	372.3037 Ops/s	357.7761 Ops/s	$\color{#35bf28}+4.06\%$
test_sac_speed	8.3743ms	7.9211ms	126.2443 Ops/s	122.1736 Ops/s	$\color{#35bf28}+3.33\%$
test_redq_speed	11.2463ms	10.3042ms	97.0481 Ops/s	94.3980 Ops/s	$\color{#35bf28}+2.81\%$
test_redq_deprec_speed	11.5116ms	10.9699ms	91.1584 Ops/s	87.5250 Ops/s	$\color{#35bf28}+4.15\%$
test_td3_speed	8.0718ms	7.8587ms	127.2471 Ops/s	122.7112 Ops/s	$\color{#35bf28}+3.70\%$
test_cql_speed	25.5976ms	24.8281ms	40.2770 Ops/s	38.5172 Ops/s	$\color{#35bf28}+4.57\%$
test_a2c_speed	5.7564ms	5.5051ms	181.6505 Ops/s	173.3668 Ops/s	$\color{#35bf28}+4.78\%$
test_ppo_speed	7.5775ms	5.9029ms	169.4082 Ops/s	162.9988 Ops/s	$\color{#35bf28}+3.93\%$
test_reinforce_speed	4.7161ms	4.4651ms	223.9594 Ops/s	212.7428 Ops/s	$\textbf{\color{#35bf28}+5.27\%}$
test_iql_speed	20.1241ms	19.4013ms	51.5431 Ops/s	49.3363 Ops/s	$\color{#35bf28}+4.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0067ms	2.9097ms	343.6826 Ops/s	344.4284 Ops/s	$\color{#d91a1a}-0.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1359ms	0.5489ms	1.8218 KOps/s	1.5991 KOps/s	$\textbf{\color{#35bf28}+13.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7151ms	0.5255ms	1.9029 KOps/s	1.8845 KOps/s	$\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0358ms	2.8810ms	347.0966 Ops/s	341.5535 Ops/s	$\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2487ms	0.5387ms	1.8563 KOps/s	1.8124 KOps/s	$\color{#35bf28}+2.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7206ms	0.5188ms	1.9276 KOps/s	1.9078 KOps/s	$\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	3.9555ms	1.5256ms	655.4809 Ops/s	640.7736 Ops/s	$\color{#35bf28}+2.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6519ms	1.4653ms	682.4329 Ops/s	677.6706 Ops/s	$\color{#35bf28}+0.70\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.2755ms	3.0188ms	331.2605 Ops/s	329.5279 Ops/s	$\color{#35bf28}+0.53\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8640ms	0.6712ms	1.4899 KOps/s	1.4676 KOps/s	$\color{#35bf28}+1.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.3537ms	0.6574ms	1.5211 KOps/s	1.5148 KOps/s	$\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.9980ms	2.8689ms	348.5608 Ops/s	340.9461 Ops/s	$\color{#35bf28}+2.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7266ms	0.5493ms	1.8205 KOps/s	1.8032 KOps/s	$\color{#35bf28}+0.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.2660ms	0.5320ms	1.8798 KOps/s	1.8986 KOps/s	$\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0656ms	2.9186ms	342.6340 Ops/s	339.5547 Ops/s	$\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3450ms	0.5456ms	1.8328 KOps/s	1.8290 KOps/s	$\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6943ms	0.5222ms	1.9151 KOps/s	1.8987 KOps/s	$\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.1369ms	3.0185ms	331.2862 Ops/s	329.0969 Ops/s	$\color{#35bf28}+0.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7754ms	0.6796ms	1.4715 KOps/s	1.4710 KOps/s	$\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.1073s	0.7898ms	1.2661 KOps/s	1.5149 KOps/s	$\textbf{\color{#d91a1a}-16.42\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1024s	6.7736ms	147.6321 Ops/s	108.7574 Ops/s	$\textbf{\color{#35bf28}+35.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	16.6977ms	14.5490ms	68.7334 Ops/s	67.0840 Ops/s	$\color{#35bf28}+2.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.0490ms	1.0767ms	928.7430 Ops/s	923.0038 Ops/s	$\color{#35bf28}+0.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1038s	8.6740ms	115.2864 Ops/s	144.7352 Ops/s	$\textbf{\color{#d91a1a}-20.35\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.8384ms	14.6815ms	68.1129 Ops/s	59.2094 Ops/s	$\textbf{\color{#35bf28}+15.04\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.1683ms	1.0817ms	924.4898 Ops/s	823.0024 Ops/s	$\textbf{\color{#35bf28}+12.33\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1053s	7.2525ms	137.8826 Ops/s	137.8913 Ops/s	$-0.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.0911ms	14.8965ms	67.1299 Ops/s	65.5654 Ops/s	$\color{#35bf28}+2.39\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.5594ms	1.4411ms	693.9110 Ops/s	656.8176 Ops/s	$\textbf{\color{#35bf28}+5.65\%}$