Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Make sure keys are exclusive in envs #1912

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

[BugFix] Make sure keys are exclusive in envs #1912

wants to merge 7 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 15, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 15, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1912

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 6848746 with merge base aebc6a2 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 15, 2024
@vmoens vmoens changed the title [Feature] Make sure keys are exclusive in. envs [BugFix] Make sure keys are exclusive in envs Feb 15, 2024
@vmoens vmoens added bug Something isn't working Suitable for minor Suitable to be integrated in minor release (no new feature) labels Feb 15, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 15, 2024 •
edited

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1165s 65.6997ms 15.2208 Ops/s 16.2664 Ops/s $\textbf{\color{#d91a1a}-6.43\%}$
test_sync 46.9739ms 38.4868ms 25.9829 Ops/s 29.8788 Ops/s $\textbf{\color{#d91a1a}-13.04\%}$
test_async 68.4339ms 31.6644ms 31.5812 Ops/s 30.9924 Ops/s $\color{#35bf28}+1.90\%$
test_simple 0.4305s 0.4270s 2.3420 Ops/s 2.2608 Ops/s $\color{#35bf28}+3.59\%$
test_transformed 0.6400s 0.5948s 1.6812 Ops/s 1.7128 Ops/s $\color{#d91a1a}-1.85\%$
test_serial 1.4950s 1.4442s 0.6924 Ops/s 0.6858 Ops/s $\color{#35bf28}+0.97\%$
test_parallel 1.4509s 1.3977s 0.7155 Ops/s 0.7018 Ops/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-True-True-True-True] 0.1441ms 21.0126μs 47.5905 KOps/s 47.0664 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[True-True-True-True-False] 65.3220μs 13.0348μs 76.7178 KOps/s 78.6228 KOps/s $\color{#d91a1a}-2.42\%$
test_step_mdp_speed[True-True-True-False-True] 33.2920μs 12.3282μs 81.1148 KOps/s 80.5780 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-True-True-False-False] 27.1320μs 7.5768μs 131.9811 KOps/s 134.4723 KOps/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-True-False-True-True] 56.4590μs 22.3380μs 44.7668 KOps/s 44.3837 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-False-True-False] 64.9470μs 14.2326μs 70.2614 KOps/s 70.8490 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-False-False-True] 36.7610μs 13.4179μs 74.5273 KOps/s 72.9250 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-True-False-False-False] 36.2600μs 8.8195μs 113.3856 KOps/s 114.4306 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-False-True-True-True] 57.7320μs 23.7742μs 42.0624 KOps/s 41.8060 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-True-True-False] 38.1330μs 15.6852μs 63.7544 KOps/s 64.4015 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[True-False-True-False-True] 47.6030μs 13.4446μs 74.3790 KOps/s 72.8829 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[True-False-True-False-False] 52.8110μs 8.6942μs 115.0186 KOps/s 114.5687 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-False-True-True] 57.7480μs 24.7151μs 40.4611 KOps/s 39.8302 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-False-True-False] 46.3160μs 16.7576μs 59.6745 KOps/s 60.3161 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-False-False-False-True] 38.7830μs 14.6571μs 68.2263 KOps/s 67.5936 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[True-False-False-False-False] 36.9580μs 9.9959μs 100.0406 KOps/s 101.4655 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-True-True-True] 59.6420μs 23.5651μs 42.4357 KOps/s 41.4637 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[False-True-True-True-False] 40.9860μs 15.6216μs 64.0140 KOps/s 65.0245 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[False-True-True-False-True] 42.6700μs 15.6338μs 63.9641 KOps/s 62.2534 KOps/s $\color{#35bf28}+2.75\%$
test_step_mdp_speed[False-True-True-False-False] 31.4920μs 9.9078μs 100.9302 KOps/s 101.1841 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-True-False-True-True] 48.9430μs 25.2616μs 39.5858 KOps/s 38.8883 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-True-False-True-False] 40.4760μs 16.8280μs 59.4248 KOps/s 60.0518 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-True-False-False-True] 39.3140μs 16.9587μs 58.9667 KOps/s 57.6789 KOps/s $\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-True-False-False-False] 35.1960μs 11.1203μs 89.9257 KOps/s 90.6459 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-False-True-True-True] 64.9020μs 26.1048μs 38.3071 KOps/s 37.5347 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-False-True-True-False] 45.2260μs 18.0228μs 55.4853 KOps/s 56.1505 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[False-False-True-False-True] 39.8350μs 17.0289μs 58.7237 KOps/s 58.3334 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-False-False] 33.5930μs 11.2287μs 89.0575 KOps/s 89.9389 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[False-False-False-True-True] 55.0540μs 27.1334μs 36.8550 KOps/s 36.4013 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-False] 43.8230μs 19.1531μs 52.2108 KOps/s 53.3120 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-False-False-False-True] 45.8470μs 18.0293μs 55.4653 KOps/s 55.3723 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-False-False-False] 32.7810μs 12.2824μs 81.4171 KOps/s 82.7467 KOps/s $\color{#d91a1a}-1.61\%$
test_values[generalized_advantage_estimate-True-True] 10.0375ms 9.3159ms 107.3433 Ops/s 109.5767 Ops/s $\color{#d91a1a}-2.04\%$
test_values[vec_generalized_advantage_estimate-True-True] 40.0113ms 35.5001ms 28.1689 Ops/s 29.8896 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_values[td0_return_estimate-False-False] 0.2616ms 0.1688ms 5.9238 KOps/s 5.9900 KOps/s $\color{#d91a1a}-1.11\%$
test_values[td1_return_estimate-False-False] 25.8573ms 23.3084ms 42.9030 Ops/s 43.8917 Ops/s $\color{#d91a1a}-2.25\%$
test_values[vec_td1_return_estimate-False-False] 39.2824ms 35.5141ms 28.1578 Ops/s 29.7511 Ops/s $\textbf{\color{#d91a1a}-5.36\%}$
test_values[td_lambda_return_estimate-True-False] 36.5446ms 33.7961ms 29.5892 Ops/s 30.1021 Ops/s $\color{#d91a1a}-1.70\%$
test_values[vec_td_lambda_return_estimate-True-False] 42.1465ms 35.8911ms 27.8620 Ops/s 29.7248 Ops/s $\textbf{\color{#d91a1a}-6.27\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.2684ms 8.1558ms 122.6128 Ops/s 125.9801 Ops/s $\color{#d91a1a}-2.67\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2576ms 1.7885ms 559.1159 Ops/s 492.4398 Ops/s $\textbf{\color{#35bf28}+13.54\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4293ms 0.3504ms 2.8539 KOps/s 2.8961 KOps/s $\color{#d91a1a}-1.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.4929ms 46.3622ms 21.5693 Ops/s 23.2424 Ops/s $\textbf{\color{#d91a1a}-7.20\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.5472ms 3.0366ms 329.3104 Ops/s 330.9004 Ops/s $\color{#d91a1a}-0.48\%$
test_dqn_speed 70.0745ms 1.4863ms 672.8061 Ops/s 721.2645 Ops/s $\textbf{\color{#d91a1a}-6.72\%}$
test_ddpg_speed 3.4095ms 2.7267ms 366.7381 Ops/s 370.0429 Ops/s $\color{#d91a1a}-0.89\%$
test_sac_speed 9.9664ms 8.3284ms 120.0708 Ops/s 121.4954 Ops/s $\color{#d91a1a}-1.17\%$
test_redq_speed 14.2568ms 13.1444ms 76.0778 Ops/s 76.0481 Ops/s $\color{#35bf28}+0.04\%$
test_redq_deprec_speed 14.3908ms 13.3914ms 74.6746 Ops/s 75.7453 Ops/s $\color{#d91a1a}-1.41\%$
test_td3_speed 8.9125ms 8.2597ms 121.0703 Ops/s 122.8072 Ops/s $\color{#d91a1a}-1.41\%$
test_cql_speed 38.1084ms 36.4245ms 27.4541 Ops/s 27.5882 Ops/s $\color{#d91a1a}-0.49\%$
test_a2c_speed 8.6813ms 7.4245ms 134.6889 Ops/s 136.0687 Ops/s $\color{#d91a1a}-1.01\%$
test_ppo_speed 8.3361ms 7.6446ms 130.8113 Ops/s 131.4882 Ops/s $\color{#d91a1a}-0.51\%$
test_reinforce_speed 7.6084ms 6.5660ms 152.3007 Ops/s 154.0223 Ops/s $\color{#d91a1a}-1.12\%$
test_iql_speed 33.2768ms 32.4750ms 30.7929 Ops/s 30.7012 Ops/s $\color{#35bf28}+0.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3415ms 2.2816ms 438.2799 Ops/s 434.3861 Ops/s $\color{#35bf28}+0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8310ms 0.4955ms 2.0181 KOps/s 2.0156 KOps/s $\color{#35bf28}+0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7924ms 0.4695ms 2.1298 KOps/s 2.1308 KOps/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.5095ms 2.3066ms 433.5383 Ops/s 436.3972 Ops/s $\color{#d91a1a}-0.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9788ms 0.4906ms 2.0383 KOps/s 2.0355 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7802ms 0.4637ms 2.1566 KOps/s 2.1610 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6942ms 2.3662ms 422.6232 Ops/s 401.3040 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0992ms 0.6124ms 1.6328 KOps/s 1.6475 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8397ms 0.5857ms 1.7074 KOps/s 1.7087 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3422ms 2.1995ms 454.6488 Ops/s 448.6572 Ops/s $\color{#35bf28}+1.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 95.6312ms 0.5841ms 1.7120 KOps/s 2.0267 KOps/s $\textbf{\color{#d91a1a}-15.53\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5757ms 0.4724ms 2.1170 KOps/s 2.1152 KOps/s $\color{#35bf28}+0.08\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.5304ms 2.3925ms 417.9778 Ops/s 436.4131 Ops/s $\color{#d91a1a}-4.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9926ms 0.4922ms 2.0319 KOps/s 2.0748 KOps/s $\color{#d91a1a}-2.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7356ms 0.4629ms 2.1601 KOps/s 2.1198 KOps/s $\color{#35bf28}+1.90\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4519ms 2.3461ms 426.2390 Ops/s 415.1748 Ops/s $\color{#35bf28}+2.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8793ms 0.6163ms 1.6227 KOps/s 1.6344 KOps/s $\color{#d91a1a}-0.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.6310ms 0.5912ms 1.6915 KOps/s 1.7079 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1088s 5.7170ms 174.9169 Ops/s 139.5698 Ops/s $\textbf{\color{#35bf28}+25.33\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.0620ms 11.9949ms 83.3690 Ops/s 83.5697 Ops/s $\color{#d91a1a}-0.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5668ms 1.0423ms 959.4460 Ops/s 985.0049 Ops/s $\color{#d91a1a}-2.59\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 93.8191ms 7.1894ms 139.0935 Ops/s 136.4282 Ops/s $\color{#35bf28}+1.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.3516ms 11.9022ms 84.0184 Ops/s 83.7037 Ops/s $\color{#35bf28}+0.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8636ms 1.0566ms 946.4525 Ops/s 928.7216 Ops/s $\color{#35bf28}+1.91\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 97.7424ms 7.5772ms 131.9752 Ops/s 136.4097 Ops/s $\color{#d91a1a}-3.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.7276ms 12.2160ms 81.8597 Ops/s 81.6325 Ops/s $\color{#35bf28}+0.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0822ms 1.3640ms 733.1155 Ops/s 696.6578 Ops/s $\textbf{\color{#35bf28}+5.23\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 15, 2024 •
edited

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1177s 0.1161s 8.6129 Ops/s 8.8597 Ops/s $\color{#d91a1a}-2.79\%$
test_sync 95.5061ms 95.2619ms 10.4974 Ops/s 10.4594 Ops/s $\color{#35bf28}+0.36\%$
test_async 0.1802s 91.7175ms 10.9030 Ops/s 10.9353 Ops/s $\color{#d91a1a}-0.30\%$
test_single_pixels 0.1959s 0.1342s 7.4522 Ops/s 7.7539 Ops/s $\color{#d91a1a}-3.89\%$
test_sync_pixels 82.3166ms 78.8286ms 12.6858 Ops/s 12.4006 Ops/s $\color{#35bf28}+2.30\%$
test_async_pixels 0.1499s 72.5688ms 13.7800 Ops/s 13.4040 Ops/s $\color{#35bf28}+2.81\%$
test_simple 0.9015s 0.8427s 1.1866 Ops/s 1.2188 Ops/s $\color{#d91a1a}-2.64\%$
test_transformed 1.1194s 1.0696s 0.9349 Ops/s 0.9613 Ops/s $\color{#d91a1a}-2.75\%$
test_serial 2.5632s 2.4940s 0.4010 Ops/s 0.4181 Ops/s $\color{#d91a1a}-4.09\%$
test_parallel 2.1736s 2.1003s 0.4761 Ops/s 0.4871 Ops/s $\color{#d91a1a}-2.26\%$
test_step_mdp_speed[True-True-True-True-True] 95.1310μs 32.2331μs 31.0240 KOps/s 30.8101 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-True-True-False] 44.0200μs 19.3888μs 51.5761 KOps/s 51.5202 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-True-False-True] 49.8700μs 18.0835μs 55.2991 KOps/s 54.6669 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-True-False-False] 80.3810μs 10.8568μs 92.1079 KOps/s 91.5624 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-True-True] 58.2000μs 33.5323μs 29.8220 KOps/s 29.4318 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[True-True-False-True-False] 41.5500μs 21.0091μs 47.5983 KOps/s 47.7763 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-False-True] 40.0210μs 19.9664μs 50.0841 KOps/s 49.2292 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[True-True-False-False-False] 35.8900μs 12.6721μs 78.9137 KOps/s 77.7978 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-True-True-True] 67.4810μs 35.6439μs 28.0553 KOps/s 27.9172 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[True-False-True-True-False] 45.5510μs 22.8718μs 43.7219 KOps/s 43.2880 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-False-True-False-True] 52.9800μs 19.9807μs 50.0483 KOps/s 50.0253 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-False-False] 44.2400μs 12.7674μs 78.3242 KOps/s 77.6664 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-False-False-True-True] 64.7710μs 37.2077μs 26.8761 KOps/s 26.4923 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-False-False-True-False] 44.0310μs 24.5076μs 40.8036 KOps/s 40.5280 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-False-False-True] 44.9300μs 21.6050μs 46.2855 KOps/s 46.0009 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-False-False-False-False] 34.4000μs 14.4783μs 69.0689 KOps/s 68.0562 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-True-True-True-True] 71.2200μs 35.8648μs 27.8825 KOps/s 27.9714 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-True-False] 47.1710μs 22.7484μs 43.9592 KOps/s 43.3692 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-True-True-False-True] 57.4510μs 23.6732μs 42.2418 KOps/s 42.8236 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-True-True-False-False] 49.9110μs 14.4863μs 69.0307 KOps/s 69.1330 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-False-True-True] 63.8610μs 37.4865μs 26.6763 KOps/s 26.3790 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-True-False-True-False] 46.1400μs 24.8325μs 40.2698 KOps/s 39.7758 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-True-False-False-True] 51.6710μs 25.6144μs 39.0405 KOps/s 39.7080 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-True-False-False-False] 37.2910μs 16.3978μs 60.9840 KOps/s 60.7359 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-True-True-True] 89.3510μs 39.5235μs 25.3014 KOps/s 24.9417 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-False-True-True-False] 65.5700μs 26.9024μs 37.1714 KOps/s 37.2033 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-True-False-True] 50.2200μs 25.6005μs 39.0617 KOps/s 39.5106 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[False-False-True-False-False] 43.3110μs 16.4076μs 60.9474 KOps/s 61.4249 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-False-False-True-True] 71.8710μs 40.7047μs 24.5672 KOps/s 24.4329 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-False-True-False] 51.5600μs 28.5913μs 34.9757 KOps/s 35.1802 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-False-False-True] 50.4110μs 26.9222μs 37.1441 KOps/s 37.3737 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-False-False-False-False] 53.1000μs 18.1440μs 55.1145 KOps/s 54.7918 KOps/s $\color{#35bf28}+0.59\%$
test_values[generalized_advantage_estimate-True-True] 25.8830ms 25.2793ms 39.5581 Ops/s 41.5641 Ops/s $\color{#d91a1a}-4.83\%$
test_values[vec_generalized_advantage_estimate-True-True] 89.3838ms 3.3447ms 298.9804 Ops/s 309.8012 Ops/s $\color{#d91a1a}-3.49\%$
test_values[td0_return_estimate-False-False] 98.1210μs 58.2632μs 17.1635 KOps/s 16.1742 KOps/s $\textbf{\color{#35bf28}+6.12\%}$
test_values[td1_return_estimate-False-False] 54.5305ms 53.6129ms 18.6522 Ops/s 19.2062 Ops/s $\color{#d91a1a}-2.88\%$
test_values[vec_td1_return_estimate-False-False] 2.0412ms 1.7689ms 565.3330 Ops/s 570.1051 Ops/s $\color{#d91a1a}-0.84\%$
test_values[td_lambda_return_estimate-True-False] 86.4221ms 85.7150ms 11.6666 Ops/s 12.0327 Ops/s $\color{#d91a1a}-3.04\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.9704ms 1.7791ms 562.0939 Ops/s 558.6561 Ops/s $\color{#35bf28}+0.62\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.9231ms 22.5326ms 44.3801 Ops/s 43.5071 Ops/s $\color{#35bf28}+2.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8657ms 0.6727ms 1.4865 KOps/s 1.4346 KOps/s $\color{#35bf28}+3.62\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6916ms 0.6317ms 1.5832 KOps/s 1.5580 KOps/s $\color{#35bf28}+1.61\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4991ms 1.4342ms 697.2509 Ops/s 694.7089 Ops/s $\color{#35bf28}+0.37\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9174ms 0.6517ms 1.5344 KOps/s 1.5056 KOps/s $\color{#35bf28}+1.92\%$
test_dqn_speed 8.0984ms 1.4157ms 706.3845 Ops/s 637.7852 Ops/s $\textbf{\color{#35bf28}+10.76\%}$
test_ddpg_speed 3.0126ms 2.6762ms 373.6593 Ops/s 378.2620 Ops/s $\color{#d91a1a}-1.22\%$
test_sac_speed 8.3619ms 7.8777ms 126.9402 Ops/s 128.4829 Ops/s $\color{#d91a1a}-1.20\%$
test_redq_speed 11.3692ms 10.1000ms 99.0103 Ops/s 100.7166 Ops/s $\color{#d91a1a}-1.69\%$
test_redq_deprec_speed 11.1822ms 10.7369ms 93.1369 Ops/s 93.5584 Ops/s $\color{#d91a1a}-0.45\%$
test_td3_speed 15.4857ms 7.8685ms 127.0897 Ops/s 128.2191 Ops/s $\color{#d91a1a}-0.88\%$
test_cql_speed 26.3021ms 24.7957ms 40.3296 Ops/s 40.1345 Ops/s $\color{#35bf28}+0.49\%$
test_a2c_speed 5.6515ms 5.4851ms 182.3110 Ops/s 186.9011 Ops/s $\color{#d91a1a}-2.46\%$
test_ppo_speed 5.9751ms 5.7607ms 173.5910 Ops/s 176.8494 Ops/s $\color{#d91a1a}-1.84\%$
test_reinforce_speed 4.6749ms 4.4694ms 223.7421 Ops/s 226.0058 Ops/s $\color{#d91a1a}-1.00\%$
test_iql_speed 21.0787ms 19.5298ms 51.2037 Ops/s 52.1729 Ops/s $\color{#d91a1a}-1.86\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9651ms 2.8348ms 352.7626 Ops/s 353.8444 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 87.7743ms 0.5982ms 1.6716 KOps/s 1.8729 KOps/s $\textbf{\color{#d91a1a}-10.75\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7138ms 0.5097ms 1.9620 KOps/s 1.9674 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.2356ms 2.8724ms 348.1461 Ops/s 348.3321 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6587ms 0.5269ms 1.8981 KOps/s 1.8967 KOps/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6235ms 0.5086ms 1.9662 KOps/s 1.9989 KOps/s $\color{#d91a1a}-1.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0669ms 2.9750ms 336.1329 Ops/s 336.1550 Ops/s $-0.01\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8309ms 0.6586ms 1.5184 KOps/s 1.5442 KOps/s $\color{#d91a1a}-1.67\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8678ms 0.6331ms 1.5794 KOps/s 1.5974 KOps/s $\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0022ms 2.8550ms 350.2611 Ops/s 349.8474 Ops/s $\color{#35bf28}+0.12\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6563ms 0.5355ms 1.8675 KOps/s 1.8769 KOps/s $\color{#d91a1a}-0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6667ms 0.5170ms 1.9343 KOps/s 1.9420 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0403ms 2.8699ms 348.4440 Ops/s 348.9179 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6490ms 0.5297ms 1.8879 KOps/s 1.9071 KOps/s $\color{#d91a1a}-1.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.5948ms 0.5099ms 1.9612 KOps/s 1.9836 KOps/s $\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1018ms 2.9795ms 335.6218 Ops/s 334.5113 Ops/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8244ms 0.6528ms 1.5319 KOps/s 1.5411 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8372ms 0.6316ms 1.5832 KOps/s 1.3903 KOps/s $\textbf{\color{#35bf28}+13.87\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1125s 8.6599ms 115.4746 Ops/s 151.0408 Ops/s $\textbf{\color{#d91a1a}-23.55\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 24.0859ms 14.6245ms 68.3784 Ops/s 70.6284 Ops/s $\color{#d91a1a}-3.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2468ms 1.0771ms 928.4096 Ops/s 872.8319 Ops/s $\textbf{\color{#35bf28}+6.37\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1013s 8.5173ms 117.4077 Ops/s 117.9165 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.4367ms 14.6807ms 68.1168 Ops/s 70.9857 Ops/s $\color{#d91a1a}-4.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1266ms 1.0418ms 959.8707 Ops/s 799.9040 Ops/s $\textbf{\color{#35bf28}+20.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 99.5601ms 6.8764ms 145.4257 Ops/s 145.0052 Ops/s $\color{#35bf28}+0.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.4921ms 14.8862ms 67.1761 Ops/s 69.1346 Ops/s $\color{#d91a1a}-2.83\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.4863ms 1.3727ms 728.4998 Ops/s 633.1578 Ops/s $\textbf{\color{#35bf28}+15.06\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot