Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NOMERG] Add @overload to forward in losses #1893

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[NOMERG] Add @overload to forward in losses #1893

wants to merge 2 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 9, 2024

Loss functions accept non-tensordict data thanks to the tensordict.dispatch decorator.

However, we do not provide an overloaded forward, which could be useful to let users know about the typical signature using default settings.

In this PR, I propose that feature for A2C. The list input keys can be found in A2CLoss.in_keys property and also extrapolated from test/test_cost.py:TestA2C:test_a2c_notensordict.

All losses that are tested with test_<smth>_notensordict can benefit from this overload.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 9, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1893

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (13 Unrelated Failures)

As of commit 738d971 with merge base 4d52d5f (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 9, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 9, 2024

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1303s 66.3885ms 15.0628 Ops/s 15.8959 Ops/s $\textbf{\color{#d91a1a}-5.24\%}$
test_sync 39.6712ms 33.6040ms 29.7584 Ops/s 29.2267 Ops/s $\color{#35bf28}+1.82\%$
test_async 71.7394ms 33.9623ms 29.4444 Ops/s 29.9630 Ops/s $\color{#d91a1a}-1.73\%$
test_simple 0.4846s 0.4286s 2.3329 Ops/s 2.2666 Ops/s $\color{#35bf28}+2.93\%$
test_transformed 0.6412s 0.5919s 1.6895 Ops/s 1.6860 Ops/s $\color{#35bf28}+0.21\%$
test_serial 1.4869s 1.4373s 0.6958 Ops/s 0.7091 Ops/s $\color{#d91a1a}-1.88\%$
test_parallel 1.4512s 1.3922s 0.7183 Ops/s 0.7215 Ops/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-True-True-True-True] 0.1414ms 21.4208μs 46.6837 KOps/s 46.8440 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-True-False] 54.9430μs 12.9805μs 77.0387 KOps/s 75.4129 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-False-True] 38.3320μs 12.5063μs 79.9599 KOps/s 76.2871 KOps/s $\color{#35bf28}+4.81\%$
test_step_mdp_speed[True-True-True-False-False] 49.8730μs 7.4882μs 133.5442 KOps/s 129.4157 KOps/s $\color{#35bf28}+3.19\%$
test_step_mdp_speed[True-True-False-True-True] 50.6050μs 22.7372μs 43.9809 KOps/s 44.0930 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-True-False-True-False] 58.3790μs 14.1629μs 70.6072 KOps/s 68.8173 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-False-False-True] 38.1420μs 13.8280μs 72.3169 KOps/s 72.8700 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-False-False-False] 33.2420μs 8.7771μs 113.9329 KOps/s 111.1092 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-True-True-True] 47.6590μs 24.1745μs 41.3660 KOps/s 41.3405 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-True-True-False] 64.6010μs 15.7488μs 63.4968 KOps/s 63.2408 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-False-True-False-True] 46.2660μs 13.8574μs 72.1637 KOps/s 72.4645 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-True-False-False] 41.1770μs 8.7989μs 113.6504 KOps/s 110.8157 KOps/s $\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-False-False-True-True] 58.9500μs 25.3890μs 39.3872 KOps/s 39.7925 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-False-False-True-False] 38.7620μs 16.9666μs 58.9392 KOps/s 59.5740 KOps/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[True-False-False-False-True] 61.8750μs 15.0093μs 66.6255 KOps/s 66.4721 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-False-False-False] 36.1980μs 10.0089μs 99.9112 KOps/s 98.4996 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-True-True-True] 0.1003ms 23.8370μs 41.9515 KOps/s 41.4268 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-True-True-True-False] 60.9240μs 15.6321μs 63.9711 KOps/s 63.6936 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-True-False-True] 43.4010μs 15.9229μs 62.8026 KOps/s 61.8748 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-True-False-False] 37.6900μs 10.0627μs 99.3765 KOps/s 99.3774 KOps/s $-0.00\%$
test_step_mdp_speed[False-True-False-True-True] 34.7760μs 25.5874μs 39.0817 KOps/s 38.7370 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-True-False-True-False] 44.1520μs 16.9583μs 58.9681 KOps/s 58.4829 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-False-False-True] 62.1160μs 17.0905μs 58.5122 KOps/s 57.4664 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-True-False-False-False] 38.2210μs 11.2944μs 88.5392 KOps/s 88.6733 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-False-True-True-True] 57.9980μs 26.5316μs 37.6909 KOps/s 37.3737 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-False-True-True-False] 62.6770μs 18.2474μs 54.8025 KOps/s 54.8632 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-False-True] 40.1750μs 17.1780μs 58.2139 KOps/s 57.9089 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-False-True-False-False] 66.2840μs 11.2855μs 88.6096 KOps/s 88.8253 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-False-False-True-True] 57.2170μs 27.5107μs 36.3494 KOps/s 36.2075 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-False-False-True-False] 67.0730μs 19.4044μs 51.5346 KOps/s 51.8225 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-False-False-True] 49.0210μs 18.2609μs 54.7617 KOps/s 54.6922 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-False-False-False] 36.5780μs 12.5190μs 79.8788 KOps/s 80.5668 KOps/s $\color{#d91a1a}-0.85\%$
test_values[generalized_advantage_estimate-True-True] 9.7599ms 9.3421ms 107.0418 Ops/s 106.4952 Ops/s $\color{#35bf28}+0.51\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.7538ms 33.3543ms 29.9812 Ops/s 28.6007 Ops/s $\color{#35bf28}+4.83\%$
test_values[td0_return_estimate-False-False] 0.2339ms 0.1733ms 5.7717 KOps/s 5.4414 KOps/s $\textbf{\color{#35bf28}+6.07\%}$
test_values[td1_return_estimate-False-False] 27.2092ms 23.6814ms 42.2272 Ops/s 42.2110 Ops/s $\color{#35bf28}+0.04\%$
test_values[vec_td1_return_estimate-False-False] 48.7936ms 34.1361ms 29.2945 Ops/s 28.3849 Ops/s $\color{#35bf28}+3.20\%$
test_values[td_lambda_return_estimate-True-False] 36.9309ms 33.7608ms 29.6202 Ops/s 29.7198 Ops/s $\color{#d91a1a}-0.34\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.4871ms 33.4906ms 29.8591 Ops/s 28.3773 Ops/s $\textbf{\color{#35bf28}+5.22\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.4548ms 8.2140ms 121.7427 Ops/s 120.0647 Ops/s $\color{#35bf28}+1.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.0405ms 1.8902ms 529.0500 Ops/s 502.9398 Ops/s $\textbf{\color{#35bf28}+5.19\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5485ms 0.3507ms 2.8516 KOps/s 2.8184 KOps/s $\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 58.6037ms 42.6974ms 23.4206 Ops/s 21.3543 Ops/s $\textbf{\color{#35bf28}+9.68\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5158ms 3.0329ms 329.7148 Ops/s 331.0230 Ops/s $\color{#d91a1a}-0.40\%$
test_dqn_speed 1.8210ms 1.3844ms 722.3180 Ops/s 672.3417 Ops/s $\textbf{\color{#35bf28}+7.43\%}$
test_ddpg_speed 3.2985ms 2.7537ms 363.1486 Ops/s 361.7231 Ops/s $\color{#35bf28}+0.39\%$
test_sac_speed 9.8950ms 8.6338ms 115.8234 Ops/s 117.1354 Ops/s $\color{#d91a1a}-1.12\%$
test_redq_speed 14.5317ms 13.3239ms 75.0531 Ops/s 74.1663 Ops/s $\color{#35bf28}+1.20\%$
test_redq_deprec_speed 13.9748ms 13.4093ms 74.5750 Ops/s 71.2035 Ops/s $\color{#35bf28}+4.73\%$
test_td3_speed 9.2443ms 8.7331ms 114.5071 Ops/s 115.5177 Ops/s $\color{#d91a1a}-0.87\%$
test_cql_speed 38.3614ms 36.5713ms 27.3439 Ops/s 27.5464 Ops/s $\color{#d91a1a}-0.74\%$
test_a2c_speed 9.4082ms 7.4174ms 134.8175 Ops/s 137.5849 Ops/s $\color{#d91a1a}-2.01\%$
test_ppo_speed 9.1976ms 7.5151ms 133.0659 Ops/s 130.6164 Ops/s $\color{#35bf28}+1.88\%$
test_reinforce_speed 7.6339ms 6.5486ms 152.7053 Ops/s 152.9957 Ops/s $\color{#d91a1a}-0.19\%$
test_iql_speed 0.1145s 35.3986ms 28.2497 Ops/s 28.3106 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9952ms 2.7101ms 368.9894 Ops/s 376.7213 Ops/s $\color{#d91a1a}-2.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7184ms 0.5133ms 1.9483 KOps/s 1.9391 KOps/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6487ms 0.4873ms 2.0521 KOps/s 2.0553 KOps/s $\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9379ms 2.6949ms 371.0766 Ops/s 376.3810 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7433ms 0.5054ms 1.9787 KOps/s 1.9773 KOps/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7546ms 0.4811ms 2.0787 KOps/s 2.0849 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9471ms 2.8092ms 355.9749 Ops/s 363.3679 Ops/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7962ms 0.6289ms 1.5902 KOps/s 1.5896 KOps/s $\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 93.8742ms 0.6692ms 1.4943 KOps/s 1.4827 KOps/s $\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0932ms 2.6581ms 376.2113 Ops/s 379.5852 Ops/s $\color{#d91a1a}-0.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6653ms 0.5108ms 1.9578 KOps/s 1.9514 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7063ms 0.4891ms 2.0444 KOps/s 2.0439 KOps/s $\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.1374ms 2.8413ms 351.9552 Ops/s 374.7949 Ops/s $\textbf{\color{#d91a1a}-6.09\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7734ms 0.5145ms 1.9437 KOps/s 1.9677 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6583ms 0.4843ms 2.0650 KOps/s 2.0583 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.2639ms 2.8170ms 354.9880 Ops/s 360.8408 Ops/s $\color{#d91a1a}-1.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7359ms 0.6286ms 1.5909 KOps/s 1.5926 KOps/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8025ms 0.6049ms 1.6533 KOps/s 1.6748 KOps/s $\color{#d91a1a}-1.28\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1014s 7.9380ms 125.9756 Ops/s 100.9139 Ops/s $\textbf{\color{#35bf28}+24.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1114s 14.9890ms 66.7155 Ops/s 75.4702 Ops/s $\textbf{\color{#d91a1a}-11.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.6402ms 2.5037ms 399.4106 Ops/s 397.1479 Ops/s $\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 97.7157ms 7.8784ms 126.9296 Ops/s 126.0975 Ops/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.3663ms 13.0652ms 76.5394 Ops/s 75.0328 Ops/s $\color{#35bf28}+2.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.2462ms 2.5388ms 393.8920 Ops/s 395.6189 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1010s 8.1551ms 122.6223 Ops/s 99.1567 Ops/s $\textbf{\color{#35bf28}+23.67\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.5738ms 13.3918ms 74.6724 Ops/s 73.7236 Ops/s $\color{#35bf28}+1.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 96.0599ms 4.6369ms 215.6608 Ops/s 355.5665 Ops/s $\textbf{\color{#d91a1a}-39.35\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 9, 2024

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1167s 0.1154s 8.6673 Ops/s 8.6901 Ops/s $\color{#d91a1a}-0.26\%$
test_sync 0.1753s 0.1031s 9.7015 Ops/s 9.6204 Ops/s $\color{#35bf28}+0.84\%$
test_async 0.2602s 92.2162ms 10.8441 Ops/s 10.8466 Ops/s $\color{#d91a1a}-0.02\%$
test_single_pixels 0.1306s 0.1277s 7.8304 Ops/s 7.8131 Ops/s $\color{#35bf28}+0.22\%$
test_sync_pixels 82.7875ms 80.0681ms 12.4894 Ops/s 12.7804 Ops/s $\color{#d91a1a}-2.28\%$
test_async_pixels 0.2067s 75.9462ms 13.1672 Ops/s 12.7277 Ops/s $\color{#35bf28}+3.45\%$
test_simple 0.8932s 0.8196s 1.2202 Ops/s 1.2479 Ops/s $\color{#d91a1a}-2.22\%$
test_transformed 1.1362s 1.0721s 0.9328 Ops/s 0.9382 Ops/s $\color{#d91a1a}-0.58\%$
test_serial 2.4674s 2.4012s 0.4165 Ops/s 0.4197 Ops/s $\color{#d91a1a}-0.76\%$
test_parallel 2.1133s 2.0476s 0.4884 Ops/s 0.4918 Ops/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-True-True-True-True] 0.3192ms 32.7781μs 30.5082 KOps/s 29.9541 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-True-True-False] 0.2068ms 19.6102μs 50.9939 KOps/s 50.5160 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-True-True-False-True] 48.4510μs 18.3739μs 54.4252 KOps/s 54.0197 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-True-True-False-False] 0.2167ms 11.1904μs 89.3623 KOps/s 88.9091 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-False-True-True] 0.2237ms 34.4366μs 29.0389 KOps/s 28.4482 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-False-True-False] 37.1810μs 21.3439μs 46.8517 KOps/s 46.5735 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-False-True] 0.2067ms 20.5342μs 48.6992 KOps/s 48.6086 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-True-False-False-False] 0.2299ms 12.9835μs 77.0207 KOps/s 76.7395 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-True-True-True] 67.3100μs 36.6327μs 27.2980 KOps/s 26.9038 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-True-False] 44.0810μs 23.5789μs 42.4108 KOps/s 42.2569 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-False-True] 0.2203ms 20.3194μs 49.2141 KOps/s 48.6601 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-True-False-False] 30.7100μs 12.9263μs 77.3619 KOps/s 77.1211 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-True-True] 56.9910μs 38.0128μs 26.3069 KOps/s 26.0373 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-False-False-True-False] 56.7210μs 25.0061μs 39.9902 KOps/s 39.5085 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-False-False-True] 0.2086ms 22.2437μs 44.9565 KOps/s 45.3633 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-False-False] 0.2102ms 14.8104μs 67.5202 KOps/s 67.5310 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[False-True-True-True-True] 55.9800μs 36.2945μs 27.5524 KOps/s 26.8069 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-True-True-False] 0.2206ms 23.5319μs 42.4955 KOps/s 42.3483 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-True-False-True] 0.2323ms 24.3595μs 41.0518 KOps/s 40.5500 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-True-True-False-False] 43.9400μs 14.9445μs 66.9143 KOps/s 67.6034 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-True-False-True-True] 0.2429ms 38.3809μs 26.0546 KOps/s 25.8198 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-True-False] 47.5400μs 25.1352μs 39.7849 KOps/s 39.3164 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-True-False-False-True] 0.2204ms 25.3190μs 39.4960 KOps/s 38.4324 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[False-True-False-False-False] 0.2174ms 16.5479μs 60.4308 KOps/s 60.0724 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-True-True-True] 0.2351ms 40.3334μs 24.7933 KOps/s 24.8202 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-True-False] 77.6100μs 27.3167μs 36.6077 KOps/s 37.0217 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-False-True-False-True] 46.9600μs 25.8418μs 38.6970 KOps/s 38.0110 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-False-True-False-False] 0.2108ms 16.5733μs 60.3380 KOps/s 60.8871 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-False-True-True] 0.2453ms 41.3702μs 24.1720 KOps/s 23.6987 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[False-False-False-True-False] 46.3100μs 28.8254μs 34.6916 KOps/s 35.0233 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-False-False-True] 0.2283ms 27.2376μs 36.7140 KOps/s 35.6421 KOps/s $\color{#35bf28}+3.01\%$
test_step_mdp_speed[False-False-False-False-False] 35.7610μs 18.2166μs 54.8950 KOps/s 55.0140 KOps/s $\color{#d91a1a}-0.22\%$
test_values[generalized_advantage_estimate-True-True] 23.9781ms 23.6291ms 42.3208 Ops/s 41.5408 Ops/s $\color{#35bf28}+1.88\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.0216ms 3.2136ms 311.1734 Ops/s 307.7660 Ops/s $\color{#35bf28}+1.11\%$
test_values[td0_return_estimate-False-False] 96.7200μs 59.6120μs 16.7751 KOps/s 17.0290 KOps/s $\color{#d91a1a}-1.49\%$
test_values[td1_return_estimate-False-False] 51.2835ms 50.6817ms 19.7310 Ops/s 19.3762 Ops/s $\color{#35bf28}+1.83\%$
test_values[vec_td1_return_estimate-False-False] 2.0426ms 1.7484ms 571.9576 Ops/s 573.6847 Ops/s $\color{#d91a1a}-0.30\%$
test_values[td_lambda_return_estimate-True-False] 81.6269ms 80.4830ms 12.4250 Ops/s 12.1046 Ops/s $\color{#35bf28}+2.65\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.9223ms 1.7767ms 562.8370 Ops/s 556.2869 Ops/s $\color{#35bf28}+1.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.4399ms 22.1319ms 45.1836 Ops/s 44.4375 Ops/s $\color{#35bf28}+1.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8700ms 0.6876ms 1.4542 KOps/s 1.4431 KOps/s $\color{#35bf28}+0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7062ms 0.6362ms 1.5718 KOps/s 1.5748 KOps/s $\color{#d91a1a}-0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4686ms 1.4401ms 694.3831 Ops/s 694.5938 Ops/s $\color{#d91a1a}-0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9160ms 0.6555ms 1.5256 KOps/s 1.5108 KOps/s $\color{#35bf28}+0.98\%$
test_dqn_speed 8.2734ms 1.4501ms 689.6217 Ops/s 715.6738 Ops/s $\color{#d91a1a}-3.64\%$
test_ddpg_speed 3.1872ms 2.7655ms 361.5976 Ops/s 364.8486 Ops/s $\color{#d91a1a}-0.89\%$
test_sac_speed 9.0553ms 8.5094ms 117.5167 Ops/s 117.8311 Ops/s $\color{#d91a1a}-0.27\%$
test_redq_speed 11.5202ms 10.7428ms 93.0856 Ops/s 93.5116 Ops/s $\color{#d91a1a}-0.46\%$
test_redq_deprec_speed 12.3600ms 11.7019ms 85.4559 Ops/s 83.3673 Ops/s $\color{#35bf28}+2.51\%$
test_td3_speed 17.9667ms 8.7836ms 113.8486 Ops/s 113.9407 Ops/s $\color{#d91a1a}-0.08\%$
test_cql_speed 27.3916ms 26.3288ms 37.9813 Ops/s 38.3736 Ops/s $\color{#d91a1a}-1.02\%$
test_a2c_speed 6.3754ms 5.5288ms 180.8707 Ops/s 183.1007 Ops/s $\color{#d91a1a}-1.22\%$
test_ppo_speed 6.8169ms 5.7620ms 173.5515 Ops/s 174.9248 Ops/s $\color{#d91a1a}-0.79\%$
test_reinforce_speed 5.1210ms 4.5521ms 219.6809 Ops/s 222.9327 Ops/s $\color{#d91a1a}-1.46\%$
test_iql_speed 20.5071ms 19.7811ms 50.5533 Ops/s 49.6857 Ops/s $\color{#35bf28}+1.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7297ms 3.5699ms 280.1181 Ops/s 280.2185 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8728ms 0.5586ms 1.7901 KOps/s 1.7914 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7401ms 0.5322ms 1.8791 KOps/s 1.8753 KOps/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9179ms 3.5918ms 278.4085 Ops/s 279.0021 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7801ms 0.5507ms 1.8159 KOps/s 1.8075 KOps/s $\color{#35bf28}+0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6663ms 0.5287ms 1.8916 KOps/s 1.8826 KOps/s $\color{#35bf28}+0.48\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.1497ms 3.7462ms 266.9391 Ops/s 272.2238 Ops/s $\color{#d91a1a}-1.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0272ms 0.6818ms 1.4667 KOps/s 1.4511 KOps/s $\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8949ms 0.6561ms 1.5242 KOps/s 1.5189 KOps/s $\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7655ms 3.5733ms 279.8540 Ops/s 279.8819 Ops/s $-0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8293ms 0.5605ms 1.7841 KOps/s 1.7946 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7943ms 0.5324ms 1.8783 KOps/s 1.8660 KOps/s $\color{#35bf28}+0.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8931ms 3.5937ms 278.2674 Ops/s 275.2306 Ops/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6916ms 0.5589ms 1.7891 KOps/s 1.8012 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7985ms 0.5407ms 1.8496 KOps/s 1.8705 KOps/s $\color{#d91a1a}-1.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9730ms 3.7190ms 268.8886 Ops/s 266.1486 Ops/s $\color{#35bf28}+1.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9916ms 0.6913ms 1.4465 KOps/s 1.4509 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8913ms 0.6604ms 1.5142 KOps/s 1.5045 KOps/s $\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1294s 10.3725ms 96.4092 Ops/s 94.7016 Ops/s $\color{#35bf28}+1.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.1887ms 15.7249ms 63.5933 Ops/s 64.1607 Ops/s $\color{#d91a1a}-0.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.7690ms 3.0538ms 327.4586 Ops/s 320.7710 Ops/s $\color{#35bf28}+2.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1213s 10.0617ms 99.3864 Ops/s 98.9314 Ops/s $\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1326s 17.9593ms 55.6816 Ops/s 63.8321 Ops/s $\textbf{\color{#d91a1a}-12.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2345ms 3.0703ms 325.6960 Ops/s 329.8069 Ops/s $\color{#d91a1a}-1.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1204s 10.3593ms 96.5314 Ops/s 78.2184 Ops/s $\textbf{\color{#35bf28}+23.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.1797ms 15.9012ms 62.8885 Ops/s 62.9341 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.0055ms 3.3513ms 298.3884 Ops/s 299.0980 Ops/s $\color{#d91a1a}-0.24\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot