Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature,WIP] unique_traj arg in slice sampler #1962

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

[Feature,WIP] unique_traj arg in slice sampler #1962

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 24, 2024

Description

Describe your changes in detail.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

  • I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds core functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (update in the documentation)
  • Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

  • I have read the CONTRIBUTION guide (required)
  • My change requires a change to the documentation.
  • I have updated the tests accordingly (required for a bug fix or a new feature).
  • I have updated the documentation accordingly.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 24, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1962

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 5c4730d with merge base 7782751 (image):

NEW FAILURE - The following job has failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 24, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.4491ms 62.7070ms 15.9472 Ops/s 16.2182 Ops/s $\color{#d91a1a}-1.67\%$
test_sync 45.0642ms 34.0142ms 29.3995 Ops/s 30.2526 Ops/s $\color{#d91a1a}-2.82\%$
test_async 73.9111ms 33.3251ms 30.0074 Ops/s 31.6975 Ops/s $\textbf{\color{#d91a1a}-5.33\%}$
test_simple 0.4923s 0.4435s 2.2548 Ops/s 2.3171 Ops/s $\color{#d91a1a}-2.69\%$
test_transformed 0.6453s 0.5975s 1.6736 Ops/s 1.7459 Ops/s $\color{#d91a1a}-4.14\%$
test_serial 1.5137s 1.4626s 0.6837 Ops/s 0.7157 Ops/s $\color{#d91a1a}-4.47\%$
test_parallel 1.4890s 1.4327s 0.6980 Ops/s 0.7157 Ops/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-True-True-True-True] 0.2259ms 21.9601μs 45.5372 KOps/s 45.3811 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-True-True-False] 42.9500μs 13.5147μs 73.9936 KOps/s 74.9597 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[True-True-True-False-True] 39.5340μs 13.0216μs 76.7956 KOps/s 77.7775 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-True-True-False-False] 33.4320μs 7.6766μs 130.2662 KOps/s 128.8503 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-False-True-True] 46.5970μs 23.5393μs 42.4822 KOps/s 43.2852 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-True-False-True-False] 39.4940μs 14.8832μs 67.1897 KOps/s 68.5475 KOps/s $\color{#d91a1a}-1.98\%$
test_step_mdp_speed[True-True-False-False-True] 36.5980μs 14.2212μs 70.3174 KOps/s 70.9952 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-True-False-False-False] 29.2940μs 9.0971μs 109.9246 KOps/s 109.6381 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-False-True-True-True] 62.3660μs 25.1156μs 39.8159 KOps/s 40.4995 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-False-True-True-False] 44.3330μs 16.3845μs 61.0333 KOps/s 62.1376 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-False-True-False-True] 35.8370μs 14.3441μs 69.7151 KOps/s 70.5602 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[True-False-True-False-False] 31.4490μs 9.1896μs 108.8181 KOps/s 109.3635 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-False-False-True-True] 53.8300μs 26.4100μs 37.8644 KOps/s 38.3903 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-False-False-True-False] 48.0300μs 17.7459μs 56.3511 KOps/s 57.6762 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-False-False-False-True] 98.7540μs 15.4888μs 64.5627 KOps/s 64.7520 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-False-False-False] 38.9930μs 10.3446μs 96.6687 KOps/s 96.3783 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[False-True-True-True-True] 53.5200μs 25.1503μs 39.7609 KOps/s 40.3737 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[False-True-True-True-False] 47.0080μs 16.2752μs 61.4432 KOps/s 61.6434 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-False-True] 40.0650μs 16.9230μs 59.0913 KOps/s 60.8516 KOps/s $\color{#d91a1a}-2.89\%$
test_step_mdp_speed[False-True-True-False-False] 33.9230μs 10.4888μs 95.3400 KOps/s 96.5619 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[False-True-False-True-True] 36.2470μs 26.9507μs 37.1048 KOps/s 38.4529 KOps/s $\color{#d91a1a}-3.51\%$
test_step_mdp_speed[False-True-False-True-False] 53.7300μs 17.8998μs 55.8666 KOps/s 57.7295 KOps/s $\color{#d91a1a}-3.23\%$
test_step_mdp_speed[False-True-False-False-True] 41.9780μs 18.1334μs 55.1467 KOps/s 56.4844 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[False-True-False-False-False] 41.4170μs 11.8603μs 84.3146 KOps/s 86.9257 KOps/s $\color{#d91a1a}-3.00\%$
test_step_mdp_speed[False-False-True-True-True] 62.9180μs 27.9785μs 35.7417 KOps/s 37.2360 KOps/s $\color{#d91a1a}-4.01\%$
test_step_mdp_speed[False-False-True-True-False] 46.1460μs 19.1049μs 52.3426 KOps/s 53.6021 KOps/s $\color{#d91a1a}-2.35\%$
test_step_mdp_speed[False-False-True-False-True] 84.6880μs 18.0072μs 55.5335 KOps/s 56.6268 KOps/s $\color{#d91a1a}-1.93\%$
test_step_mdp_speed[False-False-True-False-False] 43.1810μs 11.8442μs 84.4296 KOps/s 85.7544 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-False-False-True-True] 70.5220μs 28.8477μs 34.6648 KOps/s 35.7740 KOps/s $\color{#d91a1a}-3.10\%$
test_step_mdp_speed[False-False-False-True-False] 44.7230μs 20.1186μs 49.7052 KOps/s 51.0493 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-False-False-False-True] 49.7530μs 19.3195μs 51.7611 KOps/s 53.4878 KOps/s $\color{#d91a1a}-3.23\%$
test_step_mdp_speed[False-False-False-False-False] 44.1830μs 12.9707μs 77.0971 KOps/s 78.0857 KOps/s $\color{#d91a1a}-1.27\%$
test_values[generalized_advantage_estimate-True-True] 9.5864ms 9.3713ms 106.7089 Ops/s 106.2624 Ops/s $\color{#35bf28}+0.42\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.8239ms 33.4212ms 29.9211 Ops/s 28.2503 Ops/s $\textbf{\color{#35bf28}+5.91\%}$
test_values[td0_return_estimate-False-False] 0.2381ms 0.1672ms 5.9826 KOps/s 5.7549 KOps/s $\color{#35bf28}+3.96\%$
test_values[td1_return_estimate-False-False] 26.3227ms 23.5104ms 42.5344 Ops/s 42.1497 Ops/s $\color{#35bf28}+0.91\%$
test_values[vec_td1_return_estimate-False-False] 35.0062ms 33.5454ms 29.8104 Ops/s 27.7966 Ops/s $\textbf{\color{#35bf28}+7.24\%}$
test_values[td_lambda_return_estimate-True-False] 35.0647ms 33.5383ms 29.8167 Ops/s 29.1506 Ops/s $\color{#35bf28}+2.29\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.1436ms 33.3886ms 29.9504 Ops/s 28.1187 Ops/s $\textbf{\color{#35bf28}+6.51\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 13.0233ms 8.4194ms 118.7740 Ops/s 120.5877 Ops/s $\color{#d91a1a}-1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3401ms 2.0177ms 495.6076 Ops/s 509.7586 Ops/s $\color{#d91a1a}-2.78\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5624ms 0.3567ms 2.8035 KOps/s 2.8246 KOps/s $\color{#d91a1a}-0.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.2874ms 45.0139ms 22.2154 Ops/s 24.7882 Ops/s $\textbf{\color{#d91a1a}-10.38\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.5406ms 3.0374ms 329.2324 Ops/s 325.9609 Ops/s $\color{#35bf28}+1.00\%$
test_dqn_speed 6.6824ms 1.4091ms 709.6664 Ops/s 717.2766 Ops/s $\color{#d91a1a}-1.06\%$
test_ddpg_speed 3.6714ms 2.7881ms 358.6693 Ops/s 353.9757 Ops/s $\color{#35bf28}+1.33\%$
test_sac_speed 10.0249ms 8.4560ms 118.2596 Ops/s 118.6739 Ops/s $\color{#d91a1a}-0.35\%$
test_redq_speed 82.5440ms 14.1886ms 70.4792 Ops/s 73.1984 Ops/s $\color{#d91a1a}-3.71\%$
test_redq_deprec_speed 14.2058ms 13.4648ms 74.2676 Ops/s 71.9278 Ops/s $\color{#35bf28}+3.25\%$
test_td3_speed 8.6285ms 8.4254ms 118.6887 Ops/s 116.4092 Ops/s $\color{#35bf28}+1.96\%$
test_cql_speed 37.9123ms 36.5996ms 27.3227 Ops/s 27.2071 Ops/s $\color{#35bf28}+0.42\%$
test_a2c_speed 16.8153ms 7.6598ms 130.5522 Ops/s 133.7567 Ops/s $\color{#d91a1a}-2.40\%$
test_ppo_speed 8.6943ms 7.6664ms 130.4400 Ops/s 128.5183 Ops/s $\color{#35bf28}+1.50\%$
test_reinforce_speed 9.4383ms 6.6175ms 151.1152 Ops/s 148.3385 Ops/s $\color{#35bf28}+1.87\%$
test_iql_speed 34.0815ms 32.9488ms 30.3501 Ops/s 30.0497 Ops/s $\color{#35bf28}+1.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9264ms 2.1995ms 454.6548 Ops/s 453.9823 Ops/s $\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1685ms 0.5013ms 1.9950 KOps/s 1.9916 KOps/s $\color{#35bf28}+0.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6424ms 0.4742ms 2.1087 KOps/s 2.1106 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4773ms 2.1868ms 457.2889 Ops/s 448.8793 Ops/s $\color{#35bf28}+1.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6542ms 0.4928ms 2.0293 KOps/s 2.0268 KOps/s $\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.3670ms 0.4703ms 2.1262 KOps/s 2.1304 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8377ms 2.5062ms 399.0100 Ops/s 439.6729 Ops/s $\textbf{\color{#d91a1a}-9.25\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8484ms 0.6270ms 1.5949 KOps/s 1.6412 KOps/s $\color{#d91a1a}-2.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.6995ms 0.6023ms 1.6604 KOps/s 1.7042 KOps/s $\color{#d91a1a}-2.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.4617ms 2.2377ms 446.8900 Ops/s 458.9459 Ops/s $\color{#d91a1a}-2.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5887ms 0.4978ms 2.0090 KOps/s 2.0142 KOps/s $\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5050ms 0.4814ms 2.0772 KOps/s 1.7750 KOps/s $\textbf{\color{#35bf28}+17.02\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6159ms 2.1807ms 458.5650 Ops/s 465.0497 Ops/s $\color{#d91a1a}-1.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5944ms 0.4926ms 2.0299 KOps/s 2.0374 KOps/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.5179ms 0.4766ms 2.0981 KOps/s 2.1519 KOps/s $\color{#d91a1a}-2.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6697ms 2.3004ms 434.7024 Ops/s 435.6408 Ops/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1573ms 0.6230ms 1.6052 KOps/s 1.4102 KOps/s $\textbf{\color{#35bf28}+13.83\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9091ms 0.5956ms 1.6790 KOps/s 1.7202 KOps/s $\color{#d91a1a}-2.40\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 98.2705ms 7.3512ms 136.0322 Ops/s 184.5625 Ops/s $\textbf{\color{#d91a1a}-26.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3364ms 12.2158ms 81.8611 Ops/s 81.3629 Ops/s $\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.7854ms 1.0671ms 937.0874 Ops/s 949.7517 Ops/s $\color{#d91a1a}-1.33\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 88.6729ms 7.0091ms 142.6714 Ops/s 141.9201 Ops/s $\color{#35bf28}+0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.3361ms 12.0596ms 82.9216 Ops/s 81.9921 Ops/s $\color{#35bf28}+1.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.4751ms 1.1357ms 880.4758 Ops/s 905.0094 Ops/s $\color{#d91a1a}-2.71\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 88.4869ms 5.6850ms 175.9003 Ops/s 134.8942 Ops/s $\textbf{\color{#35bf28}+30.40\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 95.1084ms 14.1258ms 70.7925 Ops/s 79.9317 Ops/s $\textbf{\color{#d91a1a}-11.43\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8964ms 1.3491ms 741.2125 Ops/s 759.6094 Ops/s $\color{#d91a1a}-2.42\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1159s 0.1133s 8.8244 Ops/s 8.6516 Ops/s $\color{#35bf28}+2.00\%$
test_sync 96.0673ms 95.6168ms 10.4584 Ops/s 10.4051 Ops/s $\color{#35bf28}+0.51\%$
test_async 0.1802s 91.2232ms 10.9621 Ops/s 10.9139 Ops/s $\color{#35bf28}+0.44\%$
test_single_pixels 0.1267s 0.1263s 7.9176 Ops/s 7.3200 Ops/s $\textbf{\color{#35bf28}+8.16\%}$
test_sync_pixels 84.2579ms 80.4219ms 12.4344 Ops/s 12.2684 Ops/s $\color{#35bf28}+1.35\%$
test_async_pixels 0.1460s 74.5132ms 13.4204 Ops/s 15.3533 Ops/s $\textbf{\color{#d91a1a}-12.59\%}$
test_simple 0.8101s 0.8096s 1.2352 Ops/s 1.1367 Ops/s $\textbf{\color{#35bf28}+8.67\%}$
test_transformed 1.1053s 1.0459s 0.9561 Ops/s 0.9357 Ops/s $\color{#35bf28}+2.18\%$
test_serial 2.4404s 2.3831s 0.4196 Ops/s 0.3999 Ops/s $\color{#35bf28}+4.94\%$
test_parallel 2.0940s 2.0246s 0.4939 Ops/s 0.4783 Ops/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[True-True-True-True-True] 87.2310μs 32.9896μs 30.3126 KOps/s 30.6395 KOps/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[True-True-True-True-False] 44.0510μs 19.6159μs 50.9790 KOps/s 49.9169 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-False-True] 74.2210μs 18.8152μs 53.1485 KOps/s 53.6301 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-True-True-False-False] 35.1100μs 11.1856μs 89.4003 KOps/s 88.9297 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-True-False-True-True] 68.9520μs 34.8489μs 28.6953 KOps/s 28.5061 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-True-False-True-False] 43.9700μs 21.7015μs 46.0798 KOps/s 46.0044 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-True-False-False-True] 44.0910μs 20.5472μs 48.6684 KOps/s 48.3176 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-True-False-False-False] 33.3500μs 13.3364μs 74.9825 KOps/s 75.2644 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-False-True-True-True] 70.8020μs 36.7713μs 27.1952 KOps/s 27.0271 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-False-True-True-False] 46.3810μs 24.0010μs 41.6649 KOps/s 42.0794 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[True-False-True-False-True] 40.1000μs 20.5898μs 48.5678 KOps/s 48.6932 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-False-True-False-False] 35.2110μs 13.3065μs 75.1513 KOps/s 75.3937 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-False-True-True] 63.8810μs 38.6213μs 25.8925 KOps/s 25.8838 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[True-False-False-True-False] 53.7310μs 25.7223μs 38.8768 KOps/s 39.3853 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[True-False-False-False-True] 43.7010μs 22.6432μs 44.1634 KOps/s 45.2692 KOps/s $\color{#d91a1a}-2.44\%$
test_step_mdp_speed[True-False-False-False-False] 32.9000μs 15.1944μs 65.8139 KOps/s 66.9441 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[False-True-True-True-True] 69.5020μs 37.7197μs 26.5114 KOps/s 27.3029 KOps/s $\color{#d91a1a}-2.90\%$
test_step_mdp_speed[False-True-True-True-False] 45.1200μs 24.0315μs 41.6120 KOps/s 42.7431 KOps/s $\color{#d91a1a}-2.65\%$
test_step_mdp_speed[False-True-True-False-True] 42.5910μs 24.8447μs 40.2501 KOps/s 40.9850 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-True-True-False-False] 40.6810μs 15.0598μs 66.4018 KOps/s 67.3267 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[False-True-False-True-True] 79.6720μs 39.5522μs 25.2830 KOps/s 25.9535 KOps/s $\color{#d91a1a}-2.58\%$
test_step_mdp_speed[False-True-False-True-False] 52.6710μs 25.9232μs 38.5755 KOps/s 39.1545 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-True-False-False-True] 44.7120μs 26.5460μs 37.6705 KOps/s 37.9583 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-True-False-False-False] 33.8810μs 16.8512μs 59.3431 KOps/s 59.4034 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-False-True-True-True] 69.2610μs 40.6379μs 24.6076 KOps/s 24.7252 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-True-True-False] 51.3010μs 27.8243μs 35.9398 KOps/s 36.1491 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-True-False-True] 47.9010μs 26.1577μs 38.2297 KOps/s 39.0552 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-True-False-False] 41.3900μs 16.9313μs 59.0622 KOps/s 60.0985 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-False-False-True-True] 67.9510μs 43.0324μs 23.2383 KOps/s 23.7959 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-False-False-True-False] 45.6400μs 29.3971μs 34.0169 KOps/s 33.7005 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-False-False-False-True] 51.3010μs 28.2220μs 35.4334 KOps/s 36.0911 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-False-False-False-False] 35.5610μs 18.6720μs 53.5560 KOps/s 54.1713 KOps/s $\color{#d91a1a}-1.14\%$
test_values[generalized_advantage_estimate-True-True] 24.9039ms 24.4853ms 40.8409 Ops/s 39.7078 Ops/s $\color{#35bf28}+2.85\%$
test_values[vec_generalized_advantage_estimate-True-True] 85.2366ms 3.2613ms 306.6255 Ops/s 307.3859 Ops/s $\color{#d91a1a}-0.25\%$
test_values[td0_return_estimate-False-False] 0.1004ms 60.6726μs 16.4819 KOps/s 16.0095 KOps/s $\color{#35bf28}+2.95\%$
test_values[td1_return_estimate-False-False] 56.8005ms 56.3759ms 17.7381 Ops/s 18.7305 Ops/s $\textbf{\color{#d91a1a}-5.30\%}$
test_values[vec_td1_return_estimate-False-False] 2.1290ms 1.7536ms 570.2558 Ops/s 568.6555 Ops/s $\color{#35bf28}+0.28\%$
test_values[td_lambda_return_estimate-True-False] 91.0162ms 86.8372ms 11.5158 Ops/s 11.6829 Ops/s $\color{#d91a1a}-1.43\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.8723ms 1.7858ms 559.9816 Ops/s 556.3414 Ops/s $\color{#35bf28}+0.65\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.1089ms 22.7764ms 43.9051 Ops/s 42.4286 Ops/s $\color{#35bf28}+3.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8589ms 0.6895ms 1.4504 KOps/s 1.4248 KOps/s $\color{#35bf28}+1.80\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6999ms 0.6430ms 1.5551 KOps/s 1.5371 KOps/s $\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4964ms 1.4459ms 691.5952 Ops/s 687.7500 Ops/s $\color{#35bf28}+0.56\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.0040ms 0.6637ms 1.5068 KOps/s 1.4888 KOps/s $\color{#35bf28}+1.21\%$
test_dqn_speed 8.1271ms 1.4818ms 674.8750 Ops/s 615.0917 Ops/s $\textbf{\color{#35bf28}+9.72\%}$
test_ddpg_speed 3.5060ms 2.9192ms 342.5589 Ops/s 346.3153 Ops/s $\color{#d91a1a}-1.08\%$
test_sac_speed 8.8881ms 8.1899ms 122.1023 Ops/s 120.2488 Ops/s $\color{#35bf28}+1.54\%$
test_redq_speed 11.3841ms 10.4078ms 96.0821 Ops/s 94.7151 Ops/s $\color{#35bf28}+1.44\%$
test_redq_deprec_speed 12.2076ms 11.2778ms 88.6701 Ops/s 86.8959 Ops/s $\color{#35bf28}+2.04\%$
test_td3_speed 15.8884ms 8.3066ms 120.3869 Ops/s 119.8436 Ops/s $\color{#35bf28}+0.45\%$
test_cql_speed 27.9808ms 26.2033ms 38.1632 Ops/s 39.0219 Ops/s $\color{#d91a1a}-2.20\%$
test_a2c_speed 5.9546ms 5.6052ms 178.4071 Ops/s 180.3263 Ops/s $\color{#d91a1a}-1.06\%$
test_ppo_speed 6.1901ms 5.9351ms 168.4890 Ops/s 169.8406 Ops/s $\color{#d91a1a}-0.80\%$
test_reinforce_speed 4.8528ms 4.5992ms 217.4282 Ops/s 218.5845 Ops/s $\color{#d91a1a}-0.53\%$
test_iql_speed 20.5605ms 19.9409ms 50.1483 Ops/s 50.8064 Ops/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0125ms 2.8587ms 349.8091 Ops/s 344.6359 Ops/s $\color{#35bf28}+1.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0471ms 0.5370ms 1.8623 KOps/s 1.8372 KOps/s $\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7558ms 0.5170ms 1.9343 KOps/s 1.9106 KOps/s $\color{#35bf28}+1.24\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1413ms 2.8852ms 346.5968 Ops/s 342.6247 Ops/s $\color{#35bf28}+1.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1329ms 0.5326ms 1.8776 KOps/s 1.8681 KOps/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0268ms 0.5068ms 1.9730 KOps/s 1.9435 KOps/s $\color{#35bf28}+1.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1479ms 2.9898ms 334.4760 Ops/s 331.9552 Ops/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9008ms 0.6641ms 1.5057 KOps/s 1.4911 KOps/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6191ms 0.6449ms 1.5506 KOps/s 1.5541 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0899ms 2.8760ms 347.7096 Ops/s 345.5625 Ops/s $\color{#35bf28}+0.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6826ms 0.5414ms 1.8471 KOps/s 1.8410 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6055ms 0.5220ms 1.9158 KOps/s 1.6564 KOps/s $\textbf{\color{#35bf28}+15.66\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1700ms 2.8881ms 346.2449 Ops/s 341.7284 Ops/s $\color{#35bf28}+1.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7259ms 0.5299ms 1.8870 KOps/s 1.8554 KOps/s $\color{#35bf28}+1.71\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6040ms 0.5143ms 1.9444 KOps/s 1.9015 KOps/s $\color{#35bf28}+2.26\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4261ms 3.0184ms 331.2981 Ops/s 330.1574 Ops/s $\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8808ms 0.6626ms 1.5093 KOps/s 1.4969 KOps/s $\color{#35bf28}+0.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.4315ms 0.6380ms 1.5674 KOps/s 1.5365 KOps/s $\color{#35bf28}+2.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1166s 8.8613ms 112.8498 Ops/s 147.6194 Ops/s $\textbf{\color{#d91a1a}-23.55\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.6858ms 14.1456ms 70.6936 Ops/s 68.3861 Ops/s $\color{#35bf28}+3.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1068ms 1.0710ms 933.7468 Ops/s 929.0495 Ops/s $\color{#35bf28}+0.51\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1039s 8.6737ms 115.2915 Ops/s 113.9791 Ops/s $\color{#35bf28}+1.15\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.6493ms 14.2662ms 70.0959 Ops/s 68.0225 Ops/s $\color{#35bf28}+3.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1193ms 1.0407ms 960.9078 Ops/s 926.8114 Ops/s $\color{#35bf28}+3.68\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1003s 7.0092ms 142.6689 Ops/s 140.3149 Ops/s $\color{#35bf28}+1.68\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.9790ms 14.5110ms 68.9132 Ops/s 58.9761 Ops/s $\textbf{\color{#35bf28}+16.85\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.5584ms 1.3969ms 715.8718 Ops/s 703.3637 Ops/s $\color{#35bf28}+1.78\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot