You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, sorry in advance if this isn't the right place to ask these kinds of questions.
I have been playing with VMAS in its vanilla form (no torchRL/RLLib) to try and understand how to implement my own Scenarios, and currently I am confused with how VMAS handles resetting the environment. The reset() function docstring states that it handles resetting "in a vectorized way". From my testing, it seems to me that it resets all vectorized environments.
I was hoping "in a vectorized way" meant that it only reset the environments which were done and left the others alone. I would like it to behave this way to collect episode reward from episodes that are allowed to run until termination, for instance. Does VMAS have this functionality built-in? Am I misunderstanding reset()?
Thank you for the great library, by the way!
The text was updated successfully, but these errors were encountered:
To improve efficiency and avoid this for loop. It would be awsome if the reset_at function also accepted a mask.
Something like:
env.reset_at(done)
This would be amazing. The only problem is that the reset_at function of all current scenarios and a major bit of simulator logic will need to be rewritten. So it is not a quick or easy effort.
A consideration
What I do for some scenarios I create is to not implment a done function and let all environments be only done after max_steps. This makes it so that you can always call env.reset(). I understand that this does not fit all tasks, but I figured I would mention this in case it is helpful.
Hi, sorry in advance if this isn't the right place to ask these kinds of questions.
I have been playing with VMAS in its vanilla form (no torchRL/RLLib) to try and understand how to implement my own Scenarios, and currently I am confused with how VMAS handles resetting the environment. The reset() function docstring states that it handles resetting "in a vectorized way". From my testing, it seems to me that it resets all vectorized environments.
I was hoping "in a vectorized way" meant that it only reset the environments which were done and left the others alone. I would like it to behave this way to collect episode reward from episodes that are allowed to run until termination, for instance. Does VMAS have this functionality built-in? Am I misunderstanding reset()?
Thank you for the great library, by the way!
The text was updated successfully, but these errors were encountered: