Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Add support for gymnasium.spaces.Text #1704

Open
2 tasks done
jvasso opened this issue Oct 2, 2023 · 1 comment
Open
2 tasks done

[Feature Request] Add support for gymnasium.spaces.Text #1704

jvasso opened this issue Oct 2, 2023 · 1 comment
Labels
check the checklist You have checked the required items in the checklist but you didn't do what is written... enhancement New feature or request

Comments

@jvasso
Copy link

jvasso commented Oct 2, 2023

馃殌 Feature

Unless I'm mistaken, stable-baselines3 only supports Box, Discrete, MultiDiscrete, MultiBinary and Dict spaces from gymnasium.
It seems like a new fundamental space has been introduced in Gymnasium ( gymnasium.spaces.Text ), which seems to be a more natural way to deal with text data.

Motivation

This new support would ease the training of LLMs with RL.

Pitch

No response

Alternatives

One alternative is to use gymnasium.spaces.Box(low=0, high=vocab_size-1, dtype=np.int).
However this requires padding to support smaller texts.

Additional context

No response

Checklist

  • I have checked that there is no similar issue in the repo
  • If I'm requesting a new feature, I have proposed alternatives
@jvasso jvasso added the enhancement New feature or request label Oct 2, 2023
@araffin
Copy link
Member

araffin commented Oct 2, 2023

If I'm requesting a new feature, I have proposed alternatives

@araffin araffin added the check the checklist You have checked the required items in the checklist but you didn't do what is written... label Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
check the checklist You have checked the required items in the checklist but you didn't do what is written... enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants