Experimental StableLM support #1719
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces two improvements:
1. Support StableLM-Epoch for ONNX export:
- A new class StableLMEpochOnnxConfig is added in optimum/exporters/onnx/model_configs.py to define the configuration for exporting StableLM-Epoch models to ONNX format.
- This aligns with existing configs for other supported models.
2. Enable StableLM-Epoch tasks in optimum/exporters/tasks.py:
- The supported_tasks_mapping function now includes "stablelm-epoch" with supported tasks like "text-generation" and "text-classification".
- This allows exporting StableLM-Epoch models for various tasks using the appropriate ONNX configuration.
These changes expand the capabilities of optimum by enabling ONNX export and task handling for StableLM-Epoch models.
This commit adds NormalizedTextConfig as the normalized configuration class for StableLM-Epoch models in optimum/utils/normalized_config.py. This ensures consistency with other supported models and improves the overall compatibility and functionality of optimum when working with StableLM-Epoch models.
This commit updates MODEL_TYPES_REQUIRING_POSITION_IDS in optimum/exporters/onnx/utils.py to include "stablelm-epoch". This ensures that position IDs are properly handled during ONNX export for StableLM-Epoch models, aligning with other models requiring this information.
- Adds StableLmOnnxConfig for exporting StableLM models to ONNX. - Updates TasksManager to use StableLmOnnxConfig for "stablelm" task. - Updates NormalizedConfigManager to recognize "stablelm". - Requires Transformers version >= 4.37.99 for StableLM export. This commit partially enables exporting StableLM models to ONNX, allowing them to be used in various inference pipelines.
- Replaces "stablelm-epoch" with "stablelm" in MODEL_TYPES_REQUIRING_POSITION_IDS. This commit reflects the updated model name "stablelm" for consistency with other parts of the codebase.
- Adds "stablelm": "gpt2" mapping to ORTConfigManager. This commit specifies that StableLM models should be treated similarly to GPT-2 models during ONNX Runtime inference for consistency and potentially improved optimization.
kazssym
commented
Feb 27, 2024
|
@kazssym let me know if you'd like a review. |
4 tasks
2 tasks
|
Can confirm that this PR works for transformers.js models 👍 @kazssym Just a reminder: you can add https://huggingface.co/hf-internal-testing/tiny-random-StableLmForCausalLM as a unit test. |
This commit sets the default ONNX opset version to 13 for StableLM models exported with `optimum.onnx.export`. This change ensures compatibility with a wider range of ONNX runtimes that might not support the latest opsets.
…into feature/stablelm
This commit adds the StableLM model `"hf-internal-testing/tiny-random-StableLmForCausalLM"` to the dictionaries of models for testing PyTorch ONNX export: * `PYTORCH_EXPORT_MODELS_TINY` * `PYTORCH_EXPORT_MODELS_LARGE` This allows for including StableLM in regression tests to ensure compatibility with the updated ONNX exporter.
kazssym
force-pushed
the
feature/stablelm
branch
from
142a558
to
ebbf9ba
Compare
…into feature/stablelm
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This pull request tries to add support for the StableLM model. It is still incomplete.
Fixes # (issue)
Before submitting