Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to resume adapter training from an existing adapter weights #3983

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

amankhandelia
Copy link
Contributor

Minor Changes to enable continue training a previously trained adapter. This should resolve #3833

@amankhandelia
Copy link
Contributor Author

amankhandelia commented Apr 4, 2024

@arnavgarg1 I have made these changes to resume training on already trained adapter (using safetensors weights and not the checkpoint) which works for me, now the question I have for you is, will this take care of the TODO mentioned here, if not, does this PR sits well with current arch design ethos of the project, if this PR is acceptable, I will go ahead, and add any tests as required for this PR

Copy link

github-actions bot commented May 3, 2024

Unit Test Results

  5 files   -   1    5 suites   - 1   11m 32s ⏱️ - 5m 35s
12 tests ±  0    7 ✔️  -   2    5 💤 +2  0 ±0 
48 runs   - 12  23 ✔️  - 19  25 💤 +7  0 ±0 

Results for commit 6fc53d6. ± Comparison against base commit b6df715.

This pull request skips 2 tests.
tests.regression_tests.benchmark.test_model_performance ‑ test_performance[ames_housing.gbm.yaml]
tests.regression_tests.benchmark.test_model_performance ‑ test_performance[mercedes_benz_greener.gbm.yaml]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-training PEFT model fails after loading with Linear4bit error
1 participant