-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cannot use RWKV models #121
Comments
I just learned that RWKV-5 models are not yet supported by rwkv.cpp. So I tried RWKV-4 instead - took the .pth model and converted it to .bin following the docs. Unfortunately, however, the result is the same:
Using the same model with |
hi there, could you please check your ggml version, it my not work if you are using the recent ggml version |
how do I check my GGML version? I'm using the current version of rwkv.cpp |
I just found a section in th rwkv.cpp README.md which says:
may this be the reason for misbehaviour? |
FYI: I just used the version of rwkv.cpp from from Sept, 20th (before they restructured the Python API) and tried again - with the same results. Which means: no, the API restructuring is not the reason for not loading the RWKV model |
FYI: going back to the latest commit (of rwkv.cpp) before "update ggml" fails because the resulting code can not be compiled. Thus, in order to test if "llama-node" does work with RWKV actually means to go back to commit "update ggml" (8db73b1) and manually revert any changes related to GGML Damn... Not being a C++ developer, I have to give up here - I'll mention this problem in rwkv.cpp as well (see issue 144), let's see who will be able to fix it |
Hi! The module |
If it helps debugging, for some reason
|
That was quick - thank you very much. Unfortunately, I cannot get rwkv.cpp @ 363dfb1 to compile. Unless I manage to find out why, I may have to wait for RWKV-5 support. Nevertheless, thank you very much for your effort! |
FYI: I managed to compile rwkv.cpp again - my mistake was to only Now I'm trying to use it - a first attempt with the current version of llama-node failed with the same error message as before. Let's see what the detail |
Ok, I think I have to give up - now RWKV crashes with
I installed llama-node using
which seems to be wrong anyway as the RWKV inference example refers to a file ( |
@rozek I think this is because of your rwkv model was quantified by wrong version of rwkv.cpp, you may have last try on quantify the model file with rwkv.cpp @ 363dfb1. |
FYI: only rwkv-4-raven been tested |
Well, I meanwhile used rwkv.cpp @ 363dfb1 with ggml @ 00b49ec, as mentioned above. But, as described before
|
@rozek I used https://drive.google.com/file/d/1JyUmwZ9npQJDdYaAd4XKBfTNwUQpQXn9/view?usp=sharing before, it should work well |
and been tested in "llama-node": "^0.1.6", lol maybe too old |
I just tried to use the current version of "llama-node" with the "rwkv.cpp" backend and failed.
The link found in the docs where I should be able to download RWKV models points to nowhere.
Since I could not find pre-quantized models anywhere, I followed the instructions found in the rwkv.cpp repo to download, convert and quantize the 1.5B and 0.1B models - I even uploaded them to HuggingFace.
Then, I copied the example found in your docs added a path to my quantized model, changed the
template
and tried to run the result.Unfortunately, I got nothing but an error message:
Do you have any idea what could be wrong?
The text was updated successfully, but these errors were encountered: