You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Your PowerInfer is an amazing work to achieve great performance! Inspired by your brilliant ideas, I myself am thinking about development new features based on llama.cpp.
However, it is a bit hard for me to fully understand the structure of llama.cpp. As you guys have the experience of developing PowerInfer, im sincerely asking for your help:
is there any docs or videos suitable for a beginner to understand the whole structure llama.cpp? (even your own understanding would be helpful! )
could you share some tips for development based on llama.cpp?
I would be really grateful if you can give me a helping hand. Thanks in advance!
The text was updated successfully, but these errors were encountered:
Thank you for your interest in PowerInfer and we are more than happy to inspire more people!
The code structure of PowerInfer is consistent with that of llama.cpp, including aspects such as organizing the computation graph, external I/O (in llama.cpp), different operator implementations (ggml.c, ggml-cuda.cu, etc.), specific sub-function implementations (ggml-alloc.c), and high-level applications (under examples/). Therefore, I recommend focusing on understanding the architecture of llama.cpp.
Unfortunately, llama.cpp itself doesn't have extensive documentation, let alone textual or video tutorials. If you are keen to learn, you might find this community discussion helpful. This is similar to how we onboard new collaborators in our team, through collaborative learning and discussions.
Your PowerInfer is an amazing work to achieve great performance! Inspired by your brilliant ideas, I myself am thinking about development new features based on
llama.cpp
.However, it is a bit hard for me to fully understand the structure of
llama.cpp
. As you guys have the experience of developing PowerInfer, im sincerely asking for your help:llama.cpp
? (even your own understanding would be helpful! )llama.cpp
?I would be really grateful if you can give me a helping hand. Thanks in advance!
The text was updated successfully, but these errors were encountered: