Calling ggml from a Python script

[StellaAthena]

C n00b here, but I was wondering how nicely this library plays with Python. I have some existing codebases that are written in Python and would like make ggml an optional backend for people who don’t have GPUs.

Specifically, the main functionality I would like to be able to call would be:

1. Computing a continuation based on an input prompt
2. Computing the loglikelihood of a specific target continuation based on an input prompt

How easy is this to do?

[pikalover6]

Some things that might help you out:
gpt-j python bindings
llama python bindings
similar issue
You could look around in kobold.cpp to see how they deal with it