Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
-
Updated
May 17, 2023 - Python
Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.
Add a description, image, and links to the dynamic-batching topic page so that developers can more easily learn about it.
To associate your repository with the dynamic-batching topic, visit your repo's landing page and select "manage topics."