Skip to content

EdgeInfer enables efficient edge intelligence by running small AI models, including embeddings and OnnxModels, on resource-constrained devices like Android, iOS, or MCUs for real-time decision-making. EdgeInfer 旨在资源受限的设备上运行小型 AI 模型(包括向量化和 Onnx 模型),如 Android、iOS 或 MCUs,实现高效的边缘智能,用于实时决策。

License

Notifications You must be signed in to change notification settings

unit-mesh/edge-infer

Repository files navigation

logo

Edge Infer

Build Crates.io

EdgeInfer enables efficient edge intelligence by running small AI models, including embeddings and OnnxModels, on resource-constrained devices like Android, iOS, or MCUs for real-time decision-making.

Architecture:

EdgeMind Architecture

Platform support (by Design):

  • Android, iOS
  • Linux, Windows, Mac,
  • Raspberry Pi, MCU

Todos

  • Inference wrapper
    • Onnx Runtime
    • Tokenizer
  • UniFFI, is a toolkit for building cross-platform software components in Rust.
  • GRPC server with tonic
  • Multiple OS support:
    • Desktop: Windows, Mac, Linux (x86, x64)
    • Mobile: Android, iOS, Linux (ARM)
    • Embedded Linux (ARM).
  • Flexible Configuration: Easily configurable via command-line parameters, including listening port, batch size, thread count, and others.

Usecases

Resources

Examples:

  • Modern cross-platform telemetry: Glean

MCU

ToSpike

Not working:

License

This project is licensed under the MIT License, See LICENSE for the full license text.

About

EdgeInfer enables efficient edge intelligence by running small AI models, including embeddings and OnnxModels, on resource-constrained devices like Android, iOS, or MCUs for real-time decision-making. EdgeInfer 旨在资源受限的设备上运行小型 AI 模型(包括向量化和 Onnx 模型),如 Android、iOS 或 MCUs,实现高效的边缘智能,用于实时决策。

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published