FastAPI-BitNet

FastAPI-BitNet provides a robust REST API for managing and interacting with `llama.cpp`-based BitNet model instances. Designed for developers and researchers, it allows programmatic control of automated testing, benchmarking, and interactive chat sessions. Key features include session management, batch operations, and model benchmarking.

GitHub Website Docs

GitHub Stars

User Rating

Not Rated

Favorites

Views

Forks

Issues

Technical Information

Programming Languages

PythonPrimary Language

System Requirements

Docker Desktop: Latest version

Conda: Latest version (or another Python environment manager)

Python: 3.10+

Provided Features

セッション管理: 複数の`llama-cli`および`llama-server`セッションを開始、停止、状態確認

バッチ操作: 一度のAPI呼び出しで複数のインスタンスを初期化、シャットダウン、チャット

インタラクティブチャット: 実行中のBitNetセッションにプロンプトを送信し、クリーンなモデル応答を受信

モデルベンチマーキング: GGUFモデルのベンチマークをプログラム的に実行し、パープレキシティを計算

リソース推定: 利用可能なシステムRAMとCPUスレッドに基づいて最大サーバー容量を推定

VS Code統合: Model Context Protocolを介してGitHub Copilot Chatに直接接続

自動APIドキュメント: Swagger UIおよびReDocによるインタラクティブなAPIドキュメント

Safety Analysis

Safety Score

Safe100/100

Recommended For

Recommended for developers and researchers interested in AI/LLM, particularly those involved in developing applications using FastAPI and Docker.

Maintenance Status

Active

GitHub Topics

1-bitbenchmarkingbitnetdockerfastapiinferencellmmodel-context-protocolmulti-chatperplexitypythonserver-orchestrationuvicorn

Author Information

Full stack developer

GitHub

Followers

Repositories

Gists

Total Contributions

Tags

1-bit benchmarking bitnet docker fastapi inference llm model-context-protocol multi-chat perplexity python server-orchestration uvicorn

Related MCPs

utcp-mcp

128

All-in-one MCP server that can connect your AI agents to any native endpoint, powered by UTCP

Python

magg

Magg is a Python library designed to aggregate Machine Control Protocol (MCP) data. It streamlines the collection and management of data, offering functionalities that support various MCPs. Through an easy-to-use API, users can effortlessly manipulate MCP data.

Python

AGENTIX

AGENTIX is an automation tool built with Python, designed to enable users to efficiently execute specific tasks. With an intuitive interface and powerful features, it is accessible even to users with limited programming knowledge. It offers a wide range of functionalities, including task automation, data processing, and API integration.

Python