by Antoine Lemor · GitHub
🚀 About

High-performance inference API for transformer models trained with LLM Tool. Deploy your classification, generation, and embedding models with automatic resource management, concurrent request handling, and Ollama integration for generative AI.

📊 Text Classification
🤖 Ollama Integration
⚡ GPU Acceleration
🔒 API Key Auth
📦 LLM Tool Ecosystem
Core Package
LLM Tool
Core framework for training transformer models. Fine-tune classification, generation, and embedding models with ease.
Core
infer-client
Python SDK with Rich CLI for interactive classification and batch processing.
Python
inferclientllmtool
R package for data science workflows with DataFrame support.
R
infer-api
FastAPI server with multi-model support and resource monitoring.
API
🤖 Available Models
Loading models...
🏷️ Annotation Tool

Doccano — open-source annotation tool for text classification, sequence labeling, and sequence-to-sequence tasks.

Open Doccano
Requires a separate Doccano account
🧪 Playground
🔐
Test the LLM Tool inference API
Your API key is stored locally and never sent to third parties
API Online
Version 2.1.0 • Endpoints: /health, /models, /infer
Part of the LLM Tool ecosystem