Installation

Different ways to install and deploy InferXgate for your environment.

InferXgate can be deployed in several ways depending on your needs. Choose the method that best fits your infrastructure.

The easiest way to run InferXgate is with Docker Compose.

Prerequisites

  • Docker 20.10+
  • Docker Compose 2.0+

Quick Install

# Clone the repository
git clone https://github.com/jasmedia/inferxgate.git
cd inferxgate

# Copy environment file
cp .env.example .env
# Edit .env with your API keys

# Start all services
docker-compose up -d

Docker Compose Services

The default docker-compose.yml includes:

ServicePortDescription
inferxgate3000Main API gateway
dashboard80React dashboard
db5432PostgreSQL database
redis6379Redis cache

Custom Docker Compose

For production, you may want to customize the compose file:

version: '3.8'

services:
  inferxgate:
    image: ghcr.io/jasmedia/inferxgate:latest
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/inferxgate
      - REDIS_URL=redis://redis:6379
      - JWT_SECRET=${JWT_SECRET}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    depends_on:
      - db
      - redis
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
      - POSTGRES_DB=inferxgate
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:

Build from Source

For development or custom builds.

Prerequisites

  • Rust 1.75+ (install via rustup)
  • PostgreSQL 14+
  • Redis 7+

Build Steps

# Clone the repository
git clone https://github.com/jasmedia/inferxgate.git
cd inferxgate

# Build release binary
cargo build --release

# Binary is at ./target/release/inferxgate

Development Mode

# Run with hot reloading
cargo watch -x run

# Run tests
cargo test

# Run with debug logging
RUST_LOG=debug cargo run

Kubernetes (Coming Soon)

Helm chart for large-scale Kubernetes deployments is currently in development. Stay tuned for updates.

Verifying Installation

After installation, verify everything is working:

# Health check
curl http://localhost:3000/health

# List available models
curl http://localhost:3000/v1/models

# Test a completion
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-3-haiku-20240307", "messages": [{"role": "user", "content": "Hi"}]}'

Next Steps