OpenAI

InferXgate provides full support for OpenAI’s GPT models.

Configuration

OPENAI_API_KEY=sk-...

# Optional: Organization ID
OPENAI_ORG_ID=org-...

Available Models

Model ID	Description	Context Window
gpt-4-turbo	Latest GPT-4 Turbo	128K
gpt-4	GPT-4 base	8K
gpt-4-32k	GPT-4 extended context	32K
gpt-3.5-turbo	Fast and affordable	16K

Usage Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3000/v1",
    api_key="your-api-key"
)

response = client.chat.completions.create(
    model="gpt-4-turbo",
    messages=[
        {"role": "user", "content": "What is machine learning?"}
    ]
)

print(response.choices[0].message.content)

Function Calling

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-4-turbo",
    messages=[{"role": "user", "content": "Weather in Tokyo?"}],
    tools=tools
)

JSON Mode

response = client.chat.completions.create(
    model="gpt-4-turbo",
    messages=[{"role": "user", "content": "List 3 colors as JSON"}],
    response_format={"type": "json_object"}
)

Supported Features

• Chat completions
• Streaming
• Function/tool calling
• JSON mode
• Vision (GPT-4 Vision)
• Embeddings

Pricing

Model	Input (per 1M tokens)	Output (per 1M tokens)
GPT-4 Turbo	$10.00	$30.00
GPT-4	$30.00	$60.00
GPT-3.5 Turbo	$0.50	$1.50