Providers

Every provider in infy implements the same ChatModel protocol, so you can swap one for another without touching the rest of your code. A model built from any provider drops into the same chains, agents, and graphs.

The ChatModel protocol

Each provider class exposes an identical surface:

Method	Purpose
`generate`	Synchronous completion.
`agenerate`	Asynchronous completion.
`stream`	Synchronous streaming.
`astream`	Asynchronous streaming.
`bind_tools`	Attach tools for tool calling.
`with_structured_output`	Return a JSON-schema dict or a pydantic model.

Because the surface is shared, providers are interchangeable. Write your chain, agent, or graph once, then point it at whichever model you want.

from infy import HumanMessage
from infy.providers.openai import OpenAIChat

model = OpenAIChat("gpt-4o")

reply = model.generate([HumanMessage(content="Summarize bulk-synchronous parallelism in one sentence.")])
print(reply.text)

Available providers

Provider	Class	Extra	Notes
OpenAI	`infy.providers.openai.OpenAIChat`	`infy[openai]`	Chat, tools, structured output.
Anthropic	`infy.providers.anthropic.AnthropicChat`	`infy[anthropic]`	Chat, tools, structured output.
Google Gemini	`infy.providers.gemini.GeminiChat`	`infy[gemini]`	AI Studio and Vertex (`vertexai=True`).
Ollama	`infy.providers.ollama.OllamaChat`	`infy[ollama]`	Local models.

Provider SDKs are optional extras. The infy core has no third-party runtime dependencies, so you install only the providers you use.

Installing a provider

Pick the extra that matches the provider you want.

pip install "infy[openai]"

pip install "infy[anthropic]"

pip install "infy[gemini]"

pip install "infy[ollama]"

pip install "infy[all]"

Every provider needs Python 3.10 or later.

Provider notes

OpenAI
Anthropic
Gemini
Ollama

Install infy[openai], then construct OpenAIChat with a model name.

from infy.providers.openai import OpenAIChat

model = OpenAIChat("gpt-4o")

Supports chat, tool calling, and structured output.

Install infy[anthropic], then construct AnthropicChat with a model name.

from infy.providers.anthropic import AnthropicChat

model = AnthropicChat("claude-sonnet-4-5")

Supports chat, tool calling, and structured output.

Install infy[gemini], then construct GeminiChat. The same class targets both Google AI Studio and Vertex AI.

from infy.providers.gemini import GeminiChat

# Google AI Studio (default)
model = GeminiChat("gemini-1.5-pro")

# Vertex AI
vertex_model = GeminiChat("gemini-1.5-pro", vertexai=True)

Pass vertexai=True to route through Vertex.

Install infy[ollama], then construct OllamaChat to run local models.

from infy.providers.ollama import OllamaChat

model = OllamaChat("llama3")

Streaming

The same model instance streams synchronously or asynchronously.

async for chunk in model.astream([HumanMessage(content="Stream me a haiku about schedulers.")]):
    print(chunk.content, end="", flush=True)

Structured output

with_structured_output accepts either a JSON-schema dict (parsed by the Rust JsonParser, no validation) or a pydantic model (parsed and validated by pydantic-core). The return type follows what you pass in.

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

model.with_structured_output(Person).generate([HumanMessage(content="Maria Chen is 42.")])
# Person(name='Maria Chen', age=42)

Validated structured output needs pydantic. Install it with pip install "infy[pydantic]".

Where to go next

Tools and agents

Build tool-calling agents with create_agent on any provider.

Governance

Policy-check every tool call and write a tamper-evident audit.

Getting Started

Core concepts

Graph runtime

Governance

Integrations

Reference

The ChatModel protocol

Available providers

Installing a provider

Provider notes

Streaming

Structured output

Where to go next

Tools and agents

Governance

​The ChatModel protocol

​Available providers

​Installing a provider

​Provider notes

​Streaming

​Structured output

​Where to go next

Tools and agents

Governance

The ChatModel protocol

Available providers

Installing a provider

Provider notes

Streaming

Structured output

Where to go next