Skip to main content
infy is a from-scratch runtime for LLM applications and agentic systems. The Python core has no third-party runtime dependencies. The hot paths (JSON parsing, similarity, tokenization) are accelerated by a compiled Rust extension that degrades gracefully to pure Python when it is absent. You get a complete chat-model abstraction, composable runnables, structured output, tool-calling agents, and a Pregel-style stateful graph executor with checkpointing and human-in-the-loop interrupts, sync and async throughout. It is LangGraph-compatible and Apache-2.0, open core.

Two things set it apart

Weight

infy targets the costs that show up in production: cold-start latency, memory footprint, and per-invocation overhead. Across a corpus of about 37 real agents it runs a median of 8.6x faster on cold start and 5.4x lighter on memory at line-of-code parity.

Control

An optional, in-process governance layer policy-checks every tool call and writes a tamper-evident audit. You can give an agent real authority (shell, deploys, money, customer data) safely, and prove what it did. The same governance wraps agents built on other frameworks.

Install

Requires Python 3.10+.
infy is alpha. Until the first tagged PyPI release, install from source. The model, runnable, structured-output, tool, agent, graph, governance, and integration APIs are stable and covered by the test suite. Treat minor releases as potentially breaking until 1.0.

A first agent

create_agent is a tight ReAct loop with tool calls executed in parallel by default. When you need control flow that branches, loops, persists, or pauses, reach for the graph runtime.

The pillars

Models

A single ChatModel protocol (generate, agenerate, stream, astream, bind_tools, with_structured_output) across OpenAI, Anthropic, Gemini / Vertex, and Ollama. Providers are interchangeable in chains, agents, and graphs.

Composition

Runnables compose with the | operator into a Sequence. Plain callables, dicts, and tools are coerced automatically. Every chain is both sync (invoke) and async (ainvoke).

Agents

create_agent gives you a ReAct loop with parallel tool execution, structured results, and optional governance wired straight into the tool chokepoint.

Graph

StateGraph compiles to a bulk-synchronous superstep executor with typed channels, reducers, conditional routing, dynamic fan-out, checkpointing, and interrupt or resume.

Governance

Deny-by-default policy, risk tiering, human approval (including durable, out-of-band approval), and a SHA-256 or HMAC hash-chained audit with verify(). About 50 microseconds per tool call.

Integrations

infy.integrations wraps smolagents, LangChain, and OpenHands agents with the same governance, without changing them.

Structured output

with_structured_output accepts either a JSON-schema dict (parsed by the Rust JsonParser, no validation) or a pydantic model (pydantic-core fused parse and validate). The return type follows the input.

Governance you can prove

Giving an agent real authority raises one question: how do you make that safe, and prove what it did? infy answers with an optional control plane wired into the agent loop. Omit it and nothing changes and nothing is imported. Opt in and every tool call is policy-checked in-process, and every step is written to a tamper-evident audit trail.
Governance is deny-by-default and fail-closed. Any error in policy, risk, or approval yields a deny, and is still audited. There is no path to a silent allow. Enforcement is in-process by design, so it adds microseconds, not a network hop.

Honest performance

Framework overhead is isolated by porting real LangChain and LangGraph projects to infy over a shared, deterministic, offline leaf, and verifying byte-identical output before any number is trusted. Only the orchestration differs, so the difference is the framework.
Per-invocation multiples are real, but they amortize into network latency once a live model call dominates the request, so end-to-end wall clock is at parity (measured at 1.05x to 1.14x on a live, billed run). What survives to production is cold start and memory footprint, paid on every request and per running agent. That is exactly why infy targets serverless, edge, and high-density multi-tenant deployments.

Next steps

Quickstart

Install infy, wire up a provider, and run your first agent.

Governance overview

Policy, risk tiering, human approval, durable approval, and the tamper-evident audit.

Integrations

Add deny-by-default policy and a verifiable audit to smolagents, LangChain, and OpenHands agents.