AI Providers

Kairo supports multiple AI providers through specialized packages. These packages implement the LanguageModel and Provider interfaces defined in @kairo/core.

Available Providers

OpenAI (`@kairo/provider-openai`)

This provider enables communication with OpenAI's API.

Supported Models: GPT-4o, GPT-4, GPT-3.5-Turbo, etc.
Capabilities: Tool calling, Streaming, System messages, Vision (model dependent)

Configuration:

const provider = new OpenAIProvider({
  apiKey: process.env.OPENAI_API_KEY,
  organization: "org-...", // Optional
});

Azure OpenAI (`@kairo/provider-azure`)

Connect to Azure's OpenAI service instance.

Configuration:

const provider = new AzureOpenAIProvider({
  apiKey: process.env.AZURE_API_KEY,
  resourceName: "my-resource",
  deploymentName: "my-deployment",
  apiVersion: "2024-05-01-preview",
});

Architecture: How Providers Work

Kairo's provider architecture abstracts the underlying complexity of different LLM networks to offer a unified pipeline. To integrate a new AI service, you need to understand two key interfaces defined in @kairo/core:

Provider: Acts as a factory and configuration registry. It instantiates the connection pool/clients and advertises which models it supports.
LanguageModel: Represents a specific model instance (e.g., gpt-4o vs gpt-3.5). It holds the actual generation logic, declares what capabilities it has (like tool calling or streaming), and maps Kairo's unified formatting to the vendor's specific REST/RPC payloads.

The Request Lifecycle

When LmPipeline runs, it:

Validates its context
Extracts generation options (messages, tools, stop sequences)
Calls the LanguageModel.generate(options) method
Consumes the returned ReadableStream<LanguageModelStreamPart>

Implementing a Custom Provider

To create a new text generation provider (for example, for a local Ollama server or a proprietary REST API), implement both interfaces as shown below.

1. The LanguageModel Implementation

The LanguageModel must return a WHATWG ReadableStream yielding LanguageModelStreamPart chunks (text deltas, tool calls, usage data).

import {
  LanguageModel,
  LanguageModelCapability,
  LanguageModelOptions,
  LanguageModelStreamPart,
} from "@kairo/core";

export class CustomLanguageModel implements LanguageModel {
  readonly type = "language-model";
  // Declare supported capabilities (e.g., streaming, tool calling)
  readonly capabilities = new Set<LanguageModelCapability>(["streaming"]);

  constructor(
    public readonly id: string,
    public readonly name: string,
    public readonly maxOutputTokens: number,
    public readonly maxContextLength: number,
    public readonly provider: CustomProvider,
  ) {}

  async generate(
    options: LanguageModelOptions,
  ): Promise<ReadableStream<LanguageModelStreamPart>> {
    // 1. Transform Kairo's universal 'options.messages' to the vendor's API format
    const vendorMessages = options.messages.map((msg) => ({
      role: msg.role,
      content: msg.content,
    }));

    // 2. Make the HTTP request
    const response = await fetch(`${this.provider.baseUrl}/v1/chat`, {
      method: "POST",
      headers: { Authorization: `Bearer ${this.provider.apiKey}` },
      body: JSON.stringify({
        model: this.id,
        messages: vendorMessages,
        stream: true, // Requested stream
      }),
    });

    if (!response.ok) throw new Error("API Request Failed");

    // 3. Return a ReadableStream parsing the vendor's raw stream chunks
    return new ReadableStream<LanguageModelStreamPart>({
      async start(controller) {
        // Imagine a helper that parses SSE (Server-Sent Events) from the raw stream
        const reader = response.body.getReader();
        while (true) {
          const { done, value } = await reader.read();
          if (done) break;

          // Map the vendor chunk to Kairo's LanguageModelStreamPart shape
          const textDelta = parseVendorChunk(value);
          if (textDelta) {
            controller.enqueue({
              type: "text-delta",
              textDelta: textDelta,
            });
          }
        }

        // Always yield usage metrics and the finish reason at the end!
        controller.enqueue({
          type: "finish",
          finishReason: "stop",
          usage: { promptTokens: 10, completionTokens: 5 }, // Map from vendor response
        });

        controller.close();
      },
    });
  }
}

2. The Provider Factory

The Provider itself is simple. It holds the API configuration and instances of LanguageModel.

import { Provider } from "@kairo/core";

export class CustomProvider implements Provider {
  readonly name = "MyCustomProvider";
  readonly version = "1.0.0";
  readonly models: CustomLanguageModel[];

  constructor(
    public baseUrl: string,
    public apiKey: string,
  ) {
    // Pre-initialize the models supported by this provider
    this.models = [
      new CustomLanguageModel(
        "custom-model-fast",
        "Custom Fast",
        4096,
        32000,
        this,
      ),
      new CustomLanguageModel(
        "custom-model-pro",
        "Custom Pro",
        8192,
        128000,
        this,
      ),
    ];
  }

  // Optional: A helper to quickly select a model by ID
  getModel(id: string): CustomLanguageModel {
    const model = this.models.find((m) => m.id === id);
    if (!model) throw new Error(`Model ${id} not found`);
    return model;
  }
}

3. Using it in Kairo

Now you can mount your custom provider straight into an LmPipeline!

import { LmPipeline } from "@kairo/core";

const myProvider = new CustomProvider("http://localhost:8080", "secret-key");

const pipeline = new LmPipeline({
  model: myProvider.getModel("custom-model-fast"),
});

Fast Track: OpenAI-Compatible Providers

Often you don't need to write a custom LanguageModel string parser from scratch! The industry has largely converged on OpenAI's API format. For providers that are completely or mostly API compatible with OpenAI (such as DeepSeek, Groq, local vLLM, or Azure), Kairo provides highly reusable building blocks in @kairo/provider-openai.

You can significantly reduce your development effort by directly importing and wrapping OpenaiLanguageModelApiClient and OpenaiLanguageModel.

Reusing OpenAI's Implementation

Here is an example showing how you only need to write a Provider factory that configures a custom URL endpoint and authentication headers, while letting the openai package handle all the heavy stream parsing lifting:

import type { Provider } from "@kairo/core";
import {
  OpenaiLanguageModel,
  OpenaiLanguageModelApiClient,
} from "@kairo/provider-openai";

export class CompatibleProvider implements Provider {
  readonly name = "acme-corp";
  readonly version = "1.0.0";
  readonly models: OpenaiLanguageModel[];

  constructor(apiKey: string) {
    // 1. Initialize the shared OpenAI API client with your custom endpoint mapping
    const client = new OpenaiLanguageModelApiClient((path, init) => {
      const headers = new Headers(init.headers);
      headers.set("Authorization", `Bearer ${apiKey}`);

      // Override the OpenAI baseUrl with your provider's URL
      const url = new URL(`https://api.acmecorp.dev/v1/chat${path}`);

      return fetch(url.toString(), { ...init, headers });
    });

    // 2. Delegate the model generation implementation entirely to OpenaiLanguageModel
    this.models = [
      new OpenaiLanguageModel(client, "acme-fast-1", this as Provider),
      new OpenaiLanguageModel(client, "acme-pro-2", this as Provider),
    ];
  }
}

This exact pattern is how the @kairo/provider-azure package in the Kairo codebase is built. Check its source code for a real-world example of extending the OpenAI provider.

Providers

On this page