|> GenAI

Compose model conversations in Elixir

Chat, stream, call tools and classify conversation state through a shared thread API. Fork independent branches, then compare or combine their results explicitly.

Forks, classifiers and native DashScope are available in merged source. Install the source preview or choose the published Hex v0.5.0 package.

GenAI.chat(:session)
|> GenAI.with_model(GenAI.Provider.Anthropic.Models.claude_sonnet())
|> GenAI.with_message(GenAI.Message.user("Explain quantum computing simply."))
|> GenAI.run()

Documentation GitHub ♥ Donate

Provider-specific APIs. Shared thread protocols.

Anthropic OpenAI Google Gemini Mistral Groq xAI DeepSeek Cerebras Qwen / DashScope TypeSafe Jev OpenRouter ZAI LiteLLM Ollama ex_llama + Your own

Why GenAI

|> Automatic format mapping

Choose a model with its own encoder: OpenAI-compatible, Anthropic Messages or native Alibaba DashScope. Re-encode supported messages and tools for the selected API; model capabilities still determine which options are accepted.

|> Bounded tool loops

Register application tools once, then use run_with_tools to execute calls, append correlated results and continue inference. Set iteration and call limits per branch, with shared fork concurrency bounds.

|> Media generation

Text-to-image, speech synthesis and transcription through GenAI.Media.Request and generate_media/1. The media router selects a provider by its declared capabilities.

|> Provider-agnostic threads

Conversation threads are provider-independent graph structures. Swap with_model/2 and re-run — the full message history re-encodes for the new provider automatically.

|> Local inference

Connect Ollama for local chat, or install ExLLama separately for text-only Jev-Omni classification through llama.cpp. Native build tools, a compatible GGUF and the exported decision head are separate requirements.

|> OTP-native concurrency

Forks are conversation threads. Run or stream every leaf with a global concurrency limit, keep each branch's settings and history, and identify results and stream events by their full branch path.

|> Extract, aggregate, transform

Extract the last message, JSON key/index paths or fenced blocks; aggregate across a conversation; or request validated structured extraction from a chat model. Transform the continuing thread explicitly.

|> Typed classifier questions

Ask boolean, ordered-choice or numeric-rubric questions with GenAI.classify. Keep selected probabilities, provider confidence and expected scores distinct. Classify directly or inside a conversation pipeline.

|> Hosted and local decisions

Use TypeSafe's hosted Jev, Alibaba's decision-model-preview through System One, or the independent akhilaaa3/Jev-Omni checkpoint through ExLLama. These integrations currently execute text requests.

In practice — latest merged source

The examples below use features merged to develop. They are not yet a new Hex release; use the pinned source install below.

Choose the API style explicitly

alias GenAI.Provider.{Anthropic, Qwen}

thread =
  GenAI.chat(:session)
  |> GenAI.with_message(GenAI.Message.user("Summarize this ticket."))

# Configure credentials for the provider you choose.
GenAI.with_model(thread, Anthropic.Models.claude_sonnet())
GenAI.with_model(thread, Qwen.Models.model("qwen-plus"))
GenAI.with_model(thread, Qwen.Models.native("qwen-plus"))
# Native DashScope additionally needs a matching native_base_url.

Forks keep each conversation state and return ordered outcomes

thread =
  GenAI.chat(:session)
  |> GenAI.with_model(GenAI.Provider.Qwen.Models.model("qwen-plus"))
  |> GenAI.with_message(GenAI.Message.user("Suggest a clear refund policy."))

fork = GenAI.Pipeline.split(thread, 3)
{:ok, {results, updated_fork}} =
  GenAI.run(fork, nil, max_concurrency: 2)

{:ok, completion} = GenAI.Thread.Fork.Result.fetch(results, [0])
# Join or select branches explicitly; updated_fork keeps all continuations.

Extract conversation state, classify it and insert the result

alias GenAI.Classifier.Question
alias GenAI.Pipeline

ticket =
  GenAI.chat(:session)
  |> GenAI.with_message(GenAI.Message.user("I was charged twice."))

# Set JEV_API_KEY for TypeSafe's hosted classifier.
{:ok, pipeline} =
  ticket
  |> Pipeline.extract(:last_message)
  |> Pipeline.classify(
    model: GenAI.Provider.Jev.Models.latest(),
    questions: [%Question{id: "route", choices: ["billing", "technical"]}]
  )
  |> Pipeline.append_result(final: :json)
# The result is a correlated tool exchange plus a JSON assistant message.

Stream or run bounded tool loops on the same fork

{:ok, {streams, fork}} =
  GenAI.stream(updated_fork, nil, stream_sink: self())
# PID sinks receive {:genai, {:branch, path, event}}.

# registry is your configured GenAI.Tool.Registry.
{:ok, {tools, fork}} =
  GenAI.run_with_tools(fork, registry, nil,
    max_iterations: 4, max_tool_calls: 8, max_concurrency: 2)

{:ok, %{completion: completion, summary: summary}} =
  GenAI.Thread.Fork.Result.fetch(tools, [0])
# Partial failures return Fork.Error with outcomes and the updated fork.

Conversation processing, with explicit boundaries

Use Pipeline.aggregate to collect conversation payloads as lists, text, JSON or YAML; use structured extraction to prepare payloads and grading criteria. Joins combine, append or select branches explicitly. GenAI.run performs one inference; Pipeline.next_turn appends the assistant response. See the fork guide and classifier pipeline guide.

Get started with the latest source

The latest published package is GenAI v0.5.0: {:genai, "~> 0.5.0"} (GenAI Core 0.4.0 is on Hex and installs transitively). The fork, classifier and native DashScope examples on this page are part of the published release; the pinned Git source below remains the source-install alternative.

mix.exs — pinned source

defp deps do
  [
    {:genai_core,
     git: "https://github.com/noizu-labs-ml/genai-core.git",
     ref: "94a9263b6eeeecaee2d93bbd5d9f1ea9440afa17", override: true},
    {:genai,
     git: "https://github.com/noizu-labs-ml/genai.git",
     ref: "ced934a41781e5306132ee84a4a3cb7272a8b281"}
  ]
end

config/config.exs — chosen providers only

config :genai, :anthropic,
  api_key: System.get_env("ANTHROPIC_API_KEY")

config :genai, :qwen,
  api_key: System.get_env("QWEN_API_KEY"),
  native_base_url: System.get_env("QWEN_NATIVE_BASE_URL")

config :genai, :jev,
  api_key: System.get_env("JEV_API_KEY")

Native DashScope needs the native /api/v1 base for your workspace and region; OpenAI-compatible generation and System One classification use separate endpoints. Alibaba classification needs QWEN_DECISION_BASE_URL. Local Jev-Omni additionally needs ExLLama, a compatible GGUF and its exported head. If GenAI and ExLLama are sibling dependencies, keep the direct Core override above. Classifier setup and limits.

Native DashScope guide → Published API docs → Source on GitHub →