AI¶

{
    "id": "<string>",
    "customer_id": "<string>",
    "name": "<string>",
    "detail": "<string>",
    "type": "<string>",
    "is_insight_active": <boolean>,
    "engine_model": "<string>",
    "parameter": "<object>",
    "has_engine_key": <boolean>,
    "engine_base_url": "<string>",
    "rag_id": "<string>",
    "init_prompt": "<string>",
    "current_prompt_history_id": "<string>",
    "tts_type": "<string>",
    "tts_voice_id": "<string>",
    "stt_type": "<string>",
    "stt_language": "<string>",
    "vad_config": {
        "confidence": <number>,
        "start_secs": <number>,
        "stop_secs": <number>,
        "min_volume": <number>
    },
    "smart_turn_enabled": <boolean>,
    "auto_aicall_audit_enabled": <boolean>,
    "tool_names": ["<string>"],
    "mcp_server_ids": ["<string>"],
    "direct_hash": "<string>",
    "tm_create": "<string>",
    "tm_update": "<string>",
    "tm_delete": "<string>"
}

id (UUID): The AI configuration’s unique identifier. Returned when creating an AI via POST /ais or when listing AIs via GET /ais.
customer_id (UUID): The customer that owns this AI configuration. Obtained from the id field of GET /customers.
name (String, Required): A human-readable name for the AI configuration (e.g., "Sales Assistant").
detail (String, Optional): A description of the AI’s purpose or additional notes.
type (enum string, Optional): The AI’s operating mode. normal (default) is a general-purpose AI usable in calls, tasks, and conversations. insight limits the AI’s built-in tools to the Insight tool set (AllInsightToolNames), adds the tools of the MCP servers in its own mcp_server_ids, and uses a dedicated system prompt tailored for agent-facing Q&A over a contact-manager Case. See Type.
is_insight_active (Boolean): Whether this is the customer’s active Insight AI (the one the Case Insight Assistant panel auto-attaches to a Case). Only meaningful when type is insight; always false for type=normal AIs. A customer may keep any number of Insight AIs (e.g. to prepare a new prompt or model before switching to it), but at most one may be active at a time. Newly created Insight AIs are always inactive. Activate one with POST /ais/{id}/activate_insight. When a customer has no active Insight AI, the most recently created one is used. See Insight AI activation.
engine_model (String, Required): The LLM provider and model. Format: <provider>.<model> (e.g., openai.gpt-5-mini, gemini.gemini-2.5-flash). Use an id returned by GET /ai_models. See Engine Models and AI Models.
parameter (Object, Optional): Custom key-value parameter data for the AI configuration. Supports flow variable substitution at runtime. Typically left as {}.
engine_key (String, Required on POST /ais, optional on PUT /ais/{id}, request only): The API key for the LLM provider. Must be a valid key from the provider’s dashboard. For models whose platform_managed field in GET /ai_models is true (OpenRouter models and local models), the platform supplies the credentials: send an empty string, and any value you send is ignored. For a custom OpenRouter model (engine_model starts with openrouter.), your OpenRouter API key is required: when no key is stored or sent, the request is rejected with 400 ENGINE_KEY_REQUIRED. On PUT /ais/{id} the key has three states: omit the field (or send null) to keep the stored key, send an empty string to delete the stored key, or send a value to replace it. An omitted key is not kept when the provider prefix of engine_model (the part before the first dot) changes to a model whose key_mode is not platform: the stored key is deleted, so a key is never carried over to another provider. Moving to a platform model keeps the stored key, which is not used while that model is selected; moving on to a model with a different prefix that is not platform deletes it, and only an explicit empty string deletes it otherwise. The key is write-only: it is never returned by any response, event or webhook, so it does not appear in the structure above (see has_engine_key).
has_engine_key (Boolean): Whether the AI holds an engine_key. It does not describe engine_base_key of customendpoint. models. A key that is kept on a platform model also counts, although it is not used. The key itself is never returned.
engine_base_url (String, Optional): The base URL of an OpenAI-compatible language model server that you host. Required when engine_model starts with customendpoint. (400 ENGINE_BASE_URL_REQUIRED if empty), and not allowed for any other model (400 ENGINE_BASE_URL_NOT_ALLOWED). Only https is accepted. Do not put credentials or other secrets in the URL: the URL is not secret and is returned in API responses, events and webhooks. Send the key in engine_base_key instead. See Custom endpoint models.
engine_base_key (String, Optional, request only): The API key for the server at engine_base_url. Write-only: it is accepted on POST /ais and PUT /ais/{id} and is never returned by any response, event or webhook, so it does not appear in the structure above. See Custom endpoint models for what omitting the field, null, an empty string and a value mean.
rag_id (UUID, Optional): The knowledge base ID for the search_knowledge tool. Obtained from the id field of GET https://api.voipbin.net/v1.0/rags. When set, the AI assistant can search this knowledge base during voice calls. Set to 00000000-0000-0000-0000-000000000000 or omit to disable.
init_prompt (String, Required): The system prompt that defines the AI’s behavior, persona, and instructions. No enforced length limit.
current_prompt_history_id (string/UUID): UUID of the most-recent ai_ai_prompt_histories entry for this AI. Included in webhook events so callers can correlate each AI event with the exact prompt version that was active. Zero UUID (00000000-0000-0000-0000-000000000000) means no versioned prompt history has been recorded yet.
tts_type (enum string, Required): Text-to-Speech provider. See TTS Types.
tts_voice_id (String, Optional): Voice ID for the selected TTS provider. If omitted, the default voice for the chosen TTS type is used. See default voices in TTS Types.
stt_type (enum string, Required): Speech-to-Text provider. See STT Types.
stt_language (String, Optional): STT language in BCP-47 format (e.g., ko-KR, en-US). Controls which language the Speech-to-Text engine listens for. When set, the STT provider is configured to recognize this specific language, improving accuracy for non-English calls. Empty string or omitted means auto-detect (provider default).
vad_config (Object, Optional): Voice Activity Detection configuration. All fields are optional — omitted fields use Pipecat defaults. See VAD Config.
smart_turn_enabled (Boolean, Optional): Enable smart turn detection using Pipecat’s LocalSmartTurnAnalyzerV3 for more natural turn-taking. When true, the VAD stop_secs parameter is automatically forced to 0.2 regardless of vad_config settings. Defaults to false. See Smart Turn.
auto_aicall_audit_enabled (Boolean, Optional): When true, any AICall that finishes while using this AI configuration automatically triggers an AICall audit. Defaults to false (opt-in).
tool_names (Array of String, Optional): List of enabled tool functions. Use ["all"] to enable all tools, [] to disable all tools, or list specific tool names. For type=insight AIs, only Insight tool names are permitted (currently get_contact_interactions, get_conversation_content, get_related_cases, get_case_notes, get_contact_profile, get_call_transcript, emit_info_card, notify_agent, search_knowledge). Seven further Insight tools send, call or propose an SMS, an email or an outbound call: send_message (sends an SMS or MMS right away), send_email (sends an email right away from the platform’s default sender address), create_call (places an outbound call right away), draft_message (shows an SMS draft card), draft_email (shows an email draft card), draft_call (shows a call draft card) and describe_action (looks up the options of a flow action, for the actions of create_call and draft_call); for the three draft tools an agent approves or rejects the card and nothing is sent or called until the agent approves. They can be named in tool_names, and ["all"] is valid for Insight AIs and selects all sixteen tools: the nine above plus these seven. Because ["all"] includes send_message, send_email and create_call, an Insight AI that should only draft must list its tools by name and leave those three out (and name describe_action if it should assemble actions). For type=normal AIs, any Normal tool name or ["all"] is permitted; Insight-only tool names are rejected. Mismatched combinations return 400. See Tool Functions.
mcp_server_ids (Array of UUID, Optional): List of customer-owned MCP (Model Context Protocol) server IDs whose tools are made available to this AI, in addition to tool_names. Adding an ID requires an MCP Server that exists, is owned by the same customer, and has not been deleted; a cross-customer, nonexistent, or deleted ID is rejected with 400. Tool use scope: a whitelisted server’s tools are presented to this AI’s single-AI sessions, including those of type=insight AIs; team-typed AI calls never receive MCP tools regardless of this list. Sessions that do discover and advertise include chat, voice calls, ai_task flow actions and API-created single-AI sessions, so expect tools/list (and, on a tool call, tools/call) requests authenticated per the server’s auth_type in its logs. Of what is discovered, the tool names are stored on the AI session record and outlive the request, while the input schemas and descriptions are not stored; see MCP Server for retention. Tools discovered from a usable MCP server are namespaced as mcp_<first 8 hex chars of the server id>_<tool name> when presented to the LLM, up to 256 tools per session across this AI’s whitelisted servers. At call time a server is skipped silently, rather than failing the call, when its status is not active, when it has been deleted, or when it is no longer owned by this customer; a result the remote server itself reports as an error surfaces to the model as a failure, never a success. Deleting a server does not rewrite the AIs referencing it; see the note below. Defaults to [].
direct_hash (String): Hash for direct AI access. Empty string when direct access is disabled. When enabled, this hash forms the direct SIP URI: sip:direct.<hash>@sip.voipbin.net. Regenerate via POST /ais/{id}/direct-hash-regenerate.
tm_create (String, ISO 8601): Timestamp when the AI configuration was created.
tm_update (String, ISO 8601): Timestamp when the AI configuration was last updated.
tm_delete (String or null, ISO 8601): Timestamp when the AI configuration was deleted. null while the AI configuration has not been deleted.

Note

AI Implementation Hint

The engine_key field contains the LLM provider’s API key and is accepted on POST /ais and PUT /ais/{id}. It is a secret: do not log or store it, and use has_engine_key to find out whether a key is held. The key is never returned in responses, events or webhooks. To change the key, send the new value in a PUT. A PUT that omits engine_key keeps the stored key, and an empty string deletes it, so a client that sends back a full response body unchanged must drop engine_key or the empty value will delete the key.

Note

AI Implementation Hint

AI configurations do not use the 9999-01-01 00:00:00.000000 sentinel that some older VoIPBin resources use for “not yet occurred.” tm_update and tm_delete are null until the configuration is first updated or deleted, so test for null rather than comparing against a sentinel date.

Note

Deleting an MCP server an AI still references

Deleting an MCP server does not rewrite the AIs that reference it. The ID stays in mcp_server_ids so that saving an unrelated field on the AI keeps working, but the server contributes no tools from the moment it is deleted, and the ID is no longer offered in the admin UI’s server picker. To remove it, submit mcp_server_ids without that ID.

Example¶

{
    "id": "a092c5d9-632c-48d7-b70b-499f2ca084b1",
    "customer_id": "5e4a0680-804e-11ec-8477-2fea5968d85b",
    "name": "Sales Assistant AI",
    "detail": "AI assistant for handling sales inquiries",
    "type": "normal",
    "is_insight_active": false,
    "engine_model": "openai.gpt-5-mini",
    "parameter": {},
    "has_engine_key": true,
    "rag_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "init_prompt": "You are a friendly sales assistant. Help customers find the right products.",
    "tts_type": "elevenlabs",
    "tts_voice_id": "EXAVITQu4vr4xnSDxMaL",
    "stt_type": "deepgram",
    "stt_language": "en-US",
    "vad_config": {
        "stop_secs": 0.5
    },
    "smart_turn_enabled": true,
    "auto_aicall_audit_enabled": false,
    "tool_names": ["connect_call", "send_email", "stop_service"],
    "direct_hash": "",
    "tm_create": "2024-02-09 07:01:35.666687",
    "tm_update": "9999-01-01 00:00:00.000000",
    "tm_delete": "9999-01-01 00:00:00.000000"
}

Type¶

The type field controls the AI’s operating mode.

Type	Description
normal	General-purpose AI. Usable in calls, tasks, and conversations, with the tool set configured via `tool_names`. This is the default when `type` is omitted.
insight	Built-in tools restricted to the Insight tool set (`get_contact_interactions`, `get_conversation_content`, `get_related_cases`, `get_case_notes`, `get_contact_profile`, `get_call_transcript`, `emit_info_card`, `notify_agent`, `search_knowledge`) and a dedicated system prompt. Intended for agent-facing Q&A over a contact-manager Case (e.g., an agent asking the AI questions about a Case while working it in square-talk), not for handling live calls.

Type

Description

normal

General-purpose AI. Usable in calls, tasks, and conversations, with the tool set configured via tool_names. This is the default when type is omitted.

insight

Built-in tools restricted to the Insight tool set (get_contact_interactions, get_conversation_content, get_related_cases, get_case_notes, get_contact_profile, get_call_transcript, emit_info_card, notify_agent, search_knowledge) and a dedicated system prompt. Intended for agent-facing Q&A over a contact-manager Case (e.g., an agent asking the AI questions about a Case while working it in square-talk), not for handling live calls.

Note

AI Implementation Hint

Updating an AI via PUT /ais/{id} without sending type leaves the existing type unchanged — it does not reset the AI to normal. To change an AI’s type, explicitly send the desired type value in the PUT body.

Insight AI activation¶

A customer may keep multiple type=insight AI configurations (for example, to prepare a second Insight assistant with a different prompt, model, or tool set before switching to it), but at most one may be active at a time. The is_insight_active field marks which one.

The active Insight AI is the one the Case Insight Assistant panel auto-attaches when an agent opens a Case. Testing an Insight AI directly (by passing its assistance_id explicitly) works against any Insight AI regardless of its active status, so a draft configuration can be tried out without disturbing the one in production use.

Activating one

$ curl --location --request POST 'https://api.voipbin.net/v1.0/ais/a092c5d9-632c-48d7-b70b-499f2ca084b1/activate_insight?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'

{
    "id": "a092c5d9-632c-48d7-b70b-499f2ca084b1",
    "customer_id": "5e4a0680-804e-11ec-8477-2fea5968d85b",
    "name": "Case Insight Assistant v2",
    "type": "insight",
    "is_insight_active": true,
    ...
}

Activating an AI automatically deactivates whichever Insight AI was active before. There is no separate deactivate call. Activating the already-active AI succeeds and changes nothing.

Behavior rules

Event	Effect on `is_insight_active`
AI created (`POST /ais`)	Always `false`, even for the customer’s first Insight AI.
AI activated	Set to `true`; the previously active one is set to `false`.
AI deleted (`DELETE /ais/{id}`)	Cleared to `false`.
AI type changed away from `insight`	Cleared to `false`.
No Insight AI is active	The Case panel falls back to the most recently created Insight AI.

Errors

Status / reason	Cause
`400 AI_NOT_INSIGHT_TYPE`	The target AI is not `type=insight`. Only Insight AIs can be activated.
`400 INVALID_ENGINE_MODEL`	The `engine_model` is not returned by `GET /ai_models` (this includes a `local.` model that the operator has not registered), does not use an allowed provider prefix (`openai`, `gemini`, `grok`), and is not a valid `openrouter.<author>/<slug>` custom model. The OpenRouter model ID must have exactly one `/` and use only letters, digits, `.`, `_` and `-`. Variant suffixes such as `:free` and router IDs such as `openrouter/auto` are not supported. Returned on create and update.
`400 ENGINE_KEY_REQUIRED`	The `engine_model` is a custom OpenRouter model and no engine key is stored or sent (an omitted `engine_key` on update counts as the stored key). Returned on create and update, including an update that does not change the model.
`400 ENGINE_BASE_URL_REQUIRED`	The `engine_model` starts with `customendpoint.` and `engine_base_url` is empty. Returned on create and update, including a `PUT` from a client that does not send `engine_base_url` for an existing custom endpoint AI. Nothing is changed in that case.
`400 ENGINE_BASE_URL_NOT_ALLOWED`	`engine_base_url` is set but `engine_model` does not start with `customendpoint.`.
`400 INVALID_ENGINE_BASE_URL`	`engine_base_url` breaks the format rules (see Custom endpoint models). The message states the rules and does not repeat the value.
`400 ENGINE_BASE_URL_REJECTED`	The host could not be resolved to a public address. The message is the same for an unknown name, a name that resolves to a private, loopback or link-local address, and a lookup that failed or timed out. The same error is returned when the platform is busy with other address checks, so retry the request once before changing the address.
`400 INVALID_ENGINE_BASE_KEY`	`engine_base_key` is not 1 to 512 printable ASCII characters.
`400 ENGINE_BASE_KEY_NOT_ALLOWED`	`engine_base_key` is set but `engine_model` does not start with `customendpoint.`.
`404 AI_NOT_FOUND`	The target AI does not exist or has been deleted.
`409 AI_INSIGHT_ACTIVATION_CONFLICT`	Another activation for the same customer was in flight. Retry the request.

Engine Model¶

The engine_model field specifies which LLM provider and model to use. Format: <provider>.<model>.

The set of selectable models is a curated catalog. Retrieve it with GET /ai_models (see AI Models) and use the returned id as engine_model.

Supported Providers

Provider	Format	Examples	API key (`engine_key`)
OpenAI	`openai.<model>`	openai.gpt-5, openai.gpt-5-mini	Your own key
Google Gemini	`gemini.<model>`	gemini.gemini-2.5-flash	Your own key
xAI Grok	`grok.<model>`	grok.grok-3-mini	Your own key
Anthropic	`anthropic.<model>`	anthropic.claude-sonnet-4.5	Not required
Meta	`meta.<model>`	meta.llama-3.3-70b-instruct	Not required
DeepSeek	`deepseek.<model>`	deepseek.deepseek-v3.2	Not required
Qwen	`qwen.<model>`	qwen.qwen3-235b-a22b-2507	Not required
Mistral	`mistral.<model>`	mistral.mistral-small-3.2-24b-instruct	Not required

Models listed as “Not required” are managed by the platform (key_mode is platform in GET /ai_models). On create send an empty engine_key; on update you may omit it (a stored key is kept) or send an empty string (a stored key is deleted).

Local models

When the platform operator registers self-hosted models, they are listed in GET /ai_models with the format local.<endpoint>.<model> and key_mode platform. On create send an empty engine_key; on update you may omit it. The server address is not shown, and a local. value that is not in the list is rejected.

Custom endpoint models

To use a model on your own OpenAI-compatible server, set engine_model to customendpoint.<model-id> and set engine_base_url. See Custom endpoint models. These models are not listed in GET /ai_models.

Custom OpenRouter models

To use any other model offered by OpenRouter, set engine_model to openrouter.<author>/<slug> (for example openrouter.mistralai/mistral-large-2411) and set engine_key to your own OpenRouter API key. The key is required. GET /ai_models lists one entry for this option, with key_mode own_required and model_id_prefix openrouter.. That entry’s id cannot be used as engine_model directly: combine model_id_prefix with the model ID.

Calls use your key and are billed to your OpenRouter account. The platform key is never used.
Zero data retention (ZDR) routing is enforced on every request. If no ZDR provider supports the model, the call fails.
For voice calls, a model without a ZDR provider can make the call silent, because the LLM request fails. The failure is recorded on the AI call as a pipeline_error message, with a status category such as not_found or server_error (see AI Message), and the caller does not hear it. Before you connect a custom model to a call, check the model page on OpenRouter for a ZDR provider, or try the model in a text chat first.

Validation

On POST /ais, engine_model must be an id from GET /ai_models. Models that use the openai., gemini., or grok. prefix and are not in the list are also accepted and run with your own engine_key. A value in the form openrouter.<author>/<slug> is also accepted and requires your OpenRouter engine_key.
A value in the form customendpoint.<model-id> is also accepted and requires engine_base_url.
Any other value (for example an anthropic. model that is not in the list) is rejected.
On PUT /ais/{id}, the model check runs only when engine_model changes, so an existing AI that already stores an older value can still be updated without touching the model. The engine_key requirement for custom OpenRouter models is checked on every update, against the key that results from the request (the stored key when engine_key is omitted).

Custom endpoint models¶

A custom endpoint model runs on a server that you host and that implements the OpenAI chat completions API (for example vLLM, llama.cpp or Ollama). Set engine_model to customendpoint.<model-id> and engine_base_url to the base URL of your server, for example https://llm.example.com/v1. The model id is the name your server expects. It starts with a letter or digit and may contain letters, digits and . _ : / -. The whole engine_model value is at most 255 bytes, and the prefix is case sensitive. This option is separate from the local. models, which the platform operator registers.

Base URL rules

Only https is accepted. User information, a query string and a fragment are not allowed, and the URL may not contain credentials. Do not put a token in the path either.
ASCII only, 1 to 512 bytes. A port from 1 to 65535 is allowed.
The host must be a dotted domain name such as llm.example.com, or an IP address. A single label name such as llm and a name that ends with a dot are rejected.
The host must resolve to public addresses only. A name that resolves to a private, loopback, link-local or other internal address is rejected with ENGINE_BASE_URL_REJECTED. The address is checked again each time the platform connects to your server, and the server certificate must be valid for the host name.

engine_base_key semantics

The key is optional, because some servers need none. The same rules apply to POST /ais and PUT /ais/{id}.

Request	Result
`engine_base_key` omitted or `null`	Keeps the stored key. `null` is the same as omitting.
`engine_base_key` is `""`	Deletes the stored key.
`engine_base_key` is a value	Replaces the stored key.
`engine_base_url` changed	The stored key is discarded, unless the same request sends a non-empty `engine_base_key`.
Model changed to a non custom model	`engine_base_url` and the stored key are cleared.

The key is stored encrypted and is never returned. A key is 1 to 512 printable ASCII characters. The legacy engine_key is never sent to your server: use engine_base_key.

PUT /ais/{id} replaces the whole configuration (except that an omitted engine_key or engine_base_key keeps the stored key), so send engine_base_url again on every update of a customendpoint. AI. A client that does not know the field and omits it receives 400 ENGINE_BASE_URL_REQUIRED, and the stored settings are not changed.

When a change takes effect

A call that is already running keeps the key and URL it read when it started. A changed or deleted key, and a changed URL, apply from the next call or, for conversations, the next turn. For a long voice call this can be as long as the call. If a key leaks, replace or delete it and also end the calls that are in progress.

Failures

When the platform cannot reach your server (name does not resolve, connection refused, address not allowed, certificate not valid) a pipeline_error message with category connection is recorded. See AI Message. Other errors that your server returns over HTTP are not reported with this category: a 401 or 403 answer is reported as authentication, a 404 answer (for example an unknown model) as not_found, and a 5xx answer as server_error. A connection that breaks while the answer is being received is reported as connection.

TTS Type¶

Text-to-Speech provider for converting AI responses to audio.

Type	Description
elevenlabs	ElevenLabs high-quality voice synthesis (recommended)
deepgram	Deepgram Aura voices
openai	OpenAI TTS (alloy, echo, fable, etc.)
aws	AWS Polly voices
azure	Azure Cognitive Services TTS
google	Google Cloud Text-to-Speech
cartesia	Cartesia TTS
hume	Hume AI emotional TTS
playht	PlayHT voice synthesis

Default Voice IDs by TTS Type

TTS Type	Default Voice ID
elevenlabs	EXAVITQu4vr4xnSDxMaL (Rachel)
deepgram	aura-2-thalia-en (Thalia)
openai	alloy
aws	Joanna
azure	en-US-JennyNeural
google	en-US-Wavenet-D
cartesia	71a7ad14-091c-4e8e-a314-022ece01c121

STT Type¶

Speech-to-Text provider for converting incoming audio to text.

Type	Description
deepgram	Deepgram speech recognition (recommended)
cartesia	Cartesia speech recognition
elevenlabs	ElevenLabs speech recognition

STT Language¶

The stt_language field specifies which language the Speech-to-Text engine should recognize. The value must be in BCP-47 format (e.g., en-US, ko-KR, ja-JP).

When set, the STT provider is explicitly configured for the specified language, which improves recognition accuracy — especially for non-English conversations. When omitted or set to an empty string, the STT provider uses its default auto-detection behavior.

Common BCP-47 Language Codes

Language Code	Language
en-US	English (United States)
en-GB	English (United Kingdom)
ko-KR	Korean
ja-JP	Japanese
zh-CN	Chinese (Simplified)
de-DE	German
fr-FR	French
es-ES	Spanish (Spain)
pt-BR	Portuguese (Brazil)
it-IT	Italian
nl-NL	Dutch
ru-RU	Russian
ar-SA	Arabic
hi-IN	Hindi
pl-PL	Polish

Note

AI Implementation Hint

The stt_language is configured on the AI resource itself, not per-call. If you need different STT languages for different call scenarios, create separate AI configurations — one per language — and reference the appropriate ai_id in each flow action.

VAD Config¶

Voice Activity Detection configuration for tuning speech detection sensitivity and timing.

All fields are optional. Omitted fields use Pipecat’s native defaults.

Field	Default	Max	Description
confidence	0.7	1.0	Minimum confidence threshold to detect voice.
start_secs	0.2	30.0	Duration in seconds of continuous speech needed to confirm speaking started.
stop_secs	0.2	30.0	Duration in seconds of silence needed to confirm speaking stopped.
min_volume	0.6	1.0	Minimum audio volume for voice detection.

Note

AI Implementation Hint

When vad_config is null or omitted, Pipecat’s native defaults apply (confidence=0.7, start_secs=0.2, stop_secs=0.2, min_volume=0.6). To keep the AI responsive but avoid cutting off speech mid-sentence, increase stop_secs (e.g., 0.5). To make the AI more patient before responding, increase both stop_secs and start_secs.

Smart Turn¶

When smart_turn_enabled is true, the Pipecat pipeline uses LocalSmartTurnAnalyzerV3 — a local ONNX model that analyzes speech and transcription context to detect when the user has truly finished their turn, rather than pausing mid-sentence. This results in more natural conversations with fewer premature interruptions.

Note

AI Implementation Hint

Smart Turn detection requires VAD stop_secs=0.2. When smart_turn_enabled is true, any stop_secs value in vad_config is silently overridden to 0.2. This value matches the model’s training data and allows Smart Turn to dynamically adjust timing.

Tool Names¶

The tool_names field controls which tool functions the AI can invoke during conversations.

Configuration Options

Value	Description
`["all"]`	Enable all available tool functions (for `type=insight` this includes `send_message`, `draft_message`, `send_email`, `draft_email`, `create_call`, `draft_call` and `describe_action`)
`[]` or `null`	Disable all tool functions (AI can only converse)
`["tool1", "tool2"]`	Enable only the specified tools

Available Tools

See Tool Functions for the complete list of tools and their descriptions.

Example configurations:

// Enable all tools
"tool_names": ["all"]

// Enable only call transfer and email
"tool_names": ["connect_call", "send_email"]

// Enable conversation control tools only
"tool_names": ["stop_service", "stop_flow", "set_variables"]

// Disable all tools (conversation-only AI)
"tool_names": []

Note

Insight AI and the all selector

For a type=insight AI, ["all"] enables draft_message, send_message, draft_email, send_email, draft_call, create_call and describe_action together with the other Insight tools. The admin console “Enable All Tools” option stores ["all"]. An Insight AI that already stores ["all"] receives all seven tools after the service update is deployed. send_message sends an SMS or MMS right away, send_email sends an email right away and create_call places an outbound call right away, without an agent’s confirmation, and the cost is charged to the customer account; a placed call cannot be undone, and create_call can run any flow actions on the callee’s answer and connect to other numbers. An email is sent from one default sender address that is shared by all customers, so an email sent to an arbitrary recipient also affects the reputation of that shared sender; an email is sent asynchronously and a successful result does not confirm delivery. The tools can also be called in a turn that no agent is attending, and the text a customer wrote or what is said on a call is untrusted input. A customer who does not want direct sending or calling lists the tools by name and includes draft_message, draft_email, draft_call (and describe_action) but not send_message, send_email and create_call.