ReadonlyagentThe agent this manager is connected to, as the Agents API returned it.
Fetched once while the manager is created, so it is available before connect(). Useful for rendering the agent's name, thumbnail and idle_video. See Agent.
Read-only: the property cannot be reassigned, and the SDK never replaces it — the manager talks to the agent it was created for, for its whole life.
ReadonlystarterThe agent's starter messages.
Suggested openers to offer the user as buttons; empty when the agent defines none. Available before connect().
Read-only, and a copy of Agent.starter_message rather than the same array, so sorting or filtering a local copy of it cannot change what agent reports.
Switches the chat to another mode.
Any mode but ChatMode.Functional disconnects the stream, and a change into
Functional disconnects a session that was not built for a conversation (one opened in
ChatMode.DirectPlayback, for example); call connect()
again after such a change. onModeChange fires
once the change is applied; passing the mode already in effect does nothing.
A promise resolved when the mode is in effect and any disconnect has completed.
ValidationError Rejects on Expressive (V4) agents for ChatMode.Off and ChatMode.DirectPlayback, which those agents do not support.
Sends a message to the agent and gets a streamed video based on its answer (LLM).
The answer arrives through onNewMessage — first
as partial chunks, then as the full answer — while the video plays it. The chat is
created on the first call if the session does not have one yet.
The user's message text to send to the agent.
The ChatResponse for this turn.
ValidationError When the message is empty or too long, when the chat mode has chat disabled or is in maintenance, or when the manager is not connected yet.
ChatCreationFailed On Talks (V2) and Clips (V3) agents, when the session has no chat yet and the Agents API answers the creation request without one.
HttpError On Talks (V2) and Clips (V3) agents, and in ChatMode.Playground, when the message request comes back non-2xx. Expressive (V4) agents send the message over the data channel instead, so no HTTP request is made.
NetworkError On Talks (V2) and Clips (V3) agents, and in ChatMode.Playground, when that request never reaches the server.
Opens a new session with the agent: a new WebRTC connection, a new web socket and a new chat.
Resolves once the connection reaches 'connected', by which point onSrcObjectReady has been called with the media stream to render.
One session at a time: while a call is still in flight a second call returns that same promise rather than opening a second session, which is what makes it safe in a React StrictMode effect. Once a session exists it rejects instead — call disconnect() first to start a fresh conversation, or reconnect() to keep the current one. A call that joins an operation a disconnect() later cancels resolves without a session open; read getConnectionState() for what happened.
Resolves when the agent is connected and ready.
ValidationError When a session is already open, or when onSrcObjectReady was not supplied and the current ChatMode streams video — which a changeMode() out of a text-only mode can bring about.
HttpError When creating the stream or the chat comes back non-2xx — a client
key that is not authorized for the agent or the calling domain, or an account out of
credits. The SDK tries the initialization up to three times first, except on 429 and on an
out-of-credits response.
NetworkError When those requests never reach the server.
Removes a rating the user gave to an answer in the chat.
Id of the rating to remove, as returned by rate().
The Rating that was deleted.
ValidationError When no chat has started.
HttpError When the delete request comes back non-2xx.
NetworkError When the delete request never reaches the server.
Closes the existing connection and chat with the agent: the stream and the web socket.
Call it when the user leaves the page or the conversation, so the session does not keep running. After it resolves, connect() is needed before the agent can be used again.
Resolves once everything is closed.
Adds properties to every analytics event the SDK sends from now on.
The same thing analytics.additionalProperties does at creation time, for values you only learn later.
Advanced. Calls merge, so a property sent twice takes the later value, and events already
sent are not changed. It has no visible effect when analytics is switched off with
analytics.enabled set to false, because nothing is sent
at all.
A flat JSON object whose properties are added to every analytics event the SDK sends from now on.
Returns the chat mode in effect right now.
The same value onModeChange last reported, and
the one every mode guard in the SDK reads. It is not always the mode that was asked for: the
server can answer connect() with a different mode, which the
manager adopts, and a failed connection leaves the session in
Maintenance. Use it instead of mirroring onModeChange in
your own state.
The current ChatMode; before connect(), the mode the manager was created with (ChatMode.Functional by default).
Returns the connection state the manager last reported.
The same value onConnectionStateChange was last called with, so a component that mounts after the session is up can read the state instead of waiting for the next change.
The current ConnectionState; 'new' before the first connect().
Returns the ids of the session that is open, for support and for your own logs.
The same StreamCreatedInfo that onStreamCreated delivered for this session.
The open session's streamId, sessionId and agentId, or undefined before
connect() and after
disconnect().
Returns the kind of stream the current session negotiated.
StreamType.Fluent for a fluent stream (one video for the idle and talking
states), StreamType.Legacy for the two-element legacy mode, or undefined before
connect() has established a session.
Fetches a short-lived token for the D-ID speech-to-text service.
The SDK sends the request whenever it is called, connected or not; the service decides whether to issue a token for the agent.
The SttTokenResponse for this agent.
HttpError When the service does not answer with a token, or the request comes back non-2xx for any other reason.
NetworkError When the request never reaches the server.
Interrupts the current video stream mid-playback, so the user can talk over the agent.
Supported on a fluent stream — a Clips (V3) agent built on a Pro avatar, or any Expressive (V4) agent. It never throws: on every agent type it returns without doing anything when interrupting is not available for the session, when it is not allowed right now, and — on Talks (V2) and Clips (V3) agents — when the stream is not a fluent stream or no video is playing. Check isInterruptAvailable() before offering the control at all; on Expressive (V4) agents onInterruptibleChange tracks whether it is allowed right now.
The interrupted message is marked as such in the next onNewMessage — but only when an interrupt was actually sent. A call that finds nothing to interrupt leaves the message untouched and fires no callback.
Optionaloptions: InterruptOptions
What caused the interruption, as an InterruptOptions: text,
audio, click or manual. Optional — a stop button is the common case, so leaving it out
means { type: 'click' }. Expressive (V4) agents drop text interrupts, because the
orchestrator does not cancel the in-flight answer for them.
Returns whether the current stream supports interrupting the agent mid-answer.
True on a fluent stream — a Clips (V3) agent built on a Pro avatar, or any Expressive (V4) agent — once connected. This says the stream supports interrupting at all; onInterruptibleChange says whether interrupting is allowed right now.
true when interrupt() can do anything.
Publishes a camera video track to the session so the agent can see the user.
Call it after connect() to enable vision. Expressive (V4)
agents only; on Talks (V2) and Clips (V3) agents, and before connect(), the returned
promise rejects with a ValidationError.
A MediaStream whose video track is published to the session.
Resolves once the track is published.
ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.
Publishes a microphone audio track to the session so the agent can hear the user.
Call it after connect() to enable voice input. Expressive (V4)
agents only; on Talks (V2) and Clips (V3) agents, and before connect(), the returned
promise rejects with a ValidationError.
A MediaStream whose audio track is published to the session.
Resolves once the track is published.
ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.
Rates one of the agent's answers in the chat, for future analytics and insights.
Pass rateId to change a rating the user already gave instead of adding another.
Works on every avatar type. On an Expressive (V4) session an answer that arrived over the data channel without a server id is given a locally generated one, which D-ID's records cannot be matched against — rate the answers whose ids came from the server.
Id of the message being rated.
1 for a positive rating, -1 for a negative one.
OptionalrateId: string
Id of an existing rating to update; omit to create a new one.
The created or updated Rating.
ValidationError When no chat has started, or when no message with that id is in the transcript.
HttpError When the rating request comes back non-2xx.
NetworkError When the rating request never reaches the server.
Reopens the stream when the session expires, and continues the conversation on the same chat.
The chat id normally does not change, so the agent keeps its context. It starts a new stream — including after the server ended the previous one deliberately — rather than resuming the old one. On Expressive (V4) agents the transport is asked to reconnect first; if that fails the SDK falls back to a disconnect and a fresh connect, which starts a new chat id.
One session-opening operation runs at a time, teardown included, so this and
connect() cannot interleave — call one or the other rather
than racing them. A connect() that joins this one continues the existing chat, so
onNewChat does not fire for it.
Resolves when the new stream is connected.
ValidationError When a connect() is still in flight; wait for it to settle first.
HttpError When creating the new stream comes back non-2xx.
NetworkError When that request never reaches the server.
Registers a handler for a client tool, run in the browser when the agent's LLM calls it.
Expressive (V4) agents only: client tools travel on the real-time session's RPC channel, which Talks (V2) and Clips (V3) agents do not have. It throws a ValidationError on those rather than registering a handler the agent could never call. The check is on the agent, not on the connection, so it applies before connect() too.
The handler executes on the client and its result is returned to the LLM. Register the handlers before connect() so the agent can call them from the moment the session starts; registering the same name again replaces the handler. Progress is reported through onToolEvent.
Name of the tool, which must match the one defined in the agent's configuration.
The function that runs when the agent calls the tool. It receives the arguments the LLM produced and returns a JSON string of at most 15 KiB — the limit is the transport's, not the SDK's, and a larger result fails the call. See ClientToolHandler.
ValidationError When the agent is a Talks (V2) or Clips (V3) one.
Swaps the live microphone track without unpublishing it.
Use it when the user picks a different input device: the publication is preserved — its
LiveKit publication id (SID) and SSRC stay the same, though the MediaStreamTrack id
changes — so the server sees continuous audio across the swap rather than a stop and a
restart. Rejects with a plain Error from the transport when there is no active publication
— fall back to publishMicrophoneStream() in
that case. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and before
connect(), the returned promise rejects with a
ValidationError.
The audio track to send from now on.
Resolves once the transport has switched to the new track.
ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.
const stream = await navigator.mediaDevices.getUserMedia({ audio: { deviceId } });
const [track] = stream.getAudioTracks();
try {
await agentManager.replaceMicrophoneTrack(track);
} catch {
// Nothing published yet, so there is no track to swap: publish instead.
await agentManager.publishMicrophoneStream(stream);
}
Sends a JSON payload to the agent over a data-channel topic.
Use it for application-specific messages that are not speech, such as telling a presentation to change slide. Expressive (V4) agents only, after connect(); otherwise the returned promise rejects with a ValidationError.
Data-channel topic to send on. Either a DataChannelTopic member or
its string value — DataChannelTopic.Presentation and 'did.presentation' are both
accepted. DataChannelTopic is exported from the package root and lists every topic
this method accepts.
A plain object, sent as JSON.
Resolves once the payload has been sent. A room that has dropped since connect() reports a StreamError through onError and the promise still resolves.
ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.
Switches the speech-to-text language in the middle of a session.
Expressive (V4) agents only, after connect(); otherwise the returned promise rejects with a ValidationError.
The language to transcribe in, as a name or a BCP-47 code — "English" or
"en-US".
Resolves once the new language has been sent to the agent. A room that has dropped since connect() reports a StreamError through onError and the promise still resolves.
ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.
Makes the agent stream back a video based on the text or audio file you provide.
Unlike chat() the agent's LLM is not involved, so this is how you
script greetings and canned lines. Pass a plain string as a shorthand for a text script. See
SpeakScript, TextStreamScript and AudioStreamScript. Text
scripts also accept an optional sentiment, for Expressive (V4) agents only; if the
requested sentiment is not supported by the agent, the default sentiment is used.
A text or audio script, or a string treated as the text to speak.
The SpeakResponse for the video that was produced, or the same
response with duration 0 and an empty videoId when the call produced no discrete video
— on Expressive (V4) agents, and in a text-only chat mode.
did.speak command
sent over the data channel for Expressive (V4) agents.ValidationError When the manager is not connected to a stream yet.
HttpError On Talks (V2) and Clips (V3) agents, when the Agents API answers the request non-2xx. Expressive (V4) agents send the script over the data channel instead, so no HTTP request is made.
NetworkError On Talks (V2) and Clips (V3) agents, when that request never reaches the server.
Submits end-of-call feedback for the whole conversation.
Separate from rate(), which scores a single answer. Collect it when the user ends the call, using the agent's end-of-call feedback configuration.
Every avatar type. The agent must have end-of-call feedback switched on (Agent.end_of_call_feedback, enabled) — the Agents API rejects the request otherwise, so read the configuration before offering the form. A second submission for the same conversation replaces the first.
The user's score for the conversation: 1, 2, 3, 4 or 5. The Agents API rejects anything else, whole numbers outside the range and fractions alike.
Optionalanswer: string
The user's free-text answer to the follow-up question, when one was asked.
The stored SubmitFeedbackResponse.
ValidationError When no chat has started.
HttpError When the feedback request comes back non-2xx — including a 400
when the agent does not have end-of-call feedback enabled.
NetworkError When the feedback request never reaches the server.
Stops and removes the currently published camera track from the session.
Call it to disable vision. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and when nothing is published, it resolves without doing anything.
Resolves once the track is removed.
Stops and removes the currently published microphone track from the session.
The counterpart of publishMicrophoneStream() — use it to mute the user for the rest of the session. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and when nothing is published, it resolves without doing anything.
Resolves once the track is removed.
Removes a previously registered client tool handler.
After this the agent's calls to that tool fail rather than reaching your code. Unlike registerClientTool() it never throws — a name that was never registered, and any agent type, is a no-op — so it is safe in a cleanup path.
Name of the tool whose handler should be removed.
A live connection to one agent: its profile, its chat, and the video stream it answers on.
Created by createAgentManager. Nothing happens on the wire until connect() resolves; after that chat() makes the agent answer with its own LLM and speak() makes it say exactly what you give it. Call disconnect() when the user leaves.
Two properties, agent and starterMessages, are readable as soon as the manager exists; everything else is a method. Some methods work only with some avatar types: each one says so, and AgentAvatar is where the session's tier is read from.
The manager's own state is readable at any time: getChatMode(), getConnectionState() and getSessionInfo() answer with what the matching callback last reported, so nothing has to be mirrored in application state.
Every method that reaches the Agents API can reject with an HttpError when the request comes back non-2xx, or a NetworkError when it never reaches the server; the individual
@throwsentries below name the errors that are specific to each method.