D-ID Client SDK - v3.0.0-0
    Preparing search index...

    Interface AgentManagerCallbacks

    Handlers the SDK calls as the connection, the video stream and the chat change state.

    Pass the object as AgentManagerOptions.callbacks. Every handler is optional in the type, but onSrcObjectReady is required in practice for any chat mode that streams video — without it there is nothing to render the agent into, and createAgentManager rejects. Every handler is called from the SDK's own event handling, so keep the work inside short.

    The handlers fall into four groups: the connection (onConnectionStateChange, onConnectivityStateChange, onError), the video stream (onSrcObjectReady, onStreamCreated, onVideoStateChange, onAgentActivityStateChange, onInterruptibleChange), the chat (onNewChat, onNewMessage, onModeChange) and client tools (onToolEvent, onRunningToolCallsChange). A handler that lives outside the object can be typed with an indexed access such as AgentManagerCallbacks['onNewMessage'].

    The whole object is captured once, by createAgentManager, and the manager works from its own copy — assigning a handler to the object you passed afterwards has no effect, and neither does replacing options.callbacks. Give each handler a stable identity that reads the current state rather than closing over it: in React, keep the state in a ref and read ref.current inside the handler.

    interface AgentManagerCallbacks {
        onAgentActivityStateChange?: (state: AgentActivityState) => void;
        onConnectionStateChange?: (state: ConnectionState, reason?: string) => void;
        onConnectivityStateChange?: (state: ConnectivityState) => void;
        onError?: (error: Error, errorData?: ErrorContext) => void;
        onInterruptibleChange?: (interruptible: boolean) => void;
        onModeChange?: (mode: ChatMode) => void;
        onNewChat?: (chatId: string) => void;
        onNewMessage?: (
            messages: Message[],
            type: "user" | "answer" | "partial",
        ) => void;
        onRunningToolCallsChange?: (calls: readonly RunningToolCall[]) => void;
        onSrcObjectReady?: (srcObject: MediaStream) => void;
        onStreamCreated?: (stream: StreamCreatedInfo) => void;
        onToolEvent?: ToolEventCallback;
        onVideoStateChange?: (state: StreamingState) => void;
    }
    Index
    onAgentActivityStateChange? onConnectionStateChange? onConnectivityStateChange? onError? onInterruptibleChange? onModeChange? onNewChat? onNewMessage? onRunningToolCallsChange? onSrcObjectReady? onStreamCreated? onToolEvent? onVideoStateChange?
    onAgentActivityStateChange?: (state: AgentActivityState) => void

    Called when the agent moves between idle, loading, talking and running a tool.

    Use it to drive a typing indicator or to disable input while the agent is busy. The full set of states is reported for Expressive (V4) agents only. Talks (V2) and Clips (V3) agents report just Talking and Idle, and only on fluent streams, plus a final Idle when the connection closes.

    It is the authoritative "is the agent speaking" signal wherever it reports at all — an Expressive (V4) session or a fluent stream — because it comes from the agent rather than from the video track, so it also covers thinking and tool calls, which produce no video. On a legacy stream it reports nothing, so use onVideoStateChange there; that is the branch every consumer otherwise writes for itself, keyed on getStreamType().

    Type Declaration

    import { AgentActivityState, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onAgentActivityStateChange(state) {
            setTypingIndicatorVisible(state === AgentActivityState.Loading);
            setComposerEnabled(state === AgentActivityState.Idle);
        },
    };
    onConnectionStateChange?: (state: ConnectionState, reason?: string) => void

    Called every time the connection to the agent's stream changes state.

    Triggered by connect(), reconnect() and disconnect(). 'connected' is the point at which chat() and speak() can be called.

    Type Declaration

      • (state: ConnectionState, reason?: string): void

      • Parameters

        • state: ConnectionState

          The state just reached: one of new, connecting, connected, completed, disconnecting, disconnected, closed or fail. See ConnectionState.

        • Optionalreason: string

          Why that state was reached. On disconnected it is a StreamEndReason value when the server ended the stream on purpose, which is how you tell a deliberate end from a dropped connection. Talks (V2) and Clips (V3) agents report only ok, unknown_error, network_issue and inactivity; message_limit, time_limit and ended_by_agent come from Expressive (V4) agents. A close reason the SDK does not recognize is forwarded as-is, so compare reason against the enum rather than parsing it; any other value is an opaque transport diagnostic. reconnect() still works after a deliberate end; it starts a new stream rather than resuming the old one.

        Returns void

    import { ConnectionState, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onConnectionStateChange(state, reason) {
            console.log('onConnectionStateChange(): ', state, reason);

            if (state === ConnectionState.Connected) {
                console.log("I'm ready to go!");
            }
        },
    };
    onConnectivityStateChange?: (state: ConnectivityState) => void

    Called when the quality of the user's internet connection changes.

    The state is derived from jitter-buffer delay and freeze count on the video stream for Talks (V2) and Clips (V3) agents, and from LiveKit's reported connection quality for Expressive (V4) agents, so it reflects what the session is actually getting rather than the browser's online flag.

    Type Declaration

    import { ConnectivityState, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onConnectivityStateChange(state) {
            setQualityWarningVisible(state === ConnectivityState.Weak);
        },
    };
    onError?: (error: Error, errorData?: ErrorContext) => void

    Called when the SDK fails, so the application can surface the problem.

    Receives failures from the Agents API requests, the stream and the web socket — an HttpError when a request comes back non-2xx, a NetworkError when it never reaches the server, a WsError when the web socket itself fails, and also ChatModeDowngraded and StreamError. Two errors do not arrive here: ValidationError and ChatCreationFailed are thrown to whoever called the method (chat(), speak() and the rating methods), so they surface as a rejected promise rather than through this callback.

    Type Declaration

      • (error: Error, errorData?: ErrorContext): void

      • Parameters

        • error: Error

          The error that occurred. Narrow it with isDIDError and branch on kind; toJson() is what to log.

        • OptionalerrorData: ErrorContext

          Where the failure happened, as an ErrorContext: the endpoint and method for a failed request, the session or stream id for a failure on the stream. Every field is optional and nothing else is passed — in particular not the request body, which carries the end user's own message.

        Returns void

    import { isDIDError, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onError(error, errorData) {
            reportToYourErrorService({
                ...(isDIDError(error) ? error.toJson() : { kind: 'Error', message: error.message }),
                ...errorData,
            });
        },
    };
    onInterruptibleChange?: (interruptible: boolean) => void

    Called when the agent becomes interruptible, or stops being interruptible.

    Expressive (V4) agents only, and about whether interrupting is allowed right now: it goes false while a blocking client tool call is outstanding, because the agent is suspended waiting for it, and back to true when the call finishes. That is a different question from isInterruptAvailable(), which says whether the session supports interrupting at all. Use this one to enable or disable an interrupt button. Talks (V2) and Clips (V3) agents never report a change.

    Type Declaration

      • (interruptible: boolean): void

      • Parameters

        • interruptible: boolean

          true while there is something to interrupt, false while there is not.

        Returns void

    import type { AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onInterruptibleChange(interruptible) {
            setStopButtonEnabled(interruptible);
        },
    };
    onModeChange?: (mode: ChatMode) => void

    Called when the chat mode changes.

    Fires after changeMode(), and when the server answers with a different mode than the one requested (for example ChatMode.Maintenance).

    Type Declaration

    import { ChatMode, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onModeChange(mode) {
            // `chat()` rejects in Maintenance too, so the composer goes with the banner below.
            setComposerEnabled(
                mode !== ChatMode.Off && mode !== ChatMode.DirectPlayback && mode !== ChatMode.Maintenance
            );

            if (mode === ChatMode.Maintenance) {
                showBanner('The agent is temporarily unavailable.');
            }
        },
    };
    onNewChat?: (chatId: string) => void

    Called when a chat is created for this session.

    A chat is created while connect() runs, and lazily on the first chat() when none exists yet. Store the id if you want to correlate the conversation with your own records.

    On Talks (V2) and Clips (V3) agents the id comes from the Agents API. On Expressive (V4) agents the SDK derives it from the session id as cht_<sessionId>, which is the same id the Agents API stores the conversation under — so it correlates with D-ID's own records either way.

    Type Declaration

      • (chatId: string): void

      • Parameters

        • chatId: string

          Id of the chat that was just created.

        Returns void

    import type { AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onNewChat(chatId) {
            // Keep it to correlate the conversation with your own records.
            localStorage.setItem('did-chat-id', chatId);
        },
    };
    onNewMessage?: (
        messages: Message[],
        type: "user" | "answer" | "partial",
    ) => void

    Called with the whole chat transcript every time a message is added or updated.

    Triggered by chat(), by speak() for text scripts, and as the agent's answer streams in. It also fires once while the manager is created, with whatever initialMessages were given, and again on every connect() after the first — both with type answer. The array is a fresh copy on each call, oldest message first; every Message carries an id, a role of user or assistant (the agent), its content and a createdAt timestamp.

    Type Declaration

      • (messages: Message[], type: "user" | "answer" | "partial"): void

      • Parameters

        • messages: Message[]

          The chat so far.

        • type: "user" | "answer" | "partial"

          partial while the agent's answer is still streaming in, answer for the full answer spoken in the streamed video, user when the message just added is the user's own (a message passed to chat(), or a transcribed utterance).

        Returns void

    import type { AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onNewMessage(messages, type) {
            // `partial` fires repeatedly as the answer streams in; `answer` is the final one.
            if (type === 'answer') {
                console.log(messages[messages.length - 1].content);
            }
        },
    };
    onRunningToolCallsChange?: (calls: readonly RunningToolCall[]) => void

    Called whenever the set of tool calls running in the session changes.

    Expressive (V4) agents only. On disconnect it fires with an empty array if any call was still running, so a spinner driven by this callback always clears; with nothing outstanding there is nothing to clear and nothing is emitted. Each entry is a RunningToolCall.

    Type Declaration

      • (calls: readonly RunningToolCall[]): void

      • Parameters

        • calls: readonly RunningToolCall[]

          Every tool call running right now, empty when none is.

        Returns void

    import { isAwaitingTool, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onRunningToolCallsChange(calls) {
            setSpinnerVisible(isAwaitingTool(calls));
            setStatusText(calls.map(call => call.name).join(', '));
        },
    };
    onSrcObjectReady?: (srcObject: MediaStream) => void

    Called with the media stream carrying the agent's video and audio.

    This is the one callback a video session cannot work without: assign the value to the srcObject of your <video> element, and keep a reference to it, because onVideoStateChange has to put it back after the idle video has been shown. Triggered by connect(), reconnect() and disconnect().

    Optional only for the chat modes that never stream video — ChatMode.TextOnly, ChatMode.Playground and ChatMode.Maintenance — so a text-only application does not have to supply a stub. createAgentManager rejects with a ValidationError when it is missing in any other mode — ChatMode.Off and ChatMode.DirectPlayback included, since those create no chat but still stream video. connect() checks it again against the mode in effect then, so a manager created in a text-only mode and later moved into a video mode with changeMode() rejects rather than opening a stream with nothing to render it into.

    Type Declaration

      • (srcObject: MediaStream): void

      • Parameters

        • srcObject: MediaStream

          The live media stream to render.

        Returns void

    import type { AgentManagerCallbacks } from '@d-id/client-sdk';

    // Kept so `onVideoStateChange` can put it back after the idle video.
    let srcObject: MediaStream | null = null;

    const callbacks: AgentManagerCallbacks = {
        onSrcObjectReady(value) {
            videoElement.srcObject = value;
            srcObject = value;
        },
    };
    onStreamCreated?: (stream: StreamCreatedInfo) => void

    Called once per session when the stream has been created on the server.

    Fires before the connection reaches 'connected'. The ids are useful when correlating a session with D-ID support or with your own logs.

    Type Declaration

    import type { AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onStreamCreated({ streamId, sessionId }) {
            // The two ids D-ID support asks for when a session misbehaves.
            console.log('stream', streamId, 'session', sessionId);
        },
    };
    onToolEvent?: ToolEventCallback

    Called when the agent starts, finishes or fails a tool call.

    Expressive (V4) agents only. The handler takes two arguments and returns nothing: event, one of ToolCallEvent.Started, ToolCallEvent.Done or ToolCallEvent.Error; and data, the payload that event carries — ToolCallStartedPayload, ToolCallDonePayload or ToolCallErrorPayload respectively. The signature that does the narrowing is on ToolEventCallback, which also carries a handler that branches on all three events.

    import { ToolCallEvent, type AgentManagerCallbacks } from '@d-id/client-sdk';

    const callbacks: AgentManagerCallbacks = {
        onToolEvent(event, data) {
            if (event === ToolCallEvent.Error) {
                showBanner(`${data.name} failed: ${data.error ?? 'no reason given'}`);
            }
        },
    };
    onVideoStateChange?: (state: StreamingState) => void

    Called when the streamed video starts and stops, so the video element can switch source.

    Triggered by chat() and speak(). On STOP point the element at the agent's idle video (Agent.idle_video); on START put the stream handed to onSrcObjectReady back on it.

    On a legacy stream this is the only "is the agent speaking" signal — see onAgentActivityStateChange for which stream types report which.

    Type Declaration

    import { StreamingState, type AgentManagerCallbacks } from '@d-id/client-sdk';

    let srcObject: MediaStream | null = null;

    const callbacks: AgentManagerCallbacks = {
        onSrcObjectReady(value) {
            srcObject = value;
            videoElement.srcObject = value;
        },
        onVideoStateChange(state) {
            if (state === StreamingState.Stop) {
                videoElement.srcObject = null;
                videoElement.src = agentManager.agent.idle_video ?? '';
            } else {
                videoElement.src = '';
                videoElement.srcObject = srcObject;
            }
        },
    };

    Settings
    Member Visibility

    On This Page
    Properties