D-ID Client SDK - v3.0.0-0
    Preparing search index...

    Interface AgentManager

    A live connection to one agent: its profile, its chat, and the video stream it answers on.

    Created by createAgentManager. Nothing happens on the wire until connect() resolves; after that chat() makes the agent answer with its own LLM and speak() makes it say exactly what you give it. Call disconnect() when the user leaves.

    Two properties, agent and starterMessages, are readable as soon as the manager exists; everything else is a method. Some methods work only with some avatar types: each one says so, and AgentAvatar is where the session's tier is read from.

    The manager's own state is readable at any time: getChatMode(), getConnectionState() and getSessionInfo() answer with what the matching callback last reported, so nothing has to be mirrored in application state.

    Every method that reaches the Agents API can reject with an HttpError when the request comes back non-2xx, or a NetworkError when it never reaches the server; the individual @throws entries below name the errors that are specific to each method.

    interface AgentManager {
        agent: Agent;
        starterMessages: readonly string[];
        changeMode(mode: ChatMode): Promise<void>;
        chat(userMessage: string): Promise<ChatResponse>;
        connect(): Promise<void>;
        deleteRate(id: string): Promise<Rating>;
        disconnect(): Promise<void>;
        enrichAnalytics(properties: Record<string, unknown>): void;
        getChatMode(): ChatMode;
        getConnectionState(): ConnectionState;
        getSessionInfo(): StreamCreatedInfo | undefined;
        getStreamType(): StreamType | undefined;
        getSttToken(): Promise<SttTokenResponse>;
        interrupt(options?: InterruptOptions): void;
        isInterruptAvailable(): boolean;
        publishCameraStream(stream: MediaStream): Promise<void>;
        publishMicrophoneStream(stream: MediaStream): Promise<void>;
        rate(messageId: string, score: -1 | 1, rateId?: string): Promise<Rating>;
        reconnect(): Promise<void>;
        registerClientTool(name: string, handler: ClientToolHandler): void;
        replaceMicrophoneTrack(track: MediaStreamTrack): Promise<void>;
        sendDataChannelMessage(
            topic: "did.presentation",
            payload: Record<string, unknown>,
        ): Promise<void>;
        setSttLanguage(language: string): Promise<void>;
        speak(payload: string | SpeakScript): Promise<SpeakResponse>;
        submitFeedback(
            rating: 1 | 2 | 3 | 4 | 5,
            answer?: string,
        ): Promise<SubmitFeedbackResponse>;
        unpublishCameraStream(): Promise<void>;
        unpublishMicrophoneStream(): Promise<void>;
        unregisterClientTool(name: string): void;
    }
    Index
    agent: Agent

    The agent this manager is connected to, as the Agents API returned it.

    Fetched once while the manager is created, so it is available before connect(). Useful for rendering the agent's name, thumbnail and idle_video. See Agent.

    Read-only: the property cannot be reassigned, and the SDK never replaces it — the manager talks to the agent it was created for, for its whole life.

    nameElement.textContent = agentManager.agent.name ?? 'Agent';
    thumbnailElement.src = agentManager.agent.thumbnail ?? '';
    starterMessages: readonly string[]

    The agent's starter messages.

    Suggested openers to offer the user as buttons; empty when the agent defines none. Available before connect().

    Read-only, and a copy of Agent.starter_message rather than the same array, so sorting or filtering a local copy of it cannot change what agent reports.

    for (const message of agentManager.starterMessages) {
    addSuggestionButton(message, () => void agentManager.chat(message));
    }
    • Switches the chat to another mode.

      Any mode but ChatMode.Functional disconnects the stream, and a change into Functional disconnects a session that was not built for a conversation (one opened in ChatMode.DirectPlayback, for example); call connect() again after such a change. onModeChange fires once the change is applied; passing the mode already in effect does nothing.

      Parameters

      Returns Promise<void>

      A promise resolved when the mode is in effect and any disconnect has completed.

      ValidationError Rejects on Expressive (V4) agents for ChatMode.Off and ChatMode.DirectPlayback, which those agents do not support.

      import { ChatMode } from '@d-id/client-sdk';

      await agentManager.changeMode(ChatMode.TextOnly); // Disconnects the stream.

      await agentManager.changeMode(ChatMode.Functional);
      await agentManager.connect();
    • Sends a message to the agent and gets a streamed video based on its answer (LLM).

      The answer arrives through onNewMessage — first as partial chunks, then as the full answer — while the video plays it. The chat is created on the first call if the session does not have one yet.

      Parameters

      • userMessage: string

        The user's message text to send to the agent.

      Returns Promise<ChatResponse>

      The ChatResponse for this turn.

      ValidationError When the message is empty or too long, when the chat mode has chat disabled or is in maintenance, or when the manager is not connected yet.

      ChatCreationFailed On Talks (V2) and Clips (V3) agents, when the session has no chat yet and the Agents API answers the creation request without one.

      HttpError On Talks (V2) and Clips (V3) agents, and in ChatMode.Playground, when the message request comes back non-2xx. Expressive (V4) agents send the message over the data channel instead, so no HTTP request is made.

      NetworkError On Talks (V2) and Clips (V3) agents, and in ChatMode.Playground, when that request never reaches the server.

      const chat = await agentManager.chat('What is the distance to the moon?');
      
    • Opens a new session with the agent: a new WebRTC connection, a new web socket and a new chat.

      Resolves once the connection reaches 'connected', by which point onSrcObjectReady has been called with the media stream to render.

      One session at a time: while a call is still in flight a second call returns that same promise rather than opening a second session, which is what makes it safe in a React StrictMode effect. Once a session exists it rejects instead — call disconnect() first to start a fresh conversation, or reconnect() to keep the current one. A call that joins an operation a disconnect() later cancels resolves without a session open; read getConnectionState() for what happened.

      Returns Promise<void>

      Resolves when the agent is connected and ready.

      ValidationError When a session is already open, or when onSrcObjectReady was not supplied and the current ChatMode streams video — which a changeMode() out of a text-only mode can bring about.

      HttpError When creating the stream or the chat comes back non-2xx — a client key that is not authorized for the agent or the calling domain, or an account out of credits. The SDK tries the initialization up to three times first, except on 429 and on an out-of-credits response.

      NetworkError When those requests never reach the server.

      await agentManager.connect();
      
    • Closes the existing connection and chat with the agent: the stream and the web socket.

      Call it when the user leaves the page or the conversation, so the session does not keep running. After it resolves, connect() is needed before the agent can be used again.

      Returns Promise<void>

      Resolves once everything is closed.

      // The user left the conversation: close the session so it stops running.
      await agentManager.disconnect();

      // The manager stays usable — connect() opens a new session with a new chat.
      await agentManager.connect();
    • Adds properties to every analytics event the SDK sends from now on.

      The same thing analytics.additionalProperties does at creation time, for values you only learn later.

      Advanced. Calls merge, so a property sent twice takes the later value, and events already sent are not changed. It has no visible effect when analytics is switched off with analytics.enabled set to false, because nothing is sent at all.

      Parameters

      • properties: Record<string, unknown>

        A flat JSON object whose properties are added to every analytics event the SDK sends from now on.

      Returns void

      // The user signed in halfway through the session.
      agentManager.enrichAnalytics({ plan: 'pro', locale: navigator.language });
    • Returns the chat mode in effect right now.

      The same value onModeChange last reported, and the one every mode guard in the SDK reads. It is not always the mode that was asked for: the server can answer connect() with a different mode, which the manager adopts, and a failed connection leaves the session in Maintenance. Use it instead of mirroring onModeChange in your own state.

      Returns ChatMode

      The current ChatMode; before connect(), the mode the manager was created with (ChatMode.Functional by default).

      import { ChatMode } from '@d-id/client-sdk';

      if (agentManager.getChatMode() === ChatMode.Maintenance) {
      showBanner('The agent is temporarily unavailable.');
      }
    • Returns the kind of stream the current session negotiated.

      Returns StreamType | undefined

      StreamType.Fluent for a fluent stream (one video for the idle and talking states), StreamType.Legacy for the two-element legacy mode, or undefined before connect() has established a session.

      import { StreamType } from '@d-id/client-sdk';

      // A legacy stream is the one the application swaps the idle video in and out of.
      const swapsIdleVideo = agentManager.getStreamType() === StreamType.Legacy;
    • Fetches a short-lived token for the D-ID speech-to-text service.

      The SDK sends the request whenever it is called, connected or not; the service decides whether to issue a token for the agent.

      Returns Promise<SttTokenResponse>

      The SttTokenResponse for this agent.

      HttpError When the service does not answer with a token, or the request comes back non-2xx for any other reason.

      NetworkError When the request never reaches the server.

      const { token, region } = await agentManager.getSttToken();
      startRecognition(token, region);
    • Interrupts the current video stream mid-playback, so the user can talk over the agent.

      Supported on a fluent stream — a Clips (V3) agent built on a Pro avatar, or any Expressive (V4) agent. It never throws: on every agent type it returns without doing anything when interrupting is not available for the session, when it is not allowed right now, and — on Talks (V2) and Clips (V3) agents — when the stream is not a fluent stream or no video is playing. Check isInterruptAvailable() before offering the control at all; on Expressive (V4) agents onInterruptibleChange tracks whether it is allowed right now.

      The interrupted message is marked as such in the next onNewMessage — but only when an interrupt was actually sent. A call that finds nothing to interrupt leaves the message untouched and fires no callback.

      Parameters

      • Optionaloptions: InterruptOptions

        What caused the interruption, as an InterruptOptions: text, audio, click or manual. Optional — a stop button is the common case, so leaving it out means { type: 'click' }. Expressive (V4) agents drop text interrupts, because the orchestrator does not cancel the in-flight answer for them.

      Returns void

      stopButton.onclick = () => agentManager.interrupt();

      // The user typed over the answer instead of pressing the button.
      agentManager.interrupt({ type: 'text' });
    • Returns whether the current stream supports interrupting the agent mid-answer.

      True on a fluent stream — a Clips (V3) agent built on a Pro avatar, or any Expressive (V4) agent — once connected. This says the stream supports interrupting at all; onInterruptibleChange says whether interrupting is allowed right now.

      Returns boolean

      true when interrupt() can do anything.

      await agentManager.connect();
      setStopButtonVisible(agentManager.isInterruptAvailable());
    • Publishes a camera video track to the session so the agent can see the user.

      Call it after connect() to enable vision. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and before connect(), the returned promise rejects with a ValidationError.

      Parameters

      • stream: MediaStream

        A MediaStream whose video track is published to the session.

      Returns Promise<void>

      Resolves once the track is published.

      ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.

      if (agentManager.agent.vision?.enabled) {
      const cameraStream = await navigator.mediaDevices.getUserMedia({ video: true });
      await agentManager.publishCameraStream(cameraStream);
      }
    • Publishes a microphone audio track to the session so the agent can hear the user.

      Call it after connect() to enable voice input. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and before connect(), the returned promise rejects with a ValidationError.

      Parameters

      • stream: MediaStream

        A MediaStream whose audio track is published to the session.

      Returns Promise<void>

      Resolves once the track is published.

      ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.

      const micStream = await navigator.mediaDevices.getUserMedia({ audio: true });
      await agentManager.publishMicrophoneStream(micStream);
    • Rates one of the agent's answers in the chat, for future analytics and insights.

      Pass rateId to change a rating the user already gave instead of adding another.

      Works on every avatar type. On an Expressive (V4) session an answer that arrived over the data channel without a server id is given a locally generated one, which D-ID's records cannot be matched against — rate the answers whose ids came from the server.

      Parameters

      • messageId: string

        Id of the message being rated.

      • score: -1 | 1

        1 for a positive rating, -1 for a negative one.

      • OptionalrateId: string

        Id of an existing rating to update; omit to create a new one.

      Returns Promise<Rating>

      The created or updated Rating.

      ValidationError When no chat has started, or when no message with that id is in the transcript.

      HttpError When the rating request comes back non-2xx.

      NetworkError When the rating request never reaches the server.

      const rating = await agentManager.rate(messageId, 1);

      // The user changed their mind: pass the rating's id back rather than adding another.
      await agentManager.rate(messageId, -1, rating.id);
    • Reopens the stream when the session expires, and continues the conversation on the same chat.

      The chat id normally does not change, so the agent keeps its context. It starts a new stream — including after the server ended the previous one deliberately — rather than resuming the old one. On Expressive (V4) agents the transport is asked to reconnect first; if that fails the SDK falls back to a disconnect and a fresh connect, which starts a new chat id.

      One session-opening operation runs at a time, teardown included, so this and connect() cannot interleave — call one or the other rather than racing them. A connect() that joins this one continues the existing chat, so onNewChat does not fire for it.

      Returns Promise<void>

      Resolves when the new stream is connected.

      ValidationError When a connect() is still in flight; wait for it to settle first.

      HttpError When creating the new stream comes back non-2xx.

      NetworkError When that request never reaches the server.

      try {
      // The session expired. Start a new stream and keep the conversation.
      await agentManager.reconnect();
      } catch {
      // A connect() or another reconnect() is still in flight; let it settle first.
      }
    • Registers a handler for a client tool, run in the browser when the agent's LLM calls it.

      Expressive (V4) agents only: client tools travel on the real-time session's RPC channel, which Talks (V2) and Clips (V3) agents do not have. It throws a ValidationError on those rather than registering a handler the agent could never call. The check is on the agent, not on the connection, so it applies before connect() too.

      The handler executes on the client and its result is returned to the LLM. Register the handlers before connect() so the agent can call them from the moment the session starts; registering the same name again replaces the handler. Progress is reported through onToolEvent.

      Parameters

      • name: string

        Name of the tool, which must match the one defined in the agent's configuration.

      • handler: ClientToolHandler

        The function that runs when the agent calls the tool. It receives the arguments the LLM produced and returns a JSON string of at most 15 KiB — the limit is the transport's, not the SDK's, and a larger result fails the call. See ClientToolHandler.

      Returns void

      ValidationError When the agent is a Talks (V2) or Clips (V3) one.

      if (agentManager.agent.avatar.type === 'expressive') {
      agentManager.registerClientTool('get_cart_total', async args => {
      const total = await cart.total(args.currency as string);
      return JSON.stringify({ total });
      });
      }
    • Swaps the live microphone track without unpublishing it.

      Use it when the user picks a different input device: the publication is preserved — its LiveKit publication id (SID) and SSRC stay the same, though the MediaStreamTrack id changes — so the server sees continuous audio across the swap rather than a stop and a restart. Rejects with a plain Error from the transport when there is no active publication — fall back to publishMicrophoneStream() in that case. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and before connect(), the returned promise rejects with a ValidationError.

      Parameters

      Returns Promise<void>

      Resolves once the transport has switched to the new track.

      ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.

      const stream = await navigator.mediaDevices.getUserMedia({ audio: { deviceId } });
      const [track] = stream.getAudioTracks();

      try {
      await agentManager.replaceMicrophoneTrack(track);
      } catch {
      // Nothing published yet, so there is no track to swap: publish instead.
      await agentManager.publishMicrophoneStream(stream);
      }
    • Sends a JSON payload to the agent over a data-channel topic.

      Use it for application-specific messages that are not speech, such as telling a presentation to change slide. Expressive (V4) agents only, after connect(); otherwise the returned promise rejects with a ValidationError.

      Parameters

      • topic: "did.presentation"

        Data-channel topic to send on. Either a DataChannelTopic member or its string value — DataChannelTopic.Presentation and 'did.presentation' are both accepted. DataChannelTopic is exported from the package root and lists every topic this method accepts.

      • payload: Record<string, unknown>

        A plain object, sent as JSON.

      Returns Promise<void>

      Resolves once the payload has been sent. A room that has dropped since connect() reports a StreamError through onError and the promise still resolves.

      ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.

      import { DataChannelTopic } from '@d-id/client-sdk';

      await agentManager.sendDataChannelMessage(DataChannelTopic.Presentation, {
      type: 'navigate',
      slide: 3,
      });
    • Switches the speech-to-text language in the middle of a session.

      Expressive (V4) agents only, after connect(); otherwise the returned promise rejects with a ValidationError.

      Parameters

      • language: string

        The language to transcribe in, as a name or a BCP-47 code — "English" or "en-US".

      Returns Promise<void>

      Resolves once the new language has been sent to the agent. A room that has dropped since connect() reports a StreamError through onError and the promise still resolves.

      ValidationError When the session is not an Expressive (V4) one, or connect() has not run yet.

      await agentManager.setSttLanguage('en-US');
      await agentManager.setSttLanguage('Spanish');
    • Makes the agent stream back a video based on the text or audio file you provide.

      Unlike chat() the agent's LLM is not involved, so this is how you script greetings and canned lines. Pass a plain string as a shorthand for a text script. See SpeakScript, TextStreamScript and AudioStreamScript. Text scripts also accept an optional sentiment, for Expressive (V4) agents only; if the requested sentiment is not supported by the agent, the default sentiment is used.

      Parameters

      • payload: string | SpeakScript

        A text or audio script, or a string treated as the text to speak.

      Returns Promise<SpeakResponse>

      The SpeakResponse for the video that was produced, or the same response with duration 0 and an empty videoId when the call produced no discrete video — on Expressive (V4) agents, and in a text-only chat mode.

      ValidationError When the manager is not connected to a stream yet.

      HttpError On Talks (V2) and Clips (V3) agents, when the Agents API answers the request non-2xx. Expressive (V4) agents send the script over the data channel instead, so no HTTP request is made.

      NetworkError On Talks (V2) and Clips (V3) agents, when that request never reaches the server.

      const speak = await agentManager.speak({
      type: 'text',
      input: "Hi! I'm Alice!",
      });
      const speak = await agentManager.speak({
      type: 'text',
      input: "Hi! I'm Alice!",
      sentiment: 'friendly',
      });
      const speak = await agentManager.speak({
      type: 'audio',
      audio_url: 'https://www.yourwebsite.com/audio.mp3',
      });
    • Submits end-of-call feedback for the whole conversation.

      Separate from rate(), which scores a single answer. Collect it when the user ends the call, using the agent's end-of-call feedback configuration.

      Every avatar type. The agent must have end-of-call feedback switched on (Agent.end_of_call_feedback, enabled) — the Agents API rejects the request otherwise, so read the configuration before offering the form. A second submission for the same conversation replaces the first.

      Parameters

      • rating: 1 | 2 | 3 | 4 | 5

        The user's score for the conversation: 1, 2, 3, 4 or 5. The Agents API rejects anything else, whole numbers outside the range and fractions alike.

      • Optionalanswer: string

        The user's free-text answer to the follow-up question, when one was asked.

      Returns Promise<SubmitFeedbackResponse>

      The stored SubmitFeedbackResponse.

      ValidationError When no chat has started.

      HttpError When the feedback request comes back non-2xx — including a 400 when the agent does not have end-of-call feedback enabled.

      NetworkError When the feedback request never reaches the server.

      // The Agents API rejects the request unless the agent has end-of-call feedback on.
      if (agentManager.agent.end_of_call_feedback?.enabled) {
      await agentManager.submitFeedback(5, 'The answers were clear and quick.');
      }
    • Stops and removes the currently published camera track from the session.

      Call it to disable vision. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and when nothing is published, it resolves without doing anything.

      Returns Promise<void>

      Resolves once the track is removed.

      // No guard needed: a no-op wherever there is nothing published.
      await agentManager.unpublishCameraStream();
    • Stops and removes the currently published microphone track from the session.

      The counterpart of publishMicrophoneStream() — use it to mute the user for the rest of the session. Expressive (V4) agents only; on Talks (V2) and Clips (V3) agents, and when nothing is published, it resolves without doing anything.

      Returns Promise<void>

      Resolves once the track is removed.

      await agentManager.unpublishMicrophoneStream();
      
    • Removes a previously registered client tool handler.

      After this the agent's calls to that tool fail rather than reaching your code. Unlike registerClientTool() it never throws — a name that was never registered, and any agent type, is a no-op — so it is safe in a cleanup path.

      Parameters

      • name: string

        Name of the tool whose handler should be removed.

      Returns void

      // Safe in a cleanup path: it never throws, whatever the agent type.
      agentManager.unregisterClientTool('get_cart_total');