Chat modes

The chat mode decides how the agent answers: with a streamed video, as text only, or not at all. It is set with options.mode and changed later with changeMode(). The full list is ChatMode.

The four modes an application chooses

Mode	What it is for	`chat()`	Video for answers
ChatMode.Functional	The default. A conversation with the agent's LLM, answered in a streamed video.	works	yes
ChatMode.TextOnly	A text chat with no talking head. Talks (V2) and Clips (V3) agents.	works	no
ChatMode.DirectPlayback	The application scripts every line itself with `speak()`.	rejects	yes
ChatMode.Off	Chat switched off, video still streaming.	rejects	yes

Two more exist that applications do not normally set. ChatMode.Maintenance is what the SDK switches to when connecting fails after its retries, so a UI can say the agent is temporarily out of service; chat() rejects while it is in force. ChatMode.Playground is the test mode behind the agent playground in D-ID Studio.

What each mode does on the wire

On a Talks (V2) or Clips (V3) agent, connect() always creates the WebRTC stream; the mode decides whether a chat is created and whether the notifications web socket is opened:

Mode	Chat created	Notifications web socket
`Functional`, `TextOnly`, `Playground`, `Maintenance`	yes	yes
`Off`	no	yes
`DirectPlayback`	no	no

A ChatMode.TextOnly application that wants no stream at all simply does not call connect().

Expressive (V4) agents chat over the LiveKit data channel and the SDK builds their chat as ChatMode.Functional, so the first connect() adopts Functional and reports it through onModeChange. There is no text-only conversation with an Expressive (V4) agent: chat() in ChatMode.TextOnly has no session to send on.

Choosing the mode at creation

import * as sdk from '@d-id/client-sdk';
import { ChatMode } from '@d-id/client-sdk';

const agentManager = await sdk.createAgentManager('agt_fumf1234', {
    auth: { type: 'key', clientKey: 'YOUR_CLIENT_KEY' },
    mode: ChatMode.TextOnly, // Textual modes do not need `onSrcObjectReady`.
    callbacks: {
        onNewMessage(messages, type) {
            if (type === 'answer') {
                render(messages);
            }
        },
    },
});

mode takes the enum member only: mode: 'TextOnly' does not compile.

Changing the mode later

changeMode() resolves once the change is in effect. Any mode but ChatMode.Functional disconnects the stream, and a change into Functional disconnects a session that was not built for a conversation, so call connect() again when a change tore the session down. getChatMode() returns the mode in effect.

await agentManager.changeMode(ChatMode.TextOnly);
// The stream is gone. Nothing else is needed for a text-only conversation.

await agentManager.changeMode(ChatMode.Functional);
await agentManager.connect(); // Build a session that can carry a conversation again.

Reacting to a mode the application did not ask for

The server can answer the chat-creation request with a different mode, and a connection that fails after the SDK's retries leaves the session in ChatMode.Maintenance. Both arrive through onModeChange; a downgrade also reports a ChatModeDowngraded error through onError.

import { ChatMode, isDIDError, type AgentManagerCallbacks } from '@d-id/client-sdk';

const callbacks: AgentManagerCallbacks = {
    onSrcObjectReady(value) {
        videoElement.srcObject = value;
    },
    onModeChange(mode) {
        // `chat()` rejects in Maintenance too, so the composer goes with the banner below.
        setComposerEnabled(
            mode !== ChatMode.Off && mode !== ChatMode.DirectPlayback && mode !== ChatMode.Maintenance
        );
        setVideoVisible(mode === ChatMode.Functional);

        if (mode === ChatMode.Maintenance) {
            showBanner('The agent is temporarily unavailable.');
        }
    },
    onError(error) {
        if (isDIDError(error) && error.kind === 'ChatModeDowngraded') {
            // The server answered with a narrower mode than the one that was asked for.
            console.warn(error.message);
        }
    },
};

Expressive (V4) restrictions

ChatMode.Off and ChatMode.DirectPlayback are Talks (V2) and Clips (V3) features. On an Expressive (V4) agent both createAgentManager and changeMode() reject them with a ValidationError:

try {
    await agentManager.changeMode(ChatMode.DirectPlayback);
} catch (error) {
    console.error(error); // ValidationError on an Expressive (V4) agent.
}