A client tool is a function that runs in the user's browser when the agent's LLM decides to call it. Use one when the agent needs data or actions only the page has, such as the cart contents or the user's locale. The tool itself is declared in the agent's configuration; the SDK's job is to bind a name from that configuration to an implementation with registerClientTool() and to report what happens through onToolEvent.
Client tools are an Expressive (V4) feature. They travel on the real-time session's RPC channel, which Talks (V2) and Clips (V3) agents do not have, so registerClientTool() throws a ValidationError on those rather than registering a handler the agent could never call. The check is on the agent, not on the connection, so it applies before connect() too.
Register before connect(), so the agent can call a tool from the moment the session starts. Registering the same name again replaces the handler.
import * as sdk from '@d-id/client-sdk';
const agentManager = await sdk.createAgentManager('agt_fumf1234', {
auth: { type: 'key', clientKey: 'YOUR_CLIENT_KEY' },
callbacks: {
onSrcObjectReady(value) {
videoElement.srcObject = value;
},
},
});
agentManager.registerClientTool('get_cart_total', async args => {
const total = await cart.total(args.currency as string);
return JSON.stringify({ total, currency: args.currency });
});
// Nothing to await: a synchronous handler may return the string directly.
agentManager.registerClientTool('get_locale', () => JSON.stringify({ locale: navigator.language }));
await agentManager.connect();
A handler is a ClientToolHandler: (args: Record<string, unknown>) => string | Promise<string>.
unknown, because their shape is the agent's tool schema rather than anything the SDK knows — narrow or cast them yourself.JSON.stringify(result)): the server parses it before handing it to the LLM, and passes the raw text on when it is not JSON.timeout in the tool's configuration to allow more: up to 120 seconds for a blocking tool, 600 for an async one. A handler that has not resolved in time fails the call, and the agent is told the tool did not respond.Client tool failed. Do not swallow errors into a { "error": … } string unless you want the LLM to treat the call as a success.agentManager.registerClientTool('book_appointment', async args => {
const slot = args.slot as string;
if (!(await calendar.isFree(slot))) {
// The message reaches the agent, which can offer another time.
throw new Error(`${slot} is no longer available`);
}
const booking = await calendar.book(slot);
return JSON.stringify({ bookingId: booking.id });
});
Whether the agent waits for a call is a property of the tool in the agent's configuration, not of the handler, and it reaches the SDK as ToolExecutionMode on RunningToolCall.executionMode and ToolCallStartedPayload.executionMode:
blocking: the agent waits for the handler before it answers. Use it when the answer depends on the result.async: the agent keeps talking while the handler runs, and the call can outlive the turn that started it. The result still reaches the agent when the handler resolves. Use it for anything slow.While a blocking call runs, the server holds the turn: it stops listening to the user and the agent cannot be interrupted, so a slow blocking tool is dead air. onInterruptibleChange goes false while any blocking call is outstanding and back to true when none is.
onToolEvent is called once when a call starts, and again when it finishes or fails. It reports every tool the agent calls, server tools included, not only the client tools registered here. Its type, ToolEventCallback, pairs each ToolCallEvent with the payload that event carries — ToolCallEvent.Started with a ToolCallStartedPayload, ToolCallEvent.Done with a ToolCallDonePayload, ToolCallEvent.Error with a ToolCallErrorPayload.
Branching on event is what narrows data, inside the handler and not only at the call site, so nothing has to be asserted. callId, name and timestamp are on all three payloads and can be read before the branch. The call's input, output, durationMs and failure reason arrive only in a verbose session, which needs the agent owner's API key, so a browser application on a client key does not get them.
import { ToolCallEvent, type AgentManagerCallbacks } from '@d-id/client-sdk';
const callbacks: AgentManagerCallbacks = {
onToolEvent(event, data) {
if (event === ToolCallEvent.Started) {
console.log('started', data.callId, data.name);
} else if (event === ToolCallEvent.Done) {
console.log('done', data.callId, data.name);
} else {
// `error` is set only in a verbose session.
console.warn('failed', data.name, data.error ?? 'no reason given');
}
},
};
onRunningToolCallsChange carries the whole set of calls running right now, as RunningToolCall entries, every time it changes. A call appears when it starts and disappears when it finishes or fails — or, for a blocking call, when its turn ends. On disconnect it fires with an empty array if any call was still running, so a spinner driven by it always clears.
isAwaitingTool returns true while any running call is blocking.
A call marked unlisted is a built-in step, such as moving to the next slide. It keeps the agent busy like any other call, but leave it out of anything that names the running tools.
import { isAwaitingTool, type AgentManagerCallbacks } from '@d-id/client-sdk';
const callbacks: AgentManagerCallbacks = {
onSrcObjectReady(value) {
videoElement.srcObject = value;
},
onRunningToolCallsChange(calls) {
setSpinnerVisible(isAwaitingTool(calls));
setStatusText(
calls
.filter(call => !call.unlisted)
.map(call => call.name)
.join(', ')
);
},
onInterruptibleChange(interruptible) {
setStopButtonEnabled(interruptible);
},
};
unregisterClientTool() removes a handler. Unlike registerClientTool() it never throws: an unknown name, and any avatar type, is a no-op, so it is safe in a cleanup path.
useEffect(() => {
agentManager.registerClientTool('get_cart_total', getCartTotal);
return () => agentManager.unregisterClientTool('get_cart_total');
}, [agentManager]);
After a tool is unregistered, calls to it fail with Method not supported at destination, which is what the agent is told. The tool stays in the agent's configuration, so remove it there if the LLM should stop calling it.