# Assistant SDK Reference

{% hint style="warning" %}
⚠️ **Closed Beta**\
The Assistants feature is currently in Closed Beta and available to Enterprise plan customers on annual plans. If your organisation is on a Enterprise plan, you can request access by emailing <embedded@whereby.com>. Assistants will be made generally available after the beta.
{% endhint %}

The **Whereby Assistant SDK** is a Node.js-focused SDK for building assistants that can join Whereby rooms. It provides all the polyfills and plumbing needed to join a Whereby Room in Node, and exposes a simple API to connect assistants, manage their lifecycle and access audio and video streams and other in-room features for all participants. This SDK can be used in conjunction with our Trigger API to listen for [webhooks](https://docs.whereby.com/whereby-product-features/insights-suite-and-api/webhooks) and join the room on certain room conditions being met.

### Audience

This package is designed for **backend developers** and **AI/assistant builders** who want to:&#x20;

* Run assistants, bots or agents **headlessly** (no browser UI)
* Integrate Whereby with **real-time transcriptions** services
* Stream room audio into **LLM-based agents** for conversation, summarization or automation
* Trigger assistants to join rooms automatically using **webhooks**

### Use cases

* Capturing a single combined audio stream of all participants for transcription, captioning or analysis
* Feeding live meeting audio to real-time AI agents (eg OpenAI Realtime)
* Running a support bot that automatically joins rooms when triggered, and can perform in room actions
* Building meeting recorders or pipelines that store and process session audio

### Requirements

The Assistant SDK is intended for Node.js environments and depends on some external tooling:&#x20;

* **Node.js ≥ 20**
* [**FFmpeg**](https://ffmpeg.org/) must be installed on the host machine if you wish to make use of the **combined audio stream** functionality.&#x20;

{% hint style="info" %}
Note: The SDK can not be run in the browser - use [Core](https://docs.whereby.com/reference/core-sdk-reference) or [Browser](https://docs.whereby.com/whereby-for-web-browser/react-based-browser-sdk/quick-start) SDK for creating user-interface based integrations.&#x20;
{% endhint %}

## Getting started

### Installation

Install the `@whereby.com/assistant-sdk` package from the public [npm registry.](https://www.npmjs.com/package/@whereby.com/assistant-sdk)

{% tabs %}
{% tab title="npm" %}

```bash
npm install @whereby.com/assistant-sdk
```

{% endtab %}

{% tab title="yarn" %}

```bash
yarn add @whereby.com/assistant-sdk
```

{% endtab %}

{% tab title="pnpm" %}

```bash
pnpm add @whereby.com/assistant-sdk
```

{% endtab %}
{% endtabs %}

### Installing FFmpeg

To use the **combined audio stream** functionality you will need FFmpeg to be installed on the machine running your assistant. You can check if FFmpeg is already installed on your local machine by running:&#x20;

```bash
ffmpeg -version
```

If you see the version number, you're good to go! if not, install it using one of the methods below:&#x20;

{% tabs %}
{% tab title="macOS (Homebrew)" %}

```bash
brew install ffmpeg
```

{% endtab %}

{% tab title="Ubutntu/Debian" %}

```bash
sudo apt update
sudo apt install ffmpeg
```

{% endtab %}

{% tab title="Windows (Chocolatey)" %}

```bash
choco install ffmpeg
```

{% endtab %}
{% endtabs %}

Or download a prebuilt binary directly from [ffmpeg.org/download](https://ffmpeg.org/download.html).

{% hint style="info" %}
After installation, make sure ffmpeg is available in your system PATH so the Assistant SDK can call it.&#x20;
{% endhint %}

On the [next page](https://docs.whereby.com/reference/assistant-sdk-reference/quick-start) you can learn how to start working with Whereby Assistants in your code base.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.whereby.com/reference/assistant-sdk-reference.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.