> For the complete documentation index, see [llms.txt](https://docs.whereby.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.whereby.com/reference/assistant-sdk-reference.md).

# Assistant SDK Reference

{% hint style="warning" %}
⚠️ **Closed Beta**\
The Assistants feature is currently in Closed Beta. You can request access by emailing <embedded@whereby.com>. Assistants will be made generally available after the beta.
{% endhint %}

The **Whereby Assistant SDK** is a Node.js-focused SDK for building assistants that can join Whereby rooms. It provides all the polyfills and plumbing needed to join a Whereby Room in Node, and exposes a simple API to connect assistants, manage their lifecycle and access audio and video streams and other in-room features for all participants. This SDK can be used in conjunction with our Trigger API to listen for [webhooks](/whereby-product-features/insights-suite-and-api/webhooks.md) and join the room on certain room conditions being met.

### Audience

This package is designed for **backend developers** and **AI/assistant builders** who want to:

* Run assistants, bots or agents **headlessly** (no browser UI)
* Integrate Whereby with **real-time transcriptions** services
* Stream room audio into **LLM-based agents** for conversation, summarization or automation
* Trigger assistants to join rooms automatically using **webhooks**

### Use cases

* Capturing a single combined audio stream of all participants for transcription, captioning or analysis
* Feeding live meeting audio to real-time AI agents (eg OpenAI Realtime)
* Running a support bot that automatically joins rooms when triggered, and can perform in room actions
* Building meeting recorders or pipelines that store and process session audio

### Requirements

The Assistant SDK is intended for Node.js environments and depends on some external tooling:

* **Node.js ≥ 20**
* [**FFmpeg**](https://ffmpeg.org/) must be installed on the host machine if you wish to make use of the **combined audio stream** functionality.

{% hint style="info" %}
Note: The SDK can not be run in the browser - use [Core](/reference/core-sdk-reference.md) or [Browser](/whereby-for-web-browser/react-based-browser-sdk/quick-start.md) SDK for creating user-interface based integrations.
{% endhint %}

## Getting started

### Installation

Install the `@whereby.com/assistant-sdk` package from the public [npm registry.](https://www.npmjs.com/package/@whereby.com/assistant-sdk)

{% tabs %}
{% tab title="npm" %}

```bash
npm install @whereby.com/assistant-sdk
```

{% endtab %}

{% tab title="yarn" %}

```bash
yarn add @whereby.com/assistant-sdk
```

{% endtab %}

{% tab title="pnpm" %}

```bash
pnpm add @whereby.com/assistant-sdk
```

{% endtab %}
{% endtabs %}

### Installing FFmpeg

To use the **combined audio stream** functionality you will need FFmpeg to be installed on the machine running your assistant. You can check if FFmpeg is already installed on your local machine by running:

```bash
ffmpeg -version
```

If you see the version number, you're good to go! if not, install it using one of the methods below:

{% tabs %}
{% tab title="macOS (Homebrew)" %}

```bash
brew install ffmpeg
```

{% endtab %}

{% tab title="Ubutntu/Debian" %}

```bash
sudo apt update
sudo apt install ffmpeg
```

{% endtab %}

{% tab title="Windows (Chocolatey)" %}

```bash
choco install ffmpeg
```

{% endtab %}
{% endtabs %}

Or download a prebuilt binary directly from [ffmpeg.org/download](https://ffmpeg.org/download.html).

{% hint style="info" %}
After installation, make sure ffmpeg is available in your system PATH so the Assistant SDK can call it.
{% endhint %}

On the [next page](/reference/assistant-sdk-reference/quick-start.md) you can learn how to start working with Whereby Assistants in your code base.