Session summaries

Turn the content of Whereby sessions into meeting notes, clinical documentation and educational material with the power of AI.

Session summaries are currently in closed Beta and available to selected customers only. Email us at [email protected] to join our beta testing program (terms and conditions apply).

Session summaries are derived from session transcriptions and provided as json responses available through the customer portal or via the API.

In order to produce a summary from a Whereby session you need to first create a session transcript. You can get a live transcript directly from the session or you can transcribe any recording, which was saved in Whereby-provided storage. Learn more about transcribing sessions.

Once you have the transcript you can create the summary using the desired summary template.

You can use Whereby session summaries manually through the customer portal, or programmatically with the combination of API requests.

Summary Templates

The summary template determines the structure of the summary as well as the type of information that will be derived from the transcript. You can choose from summary templates designed to be used for Whereby sessions being run in different contexts.

The following summary templates are currently available:

• General Bulleted

• General Narrative

• SOAP

• Extended SOAP

• Educational Lecture

• Educational Tutoring

General context

General Bulleted

This is the default summary template designed for use in any context and in session with two or more participants. Works great for business or internal company meetings. The summary contains session agenda, key points discussed, follow up action items and a short summary. The content of each section is organised into bullet points, with the exception of the short summary, which comes as a single paragraph.

Sample output

"summary": {
    "summary": "Agenda\n- Discussion of current goals and ongoing projects\n- Addressing audio issues during the meeting\n- Planning for follow-up meetings and action items\n\nKey Points Discussed\n- The team page has been updated with archive information from the past two quarters.\n- Goals for the current and next quarter were discussed, including ensuring the Progressive Web App (PWA) responds to changing network conditions, improving audio reliability, and reducing infrastructure vendor costs.\n- Audio issues during the meeting were addressed, suggesting possible electromagnetic interference.\n- The RTP quality score project is ongoing, aiming to make decisions in meetings based on connection quality.\n- The audio-only mode is being tested internally, with plans to make it automatic.\n- Infrastructure cost reduction efforts are in progress, with a knowledge-sharing session scheduled.\n- Challenges with the team include knowledge concentration, keeping the team on track, and unplanned work from business-critical customer issues.\n- Ongoing projects include glitch-free reconnect, RTP quality score, audio-only mode, audio improvements, and live captions.\n- Test DevLab is used for testing media performance, but its utility is being evaluated.\n- A new quality of service metric is being considered for Posthog.\n\nFollow Up Action Items\n- Schedule a meeting to focus on the RTP quality score project details.\n- Investigate the source of audio issues during the meeting.\n- Review and possibly revise the placement of the audio-only mode toggle for embedded customers.\n- Consider what metrics to track for the audio-only mode project.\n- Discuss resource allocation for the live captions project and RTP quality score project.\n- Attend the knowledge-sharing session on infrastructure cost reduction.\n\nShort Summary\nDuring the meeting, the team discussed the goals and ongoing projects, including improving the Progressive Web App's response to network conditions, enhancing audio reliability, and reducing infrastructure costs. Audio issues experienced during the call led to a discussion about potential electromagnetic interference. The team also talked about the RTP quality score project, which is in progress and aims to improve meeting experiences based on connection quality. Challenges faced by the team were highlighted, including knowledge concentration, staying on track, and unplanned work. Ongoing projects were reviewed, with a focus on glitch-free reconnect, audio-only mode, and live captions. Test DevLab's role in testing media performance was discussed, and a new quality of service metric for Posthog is being considered. Follow-up action items were set, including scheduling detailed project meetings, investigating audio issues, and attending a session on cost reduction."
},

General Narrative

Medical context

SOAP

This template is designed for use in medical sessions involving two participants (usually a doctor and a patient). It provides the summary in the SOAP format commonly used for clinical documentation, where SOAP stands for Subjective (S), Objective (O), Assessment (A) and Plan (P).

Sample output

"summary": {
    "summary": "A 40-year-old patient reports long-term digestive issues, including bloating, constipation, and dietary restrictions. The doctor suspects constipation-predominant IBS or chronic constipation and recommends seeing a gastroenterologist for further evaluation, including a possible colonoscopy. A probiotic, increased water intake, and non-stimulant laxative are advised, alongside reviewing a list of foods that cause gas and bloating.",
    "SOAP": {
        "subjective": "40-year-old patient with long-standing digestive problems, experiencing daily bloating, constipation (less than once a week without laxatives), and dietary restrictions (avoiding lactose, gluten, fruits, vegetables, and beans). Previous medications to speed up stomach emptying were ineffective. Occasionally has bowel movements every 2-3 days for a few weeks to months without diet changes, but then reverts to constipation. Experiences pain with significant bloating. Allergic to pollen and dust, taking allergy medication and supplements vitamin D and B12 due to vegetarian diet.",
        "objective": "Not identified in the transcript",
        "assessment": "Doctor suspects constipation-predominant irritable bowel syndrome (IBS-C) or chronic constipation. The patient's symptoms have been persistent and are impacting daily life. The patient has a history of an upper endoscopy but no lower GI studies.",
        "plan": "The plan includes a referral to a gastroenterologist for further evaluation, possibly including a colonoscopy. The patient is advised to start a probiotic, increase water intake, and consider using a non-stimulant laxative (polyethylene glycol). The doctor will provide an informational packet on foods to avoid for gas and bloating, and suggests the patient review it for potential dietary changes. The doctor also educates the patient on lactose content in dairy products. Follow-up with the gastroenterologist and the referring doctor is recommended if the patient is not satisfied or if there are any changes in symptoms."
    }
},

Extended SOAP

Educational context

Educational Lecture

Sample output

"summary": {
    "summary": "Overview:\nThe session focused on transcription and its applications, such as supercharging workflows, fact-finding, source verification, content discovery, and summarization. The speaker highlighted the power of transcription and demonstrated a new feature that allows users to view and interact with transcriptions synchronized with video recordings. The session also touched upon the potential integration with other features like participant insights and the use of utterances for more nuanced transcription. Additionally, there were discussions on pre-call network tests and the possibility of incorporating enterprise-level testing tools for WebRTC infrastructure.\n\nKey Concepts:\n- Transcription is a powerful tool for content discovery and workflow enhancement.\n- Transcription APIs provide start and stop times for each word or sentence, which can be used to synchronize text with video.\n- The new feature allows viewing of recordings with synchronized transcription, enhancing the usability of cloud recordings.\n\nTranscription Features:\n- Transcription data includes speaker diarization, timestamps, and the ability to skim through videos.\n- The speaker demonstrated a new page that displays transcriptions alongside the video, allowing users to follow along karaoke-style.\n- The transcription viewer could potentially include features like search functionality, speaker IDs, and summaries.\n\nTechnical Implementation:\n- The demo used the recordings API endpoint and fetched video URLs from Cloudflare.\n- The full JSON data of the transcription is now saved, not just the markdown text.\n- The speaker discussed potential issues with loading videos and accessing Cloudflare, as well as the need to configure CORS headers correctly.\n\nPotential Enhancements:\n- Integration with participant insights to provide more context in transcriptions.\n- The use of utterances in transcription for more accurate speaker patterns and closed captioning.\n- Offering summarization as part of the transcription service.\n\nPre-call Network Test Discussion:\n- The pre-call network test is designed to detect network-related issues by setting up a peer connection and verifying data flow through the SFU.\n- There was a suggestion to explore enterprise-level testing tools for WebRTC infrastructure to better serve customers with large-scale needs.\n\nConclusion:\n- The session concluded with the idea that offering full transcription data could make cloud recordings more attractive and open up possibilities for features like searchable transcripts for educational and moderation purposes."
  },

Educational Tutoring

Manual Session Summaries

Go to "Transcriptions" page of your customer portal to access all existing transcriptions of Whereby sessions. Find the transcription of a session that you want to summarise and select the template which you want to use for the summary.

The processing time, especially for longer sessions, may take up to a minute. Once the summary is generated, you can download it in the form of a .json file.

For each session transcript, only one summary can be created at a time. If you want to summarize the session using a different template, you must first delete the existing summary of the transcript. Only the most recent summary is stored and available for download.

Programmatic Session Summaries

If you want to automate the process of summarising Whereby sessions, you can do so with a combination of API requests.

Select the transcript to summarise

Summaries are derived from session transcriptions, so you need to find the transcriptionId of the session to be summarised.

You can get the transcriptionId of an individual session from the transcription.finished webhook event. Alternatively, you can fetch the list of all transcriptions with a GET /transcriptions request.

Trigger the summary process

In order to summarise the transcript, send a POST request to /summaries endpoint with the transcriptionId and the desired summary template in the request body:

{ 
    "transcriptionId": "1a74b982-deec-4615-b1d1-3d38a37d6698", 
    "template": "General Narrative" 
}

The "template" field is optional, and - if not set - the "General Bulleted" template will be used by default.

You expect a 201 Created response with the summaryId:<id> content.

For each session transcript, only one summary can be created at a time. If you want to summarize the session using a different template, you must first delete the existing summary of the transcript. Only the most recent summary is stored and available for download.

Fetch the summary

You can fetch an individual summary with a GET /summaries/{summaryId} request.

Alternatively, you can fetch all summaries with a GET /summaries request, witch returns a paginated response containing 25 items per page.

The summarisation process is running in the background and usually takes about a minute to complete. There is no webhook event sent when the summary is finished, so you need to poll the endpoint and read the state field of the response.

For state: in_progress the summary field is empty. Once the process is completed, the state field changes to ready and the summary field contains the desired content.

Delete the summary

Session summaries are stored in Whereby database until you delete them.

You can delete a summary with a DELETE /summaries/{summaryId} request where you expect a 204 No Content response.

Known limitations

Session summaries are only compatible with Whereby-hosted transcriptions. It will not be possible to trigger session summaries for transcriptions stored in your own Amazon S3 bucket.

Session summaries are currently not considered to be HIPAA compliant. Avoid using Session Summaries in order to maintain HIPAA compliance of your Whereby sessions. Learn more about Whereby HIPAA compliant setup.