How to Stream Elevenlabs Text-to-Speech in python

Unreal Speech

Dec 4, 2023 • 3 min read

Introduction to ElevenLabs Streaming API

ElevenLabs offers a powerful API for converting text into speech using a chosen voice and streaming the audio in real-time. This capability is significant for applications requiring dynamic voice responses, such as virtual assistants, audio content generation, or interactive voice response systems.

Setting Up for Streaming

To use ElevenLabs' streaming feature, you first need to authenticate with the API using your API key. The API key is essential for accessing most of the API's endpoints programmatically. You can obtain your xi-api-key from the 'Profile' tab on the ElevenLabs website.

Choosing a Voice

The API allows you to select a voice for TTS conversion. Each voice is identified by a unique voice_id, which you can retrieve from the available voices listed at https://api.elevenlabs.io/v1/voices. This flexibility lets you tailor the voice output to suit your application's context and audience.

Optimizing Streaming Latency

A key parameter in real-time streaming is the latency from the request to the first audio byte. ElevenLabs offers options to optimize this latency at the potential cost of audio quality. The optimize_streaming_latency parameter accepts values from 0 (default, no optimization) to 4 (maximum optimization with possible compromises in pronunciation accuracy).

Specifying Output Format

The API supports various output formats for the generated audio, including different MP3 and PCM formats, as well as the μ-law format. The output_format parameter allows you to specify the desired format, with mp3_44100_128 being the default.

Providing the Text

The core of the TTS feature is the text you want to convert. This is passed as a string in the text parameter. The text can be anything from a simple greeting to a complex narrative, depending on your application's needs.

Customizing Voice Settings

For more control over the voice output, the API provides voice_settings, an object that lets you override stored settings for a given voice. These settings apply only to the current request, allowing for dynamic customization of the voice output

Additional Voice Parameters

The voice_settings object includes parameters like similarity_boost and stability to fine-tune the voice output. similarity_boost adjusts how closely the voice resembles a target voice, while stability affects the consistency of the voice output. Additionally, parameters like style and use_speaker_boost provide further customization options.

Code Example

Here's a Python code example demonstrating how to use the ElevenLabs API for streaming:

from elevenlabs import generate, stream

audio_stream = generate(
  text="Your text here",
  voice_id="your_voice_id",
  optimize_streaming_latency=1,  # Adjust as needed
  output_format="mp3_44100_128",  # Adjust as needed
  voice_settings={
    "similarity_boost": 1.0,  # Adjust as needed
    "stability": 1.0,  # Adjust as needed
    "style": 1,  # Adjust as needed
    "use_speaker_boost": True  # Adjust as needed
  }
)

stream(audio_stream)

This code snippet demonstrates how to set up a streaming request with customized settings, including voice choice, latency optimization, output format, and voice settings.

Conclusion

ElevenLabs' streaming API offers a flexible and powerful tool for real-time text-to-speech conversion. By understanding and utilizing the various parameters and settings available, developers can create tailored voice experiences for their applications. Whether for interactive voice applications, content creation, or other innovative uses, ElevenLabs provides a robust solution for real-time voice streaming.

References

This blog post referenced the following sources for accurate and detailed information: ElevenLabs API Documentation