stream in real time

for real-time playback, mint a one-shot websocket session, connect to it, and send a json frame with your synthesis parameters. when audio_format is omitted, the server streams raw pcm int16 little-endian @ 24 khz mono as binary frames, then a terminal {"type":"done"} json text frame.

mint a session

POST /v1/tts/ws-connect returns { ws_url, token }.

connect and send

connect to ws_url?token=<token> and send one json synthesis frame.

collect pcm

read binary pcm chunks until the done (or error) control frame.

choose an audio format

add audio_format to the post /v1/tts/ws-connect body to stream opus, pcm, mulaw, alaw, or mp3. the format is fixed for that one-shot session, so do not repeat it in the synthesis frame. always connect to the returned ws_url.

curl

api_key='rk_live_•••••••••'

curl --request post https://silk-api.rumik.ai/v1/tts/ws-connect \
  --header "authorization: bearer ${api_key}" \
  --header "content-type: application/json" \
  --data '{"model":"muga","text":"stream this as opus.","audio_format":"opus"}'

see audio formats for exact content types, file extensions, and a complete websocket example.

import asyncio, json, wave, requests, websockets

API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"

async def main():
    # 1. Mint a one-shot WS session -> { ws_url, token }
    s = requests.post(f"{BASE}/v1/tts/ws-connect",
                      headers={"Authorization": f"Bearer {API_KEY}"},
                      json={"model": "mulberry", "text": "Streaming in real time."}).json()

    # 2. Connect, then send the synthesis frame
    async with websockets.connect(f'{s["ws_url"]}?token={s["token"]}') as ws:
        await ws.send(json.dumps({
            "text": "Streaming in real time.",
            "description": "a calm 30s female voice, smooth timbre, slow pacing, like a narrator",
            "speaker": "ira",   # mulberry or mulberry-1.6; omit for muga / the described voice
        }))

        # 3. Collect PCM int16 (24 kHz mono) until the done frame
        pcm = bytearray()
        async for msg in ws:
            if isinstance(msg, bytes):
                pcm.extend(msg)
            elif json.loads(msg).get("type") == "done":
                break

    with wave.open("stream.wav", "wb") as w:
        w.setnchannels(1); w.setsampwidth(2); w.setframerate(24000)
        w.writeframes(pcm)

asyncio.run(main())

<!doctype html>
<html>
  <body>
    <button id="play">Speak</button>
    <script>
      const API_KEY = "rk_live_•••••••••";   // use a key with the tts:stream scope
      const BASE = "https://silk-api.rumik.ai";

      document.getElementById("play").onclick = async () => {
        // 1. Mint a one-shot WebSocket session -> { ws_url, token }
        const res = await fetch(BASE + "/v1/tts/ws-connect", {
          method: "POST",
          headers: { "Authorization": "Bearer " + API_KEY, "Content-Type": "application/json" },
          body: JSON.stringify({ model: "mulberry", text: "Hello from the browser." }),
        });
        const { ws_url, token } = await res.json();

        // 2. Set up 24 kHz mono playback
        const ctx = new AudioContext({ sampleRate: 24000 });
        let playAt = ctx.currentTime;

        // 3. Connect, send the synthesis frame, queue PCM as it arrives
        const ws = new WebSocket(ws_url + "?token=" + encodeURIComponent(token));
        ws.binaryType = "arraybuffer";

        ws.onopen = () => ws.send(JSON.stringify({
          text: "Hello from the browser.",
          description: "a warm 30s voice, smooth timbre, conversational pacing, like a friendly narrator",
          speaker: "ira",   // mulberry or mulberry-1.6; omit for muga / the described voice
        }));

        ws.onmessage = (e) => {
          if (e.data instanceof ArrayBuffer) {
            const pcm = new Int16Array(e.data);
            const buf = ctx.createBuffer(1, pcm.length, 24000);
            const ch = buf.getChannelData(0);
            for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
            const src = ctx.createBufferSource();
            src.buffer = buf;
            src.connect(ctx.destination);
            playAt = Math.max(playAt, ctx.currentTime);
            src.start(playAt);
            playAt += buf.duration;
          } else if (JSON.parse(e.data).type === "done" || JSON.parse(e.data).error) {
            ws.close();
          }
        };
      };
    </script>
  </body>
</html>

interruption (barge-in)

for a live “call with ai”, keep the websocket open for the whole call and send a new {"text": "..."} frame for each thing the agent says. if the caller talks over the agent, just send the next utterance: a new text frame while one is still generating interrupts the current one and starts the new one. this is “latest-wins”, there is no special barge-in frame.

frames

once connected, the session is a two-way stream of json text frames and binary audio. client → server

frame	meaning
`{"text": "..."}`	start a generation. sending another while one is in progress is a barge-in: it interrupts the current generation and starts the new one.
`{"type": "cancel"}`	stop the current generation with no replacement.
`{"type": "close"}`	end the session.

server → client

frame	meaning
`binary`	a chunk in the session’s negotiated format; raw 24 khz mono pcm when `audio_format` was omitted.
`{"type": "done", "request_id", "credits_used", ...}`	the generation finished normally. `credits_used` is retained as wire-compatibility metadata; use the usage dashboard for current plan usage.
`{"type": "cancelled", "request_id", "reason": ...}`	the generation was stopped. `reason` is `"interrupt"` (a new request replaced it) or `"cancel"` (explicit cancel).
`{"type": "timeout"}` / `{"error": true, ...}`	idle timeout / error.

on a barge-in you receive {"type": "cancelled", "reason": "interrupt"} for the old request, then the audio chunks and done for the new one. the old audio stops almost immediately.

behavior

latest-wins: only the newest utterance is held. a burst of requests during one generation never queues a backlog of now-stale audio, only the latest runs next.
immediate stop: on interrupt or cancel the old audio stops streaming right away, so the caller stops hearing it within a couple hundred milliseconds (mostly network round-trip).

example

python

# inside an open `ws` from the example above, for a live call:
await ws.send(json.dumps({"text": "[neutral] Aapka call connect ho gaya hai."}))

# caller interrupts -> just send the next line. the old one is cancelled,
# the new one starts.
await ws.send(json.dumps({"text": "[happy] Haan ji, bataiye main kaise help karun?"}))

# or stop the current line with no replacement:
await ws.send(json.dumps({"type": "cancel"}))

# end the call:
await ws.send(json.dumps({"type": "close"}))

silk mulberry 1.6

mulberry-1.6 streams exactly like mulberry: mint the session with "model": "mulberry-1.6", then send the same frames. barge-in, cancel and close work the same way.

speaker is ira, aisha, siya or zoya, and defaults to ira.
description is optional: style, accent, pace. see the prompting guide.
send one complete sentence or utterance per text frame, as soon as it’s ready.
<laugh>, <chuckle> and <sigh> in text are performed; any other <tag> marker is removed before synthesis.
a line too long to finish ends with an error frame, code GENERATION_TOO_LONG, instead of cut-off audio, and isn’t charged. any other failure is GENERATION_ERROR.

python

# inside an open `ws` minted with "model": "mulberry-1.6":
await ws.send(json.dumps({
    "text": "నమస్కారం, మీరు ఎలా ఉన్నారు?",
    "description": "happy, Telugu accent, fast pace",
    "speaker": "siya",
}))

its done frame carries type, finished, ttft_ms, ttfa_ms, total_tokens, frames, chunks, gen_time_ms, duration_s, audio_duration_ms, request_id and credits_used.

documentation

sdk

integrations

cookbook

api reference

choose an audio format

interruption (barge-in)

frames

behavior

example

silk mulberry 1.6

​choose an audio format

​interruption (barge-in)

​frames

​behavior

​example

​silk mulberry 1.6

choose an audio format

interruption (barge-in)

frames

behavior

example

silk mulberry 1.6