stream in real time

for real-time playback, mint a one-shot WebSocket session, connect to it, and send a JSON frame with your synthesis parameters. the server streams raw PCM int16 little-endian @ 24 kHz mono as binary frames, then a terminal {"type":"done"} JSON text frame.

mint a session

POST /v1/tts/ws-connect returns { ws_url, token }.

connect and send

connect to ws_url?token=<token> and send one JSON synthesis frame.

collect PCM

read binary PCM chunks until the done (or error) control frame.

import asyncio, json, wave, requests, websockets

API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"

async def main():
    # 1. Mint a one-shot WS session -> { ws_url, token }
    s = requests.post(f"{BASE}/v1/tts/ws-connect",
                      headers={"Authorization": f"Bearer {API_KEY}"},
                      json={"model": "mulberry", "text": "Streaming in real time."}).json()

    # 2. Connect, then send the synthesis frame
    async with websockets.connect(f'{s["ws_url"]}?token={s["token"]}') as ws:
        await ws.send(json.dumps({
            "text": "Streaming in real time.",
            "description": "a calm 30s female voice, smooth timbre, slow pacing, like a narrator",
            "speaker": "ira",   # mulberry only; omit for muga / the described voice
        }))

        # 3. Collect PCM int16 (24 kHz mono) until the done frame
        pcm = bytearray()
        async for msg in ws:
            if isinstance(msg, bytes):
                pcm.extend(msg)
            elif json.loads(msg).get("type") == "done":
                break

    with wave.open("stream.wav", "wb") as w:
        w.setnchannels(1); w.setsampwidth(2); w.setframerate(24000)
        w.writeframes(pcm)

asyncio.run(main())

<!doctype html>
<html>
  <body>
    <button id="play">Speak</button>
    <script>
      const API_KEY = "rk_live_•••••••••";   // use a key with the tts:stream scope
      const BASE = "https://silk-api.rumik.ai";

      document.getElementById("play").onclick = async () => {
        // 1. Mint a one-shot WebSocket session -> { ws_url, token }
        const res = await fetch(BASE + "/v1/tts/ws-connect", {
          method: "POST",
          headers: { "Authorization": "Bearer " + API_KEY, "Content-Type": "application/json" },
          body: JSON.stringify({ model: "mulberry", text: "Hello from the browser." }),
        });
        const { ws_url, token } = await res.json();

        // 2. Set up 24 kHz mono playback
        const ctx = new AudioContext({ sampleRate: 24000 });
        let playAt = ctx.currentTime;

        // 3. Connect, send the synthesis frame, queue PCM as it arrives
        const ws = new WebSocket(ws_url + "?token=" + encodeURIComponent(token));
        ws.binaryType = "arraybuffer";

        ws.onopen = () => ws.send(JSON.stringify({
          text: "Hello from the browser.",
          description: "a warm 30s voice, smooth timbre, conversational pacing, like a friendly narrator",
          speaker: "ira",   // mulberry only; omit for muga / the described voice
        }));

        ws.onmessage = (e) => {
          if (e.data instanceof ArrayBuffer) {
            const pcm = new Int16Array(e.data);
            const buf = ctx.createBuffer(1, pcm.length, 24000);
            const ch = buf.getChannelData(0);
            for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
            const src = ctx.createBufferSource();
            src.buffer = buf;
            src.connect(ctx.destination);
            playAt = Math.max(playAt, ctx.currentTime);
            src.start(playAt);
            playAt += buf.duration;
          } else if (JSON.parse(e.data).type === "done" || JSON.parse(e.data).error) {
            ws.close();
          }
        };
      };
    </script>
  </body>
</html>

interruption (barge-in)

for a live “call with AI”, keep the WebSocket open for the whole call and send a new {"text": "..."} frame for each thing the agent says. if the caller talks over the agent, just send the next utterance: a new text frame while one is still generating interrupts the current one and starts the new one. this is “latest-wins”, there is no special barge-in frame.

frames

once connected, the session is a two-way stream of JSON text frames and binary audio. client → server

frame	meaning
`{"text": "..."}`	start a generation. sending another while one is in progress is a barge-in: it interrupts the current generation and starts the new one.
`{"type": "cancel"}`	stop the current generation with no replacement.
`{"type": "close"}`	end the session.

server → client

frame	meaning
`binary`	a PCM audio chunk (int16, 24 kHz, mono).
`{"type": "done", "request_id", "credits_used", ...}`	the generation finished normally.
`{"type": "cancelled", "request_id", "reason": ...}`	the generation was stopped. `reason` is `"interrupt"` (a new request replaced it) or `"cancel"` (explicit cancel).
`{"type": "timeout"}` / `{"error": true, ...}`	idle timeout / error.

on a barge-in you receive {"type": "cancelled", "reason": "interrupt"} for the old request, then the audio chunks and done for the new one. the old audio stops almost immediately.

behavior

latest-wins: only the newest utterance is held. a burst of requests during one generation never queues a backlog of now-stale audio, only the latest runs next.
immediate stop: on interrupt or cancel the old audio stops streaming right away, so the caller stops hearing it within a couple hundred milliseconds (mostly network round-trip).

an interrupted or cancelled utterance is still charged. the input was submitted and partially generated, so it bills like a completed request. rapid barge-ins each cost one interrupted utterance.

example

python

# inside an open `ws` from the example above, for a live call:
await ws.send(json.dumps({"text": "[neutral] Aapka call connect ho gaya hai."}))

# caller interrupts -> just send the next line. the old one is cancelled,
# the new one starts.
await ws.send(json.dumps({"text": "[happy] Haan ji, bataiye main kaise help karun?"}))

# or stop the current line with no replacement:
await ws.send(json.dumps({"type": "cancel"}))

# end the call:
await ws.send(json.dumps({"type": "close"}))

documentation

SDK

integrations

cookbook

API reference

interruption (barge-in)

frames

behavior

example

​interruption (barge-in)

​frames

​behavior

​example

interruption (barge-in)

frames

behavior

example