stream in real time

for real-time playback, mint a one-shot websocket session, connect to it, and send a json frame with your synthesis parameters. when audio_format is omitted, the server streams raw pcm int16 little-endian @ 24 khz mono as binary frames, then a terminal {"type":"done"} json text frame.

mint a session

POST /v1/tts/ws-connect returns { ws_url, token, request_id, expires_in }. the token is single-use and expires in 300 seconds (expires_in).

connect and send

connect to ws_url?token=<token> and send one json synthesis frame.

collect pcm

read binary pcm chunks until the done (or error) control frame.

choose an audio format

add audio_format to the post /v1/tts/ws-connect body to stream opus, pcm, mulaw, alaw, or mp3. the format is fixed for that one-shot session, so do not repeat it in the synthesis frame. always connect to the returned ws_url.

curl

api_key='rk_live_•••••••••'

curl --request POST https://silk-api.rumik.ai/v1/tts/ws-connect \
  --header "Authorization: Bearer ${api_key}" \
  --header "content-type: application/json" \
  --data '{"model":"muga","text":"stream this as opus.","audio_format":"opus"}'

see audio formats for exact content types, file extensions, and a complete websocket example.

never put a rk_live_ key in a browser. call /v1/tts/ws-connect from your server and pass only the returned ws_url and token to the page. the token is single-use and expires in 5 minutes. the browser example below does this: the page asks your own endpoint (here /tts-session) for a session.

import asyncio, json, wave, requests, websockets

API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"

async def main():
    # 1. Mint a one-shot WS session -> { ws_url, token }
    s = requests.post(f"{BASE}/v1/tts/ws-connect",
                      headers={"Authorization": f"Bearer {API_KEY}"},
                      json={"model": "mulberry", "text": "Streaming in real time."}).json()

    # 2. Connect, then send the synthesis frame
    async with websockets.connect(f'{s["ws_url"]}?token={s["token"]}') as ws:
        await ws.send(json.dumps({
            "text": "Streaming in real time.",
            "description": "a calm 30s female voice, smooth timbre, slow pacing, like a narrator",
            "speaker": "ira",   # mulberry only; omit for muga / the described voice
        }))

        # 3. Collect PCM int16 (24 kHz mono) until the done frame
        pcm = bytearray()
        async for msg in ws:
            if isinstance(msg, bytes):
                pcm.extend(msg)
            else:
                frame = json.loads(msg)
                if frame.get("error"):
                    raise RuntimeError(frame)   # error frame: no `done` follows
                if frame.get("type") == "done":
                    break

    with wave.open("stream.wav", "wb") as w:
        w.setnchannels(1); w.setsampwidth(2); w.setframerate(24000)
        w.writeframes(pcm)

asyncio.run(main())

<!doctype html>
<html>
  <body>
    <button id="play">Speak</button>
    <script>
      document.getElementById("play").onclick = async () => {
        // 1. Ask your own endpoint for a one-shot session -> { ws_url, token }.
        //    It calls /v1/tts/ws-connect server-side (see "your server"), so the
        //    rk_live_ key never reaches the browser.
        const res = await fetch("/tts-session", { method: "POST" });
        const { ws_url, token } = await res.json();

        // 2. Set up 24 kHz mono playback
        const ctx = new AudioContext({ sampleRate: 24000 });
        let playAt = ctx.currentTime;

        // 3. Connect, send the synthesis frame, queue PCM as it arrives
        const ws = new WebSocket(ws_url + "?token=" + encodeURIComponent(token));
        ws.binaryType = "arraybuffer";

        ws.onopen = () => ws.send(JSON.stringify({
          text: "Hello from the browser.",
          description: "a warm 30s voice, smooth timbre, conversational pacing, like a friendly narrator",
          speaker: "ira",   // mulberry only; omit for muga / the described voice
        }));

        ws.onmessage = (e) => {
          if (e.data instanceof ArrayBuffer) {
            const pcm = new Int16Array(e.data);
            const buf = ctx.createBuffer(1, pcm.length, 24000);
            const ch = buf.getChannelData(0);
            for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
            const src = ctx.createBufferSource();
            src.buffer = buf;
            src.connect(ctx.destination);
            playAt = Math.max(playAt, ctx.currentTime);
            src.start(playAt);
            playAt += buf.duration;
          } else if (JSON.parse(e.data).type === "done" || JSON.parse(e.data).error) {
            ws.close();
          }
        };
      };
    </script>
  </body>
</html>

import requests

API_KEY = "rk_live_•••••••••"   # use a key with the tts:stream scope
BASE = "https://silk-api.rumik.ai"

# call this from your POST /tts-session handler
def tts_session() -> dict:
    r = requests.post(
        f"{BASE}/v1/tts/ws-connect",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"model": "mulberry", "text": "Hello from the browser."},
        timeout=30,
    )
    r.raise_for_status()
    s = r.json()
    return {"ws_url": s["ws_url"], "token": s["token"]}   # forward only these to the page

interruption (barge-in)

for a live “call with ai”, keep the websocket open for the whole call and send a new {"text": "..."} frame for each thing the agent says. if the caller talks over the agent, just send the next utterance: a new text frame while one is still generating interrupts the current one and starts the new one. this is “latest-wins”, there is no special barge-in frame.

frames

once connected, the session is a two-way stream of json text frames and binary audio. client → server

frame	meaning
`{"text": "..."}`	start a generation. sending another while one is in progress is a barge-in: it interrupts the current generation and starts the new one.
`{"type": "cancel"}`	stop the current generation with no replacement.
`{"type": "close"}`	end the session.

server → client

frame	meaning
`binary`	a chunk in the session’s negotiated format; raw 24 khz mono pcm when `audio_format` was omitted.
`{"type": "done", "request_id", "credits_used", ...}`	the generation finished normally. `credits_used` is retained as wire-compatibility metadata; use the usage dashboard for current plan usage.
`{"type": "cancelled", "request_id", "reason": ...}`	the generation was stopped. `reason` is `"interrupt"` (a new request replaced it) or `"cancel"` (explicit cancel).
`{"type": "timeout"}`	idle timeout.
`{"error": true, "code", "recoverable", ...}`	an error. no `done` follows it. when `recoverable` is `true` the socket stays open and you can send the next frame.

on a barge-in you receive {"type": "cancelled", "reason": "interrupt"} for the old request, then the audio chunks and done for the new one. the old audio stops almost immediately.

errors

an error frame ends the current generation, so stop waiting for done when you see one.

`code`	`recoverable`	what happened
`INVALID_TEXT`	`true`	`text` has nothing to speak: empty, whitespace, punctuation only or emoji only (no letter or digit in any script). not charged. the socket stays open.
`INVALID_MESSAGE`	`true`	the frame was not a json object. the socket stays open.

{ "error": true, "code": "INVALID_TEXT", "recoverable": true }

POST /v1/tts/ws-connect shares POST /v1/tts’s UNSUPPORTED_AUDIO_FORMAT (400) and VALIDATION_ERROR (422), but not INVALID_TEXT: the mint does not check the text, the socket does, as the frame above. see quickstart.

behavior

latest-wins: only the newest utterance is held. a burst of requests during one generation never queues a backlog of now-stale audio, only the latest runs next.
immediate stop: on interrupt or cancel the old audio stops streaming right away, so the caller stops hearing it within a couple hundred milliseconds (mostly network round-trip).

example

python

# inside an open `ws` from the example above, for a live call:
await ws.send(json.dumps({"text": "[neutral] Aapka call connect ho gaya hai."}))

# caller interrupts -> just send the next line. the old one is cancelled,
# the new one starts.
await ws.send(json.dumps({"text": "[happy] Haan ji, bataiye main kaise help karun?"}))

# or stop the current line with no replacement:
await ws.send(json.dumps({"type": "cancel"}))

# end the call:
await ws.send(json.dumps({"type": "close"}))

documentation

sdk

integrations

cookbook

api reference

choose an audio format

interruption (barge-in)

frames

errors

behavior

example

​choose an audio format

​interruption (barge-in)

​frames

​errors

​behavior

​example

choose an audio format

interruption (barge-in)

frames

errors

behavior

example