# limits
Source: https://docs.rumik.ai/agent-limits
how much capacity you have, and how much is in use right now.
`GET /v1/agent/limits` reports what the calling key is allowed to run and what
it is running. use it to queue callers gracefully instead of discovering a
`429` mid-flow.
```bash curl theme={null}
curl https://silk-api.rumik.ai/v1/agent/limits \
-H "Authorization: Bearer rk_live_•••••••••"
```
## response
how many calls your plan may run at once.
how many are running right now.
your billing mode. this is what decides how a call is paid for:
* `payg` — calls are billed per second from your credit balance.
* `unlimited` — you are on a concurrency plan; calls run on its slots and cost
nothing per second.
a legacy credit mode may also appear on older accounts and bills like `payg`.
see [billing](/voice-agents#billing) for the full comparison.
```json theme={null}
{ "concurrency_limit": 4, "active_requests": 1, "plan": "payg" }
```
## on a concurrency plan, this is your only ceiling
if `plan` is `unlimited`, `concurrency_limit` is the whole story: calls cost
nothing per second, and the limit is what stops them. a credit balance sitting
in the account is **not** a fallback — a call that arrives with every slot busy
is refused with `429`, it is not charged to your credits instead.
if `plan` is `payg`, there are two ceilings and you can hit either: this one
(`429`) and your balance (`402 insufficient_balance`).
## the pay-as-you-go default: 4 slots
without an active subscription you are pay-as-you-go, and that comes with
**4 concurrent slots out of the box** — nothing to buy, nothing to configure.
there is no separate "TTS limit" and "agent limit": it is one number for the
whole account, in the sense described just below.
a subscription raises it — each plan carries its own concurrency — and when a
subscription ends the account reverts to 4 immediately. limit changes apply from
the next request; capacity is read live, never cached.
the things sharing those 4 slots hold them for very different lengths of
time. a TTS request occupies a slot for the seconds it takes to synthesize; an
agent call occupies one for **the entire call**. four long calls will starve a
batch TTS job, and vice versa.
## capacity is account-wide
`active_requests` counts **everything** on the account, not just voice agents:
TTS requests, dashboard sessions and API calls all draw on the same pool. a
batch TTS job can therefore be the reason an agent call is refused.
that is also why this endpoint is the right thing to check: it measures exactly
what a `429` is measured against.
## using it
```python python theme={null}
import requests
BASE = "https://silk-api.rumik.ai"
HEADERS = {"Authorization": "Bearer rk_live_•••••••••"}
def has_capacity() -> bool:
limits = requests.get(f"{BASE}/v1/agent/limits", headers=HEADERS).json()
return limits["active_requests"] < limits["concurrency_limit"]
if has_capacity():
start_call()
else:
queue_caller() # or tell them to hold
```
this is a snapshot, not a reservation. between your check and your start,
another call can take the last slot — so handle
`429 concurrency_limit_exceeded` on the start anyway. the check reduces how
often you hit it; it does not replace handling it.
## raising your limit
concurrency comes from your plan. change it in
[billing](https://playground.rumik.ai/billing), or write to
[api@rumik.ai](mailto:api@rumik.ai) if you need more than the plans offer.
# coding-agent skill
Source: https://docs.rumik.ai/agent-skill
hand your ai coding agent everything it needs to integrate rumik tts correctly.
the `rumik-tts` skill packs everything an ai coding agent needs to integrate rumik
tts correctly through [pipecat](/pipecat) or [livekit](/livekit): which package and
class to use, how to configure muga and mulberry voices, the prompt rules each model
expects, and how to test the result.
a drop-in skill for claude code, cursor, and other coding agents.
## install it
unzip into your agent's skills directory to get a `rumik-tts/` folder with
`SKILL.md` and reference docs:
```bash theme={null}
unzip rumik-tts-skill.zip -d ~/.claude/skills/
```
## what's inside
| file | purpose |
| ----------------------------------- | ------------------------------------------------------- |
| `SKILL.md` | workflow, framework choice, settings, and prompt rules. |
| `references/pipecat-integration.md` | pipecat pipeline patterns and examples. |
| `references/livekit-integration.md` | the livekit `rumik_ai.TTS` plugin. |
| `references/rumik-api.md` | payloads, audio format, and response behavior. |
| `references/prompting.md` | muga and mulberry prompt guidance. |
use it when asking an agent to add rumik tts to a pipecat, livekit, or python
voice-agent project. it steers the agent toward the official packages instead of
hand-rolled http/websocket clients.
# create a streaming session
Source: https://docs.rumik.ai/api-reference/speech/create-a-streaming-session
/openapi.json post /v1/tts/ws-connect
mint a one-shot websocket session for real-time streaming synthesis. set `audio_format` here to negotiate `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`; omit it for raw 24 khz mono signed 16-bit little-endian pcm. connect only to the returned `ws_url`, append `?token=`, and send a json synthesis frame. binary messages contain the negotiated audio stream and are followed by a terminal `{"type":"done"}` json frame.
# synthesize speech
Source: https://docs.rumik.ai/api-reference/speech/synthesize-speech
/openapi.json post /v1/tts
synthesize an utterance and return binary audio. omit `audio_format` for the original 24 khz mono wav response, or request `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`. pass your api key as a bearer token. payg tts is prorated from the exact submitted `text` character count, including whitespace, punctuation, newlines, and inline tags. the output format and generated audio duration do not change the charge. unlimited usage records both character count and audio milliseconds with zero usage charge.
# synthesize speech as json
Source: https://docs.rumik.ai/api-reference/speech/synthesize-speech-as-json
/openapi.json post /v1/tts/json
synthesize an utterance and return a json object whose `audio_base64` field contains the requested audio bytes as base64. omit `audio_format` for wav, or request `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`. the remaining response fields keep their existing behavior.
# get concurrency limits
Source: https://docs.rumik.ai/api-reference/voice-agents/get-concurrency-limits
/openapi.json get /v1/agent/limits
report the calling account's concurrency ceiling and how much of it is in use. capacity is account-wide — tts, the dashboard and voice agents share one pool — so this measures exactly what a 429 is measured against.
# register a realtime call
Source: https://docs.rumik.ai/api-reference/voice-agents/register-a-realtime-call
/openapi.json post /v1/register-call
exchange your api key for a single-use token that opens one realtime socket, so a browser never holds the key. registering does not start a call: nothing is billed and no capacity is taken until the socket connects.
# start a web call
Source: https://docs.rumik.ai/api-reference/voice-agents/start-a-web-call
/openapi.json post /v1/webcall
start a voice-agent call and return the credentials a browser joins it with over webrtc. media flows browser ↔ agent directly. requires an api key with the `agent` scope and an agent that has been deployed. the returned `callId` identifies the call in conversations and usage.
# audio formats
Source: https://docs.rumik.ai/audio-formats
request wav, opus, pcm, mulaw, alaw, or mp3 audio from silk.
silk supports five explicit `audio_format` values on both `muga` and `mulberry`:
`opus`, `pcm`, `mulaw`, `alaw`, and `mp3`.
the field is optional. omit it entirely to preserve the original response:
* `post /v1/tts` returns a 24 khz mono wav file.
* `post /v1/tts/json` returns the same wav bytes in `audio_base64`.
* websocket sessions stream raw 24 khz mono signed 16-bit little-endian pcm.
do not send `"audio_format": null`. explicit `null` is invalid and returns
`400 unsupported_audio_format`. omit the field when you want the default.
## supported values
| value | output | content type | common extension |
| ------- | ----------------------------------------------- | ------------------------------------------------- | ---------------- |
| omitted | wav containing 24 khz mono signed 16-bit pcm | `audio/wav` | `.wav` |
| `opus` | opus in an ogg container | `audio/ogg; codecs=opus` | `.ogg` |
| `pcm` | raw 24 khz mono signed 16-bit little-endian pcm | `audio/pcm; rate=24000; channels=1; format=s16le` | `.pcm` |
| `mulaw` | raw 8 khz mono g.711 mu-law | `audio/x-mulaw; rate=8000; channels=1` | `.mulaw` |
| `alaw` | raw 8 khz mono g.711 a-law | `audio/x-alaw; rate=8000; channels=1` | `.alaw` |
| `mp3` | mp3 audio | `audio/mpeg` | `.mp3` |
`wav` and `ogg` are not accepted values. omit `audio_format` for wav, and use
`opus` for opus audio in an ogg container.
## binary http response
add `audio_format` to the json request body. the response body contains the
audio bytes directly, and the response `content-type` identifies the format.
```bash curl theme={null}
api_key='rk_live_•••••••••'
curl --request post https://silk-api.rumik.ai/v1/tts \
--header "authorization: bearer ${api_key}" \
--header "content-type: application/json" \
--data '{"model":"muga","text":"[happy] hello from silk.","audio_format":"opus"}' \
--output speech.ogg
```
change only the `audio_format` value and output extension to request another
format. leave the field out to receive `speech.wav` with the original behavior.
## json base64 response
the json endpoint keeps the response object and replaces `audio_base64` with
the requested audio bytes encoded as base64.
```python python theme={null}
import base64
import requests
api_key = "rk_live_•••••••••"
response = requests.post(
"https://silk-api.rumik.ai/v1/tts/json",
headers={"authorization": f"bearer {api_key}"},
json={
"model": "mulberry",
"text": "hello from silk.",
"description": "a warm conversational voice",
"audio_format": "mp3",
},
timeout=90,
)
response.raise_for_status()
result = response.json()
with open("speech.mp3", "wb") as output:
output.write(base64.b64decode(result["audio_base64"]))
```
the response also includes `request_id`, `audio_duration_ms`, `credits_used`,
and `usage_cost_nanos`. the requested format does not change billing.
## websocket streaming
set `audio_format` when minting the one-shot session. do not add it to later
synthesis frames: the gateway stores the negotiated format in the session.
```python python theme={null}
import asyncio
import json
import requests
import websockets
api_key = "rk_live_•••••••••"
async def main():
session_response = requests.post(
"https://silk-api.rumik.ai/v1/tts/ws-connect",
headers={"authorization": f"bearer {api_key}"},
json={
"model": "muga",
"text": "stream this as opus.",
"audio_format": "opus",
},
timeout=30,
)
session_response.raise_for_status()
session = session_response.json()
chunks = bytearray()
async with websockets.connect(
f'{session["ws_url"]}?token={session["token"]}'
) as websocket:
await websocket.send(json.dumps({"text": "stream this as opus."}))
async for message in websocket:
if isinstance(message, bytes):
chunks.extend(message)
elif json.loads(message).get("type") == "done":
break
with open("speech.ogg", "wb") as output:
output.write(chunks)
asyncio.run(main())
```
always connect to the returned `ws_url`. the negotiated format applies at the
silk gateway. connecting to any other endpoint is outside this contract.
for `opus` and `mp3`, concatenate binary websocket messages in order before
decoding the complete stream. individual messages are not guaranteed to be
standalone audio files.
## billing and stored audio
payg tts remains based on the exact submitted text character count. choosing a
different audio format does not add a surcharge and does not make billing depend
on encoded byte length or generated duration.
silk stores its canonical wav source for usage and audit workflows regardless
of the format returned to the client.
# cookbook
Source: https://docs.rumik.ai/cookbook/index
step-by-step guides for building with rumik tts.
practical, build-it-from-scratch guides. each one is a complete walkthrough you
can follow start to finish, in plain language, with every command and file you
need.
## build a voice agent
a voice agent listens to a person, thinks, and talks back, all in real time. the
loop is always the same four steps:
```
you speak → STT (speech to text) → LLM (the brain) → rumik TTS (text to speech) → it speaks
```
silk is the **last step**: it turns the agent's reply into natural
speech. these guides wire up the whole loop around it. pick your framework:
build a voice agent using pipecat, an open-source framework for real-time voice.
build a voice agent using livekit agents, with rooms and telephony built in.
use livekit just for webrtc transport while pipecat orchestrates the pipeline.
## which framework?
both give you a production-ready real-time agent. the difference is what they
ship around the voice loop:
| | pipecat | livekit |
| ------------ | ---------------------------------------------------- | ------------------------------------------------------ |
| best for | a lightweight, code-first pipeline you fully control | rooms, web/mobile sdks, and phone calls out of the box |
| transport | brings its own | livekit's webrtc rooms |
| rumik plugin | [`pipecat-rumik`](/pipecat) | [`livekit-plugins-rumik-ai`](/livekit) |
new to both? **start with pipecat**, it's the shortest path to hearing your agent
talk.
these guides assume you can run python and use a terminal. you do not need prior
experience with voice agents.
# build a voice agent with livekit
Source: https://docs.rumik.ai/cookbook/voice-agent-livekit
a complete, from-scratch walkthrough: a real-time livekit voice agent that talks back in hinglish.
by the end you'll have a voice agent you can talk to, that replies out loud in
natural hinglish using rumik. we'll build it with
[livekit agents](https://docs.livekit.io/agents/), which gives you rooms, web and
mobile sdks, and phone calls out of the box.
## what you'll build
a real-time loop running inside a livekit room: you speak, the agent transcribes
you, an llm writes a reply, and silk speaks it back.
```
🎙️ you speak → STT (deepgram) → LLM (openai) → rumik TTS → 🔊 it speaks
```
silk is the voice. you can swap the stt and llm for any provider livekit supports.
## before you start
you need:
* **python 3.10+** and a terminal.
* a free **livekit cloud** project from [cloud.livekit.io](https://cloud.livekit.io)
(gives you a url, api key, and secret).
* three more api keys:
* **rumik** for the voice, from [your dashboard](https://playground.rumik.ai/api-keys).
* **deepgram** for speech-to-text ([deepgram.com](https://deepgram.com)).
* **openai** for the llm ([platform.openai.com](https://platform.openai.com)).
you can swap deepgram or openai for any stt / llm that livekit supports. we use
these two because they're quick to set up.
## step 1 · set up the project
make a folder, a virtual environment, and install the packages.
```bash theme={null}
mkdir rumik-livekit-agent && cd rumik-livekit-agent
python -m venv venv
source venv/bin/activate # windows: venv\Scripts\activate
pip install "livekit-agents[deepgram,openai,silero]" livekit-plugins-rumik-ai
```
`livekit-plugins-rumik-ai` is the official rumik tts plugin.
## step 2 · add your keys
create a file called `.env`:
```bash .env theme={null}
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=•••••••••
LIVEKIT_API_SECRET=•••••••••
RUMIK_API_KEY=rk_live_•••••••••
DEEPGRAM_API_KEY=•••••••••
OPENAI_API_KEY=sk-•••••••••
```
the livekit values come from your livekit cloud project settings.
## step 3 · write the agent
create `agent.py`. livekit wires the four steps together in an `AgentSession`. the
rumik part is the `tts` line.
```python agent.py theme={null}
from dotenv import load_dotenv
from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli
from livekit.plugins import deepgram, openai, silero, rumik_ai
load_dotenv()
# muga is steered by a [tone] tag, so we tell the LLM to add one
INSTRUCTIONS = """
You write text spoken by the Silk Muga 1 text-to-speech model.
- Output only the final tagged text, no markdown or notes.
- Devanagari or Latin (romanised) both work; use whichever you prefer.
- Start every reply with one tone tag, as the first token:
[happy], [excited], [sad], [angry], [neutral], [whisper].
- Keep replies short: 1 to 2 sentences.
"""
async def entrypoint(ctx: JobContext):
await ctx.connect()
session = AgentSession(
stt=deepgram.STT(),
llm=openai.LLM(model="gpt-4o-mini"),
tts=rumik_ai.TTS(model="muga"), # the rumik voice
vad=silero.VAD.load(), # detects when you start/stop talking
)
await session.start(
agent=Agent(instructions=INSTRUCTIONS),
room=ctx.room,
)
if __name__ == "__main__":
cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```
`rumik_ai.TTS` reads `RUMIK_API_KEY` from the environment automatically. the system
prompt makes the llm emit muga's `[tone]` tags. the full rules are in
[prompting muga](/prompting-muga).
**tones are required — but there's a default.** muga speaks one `[tone]` per
line, and the `INSTRUCTIONS` above make the llm tag every reply. if a reply ever
arrives without a tag, the plugin falls back to `[neutral]` so it never errors —
or set your own fallback with `rumik_ai.TTS(model="muga", tone="happy")`.
## step 4 · run it
start the agent worker:
```bash theme={null}
python agent.py dev
```
then open the [livekit agents playground](https://agents-playground.livekit.io),
connect to your project, and talk. you'll hear muga reply in hinglish. it streams,
and livekit handles interruptions and turn-taking for you.
## customize it
use `rumik_ai.TTS(model="mulberry", description="...")` to design any voice. see
[prompting mulberry](/prompting-mulberry).
add `speaker="ira"` alongside your `description` to pin one of the twelve
named voices for the whole conversation.
edit the `INSTRUCTIONS` system prompt. that's the agent's character.
`rumik_ai.TTS(model="muga", tone="neutral")` prefixes any untagged reply with
that tone, so the llm doesn't have to tag every line.
livekit handles telephony and web/mobile sdks from the same agent.
## next steps
* [livekit integration reference](/livekit) for every constructor option.
* [prompting muga](/prompting-muga) and [prompting mulberry](/prompting-mulberry).
* prefer pipecat? build the [same agent with pipecat](/cookbook/voice-agent-pipecat).
# build a voice agent with pipecat
Source: https://docs.rumik.ai/cookbook/voice-agent-pipecat
a complete, from-scratch walkthrough: a real-time voice agent that talks back in hinglish.
by the end you'll have a voice agent you can talk to, that replies out loud in
natural hinglish using rumik. we'll build it with
[pipecat](https://github.com/pipecat-ai/pipecat), an open-source framework for
real-time voice.
## what you'll build
a real-time loop: you speak, the agent transcribes you, an llm writes a reply, and
silk speaks it back.
```
🎙️ you speak → STT (deepgram) → LLM (openai) → rumik TTS → 🔊 it speaks
```
you'll wire up three services. silk is the voice. you can swap the stt and llm for
any provider pipecat supports.
## before you start
you need:
* **python 3.10+** and a terminal.
* three api keys:
* **rumik** for the voice, from [your dashboard](https://playground.rumik.ai/api-keys).
* **deepgram** for speech-to-text ([deepgram.com](https://deepgram.com)).
* **openai** for the llm ([platform.openai.com](https://platform.openai.com)).
you can swap deepgram or openai for any stt / llm that pipecat supports. we use
these two because they're quick to set up.
## step 1 · set up the project
make a folder, a virtual environment, and install the packages.
```bash theme={null}
mkdir rumik-voice-agent && cd rumik-voice-agent
python -m venv venv
source venv/bin/activate # windows: venv\Scripts\activate
pip install "pipecat-ai[deepgram,openai,silero]" pipecat-rumik
```
`pipecat-rumik` is the official rumik tts service. the rest is pipecat plus the
stt and llm plugins.
## step 2 · add your keys
create a file called `.env` in the folder:
```bash .env theme={null}
RUMIK_API_KEY=rk_live_•••••••••
RUMIK_GATEWAY_URL=https://silk-api.rumik.ai
DEEPGRAM_API_KEY=•••••••••
OPENAI_API_KEY=sk-•••••••••
```
never hard-code keys in your script. we'll load them from this file.
## step 3 · write the agent
create `agent.py`. this builds the four-step loop. the rumik part is the `tts`
line.
```python agent.py theme={null}
import os
from dotenv import load_dotenv
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat_rumik import RumikTTSService
load_dotenv()
# speech to text
stt = DeepgramSTTService(api_key=os.environ["DEEPGRAM_API_KEY"])
# the brain. keep replies short and in hinglish (devanagari or romanised) so muga sounds natural.
llm = OpenAILLMService(
api_key=os.environ["OPENAI_API_KEY"],
model="gpt-4o-mini",
)
# the voice: rumik muga
tts = RumikTTSService(
api_key=os.environ["RUMIK_API_KEY"],
gateway_url=os.environ["RUMIK_GATEWAY_URL"],
settings=RumikTTSService.Settings(model="muga"),
)
# the loop: audio in → stt → llm → rumik tts → audio out
pipeline = Pipeline([stt, llm, tts])
# connect this pipeline to a transport (a phone call, a web room, or your mic)
# and run it. see the runnable examples linked below for a complete transport.
```
the surrounding pieces (the transport that carries audio, the system prompt, the
context aggregator) come straight from the pipecat quickstart. the
[`pipecat-rumik` examples](https://pypi.org/project/pipecat-rumik/) ship a
complete, runnable agent you can copy.
## step 4 · make the llm speak muga's language
muga is steered by a `[tone]` tag at the start of each reply. tell your llm to add
one. paste this into the llm's system prompt:
```text theme={null}
You write text spoken by the Silk Muga 1 text-to-speech model.
- Output only the final tagged text, no markdown or notes.
- Devanagari or Latin (romanised) both work; use whichever you prefer.
- Start every paragraph with one tone tag, as the first token:
[happy], [excited], [sad], [angry], [neutral], [whisper].
- Keep replies short: 1 to 2 sentences.
```
now the llm produces `[happy] Haan ji, ho gaya!` and silk speaks it with the right
emotion. the full prompt rules are in [prompting muga](/prompting-muga).
## step 5 · run it
```bash theme={null}
python agent.py
```
speak into your mic. you'll hear muga reply in hinglish. it streams, so the first
audio comes back fast, and pipecat handles interruptions for you.
## customize it
switch to `model="mulberry"` and add a `description` to design any voice. see
[prompting mulberry](/prompting-mulberry).
edit the llm system prompt. that's the agent's character.
pipecat supports many providers. change the `stt` or `llm` line.
hand the [rumik tts skill](/agent-skill) to your coding agent.
## next steps
* [pipecat integration reference](/pipecat) for every setting and both transports.
* [prompting muga](/prompting-muga) and [prompting mulberry](/prompting-mulberry).
* prefer livekit? build the [same agent with livekit](/cookbook/voice-agent-livekit).
# pipecat pipeline over livekit transport
Source: https://docs.rumik.ai/cookbook/voice-agent-pipecat-livekit
use livekit purely as the webrtc transport while pipecat runs the stt to llm to rumik tts pipeline.
want livekit just for transport (webrtc rooms, web and mobile sdks, telephony) but
pipecat to run the pipeline? **pipecat ships a livekit transport**, so the whole agent
is one pipecat pipeline that sends and receives audio over a livekit room. no glue
code needed.
## which livekit setup is this?
| approach | you write | rumik via | owns the loop |
| ------------------------------- | ------------------------ | -------------------------------------- | ------------- |
| livekit agents | a livekit `AgentSession` | [`livekit-plugins-rumik-ai`](/livekit) | livekit |
| **pipecat + livekit transport** | a pipecat `Pipeline` | [`pipecat-rumik`](/pipecat) | pipecat |
this guide is the second row: **livekit is transport only.**
## what you'll build
one pipecat pipeline, with the livekit transport at both ends:
**livekit** `transport.input()` → stt → llm → rumik tts → **livekit** `transport.output()`
## before you start
* **python 3.10+**
* a [livekit](https://cloud.livekit.io) project (url, api key, secret)
* a [rumik key](https://playground.rumik.ai/api-keys)
* an stt and llm (this guide uses deepgram + openai, both swappable)
## step 1 · install
```bash theme={null}
pip install "pipecat-ai[livekit,deepgram,openai,silero]" pipecat-rumik
```
the `livekit` extra adds pipecat's transport; `pipecat-rumik` is the voice.
## step 2 · keys
```bash .env theme={null}
LIVEKIT_URL=wss://your-project.livekit.cloud
RUMIK_API_KEY=rk_live_•••••••••
RUMIK_GATEWAY_URL=https://silk-api.rumik.ai
DEEPGRAM_API_KEY=•••••••••
OPENAI_API_KEY=sk-•••••••••
```
## step 3 · build the pipeline
the transport is the only livekit-specific line. the rest is plain pipecat with
`RumikTTSService` as the voice.
```python agent.py theme={null}
import os
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.transports.livekit.transport import LiveKitParams, LiveKitTransport
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat_rumik import RumikTTSService
# muga is steered by a [tone] tag, so the LLM must add one. give your LLM this
# system prompt (see /prompting-muga).
SYSTEM_PROMPT = "Start every reply with one tone tag ([happy], [sad], ...). Romanised Hinglish, 1-2 sentences."
async def run_agent(url: str, token: str, room_name: str):
# livekit is ONLY the transport, pipecat owns the pipeline
transport = LiveKitTransport(
url=url,
token=token,
room_name=room_name,
params=LiveKitParams(audio_in_enabled=True, audio_out_enabled=True),
)
stt = DeepgramSTTService(api_key=os.environ["DEEPGRAM_API_KEY"])
llm = OpenAILLMService(api_key=os.environ["OPENAI_API_KEY"], model="gpt-4o-mini")
tts = RumikTTSService(
api_key=os.environ["RUMIK_API_KEY"],
gateway_url=os.environ["RUMIK_GATEWAY_URL"],
settings=RumikTTSService.Settings(model="muga"),
)
# livekit in → stt → llm → rumik tts → livekit out
pipeline = Pipeline([transport.input(), stt, llm, tts, transport.output()])
await PipelineRunner().run(PipelineTask(pipeline))
```
the transport import path and runner api track your pipecat version, see the
[pipecat docs](https://docs.pipecat.ai/). `RumikTTSService` is the same as the
[pipecat integration](/pipecat) reference.
## step 4 · connect a caller
mint a livekit token per participant (see livekit's
[token docs](https://docs.livekit.io/home/get-started/authentication/)), start
`run_agent` in the room, and join from your client or the
[agents playground](https://agents-playground.livekit.io). once both are in the room
you talk to the bot, pipecat handles turn-taking over livekit's webrtc.
## next steps
* want a described voice? set `model="mulberry"` with a `description`, see
[prompting mulberry](/prompting-mulberry).
* want livekit to own the loop instead? use the
[livekit agents plugin](/livekit).
* [pipecat integration](/pipecat) for every `RumikTTSService` setting.
# overview
Source: https://docs.rumik.ai/index
the rumik silk text-to-speech api.
silk is rumik ai's text-to-speech api. it turns text into natural, expressive
speech over a simple http call or a real-time websocket stream. use the original
wav and raw pcm defaults, or request opus, pcm, mulaw, alaw, or mp3.
pick the model that fits your use case:
our more expressive model. steer delivery with a tone tag like `[happy]` and
inline events like ``.
our faster model. steer with a natural-language `description`, or pick a
preset studio voice.
## start here
get a key and synthesize your first clip in three steps.
steer muga and mulberry with tones, tags, and descriptions.
low-latency playback over websocket.
receive opus, pcm, mulaw, alaw, or mp3 from every delivery mode.
drop rumik into a pipecat pipeline with `pipecat-rumik`.
endpoints, request fields, and a live playground.
## your account
create and manage keys in your dashboard.
view usage, current spend, and your plan details.
## audio format
omit `audio_format` for the original behavior: http returns a 24 khz mono wav,
and websocket sessions stream raw 24 khz mono signed 16-bit pcm. set
`audio_format` to `opus`, `pcm`, `mulaw`, `alaw`, or `mp3` to receive that format
instead. see [audio formats](/audio-formats) for exact content types and examples.
## status
live api uptime and incident history are at
[silk-api.statuspage.io](https://silk-api.statuspage.io).
# build a voice agent with livekit
Source: https://docs.rumik.ai/livekit
drop rumik into a livekit agent with livekit-plugins-rumik-ai.
building on [livekit agents](https://docs.livekit.io/agents/)? use
[`livekit-plugins-rumik-ai`](https://pypi.org/project/livekit-plugins-rumik-ai/),
our official livekit tts plugin. it drops rumik straight into an `AgentSession`
with streaming audio and interruption handling already wired up.
```bash theme={null}
pip install livekit-plugins-rumik-ai
```
set your key from the dashboard:
```bash theme={null}
export RUMIK_API_KEY="rk_live_•••••••••"
```
## add it to an agent
the `TTS` class plugs into a livekit `AgentSession` next to your stt and llm:
```python theme={null}
from livekit.agents import AgentSession
from livekit.plugins import rumik_ai
# muga: expressive, tone-tagged hinglish
session = AgentSession(
stt=..., # your speech-to-text plugin
llm=..., # your llm plugin
tts=rumik_ai.TTS(model="muga"),
)
```
steer mulberry with a natural-language description, or pin a preset speaker:
```python theme={null}
# mulberry: description-driven voice
tts = rumik_ai.TTS(
model="mulberry",
description="a female 30s hindi voice, warm timbre, conversational pacing, like a podcast host",
)
# or pin a named studio voice. description still goes with it.
tts = rumik_ai.TTS(
model="mulberry",
description="a female 30s hindi voice, warm timbre, conversational pacing, like a podcast host",
speaker="ira",
)
```
## constructor options
| argument | applies to | notes |
| --------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------ |
| `model` | both | `"muga"` or `"mulberry"`. default `"muga"`. |
| `api_key` | both | defaults to the `RUMIK_API_KEY` environment variable. |
| `base_url` | both | defaults to `https://silk-api.rumik.ai`. |
| `full_response_aggregation` | both | buffer the full reply before synthesis. default `True` for muga, `False` for mulberry. |
| `tone` | muga | fallback tone when the input text has no `[tone]` marker. |
| `description` | mulberry | required. natural-language voice description. |
| `speaker` | mulberry | optional. a named voice, e.g. `"ira"`. send `description` too. see [preset voices](/mulberry#preset-voices). |
it also accepts the shared sampling params (`temperature`, `top_p`, `top_k`,
`repetition_penalty`, `max_new_tokens`).
muga aggregates the full tagged reply before speaking (`full_response_aggregation`
is `True` by default), so it never tries to synthesize a half-tagged sentence.
see [prompting muga](/prompting-muga) for why.
## which model?
* **muga** for short, expressive reactions you steer with `[tone]` tags. see
[prompting muga](/prompting-muga).
* **mulberry** for low-latency conversational agents you steer with a
`description`. see [prompting mulberry](/prompting-mulberry).
prefer a full walkthrough? the [livekit cookbook](/cookbook/voice-agent-livekit)
builds a working voice agent from scratch. or hand your coding agent the
[rumik tts skill](/agent-skill) so it wires all of this up on the first try.
# silk muga 1
Source: https://docs.rumik.ai/muga
our expressive text-to-speech voice.
**silk muga 1** is our expressive text-to-speech voice. you set the mood with one
`[tone]` tag and drop in inline events like a laugh or a sigh. it's tuned for short,
natural, spoken delivery: reactions, banter, support replies, storytelling.
## at a glance
| field | value |
| -------- | ---------------------------------------------------------------------------------------------------------- |
| model id | `muga` |
| language | hindi and english, code-mixed. both scripts work: devanagari (`कैसे हो`) and latin/romanised (`kaise ho`). |
| best for | short expressive lines: reactions, banter, support, storytelling |
| length | 2 to 40 seconds per utterance |
| not for | very long passages (past \~40s) |
## quickstart
one call, one `.wav` file:
```bash theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{
"model": "muga",
"text": "[happy] Yaar tumne phir wahi joke maara!",
"temperature": 0.7
}' \
--output muga.wav
```
to receive `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`, add `audio_format` to the
request. see [audio formats](/audio-formats) for exact values and examples.
a request is a `[tone]`, an optional ``, and your words, in devanagari or
romanised hinglish. the full request schema is in the [api reference](/api-reference).
## tones
six moods. pick by ear:
| tone | tag | sounds like |
| ------- | ----------- | -------------------------------- |
| neutral | `[neutral]` | flat, even, no affect |
| happy | `[happy]` | bright, smiling, mid-energy |
| excited | `[excited]` | loud, fast, pitch up |
| sad | `[sad]` | slow, breathy, low |
| angry | `[angry]` | tight, clipped, sharp |
| whisper | `[whisper]` | quiet, breathy, no voiced energy |
## parameters
`muga` is steered mainly by the tone tag in your `text`. it also accepts the shared
sampling parameters:
| field | default | notes |
| -------------------- | ------- | ------------------------------------------------------------ |
| `text` | n/a | required. up to 2000 characters. must start with a `[tone]`. |
| `temperature` | `0.6` | **set `0.7` for muga.** most consistent run to run. |
| `top_p` | `0.95` | nucleus sampling |
| `top_k` | `50` | top-k sampling |
| `repetition_penalty` | `1.2` | penalise repeated tokens |
| `max_new_tokens` | `2048` | output length cap |
→ to write good prompts, read the [prompting guide](/prompting-muga).
# silk mulberry 1.5
Source: https://docs.rumik.ai/mulberry
our faster, description-driven text-to-speech voice.
**silk mulberry 1.5** is our faster voice. you describe how it should sound in one
natural sentence, then give it your text. it streams, so it's a good fit for
low-latency voice agents.
## at a glance
| field | value |
| -------- | ------------------------------------------------------------------------ |
| model id | `mulberry` |
| language | hindi in devanagari, english in latin (code-mixed) |
| best for | low-latency narration, voice agents, and creative voices |
| steering | a natural-language `description`, optionally pinned to a named `speaker` |
| length | up to 2000 characters per request |
## quickstart
one call, one `.wav` file:
```bash theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{
"model": "mulberry",
"text": "आज का episode थोड़ा अलग है।",
"description": "a female 30s hindi voice, smooth timbre, conversational pacing, casual register, like a podcast host"
}' \
--output mulberry.wav
```
to receive `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`, add `audio_format` to the
request. see [audio formats](/audio-formats) for exact values and examples.
put the voice in `description` and your spoken text in `text`. the full request
schema is in the [api reference](/api-reference).
## preset voices
`description` is what builds the voice, and it's always required. leave `speaker`
out and mulberry generates a voice to match the description you wrote. add
`speaker` to pin one of twelve named voices instead:
| voice | gender |
| ------------------------------------------------------- | ------ |
| `emma` `mia` `sophia` `ava` `ira` `siya` `aisha` `zoya` | female |
| `lucas` `noah` `theo` `adam` | male |
send `description` alongside it, exactly as you would without a speaker:
```bash theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{
"model": "mulberry",
"text": "आज का episode थोड़ा अलग है।",
"description": "a female 30s hindi voice, smooth timbre, conversational pacing, casual register, like a podcast host",
"speaker": "siya"
}' \
--output siya.wav
```
names are case-insensitive. send a name we don't know and you get a voice built
from your `description`, not an error, so a typo sounds like the wrong voice
rather than failing loudly.
**deprecating soon.** the old numbered preset values still work for now and map
to `ira`, `siya`, `aisha` and `zoya`, in that order. they're going away, so move
to the names above.
## parameters
| field | default | notes |
| -------------------- | ------- | ------------------------------------------------------------------------------ |
| `text` | n/a | required. up to 2000 characters. |
| `description` | n/a | required. natural-language voice description. |
| `speaker` | n/a | optional. a named voice, e.g. `siya`. still send `description`. |
| `temperature` | `0.6` | sampling temperature. |
| `top_p` | `0.95` | nucleus sampling. |
| `top_k` | `50` | top-k sampling. |
| `repetition_penalty` | `1.2` | penalise repeated tokens. |
| `max_new_tokens` | `2048` | output length cap. if a long line comes back cut off, raise it (up to `8192`). |
**getting truncated audio?** if the returned speech stops before the end of your
text, the generation hit the token cap. raise `max_new_tokens` above the default
`2048`, up to a maximum of `8192`.
→ to write good descriptions, read the [prompting guide](/prompting-mulberry).
# numbers, ids & dates
Source: https://docs.rumik.ai/normalization
how to write text so silk speaks numbers, money, dates, and ids correctly.
silk normalizes the `text` you send before it synthesizes, so numbers, money, dates,
and ids come out right. `Rs 2,45,600` is read as "two lakh forty five thousand six
hundred rupees", and `09/03/2025` as "ninth march two thousand twenty five". you write
text the normal way.
there is one thing you decide: which digits should be read one at a time. wrap those in
double quotes. write everything else plainly.
normalization runs by default. set `"normalization": false` on a request to send your
text through untouched.
## the one rule
a run of digits can be read two ways, and only you know which you mean.
| you want it read as | how to write it | silk speaks |
| -------------------------------------------------------------------------------------- | ------------------------------ | ---------------------------------------------------- |
| a digit-by-digit id (otp, phone, account, card, pan, gstin, vehicle, coupon, tracking) | wrap it in double quotes `"…"` | `"482093"` reads as *four eight two zero nine three* |
| a quantity (prices, counts, years, scores) | write it plainly | `3400` reads as *three thousand four hundred* |
currency symbols, `%`, dates, and times are picked up on their own. write them the
natural way.
**why quotes?** a bare `8123456789` could be a phone number or a very large count.
rather than guess, silk spells out what you put in quotes and reads everything else
as a number. quoting is how you tell it which one you meant.
here it is in one request:
```bash theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{
"model": "muga",
"text": "[neutral] Aapka OTP \"482093\" hai. Rs 2,45,600 debit hue 09/03/2025 ko."
}' --output out.wav
# speaks: "…OTP four eight two zero nine three hai. two lakh forty five thousand six
# hundred rupees debit hue ninth March two thousand twenty five ko."
```
## quick reference
| data | write it as | spoken |
| ---------------- | ------------------------------- | -------------------------------------------------------------------- |
| amount (inr) | `Rs 2,45,600` | two lakh forty five thousand six hundred rupees |
| amount + paise | `Rs 89.75` | eighty nine rupees and seventy five paise |
| foreign currency | `$45` · `€60` · `£12` · `¥5000` | forty five dollars · sixty euros · twelve pounds · five thousand yen |
| percentage | `25%` · `7.5%` | twenty five percent · seven point five percent |
| decimal | `3.2` | three point two |
| plain number | `3400` | three thousand four hundred |
| date | `09/03/2025` | ninth march two thousand twenty five |
| time (24h) | `18:45` · `07:15` | six forty five pm · seven fifteen am |
| otp / pin | `"482093"` · `"0071"` | four eight two zero nine three · zero zero seven one |
| phone | `"8123456789"` | eight one two three four five six seven eight nine |
| phone (+91) | `plus "918123456789"` | plus nine one eight one two three… |
| account | `"000723145589"` | zero zero zero seven two three… |
| card ending | `ending "6042"` | ending six zero four two |
| pan | `"FGHIJ5678K"` | f g h i j five six seven eight k |
| gstin | `"27FGHIJ5678K1Z3"` | two seven f g h i j … one z three |
| vehicle | `"MH12AB4567"` | m h one two a b four five six seven |
| pincode | `"411045"` | four one one zero four five |
| coupon / order | `"FEST50"` · `"TRK44120983"` | f e s t five zero · t r k four four one two… |
| turn off | `"normalization": false` | (raw text, unchanged) |
## numbers & money
write quantities plainly. silk handles indian grouping (lakh/crore), decimals, and
currency symbols on its own. no quotes.
| write | spoken |
| ------------------- | ----------------------------------------- |
| `3400` | three thousand four hundred |
| `2,45,600` | two lakh forty five thousand six hundred |
| `3.2` | three point two |
| `Rs 750` or `Rs750` | seven hundred fifty rupees |
| `Rs 89.75` | eighty nine rupees and seventy five paise |
| `Rs 2,00,00,000` | two crore rupees |
| `$18.40` | eighteen dollars and forty cents |
currency symbols: `₹`, `Rs`, `Rs.`, `INR`, `$`, `€`, `£`, `¥`, with or without a space
before the amount.
**do**
* write `Rs 2,45,600`. the indian comma grouping is read as lakh/crore.
* put the symbol before the amount (`$45`, not `45$`).
**don't**
* don't quote an amount. `"3400"` reads as *"three four zero zero"*, not the price.
## percentages, decimals & time
| write | spoken |
| ------- | ------------------------ |
| `25%` | twenty five percent |
| `7.5%` | seven point five percent |
| `18:45` | six forty five pm |
| `07:15` | seven fifteen am |
| `06:05` | six oh five am |
| `22:30` | ten thirty pm |
time is read 24-hour to 12-hour with am/pm. use a period for decimals (`3.2`), never a
comma.
## dates
write `DD/MM/YYYY` or `DD-MM-YY` (india, day first). silk reads an ordinal day, then
the month name, then the year.
| write | spoken |
| ------------ | --------------------------------------- |
| `09/03/2025` | ninth march two thousand twenty five |
| `01-12-24` | first december two thousand twenty four |
| `7/6/2025` | seventh june two thousand twenty five |
use 4-digit years for anything historical or ambiguous. `24/7` and `3-2-1` are not read
as dates; they come out as their numbers.
## ids, always quote
put any identifier in double quotes. digits turn into words, letters stay as letters,
separators (spaces, `-`, `/`) are dropped, and leading zeros are kept.
| type | write | spoken |
| ----------------- | ----------------------------------- | ------------------------------------------------------------ |
| otp / pin | `"482093"` · `"0071"` | four eight two zero nine three · zero zero seven one |
| phone | `"8123456789"` | eight one two three four five six seven eight nine |
| toll-free | `"18002094321"` | one eight zero zero two zero nine four three two one |
| account | `"000723145589"` | zero zero zero seven two three one four five five eight nine |
| account (grouped) | `"5231 7789 4402 1176"` | five two three one seven seven eight nine … |
| aadhaar | `"5210 8834 9071"` | five two one zero eight eight three four nine zero seven one |
| card ending | `ending "6042"` | ending six zero four two |
| pan | `"FGHIJ5678K"` | f g h i j five six seven eight k |
| gstin | `"27FGHIJ5678K1Z3"` | two seven f g h i j five six seven eight k one z three |
| vehicle | `"MH12AB4567"` or `"MH-12-AB-4567"` | m h one two a b four five six seven |
| pincode | `"411045"` | four one one zero four five |
| coupon | `"FEST50"` | f e s t five zero |
| order / tracking | `"TRK44120983"` | t r k four four one two zero nine eight three |
### phone numbers with a country code
write the `+` as the word plus, and quote the digits:
```text theme={null}
Call plus "918123456789" → "…plus nine one eight one two three…"
```
writing `"+918123456789"` won't work; the `+` inside the quotes isn't spelled cleanly.
## per-model notes
the rules above are the same for both models. these are the only differences.
**muga.** your `[tone]` tag stays at the very start. normalization leaves tags and
`` alone, so tagging and normalizing never interfere with each other.
```text theme={null}
[excited] Aapko 3400 reward points mile hain, coupon "FEST50" abhi use karo!
```
**mulberry.** there are no tone tags to place, the voice comes from `description`.
normalization only ever touches `text`, so your `description` reaches the model exactly
as you wrote it.
```text theme={null}
Aapka refund Rs 1,899 process ho gaya. Reference "RFN20915" hai.
```
## edge cases & gotchas
* digits stuck to letters are left alone. `B2B`, `COVID19`, `12kg` are never split.
* a quoted sentence isn't spelled out. `"see you at 400"` reads *"see you at four
hundred"*, not letter by letter. only clean identifiers (every token has a digit) get
spelled.
* leading zeros survive only inside quotes. `"0083"` reads *"zero zero eight three"*; an
unquoted `0083` becomes *"eighty three"*.
* ranges: `4-6` reads *"four-six"*. write `4 to 6` if you want the word "to".
* running the same text through twice is safe; normalized text stays as it is.
## turning normalization off
set `"normalization": false` and silk reads your text as-is. nothing is spelled out or
converted.
here is the same input both ways:
```json theme={null}
// default (on): the quoted OTP is spelled out
{ "model": "muga", "text": "[neutral] Aapka OTP \"482093\" hai." }
// speaks: "…OTP four eight two zero nine three hai."
// off: the model gets the text untouched and reads it however it reads it
{ "model": "muga", "text": "[neutral] Aapka OTP \"482093\" hai.", "normalization": false }
// speaks: "…OTP hai."
```
reach for it when your text is already in the exact spoken form you want, or when you
need full control over how something is pronounced.
# build a voice agent with pipecat
Source: https://docs.rumik.ai/pipecat
drop rumik into a pipecat pipeline with pipecat-rumik.
building a real-time voice bot? skip the websocket plumbing and use
[`pipecat-rumik`](https://pypi.org/project/pipecat-rumik/), our official
[pipecat](https://github.com/pipecat-ai/pipecat) tts service. it drops rumik
straight into a pipecat pipeline with streaming audio, metrics, and interruption
handling already wired up.
```bash theme={null}
pip install pipecat-rumik
```
## pick a service
it ships two tts services. pick one for your transport:
| service | transport | best for |
| --------------------- | --------- | --------------------------------------------------------------------- |
| `RumikTTSService` | websocket | interactive voice agents that need interruption-aware, streaming tts. |
| `RumikHttpTTSService` | http | simpler request/response synthesis and batch-style flows. |
## add it to a pipeline
settings map to the same request fields as the rest api.
```python theme={null}
import os
from pipecat_rumik import RumikTTSService
# muga: our more expressive voice
tts = RumikTTSService(
api_key=os.environ["RUMIK_API_KEY"],
gateway_url=os.environ["RUMIK_GATEWAY_URL"],
settings=RumikTTSService.Settings(model="muga"),
)
# mulberry: our faster voice, steered by a description + preset speaker
tts = RumikTTSService(
api_key=os.environ["RUMIK_API_KEY"],
gateway_url=os.environ["RUMIK_GATEWAY_URL"],
settings=RumikTTSService.Settings(
model="mulberry",
voice="ira", # sent to Rumik as "speaker"
description="a warm 30s indian voice, smooth timbre, conversational pacing, like a friendly narrator",
),
)
# Then drop `tts` into your Pipecat Pipeline alongside your STT + LLM services.
```
set `RUMIK_API_KEY` to a key from your dashboard and `RUMIK_GATEWAY_URL` to
`https://silk-api.rumik.ai`. see the
[pypi package](https://pypi.org/project/pipecat-rumik/) for runnable
voice-agent examples, settings, and event handlers.
## settings
`RumikTTSSettings` is shared by `RumikTTSService.Settings` and
`RumikHttpTTSService.Settings`. it maps to rumik request fields:
| setting | request field | notes |
| ------------- | ------------- | ------------------------------------------------------ |
| `model` | `model` | `muga` or `mulberry`. |
| `voice` | `speaker` | optional. a named voice, e.g. `ira`. |
| `description` | `description` | required for `mulberry`. expressive voice description. |
| `temperature` | `temperature` | optional sampling temperature. |
| `top_p` | `top_p` | optional nucleus sampling value. |
| `top_k` | `top_k` | optional top-k sampling value. |
next: hand your coding agent the [rumik tts skill](/agent-skill) so it integrates
all of this correctly on the first try.
# pay as you go
Source: https://docs.rumik.ai/pricing/pay-as-you-go
use the silk services you need and pay for your usage.
pay only for the usage your account generates. payg has no monthly plan and
includes access to every silk model. it is designed for light or irregular
usage and has an account-wide cap of **4 concurrent requests**.
## pricing
| product | india | rest of world |
| ------------------- | ------------------: | -------------------------: |
| silk mulberry tts | ₹0.50/1k characters | inr price converted to usd |
| silk muga tts | ₹0.99/1k characters | inr price converted to usd |
| silk mulberry agent | ₹1.99/min | \$0.024/min |
| silk muga agent | ₹2.99/min | \$0.036/min |
tts is prorated for the exact submitted `text` character count. whitespace,
punctuation, newlines, and inline tags count; mulberry's `description` does not.
for rest-of-world accounts, the canonical inr tts price is converted with
silk's current shared inr exchange-rate snapshot. agents remain billed from
server-measured call milliseconds. audio milliseconds are recorded for tts
telemetry but never determine a payg tts charge. payg requests use the
plan's usage-based pricing; api requests never buy capacity automatically.
## account-wide capacity
capacity belongs to the account, not an api key, project, voice, or model.
mulberry, muga, and agents all draw from the same four request slots. if all
slots are busy, silk fails fast with http `429`, code
`concurrency_limit_exceeded`, and the active count plus the account limit.
usage is available on the dashboard.
## ready to build?
create a key and make your first request in the [quickstart](/quickstart).
# unlimited
Source: https://docs.rumik.ai/pricing/unlimited
unlimited generation with a concurrency capacity that fits your needs.
## how unlimited works
this plan gives you unlimited text-to-speech generation and unlimited silk
agents. you choose a concurrency capacity (the number of requests silk can
process simultaneously) and your plan includes that capacity every month.
there is no published minute allowance and no normal-use slowdown after a
hidden threshold. silk records generated audio duration for operations and
usage visibility, never to calculate an unlimited customer's bill.
for unlimited customers, the **usage** dashboard shows:
* **purchased lines:** your selected concurrency capacity
* **active lines:** agent sessions currently starting or running
* **failed requests:** recent rejected or failed requests, with their http
error codes
capacity belongs to the account and is shared by mulberry, muga, agents, api
keys, projects, and voices. a request or agent session releases its slot when
it completes, fails, times out, or is cancelled.
## pricing
| concurrent requests | india | rest of world |
| ------------------- | ------------- | ------------- |
| 1 at a time | ₹9,999/month | \$199/month |
| 2 at a time | ₹19,999/month | \$399/month |
| 4 at a time | ₹39,999/month | \$799/month |
| 8 at a time | ₹79,999/month | \$1,599/month |
| more than 8 | contact sales | contact sales |
upgrades take effect immediately with the provider's prorated adjustment.
downgrades and cancellation take effect at renewal; neither changes your api
keys or account configuration. each renewal is charged on the provider cycle
date, not on a fixed 30-day approximation.
# prompting silk muga 1
Source: https://docs.rumik.ai/prompting-muga
how to steer silk muga 1 with tones and inline events.
every muga prompt is three parts: a `[tone]`, an optional ``, and your
words. hear one first:
## anatomy of a prompt
```
[happy] Yaar tumne phir wahi joke maara!
│ │ │
tone event your words
(the mood) (the sound) (devanagari or romanised)
```
every prompt is these three parts.
## the 3 rules that prevent most problems
1. **start every paragraph with one `[tone]`.** it must be the first token. no
tone means flat delivery. one tone per paragraph; a blank line starts a new one.
2. **either script works.** devanagari (`कैसे हो`) and latin/romanised hinglish
(`kaise ho`) both read correctly.
3. **match the event to the tone.** laughs belong to high, positive tones; sighs to
low, reflective ones. a `` in a `[sad]` line fights the model (matrix below).
everything else is detail.
**saying numbers, ids and dates.** silk normalizes your `text` before speaking it, so
amounts, dates, times and percentages are read out for you. anything that must be read
digit by digit, like an otp or an order number, has to be wrapped in double quotes. see
[numbers, ids & dates](/normalization).
## tones
six moods. press play on any one, and copy the prompt to try it yourself.
no tag means the default tone (`[neutral]` if nothing is set). the tone covers the
whole paragraph.
## inline events
drop these in the text, where you want the sound:
| event | length | sound |
| ----------- | -------- | ----------------------------- |
| `` | 0.5-1.5s | loud, voiced laughter |
| `` | 0.3-0.7s | soft, amused, almost a breath |
| `` | 0.4-0.8s | audible exhale, breathy |
rules:
* lowercase, angle brackets, no inner spaces: ``, never `` or
`< laugh >`. get it wrong and it's spoken as a word.
* one space on each side. never mid-word.
* position matters: ` kya baat hai` lands differently from
`kya baat hai `.
* stack at most two (``) for a harder laugh. three or more gets unstable.
## tone × event compatibility
the rule that trips people up most. **laughs ride high tones; sighs ride low ones.**
mix contradictory pairs (a laugh in grief, a sigh in a hype shout) and the model
has almost no training to fall back on, so it fights itself.
## length and chunking
muga is built for **2 to 40 second** utterances.
* **2 to 30s**: the sweet spot. one to three sentences.
* **30 to 40s**: fine for a short monologue.
* **40s+**: split it. past 40s, tone drifts and you get repeats or cutoffs.
to chunk: break at sentence boundaries, keep each piece under 30s, re-state the
tone on each piece, and send them as separate calls.
## recipes
| you want | prompt |
| ----------------------- | ---------------------------------------------------------------------------------- |
| reassuring confirmation | `[neutral] Aapka order place ho gaya hai. Confirmation SMS bhej diya hai.` |
| hype reaction | `[excited] Bhai jeet gaye, vishwas nahi ho raha!` |
| warm banter | `[happy] Arre yaar, kitne din baad baat hui! Sab badhiya chal raha hai?` |
| empathy / bad news | `[sad] Yaar, samajh sakti hoon. Time lagega.` |
| secret / late night | `[whisper] Phir achanak kuch khatka hua. Maine darwaza dekha, koi nahi tha.` |
| frustration | `[angry] Tumne phir wahi kiya. Maine kitni baar bola tha.` |
## using muga in a voice agent
muga sets tone per paragraph, so it needs the **whole tagged utterance up front**.
it can't tag half a sentence.
1. buffer the tokens streaming out of your llm until the turn is complete.
2. you now have one fully tagged utterance.
3. send it to the tts endpoint and stream the audio back to the call.
never forward partial llm output. a fragment like `[hap` or a half-placed `, , . Lowercase, a space on each side,
at most one per paragraph, placed where the sound occurs.
- Match the tone: / with [happy]/[excited];
with [sad]/[angry]/[neutral]/[whisper]. Never mix contradictory emotions.
- Keep each paragraph under ~40 seconds (1 to 3 sentences). Don't be verbose.
```
## pre-flight checklist
before you ship a prompt, six yes/no checks:
1. starts with exactly one `[tone]`?
2. just the tagged line, no markdown or metadata?
3. every event lowercase, spaces around it, none mid-word?
4. event matches the tone (laugh = high, sigh = low)?
5. under \~40s (1 to 3 sentences)?
6. in an agent: full utterance buffered before sending?
## troubleshooting
| what you hear | cause | fix |
| ---------------------------- | -------------------------------------------- | --------------------------------------------- |
| flat delivery, tone ignored | no `[tone]` at the start, or placed mid-text | put one `[tone]` at the very start |
| garbled or wrong words | unsupported language, or malformed input | hindi/english, devanagari or romanised |
| a laugh or sigh sounds off | event doesn't match the tone | use the compatibility matrix |
| an event is spoken as a word | wrong casing or spacing | lowercase, no inner spaces, a space each side |
| drifts, repeats, or cuts off | utterance over \~40s | split into ≤30s chunks |
| inconsistent run to run | sampling settings | set `temperature` to 0.7 |
## faq
the selected default tone, or `[neutral]` if none is set.
not inside a paragraph. start a new paragraph (a blank line) with a new `[tone]`.
it doesn't match the tone. laughs need high, positive tones; sighs need low
ones. see the matrix.
yes. muga now reads devanagari, so `मैं ठीक हूँ` works, and romanised `main
theek hoon` works too. both scripts are supported.
2 to 40 seconds. past \~40s, split it into chunks of 30s or less.
# prompting silk mulberry 1.5
Source: https://docs.rumik.ai/prompting-mulberry
how to steer silk mulberry 1.5 with a natural-language description.
mulberry is steered by a **description**: one natural sentence that says how the
voice should sound. write the voice you want, then your spoken text. hear one:
## the description formula
weave the attributes into one sentence instead of listing them as fields. a useful
shape is:
> a `{gender}` `{age}` `{accent}` voice, `{pitch}` pitch, `{timbre}`,
> `{pacing}` pacing, `{emotion}`, `{register}` register, like a `{role}`.
the `{...}` parts are placeholders you fill in. mention only the ones you care
about; the model fills in the rest.
```text use theme={null}
a deep male 30s indian voice, slow pacing, formal register, like a corporate training narrator.
```
```text avoid theme={null}
30s. indian. deep. slow. formal. corporate narrator.
```
## hear different voices
each one follows the formula. press play, and copy any sample to start from it.
**saying numbers, ids and dates.** silk normalizes your `text` before speaking it, so
amounts, dates, times and percentages are read out for you. anything that must be read
digit by digit, like an otp or an order number, has to be wrapped in double quotes. your
`description` is never touched. see [numbers, ids & dates](/normalization).
## send it
put the voice in `description` and your spoken text in `text`. add `speaker` to use
a preset voice instead of a description.
```bash curl theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{
"model": "mulberry",
"text": "आज का episode थोड़ा अलग है।",
"description": "a female 30s hindi voice, smooth timbre, conversational pacing, casual register, like a podcast host"
}' \
--output mulberry.wav
```
```python python theme={null}
import requests
resp = requests.post(
"https://silk-api.rumik.ai/v1/tts",
headers={"Authorization": "Bearer rk_live_•••••••••"},
json={
"model": "mulberry",
"text": "आज का episode थोड़ा अलग है।",
"description": "a female 30s hindi voice, smooth timbre, conversational pacing, casual register, like a podcast host",
},
)
with open("mulberry.wav", "wb") as f:
f.write(resp.content)
```
```javascript javascript theme={null}
const resp = await fetch("https://silk-api.rumik.ai/v1/tts", {
method: "POST",
headers: {
Authorization: "Bearer rk_live_•••••••••",
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "mulberry",
text: "आज का episode थोड़ा अलग है।",
description:
"a female 30s hindi voice, smooth timbre, conversational pacing, casual register, like a podcast host",
}),
});
const fs = require("fs");
fs.writeFileSync("mulberry.wav", Buffer.from(await resp.arrayBuffer()));
```
see the [api reference](/api-reference) for the full request schema.
## voice attributes
mention any of these in your description. write hindi words in devanagari and
english words in latin, e.g. `आज का episode थोड़ा अलग है`.
* **gender**: `male`, `female`
* **age**: `20s`, `30s`, `40s`
* **pitch**: `low`, `normal`, `high`
* **pacing**: `very slow`, `slow`, `conversational`, `brisk`, `fast`, `very_fast`
* **emotion**: `neutral`, `energetic`, `excited`, `sad`, `sarcastic`, `dry`, `crying`, `angry`
* **intensity**: `low`, `med`, `high`
* **register**: `formal`, `neutral`, `casual`
**global**: `american`, `british`, `middle_eastern`, `asian_american`, `indian`
**indian regional**: `hindi`, `punjabi`, `bihari`, `south_indian`, `bengali`,
`rajasthani`, `marathi`, `gujarati`, `kashmiri`, `assamese`, `odia`, `telugu`,
`kannada`, `malayali`, `haryanvi`, `chhattisgarhi`
**realistic**: `deep`, `warm`, `gravelly`, `smooth`, `raspy`, `nasally`,
`throaty`, `harsh`, `whisper`
**creative**: adds `robotic`, `ethereal` to the realistic set
pick a role from a domain to anchor the delivery style.
* **social**: `youtube_vlogger`, `social_media_creator`, `influencer_voice`, `streamer_companion`
* **podcast**: `podcast_host`, `interviewer`
* **commercial**: `ad_narrator`, `brand_spokesperson`, `product_demo_voice`, `sales_pitch_voice`
* **education**: `elearning_instructor`, `kids_story_voice`
* **support**: `customer_support_agent`, `virtual_receptionist`, `healthcare_assistant`
* **entertainment**: `storyteller`, `social_media_reaction`, `meme_voice`
* **corporate**: `explainer_video_voice`, `event_host`, `corporate_training_narrator`
* **viral**: `short_form_narrator`, `meme_voice`
available when you want a non-realistic timbre (characters, stylized voices):
```text theme={null}
animated_cartoon ai_machine_voice alien_scifi seductively flirty anime
cyborg pirate dark_villain demon gangster mafia dramatic_narrator
mythical_godlike_magical spy vampire alpha
```
## inline tags
drop these in the `text` to trigger a sound. they render as part of the
performance, not as words.
```text theme={null}
```
## preset voices
your `description` is always required, and on its own it's what the voice is built
from. add `speaker` to pin one of the twelve named voices instead:
| voice | gender |
| ------------------------------------------------------- | ------ |
| `emma` `mia` `sophia` `ava` `ira` `siya` `aisha` `zoya` | female |
| `lucas` `noah` `theo` `adam` | male |
names are case-insensitive, and `description` still goes with the request. see
[the mulberry page](/mulberry#preset-voices) for a full request.
## using mulberry in a voice agent
mulberry reads a **complete sentence or utterance** at once, not a running token
stream. but it **streams the audio out**, so the first audio comes back before the
whole line finishes synthesizing. wait until your llm has a full sentence, send that,
and stream the audio to the caller. that makes it a good fit for low-latency
conversational agents.
**keep one voice across the conversation.** the `description` defines the voice, so
for a given bot send the **same `description` string on every request**. reusing it
keeps the persona consistent turn to turn. a preset `speaker` is the other way to
pin a fixed voice.
an unknown or misspelled inline tag is **spoken literally, not rejected**, so
validate the text before production. for transport, see [streaming](/streaming) and
the [pipecat integration](/pipecat).
### system prompt for your agent's llm
hand this to the llm that writes mulberry's lines. tune the wording, not the attributes.
```text theme={null}
You write voice descriptions and spoken text for the silk mulberry 1.5
text-to-speech model.
- "description" is ONE natural sentence describing the voice. Weave 3 to 5
attributes together; never list them as separate fields.
- Draw attributes from these only: gender (male/female); age (20s/30s/40s);
accent (e.g. american, british, indian, hindi, punjabi, bengali, south_indian);
pitch (low/normal/high); timbre (deep, warm, gravelly, smooth, raspy, ...);
pacing (slow, conversational, fast, ...); emotion (neutral, energetic, excited,
sad, ...); register (formal, neutral, casual); and an optional role
(e.g. podcast_host, customer_support_agent, storyteller).
- "text" is the spoken line: Hindi words in Devanagari, English words in Latin.
- You may add inline tags such as , , inside "text".
Return JSON: { "description": "...", "text": "..." }
```
## common mistakes
most "the voice ignored me" cases trace back to one of these:
| what you hear | likely cause | fix |
| ----------------------------- | -------------------------------------------------- | -------------------------------------------------------- |
| an attribute is ignored | the description lists fields, or packs in too many | write one sentence with 3 to 5 attributes |
| the accent is wrong | the accent is not in the supported vocabulary | pick one from the accent list above (global or regional) |
| the voice sounds generic | the description is too vague | add an age, an accent, a timbre, and a role |
| a tag is read aloud as a word | the tag is misspelled or not in the supported set | use an exact tag from the inline tags list |
| pacing or pitch is off | a value outside the listed vocabulary | use the listed `pacing` and `pitch` values |
| the audio is cut off early | the line needs more tokens than the default cap | raise `max_new_tokens` (default `2048`, up to `8192`) |
## faq
one sentence with 3 to 5 attributes beats a paragraph of vague adjectives.
mention only what you care about; the model fills in the rest.
it isn't either/or. `description` is always required; send it every time. on
its own, the voice is generated from it. add `speaker` when you want one of the
twelve named voices instead of a generated one.
send the identical `description` on every request for that bot. pinning a
`speaker` as well makes it steadier still.
yes. drop inline tags like ``, ``, `` directly in the
`text` where you want the sound.
# quickstart
Source: https://docs.rumik.ai/quickstart
synthesize your first clip in three steps.
## 1. get an api key
sign in to the [rumik dashboard](https://playground.rumik.ai), open **api keys**,
and create a new key. the full key is shown only once at creation, so copy it
somewhere safe before closing the dialog.
create and manage keys in your dashboard.
## 2. pick a model
every request takes a `model` and `text`. [`muga`](/muga) is our more expressive
model; [`mulberry`](/mulberry) is faster. each is steered differently. see the
prompting guides for [muga](/prompting-muga) and [mulberry](/prompting-mulberry)
for the full breakdown.
| model | steer with |
| ---------- | ---------------------------------------------- |
| `muga` | a tone tag prefix, e.g. `[happy]` |
| `mulberry` | a natural-language `description` (+ `speaker`) |
## 3. synthesize speech
pass your key as a bearer token. when `audio_format` is omitted, the response
body is a 24 khz mono wav.
```bash curl theme={null}
curl -X POST https://silk-api.rumik.ai/v1/tts \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{ "model": "muga", "text": "[happy] Namaste! Kaise hain aap?" }' \
--output speech.wav
```
```python python theme={null}
import requests
API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"
HEADERS = {"Authorization": f"Bearer {API_KEY}"}
# muga: expressive hinglish, tone set via a [tone] prefix on the text
# tones: neutral (default), happy, sad, excited, angry, whisper
r = requests.post(f"{BASE}/v1/tts", headers=HEADERS, json={
"model": "muga",
"text": "[happy] Namaste! Kaise hain aap?",
})
r.raise_for_status()
open("muga.wav", "wb").write(r.content) # 24 kHz mono WAV
# mulberry: steered by a description (+ optional named voice)
r = requests.post(f"{BASE}/v1/tts", headers=HEADERS, json={
"model": "mulberry",
"text": "Hi there, how can I help you today?",
"description": "a warm 30s female voice, smooth timbre, conversational pacing, like a friendly assistant",
"speaker": "siya", # optional named voice: siya, ira, adam, emma, …
})
r.raise_for_status()
open("mulberry.wav", "wb").write(r.content)
```
## choose another audio format
set `audio_format` to `opus`, `pcm`, `mulaw`, `alaw`, or `mp3`. for example,
this returns opus in an ogg container:
```bash curl theme={null}
api_key='rk_live_•••••••••'
curl --request post https://silk-api.rumik.ai/v1/tts \
--header "authorization: bearer ${api_key}" \
--header "content-type: application/json" \
--data '{"model":"muga","text":"[happy] namaste! kaise hain aap?","audio_format":"opus"}' \
--output speech.ogg
```
omit the field for wav. do not send `"audio_format": null`. see
[audio formats](/audio-formats) for binary http, json base64, and websocket
examples.
try the [api reference playground](/api-reference) to call the endpoint with
your own key right from the docs.
next: [stream audio in real time](/streaming) or
[build a voice agent with pipecat](/pipecat).
# realtime socket
Source: https://docs.rumik.ai/realtime-agent
talk to an agent over a plain websocket carrying pcm.
`wss://silk-api.rumik.ai/v1/agent/connect?token=`
a bidirectional json socket: you send microphone audio, the agent sends its
audio back, plus events telling you when it starts and stops speaking. no
webrtc, no SDK, no ICE — if your platform can open a websocket and move audio
buffers, it can run an agent.
## connect
mint a token with [`/v1/register-call`](/register-call), then open the socket
with it. the token is single-use.
```
wss://silk-api.rumik.ai/v1/agent/connect?token=wct_9f8c2b1e…
```
the call starts when the socket connects — that is the moment billing and
concurrency begin, not when you registered.
## audio format
both directions carry **base64-encoded raw pcm**:
| | |
| ----------- | ------------------------------ |
| encoding | signed 16-bit, little-endian |
| channels | 1 (mono) |
| sample rate | 24 000 Hz |
| frame size | 20 ms (480 samples, 960 bytes) |
24 kHz is fixed for this version. send whatever frame size is convenient —
20 ms is what we recommend and what we emit — but resample to 24 kHz before
sending, and expect 24 kHz back.
## lifecycle
open the socket with your `wct_` token.
the first frame is always `session.created`. do not send audio before it —
the agent is not in the room yet.
send `input_audio_buffer.append` continuously; read `output_audio.delta` and
play it. voice activity and turn-taking are handled for you.
send `session.close`, or just close the socket. you get `session.closed`
with a reason, then a normal close.
## events you receive
the call is live. carries `session_id`, `call_id`, `sample_rate` and
`channels`. `call_id` is what appears in **conversations** and your usage.
```json theme={null}
{ "type": "session.created", "session_id": "01a05b4e…", "call_id": "01a05b4e…", "sample_rate": 24000, "channels": 1 }
```
one 20 ms chunk of the agent's speech, base64 pcm. play these back-to-back.
```json theme={null}
{ "type": "output_audio.delta", "audio": "AAABAAIA…" }
```
speech boundaries, with an ISO-8601 `timestamp`. useful for driving an
animation or a "speaking" indicator.
the caller started speaking over the agent. a good cue to flush whatever
audio you have buffered for playback.
what was said, with `role` (`user` or `assistant`) and `text`. `transcript`
is a settled turn; `transcript.delta` is the turn so far.
the call is over, with a `reason` — `client_requested`, `ended`,
`max_duration` or `error`. the socket closes immediately after.
something went wrong, with a `code` and `message`. a malformed frame is
reported and the call continues; a fatal one is followed by close `4000`.
`transcript`, `transcript.delta`, `agent_start_talking`, `agent_stop_talking`
and `interruption` are best-effort: treat them as enrichment, and never gate
your audio pipeline on one arriving.
## events you send
a chunk of microphone audio: `{ "type": "input_audio_buffer.append", "audio": "" }`.
send continuously while the caller talks. one frame may not exceed one second
of audio.
marks the end of a turn. accepted for compatibility — the agent's own voice
activity detection decides turns, so you do not need it.
`{ "type": "input_text.send", "text": "…" }` — send text instead of speech.
end the call politely. you get `session.closed` back before the socket closes.
## example
a complete call: register, connect, stream a microphone, play the reply.
```python python theme={null}
import asyncio, base64, json, requests, websockets
API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"
AGENT = "ua_fe3277d8"
RATE, FRAME = 24000, 480 # 20 ms of 24 kHz mono
async def main():
# 1. mint a single-use token on the server
reg = requests.post(f"{BASE}/v1/register-call",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"agent_id": AGENT}).json()
url = f'{BASE.replace("https", "wss")}/v1/agent/connect?token={reg["access_token"]}'
async with websockets.connect(url) as ws:
reply = json.loads(await ws.recv())
assert reply["type"] == "session.created", reply
print("call", reply["call_id"], "at", reply["sample_rate"], "Hz")
async def send_mic():
# replace with real capture; here: 10 s of silence, correctly framed
for _ in range(50 * 10):
pcm = b"\x00\x00" * FRAME
await ws.send(json.dumps({
"type": "input_audio_buffer.append",
"audio": base64.b64encode(pcm).decode(),
}))
await asyncio.sleep(0.02)
await ws.send(json.dumps({"type": "session.close"}))
async def receive():
speech = bytearray()
async for raw in ws:
event = json.loads(raw)
if event["type"] == "output_audio.delta":
speech.extend(base64.b64decode(event["audio"]))
elif event["type"] in ("transcript", "transcript.delta"):
print(f'{event["role"]}: {event["text"]}')
elif event["type"] == "session.closed":
print("closed:", event["reason"])
break
elif event["type"] == "error":
print("error:", event["code"], event["message"])
return bytes(speech) # 24 kHz mono PCM — write a WAV header to save
_, audio = await asyncio.gather(send_mic(), receive())
print(f"{len(audio) / (RATE * 2):.1f}s of agent audio")
asyncio.run(main())
```
```javascript browser theme={null}
// The page never sees the API key — your server mints the token.
const { access_token } = await fetch("/api/agent-token", { method: "POST" }).then((r) => r.json());
const ws = new WebSocket(`wss://silk-api.rumik.ai/v1/agent/connect?token=${access_token}`);
const RATE = 24000;
const play = new AudioContext({ sampleRate: RATE });
let playhead = 0;
ws.onmessage = async (event) => {
const msg = JSON.parse(event.data);
if (msg.type === "session.created") {
startMic(ws); // see below
} else if (msg.type === "output_audio.delta") {
// schedule each chunk back-to-back so playback is gapless
const bytes = Uint8Array.from(atob(msg.audio), (c) => c.charCodeAt(0));
const pcm = new Int16Array(bytes.buffer);
const buffer = play.createBuffer(1, pcm.length, RATE);
const channel = buffer.getChannelData(0);
for (let i = 0; i < pcm.length; i++) channel[i] = pcm[i] / 32768;
const src = play.createBufferSource();
src.buffer = buffer;
src.connect(play.destination);
const at = Math.max(play.currentTime + 0.05, playhead);
src.start(at);
playhead = at + buffer.duration;
} else if (msg.type === "transcript") {
console.log(`${msg.role}: ${msg.text}`);
} else if (msg.type === "session.closed") {
console.log("closed:", msg.reason);
}
};
async function startMic(ws) {
const stream = await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1 } });
const ctx = new AudioContext({ sampleRate: RATE });
const source = ctx.createMediaStreamSource(stream);
// A worklet keeps capture off the main thread.
const code = `class Tap extends AudioWorkletProcessor {
process(inputs) { const c = inputs[0]?.[0]; if (c) this.port.postMessage(new Float32Array(c)); return true; }
} registerProcessor('tap', Tap);`;
await ctx.audioWorklet.addModule(URL.createObjectURL(new Blob([code], { type: "application/javascript" })));
const tap = new AudioWorkletNode(ctx, "tap");
tap.port.onmessage = (e) => {
if (ws.readyState !== WebSocket.OPEN) return;
const pcm = new Int16Array(e.data.length);
for (let i = 0; i < e.data.length; i++) {
const s = Math.max(-1, Math.min(1, e.data[i]));
pcm[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
}
const bytes = new Uint8Array(pcm.buffer);
let bin = "";
for (const b of bytes) bin += String.fromCharCode(b);
ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: btoa(bin) }));
};
source.connect(tap).connect(ctx.destination);
}
```
## connection errors
a failure before the call starts is still delivered on the socket: we accept the
connection, send one `error` frame so you know why, and then close.
| close code | meaning |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `4401` | the token was unknown, already spent, or expired — mint a new one |
| `4000` | the call could not start. the `error` frame's `code` says why: `insufficient_balance`, `concurrency_limit_exceeded`, `agent_start_failed`, `not_configured` |
| `1000` | normal end, after `session.closed` |
```json theme={null}
{ "type": "error", "code": "concurrency_limit_exceeded", "message": "Silk is already processing 4 requests for this account. …" }
```
a `4000` close with `concurrency_limit_exceeded` is the socket equivalent of a
`429`. check [`/v1/agent/limits`](/agent-limits) before connecting if you want
to queue callers rather than turn them away.
## ending and billing
the call ends when you send `session.close`, when the socket drops, when the
agent hangs up, or when it hits the maximum session length. you are billed for
the seconds it actually ran, and the transcript and recording appear in
**conversations** shortly afterwards.
# register a call
Source: https://docs.rumik.ai/register-call
mint a single-use token so a browser never holds your api key.
`POST /v1/register-call` exchanges your api key for a short-lived token that
opens exactly one [realtime socket](/realtime-agent). the token is spent the
moment it is redeemed, so leaking one costs you a single call rather than an
account.
call it from your server, hand the token to the client, and let the client
connect.
## request
the agent to run — its UUID or its `ua_…` handle. `agentId` is accepted too.
```bash curl theme={null}
curl -X POST https://silk-api.rumik.ai/v1/register-call \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{ "agent_id": "ua_fe3277d8" }'
```
## response
returns `201 Created`.
the single-use token, prefixed `wct_`. pass it as the `token` query parameter
when opening the socket.
how many seconds the token stays redeemable. it only has to survive the trip
from your server to the client opening the socket.
the audio sample rate the session will use, in Hz.
```json theme={null}
{
"access_token": "wct_9f8c2b1e4d5a4e6fa7b8c9d0e1f2a3b4",
"expires_in": 300,
"sample_rate": 24000
}
```
## what it does not do
registering a call **does not start one**. nothing is billed, no capacity slot
is taken, and no agent is spawned until the socket actually connects — so a
token your user never redeems costs nothing and blocks nobody.
that also means a `429` cannot happen here. if you are out of capacity, you find
out when the socket opens: it accepts, sends an `error` frame, and closes with
code `4000`. see [connection errors](/realtime-agent#connection-errors).
## the agent must be deployed
registering checks that the agent has a live version and returns
`409 agent_not_deployed` if not. that check happens here rather than at the
socket on purpose: the token is single-use, so a caller refused *after*
redeeming one would have to come back for another.
## single use
```python theme={null}
token = register_call(agent)["access_token"]
open_socket(token) # works
open_socket(token) # 4401 — the token was already spent
```
a redeemed, expired or unknown token are indistinguishable from one another, on
purpose. mint a fresh token per call; they are cheap.
## example
```python python theme={null}
import requests
API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"
def register_call(agent: str) -> dict:
r = requests.post(
f"{BASE}/v1/register-call",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"agent_id": agent},
timeout=30,
)
r.raise_for_status()
return r.json() # {access_token, expires_in, sample_rate}
```
```javascript node theme={null}
// Your endpoint: mint a token for the browser, keep the API key on the server.
app.post("/api/agent-token", async (req, res) => {
const upstream = await fetch("https://silk-api.rumik.ai/v1/register-call", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.RUMIK_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ agent_id: "ua_fe3277d8" }),
});
res.status(upstream.status).json(await upstream.json());
});
```
next: [open the socket](/realtime-agent).
# sdk reference
Source: https://docs.rumik.ai/sdk/api-reference
every public class, method, exception, and type in rumik-ai.
the import name is `rumikai`. everything below is importable from the top level, for
example `from rumikai import Rumik, AudioChunk, RateLimitError`.
## clients
```python theme={null}
Rumik(api_key=None, base_url=None, timeout=60, max_retries=2,
default_headers=None, http_client=None)
AsyncRumik(api_key=None, base_url=None, timeout=60, max_retries=2,
default_headers=None, http_client=None)
```
`Rumik` is synchronous, `AsyncRumik` is asynchronous with the same surface. both are
context managers (`with` / `async with`) and expose `.close()`. synthesis lives under
the `client.speech` namespace.
| argument | default | description |
| ----------------- | --------------------------- | -------------------------------------- |
| `api_key` | `RUMIK_API_KEY` | your api key. |
| `base_url` | `https://silk-api.rumik.ai` | api base url (or `RUMIK_BASE_URL`). |
| `timeout` | `60` | request timeout in seconds. |
| `max_retries` | `2` | automatic retries on transient errors. |
| `default_headers` | `None` | headers sent with every request. |
| `http_client` | `None` | a custom http client. |
## client.speech
```python theme={null}
client.speech.create(
text, model="muga", description=None, speaker=None,
temperature=None, top_p=None, top_k=None, repetition_penalty=None,
max_new_tokens=None, timeout=None, extra_headers=None,
) -> Audio
client.speech.stream(
text, model="muga", description=None, speaker=None,
timeout=None, idle_timeout=30,
) -> SpeechStream
client.speech.session(
model="muga", description=None, speaker=None,
timeout=None, idle_timeout=30,
) -> SpeechSession
```
`stream` and `session` require the `ws` extra. see [synthesis](/sdk/synthesis) for
usage and parameters.
## types
### audio
| member | description |
| --------------- | --------------------------------------- |
| `save(path)` | write the audio to a wav file. |
| `__bytes__` | `bytes(audio)` returns the raw bytes. |
| `__len__` | `len(audio)` returns the byte count. |
| `.is_wav` | whether the bytes include a wav header. |
| `.content_type` | the response content type. |
| `.request_id` | the server request id. |
### speechstream
a context manager. iterate it for raw pcm chunks.
| member | description |
| --------------- | ----------------------------------- |
| iterate | yields raw pcm chunks (`bytes`). |
| `.read()` | all remaining pcm as one `bytes`. |
| `.save(path)` | drain and write a wav file. |
| `.request_id` | server request id. |
| `.credits_used` | legacy wire-compatibility metadata. |
### speechsession
a context manager. iterate it for events.
| member | description |
| -------------------- | -------------------------------------------------------- |
| `send(text)` | synthesize a new utterance (barge-in if one is playing). |
| `interrupt()` | stop the current utterance. |
| `close()` | end the session. |
| iterate | yields session events. |
| `iter_audio()` | yields only pcm chunks. |
| `.last_request_id` | request id of the latest utterance. |
| `.last_credits_used` | legacy wire-compatibility metadata. |
### events
| event | attributes |
| -------------------- | ------------------------------------------------------ |
| `AudioChunk` | `.data` (raw pcm bytes) |
| `UtteranceDone` | `.request_id`, `.credits_used` |
| `UtteranceCancelled` | `.request_id`, `.reason` (`"interrupt"` \| `"cancel"`) |
### helpers
```python theme={null}
pcm_to_wav(pcm_bytes) -> bytes # wrap raw PCM in a 24 kHz mono 16-bit WAV header
```
## exceptions
all inherit from `RumikError`. see [error handling](/sdk/errors) for the full tree.
| exception | status | notes |
| -------------------------- | ------- | ------------------------------------------------------------------------------ |
| `RumikError` | base | base class. |
| `APIConnectionError` | network | connection failure. |
| `APITimeoutError` | network | request timed out. |
| `APIStatusError` | 4xx/5xx | base for http errors; has `.status_code`, `.code`, `.request_id`, `.response`. |
| `BadRequestError` | 400 | |
| `AuthenticationError` | 401 | |
| `PermissionDeniedError` | 403 | |
| `NotFoundError` | 404 | |
| `UnprocessableEntityError` | 422 | |
| `RateLimitError` | 429 | adds `.retry_after`. |
| `InternalServerError` | 500 | |
| `ServiceUnavailableError` | 503 | |
| `StreamError` | stream | streaming/session failure. |
# errors and configuration
Source: https://docs.rumik.ai/sdk/errors
the exception tree, retries, timeouts, headers, and debug logging.
## error handling
every error the sdk raises inherits from `RumikError`. handle the specific cases you can
recover from, like a bad key or a rate limit, and keep `RumikError` as a backstop so
nothing slips through unhandled.
```python theme={null}
from rumikai import (
Rumik,
RateLimitError,
AuthenticationError,
UnprocessableEntityError,
RumikError,
)
client = Rumik()
try:
audio = client.speech.create(text="Hello")
except AuthenticationError:
... # bad/missing key
except RateLimitError as e:
wait(e.retry_after)
except UnprocessableEntityError as e:
print(e.code, e) # e.g. "text: String should have at most 2000 characters"
except RumikError as e:
print(f"error {getattr(e, 'request_id', None)}: {e}")
```
### the exception tree
```
RumikError
├─ APIConnectionError
│ └─ APITimeoutError
├─ APIStatusError
│ ├─ BadRequestError 400
│ ├─ AuthenticationError 401
│ ├─ PermissionDeniedError 403
│ ├─ NotFoundError 404
│ ├─ UnprocessableEntityError 422
│ ├─ RateLimitError 429 (.retry_after)
│ ├─ InternalServerError 500
│ └─ ServiceUnavailableError 503
└─ StreamError
```
every `APIStatusError` (the 4xx/5xx group) carries `.status_code`, `.code`,
`.request_id`, and `.response`. `RateLimitError` adds `.retry_after` (seconds to wait).
passing empty `text` raises a plain `ValueError` before any request is sent.
always log `request_id` so support can trace a failure.
### account capacity
`429` can also mean your account has reached its purchased concurrent-request
capacity. in that case the response code is `concurrency_limit_exceeded` and
the json response includes `active_requests` and `limit`:
```json theme={null}
{
"error": "Silk is already processing 2 requests for this account. Try again when one finishes, or increase your plan capacity.",
"code": "concurrency_limit_exceeded",
"active_requests": 2,
"limit": 2
}
```
wait for an active request to finish or increase account capacity. retrying
immediately cannot create capacity and never triggers an automatic purchase.
## configuration
set these once when you build the client. a few can be overridden per request where it
makes sense.
```python theme={null}
client = Rumik(
api_key="rk_live_...", # or RUMIK_API_KEY
base_url="https://silk-api.rumik.ai",
timeout=60.0,
max_retries=2,
default_headers={"X-App": "my-app"},
)
audio = client.speech.create(text="Hi", extra_headers={"X-Trace-Id": "abc"})
print(audio.request_id) # server x-request-id
```
| option | default | description |
| ----------------- | --------------------------- | -------------------------------------- |
| `api_key` | `RUMIK_API_KEY` | your key. |
| `base_url` | `https://silk-api.rumik.ai` | api base url (or `RUMIK_BASE_URL`). |
| `timeout` | `60` | request timeout in seconds. |
| `max_retries` | `2` | automatic retries on transient errors. |
| `default_headers` | `None` | headers added to every request. |
| `http_client` | `None` | supply your own http client. |
**retries** happen automatically on 408, 429, and 5xx responses and on network errors,
using jittered exponential backoff and honoring `Retry-After`. **`default_headers`**
apply to every request; **`extra_headers`** apply to a single call. every response
carries a `.request_id` from `x-request-id`.
### debug logging
set `RUMIK_LOG=debug` to log every request, response, and retry:
```bash theme={null}
export RUMIK_LOG=debug
```
debug logging never includes your api key.
# python sdk
Source: https://docs.rumik.ai/sdk/introduction
the official rumik-ai python package for silk text-to-speech.
`rumik-ai` is the official python client for the silk tts api. it gives you three ways
to turn text into speech, a single batch call, a real-time stream, or a long-lived
voice-agent session, all behind one small, fully typed client. you write a few lines
of python instead of managing http requests and websocket frames yourself.
```python theme={null}
from rumikai import Rumik
client = Rumik() # reads RUMIK_API_KEY
audio = client.speech.create(text="[happy] Namaste! Kaise hain aap?", model="muga")
audio.save("hello.wav")
```
the package installs as `rumik-ai` but you **import `rumikai`**. the api key env var
is `RUMIK_API_KEY`.
## three ways to synthesize
one call in, one wav out. best for pre-generated audio.
raw pcm over a websocket as it is generated. lowest time to first audio.
a persistent connection with barge-in, for real-time voice agents.
## highlights
* **sync and async** clients (`Rumik`, `AsyncRumik`) with the same surface.
* **fully typed**: ships `py.typed`, so type checkers see everything.
* **automatic retries** on transient errors with jittered backoff, honoring `Retry-After`.
* **two models**: expressive `muga` and faster `mulberry`.
* **default audio contract**: 24 khz mono 16-bit pcm; `create` returns a
ready-to-play wav.
## requirements
python **3.9 to 3.13**. streaming and sessions need the `ws` extra
(`pip install "rumik-ai[ws]"`).
## next
install, authenticate, and synthesize your first clip.
batch, streaming, sessions, and async.
the direct api also supports opus, pcm, mulaw, alaw, and mp3. see
[audio formats](/audio-formats) for request and response examples.
# quickstart
Source: https://docs.rumik.ai/sdk/quickstart
install the sdk, authenticate, and synthesize your first clip.
## install
```bash theme={null}
pip install rumik-ai # core: batch synthesis
pip install "rumik-ai[ws]" # + streaming and sessions
```
streaming (`stream`) and sessions (`session`) need the `ws` extra. install
`"rumik-ai[ws]"` or those calls will fail.
python **3.9 to 3.13**. the package ships `py.typed`, so type checkers and editors see
full types. remember: you install `rumik-ai` but **import `rumikai`**.
## authenticate
the client reads your key from `RUMIK_API_KEY`:
```bash theme={null}
export RUMIK_API_KEY=rk_live_...
```
```python theme={null}
from rumikai import Rumik
client = Rumik() # uses RUMIK_API_KEY
```
or pass it explicitly with `Rumik(api_key="rk_live_...")`. the base url defaults to
`https://silk-api.rumik.ai`; override it with `RUMIK_BASE_URL` or `base_url=`.
your key is never logged, even with debug logging enabled.
## your first clip
```python theme={null}
from rumikai import Rumik
client = Rumik()
# muga: expressive, steer with an inline tone tag
audio = client.speech.create(text="[happy] Namaste! Kaise hain aap?", model="muga")
audio.save("hello.wav")
# mulberry: faster, steer with a description + named voice
audio = client.speech.create(
text="Hi there, how can I help you today?",
model="mulberry",
description="a warm 30s female voice, conversational pacing",
speaker="siya",
)
audio.save("greeting.wav")
```
`create` returns an [`Audio`](/sdk/synthesis#the-audio-object), a 24 khz mono wav.
## reuse and close the client
each client keeps a pool of http connections open, so create one and **reuse it across
requests** instead of building a new client each time. wrap it in a `with` block and it
closes itself when you are done:
```python theme={null}
with Rumik() as client:
audio = client.speech.create(text="Hello")
audio.save("hello.wav")
```
or call `client.close()` yourself. `AsyncRumik` works the same with `async with` and
`await client.close()`.
## next
batch, streaming, sessions, and async.
exceptions, retries, and configuration.
# synthesis
Source: https://docs.rumik.ai/sdk/synthesis
batch, streaming, sessions, and async with the rumik-ai client.
you can synthesize in three ways, depending on how soon you need the audio and whether
it is a one-shot clip or an ongoing conversation: **batch** (`create`), **streaming**
(`stream`), and **sessions** (`session`). all three come in a sync and an async form.
## batch synthesis
`create` is the simplest path: send text, get an [`Audio`](#the-audio-object) object
back (a ready-to-play 24 khz mono wav). reach for it whenever you can wait for the whole
clip before playing it.
```python theme={null}
from rumikai import Rumik
client = Rumik()
audio = client.speech.create(text="[happy] Namaste! Kaise hain aap?", model="muga")
audio.save("hello.wav")
```
```python theme={null}
import asyncio
from rumikai import AsyncRumik
async def main():
async with AsyncRumik() as client:
audio = await client.speech.create(
text="[happy] Namaste! Kaise hain aap?", model="muga"
)
audio.save("hello.wav")
asyncio.run(main())
```
`create(text, model="muga", ...)`. only `text` is required.
| parameter | type | default | notes |
| -------------------- | ------- | -------- | ----------------------------------------------------------------------------------------- |
| `text` | `str` | required | the text to synthesize. empty text raises `ValueError`. |
| `model` | `str` | `"muga"` | `"muga"` or `"mulberry"`. |
| `description` | `str` | `None` | required for `mulberry`. natural-language voice description. |
| `speaker` | `str` | `None` | `mulberry` only, optional. a named voice, e.g. `"ira"`. |
| `temperature` | `float` | `None` | sampling temperature. |
| `top_p` | `float` | `None` | nucleus sampling. |
| `top_k` | `int` | `None` | top-k sampling. |
| `repetition_penalty` | `float` | `None` | penalize repeated tokens. |
| `max_new_tokens` | `int` | `None` | output length cap. raise it (up to `8192`) if long `mulberry` audio comes back truncated. |
| `timeout` | `float` | `None` | per-request timeout override, in seconds. |
| `extra_headers` | `dict` | `None` | extra http headers for this request only. |
**steering by model:** `muga` uses an inline tone tag (`[happy]`, `[sad]`, `[excited]`,
`[angry]`, `[whisper]`, or `neutral`); `mulberry` always takes a `description`, plus
an optional named `speaker`. see [prompting muga](/prompting-muga) and
[prompting mulberry](/prompting-mulberry).
### the audio object
`create` returns an `Audio` holding the wav bytes and request metadata.
| member | description |
| --------------- | -------------------------------------------- |
| `save(path)` | write the audio to a `.wav` file. |
| `bytes(audio)` | the raw bytes (`__bytes__`). |
| `len(audio)` | number of bytes (`__len__`). |
| `.is_wav` | whether the bytes include a wav header. |
| `.content_type` | the response content type. |
| `.request_id` | the server request id (from `x-request-id`). |
the current python sdk uses the original wav and raw pcm defaults. to request
opus, pcm, mulaw, alaw, or mp3, use the direct api examples in
[audio formats](/audio-formats).
## streaming
`stream` opens a websocket and hands you raw pcm (24 khz mono 16-bit) as it is
generated, so playback can start before the sentence finishes synthesizing. it returns
a `SpeechStream` you iterate over inside a `with` block.
streaming needs the `ws` extra: `pip install "rumik-ai[ws]"`.
```python theme={null}
with client.speech.stream(text="Streaming in real time.", model="mulberry") as stream:
for chunk in stream: # raw PCM (24kHz mono 16-bit) as it's generated
play(chunk) # feed your audio device
# or: stream.save("out.wav")
```
```python theme={null}
async with client.speech.stream(text="Streaming in real time.", model="mulberry") as stream:
async for chunk in stream:
await play(chunk)
```
`stream(text, model="muga", description=none, speaker=none, timeout=none,
idle_timeout=30)`. `idle_timeout` closes the socket after that many seconds with no
audio.
| member | description |
| --------------- | ---------------------------------------- |
| iterate | yields raw pcm chunks (`bytes`). |
| `.read()` | all remaining pcm as one `bytes` object. |
| `.save(path)` | drain the stream and write a wav file. |
| `.request_id` | the server request id. |
| `.credits_used` | legacy wire-compatibility metadata. |
## sessions and voice agents
a session keeps a single websocket open across a whole conversation. you `send` text,
read back **events** as they arrive, and call `interrupt()` the moment the user starts
talking over the agent. this is what you build a real-time voice agent on.
sessions need the `ws` extra: `pip install "rumik-ai[ws]"`.
```python theme={null}
from rumikai import Rumik, AudioChunk, UtteranceDone
client = Rumik()
with client.speech.session(
model="mulberry",
description="a warm 30s female voice, smooth timbre, conversational pacing",
speaker="ira",
) as session:
session.send("Hello! How can I help you today?")
for event in session:
if isinstance(event, AudioChunk):
play(event.data)
if user_started_talking(): # barge-in
session.interrupt()
session.send("Sorry, go ahead.")
elif isinstance(event, UtteranceDone):
break
```
`session(model="muga", description=None, speaker=None, timeout=None, idle_timeout=30)`
returns a `SpeechSession`, a context manager.
| member | description |
| -------------------- | ---------------------------------------------------------------------- |
| `send(text)` | synthesize a new utterance. sending again mid-utterance is a barge-in. |
| `interrupt()` | stop the current utterance immediately. |
| `close()` | end the session (or leave the `with` block). |
| iterate | yields session events as they arrive. |
| `iter_audio()` | yields only the pcm chunks, skipping control events. |
| `.last_request_id` | request id of the most recent utterance. |
| `.last_credits_used` | legacy wire-compatibility metadata. |
**events** yielded while iterating:
| event | attributes | meaning |
| -------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `AudioChunk` | `.data` | a raw pcm chunk (24 khz mono 16-bit). |
| `UtteranceDone` | `.request_id`, `.credits_used` | the utterance finished normally. `credits_used` is retained as wire-compatibility metadata. |
| `UtteranceCancelled` | `.request_id`, `.reason` | the utterance was stopped. `.reason` is `"interrupt"` (a new `send` replaced it) or `"cancel"` (you called `interrupt()`). |
a session does **not** auto-reconnect if the socket drops. detect the error, open a
new session, and resend. input is always **whole utterances**; there is no partial or
streaming text input.
## async
`AsyncRumik` is the same client with awaitable calls, so it drops straight into an async
app. `create`, `stream`, and `session` all work with `await`, `async with`, and
`async for`.
```python theme={null}
import asyncio
from rumikai import AsyncRumik
async def main():
async with AsyncRumik() as client:
clips = await asyncio.gather(
client.speech.create(text="One"),
client.speech.create(text="Two"),
client.speech.create(text="Three"),
)
for i, audio in enumerate(clips):
audio.save(f"clip_{i}.wav")
asyncio.run(main())
```
`asyncio.gather` runs requests in parallel over one shared connection pool. reuse a
single `AsyncRumik` rather than one per request.
# stream in real time
Source: https://docs.rumik.ai/streaming
low-latency playback over a websocket session.
for real-time playback, mint a one-shot websocket session, connect to it, and send
a json frame with your synthesis parameters. when `audio_format` is omitted, the
server streams raw **pcm int16 little-endian @ 24 khz mono** as binary frames,
then a terminal `{"type":"done"}` json text frame.
`POST /v1/tts/ws-connect` returns `{ ws_url, token }`.
connect to `ws_url?token=` and send one json synthesis frame.
read binary pcm chunks until the `done` (or `error`) control frame.
## choose an audio format
add `audio_format` to the `post /v1/tts/ws-connect` body to stream `opus`, `pcm`,
`mulaw`, `alaw`, or `mp3`. the format is fixed for that one-shot session, so do
not repeat it in the synthesis frame. always connect to the returned `ws_url`.
```bash curl theme={null}
api_key='rk_live_•••••••••'
curl --request post https://silk-api.rumik.ai/v1/tts/ws-connect \
--header "authorization: bearer ${api_key}" \
--header "content-type: application/json" \
--data '{"model":"muga","text":"stream this as opus.","audio_format":"opus"}'
```
see [audio formats](/audio-formats) for exact content types, file extensions,
and a complete websocket example.
```python python theme={null}
import asyncio, json, wave, requests, websockets
API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"
async def main():
# 1. Mint a one-shot WS session -> { ws_url, token }
s = requests.post(f"{BASE}/v1/tts/ws-connect",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"model": "mulberry", "text": "Streaming in real time."}).json()
# 2. Connect, then send the synthesis frame
async with websockets.connect(f'{s["ws_url"]}?token={s["token"]}') as ws:
await ws.send(json.dumps({
"text": "Streaming in real time.",
"description": "a calm 30s female voice, smooth timbre, slow pacing, like a narrator",
"speaker": "ira", # mulberry only; omit for muga / the described voice
}))
# 3. Collect PCM int16 (24 kHz mono) until the done frame
pcm = bytearray()
async for msg in ws:
if isinstance(msg, bytes):
pcm.extend(msg)
elif json.loads(msg).get("type") == "done":
break
with wave.open("stream.wav", "wb") as w:
w.setnchannels(1); w.setsampwidth(2); w.setframerate(24000)
w.writeframes(pcm)
asyncio.run(main())
```
```html browser theme={null}
```
## interruption (barge-in)
for a live "call with ai", keep the websocket open for the whole call and send a
new `{"text": "..."}` frame for each thing the agent says. if the caller talks over
the agent, just send the next utterance: a new `text` frame while one is still
generating **interrupts** the current one and starts the new one. this is
"latest-wins", there is no special barge-in frame.
### frames
once connected, the session is a two-way stream of json text frames and binary
audio.
**client → server**
| frame | meaning |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `{"text": "..."}` | start a generation. sending another while one is in progress is a **barge-in**: it interrupts the current generation and starts the new one. |
| `{"type": "cancel"}` | stop the current generation with no replacement. |
| `{"type": "close"}` | end the session. |
**server → client**
| frame | meaning |
| ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `binary` | a chunk in the session's negotiated format; raw 24 khz mono pcm when `audio_format` was omitted. |
| `{"type": "done", "request_id", "credits_used", ...}` | the generation finished normally. `credits_used` is retained as wire-compatibility metadata; use the usage dashboard for current plan usage. |
| `{"type": "cancelled", "request_id", "reason": ...}` | the generation was stopped. `reason` is `"interrupt"` (a new request replaced it) or `"cancel"` (explicit cancel). |
| `{"type": "timeout"}` / `{"error": true, ...}` | idle timeout / error. |
on a barge-in you receive `{"type": "cancelled", "reason": "interrupt"}` for the old
request, then the audio chunks and `done` for the new one. the old audio stops
almost immediately.
### behavior
* **latest-wins**: only the newest utterance is held. a burst of requests during one
generation never queues a backlog of now-stale audio, only the latest runs next.
* **immediate stop**: on interrupt or cancel the old audio stops streaming right
away, so the caller stops hearing it within a couple hundred milliseconds (mostly
network round-trip).
### example
```python python theme={null}
# inside an open `ws` from the example above, for a live call:
await ws.send(json.dumps({"text": "[neutral] Aapka call connect ho gaya hai."}))
# caller interrupts -> just send the next line. the old one is cancelled,
# the new one starts.
await ws.send(json.dumps({"text": "[happy] Haan ji, bataiye main kaise help karun?"}))
# or stop the current line with no replacement:
await ws.send(json.dumps({"type": "cancel"}))
# end the call:
await ws.send(json.dumps({"type": "close"}))
```
# voice agents
Source: https://docs.rumik.ai/voice-agents
run a conversational agent over the phone-free web, from your own app.
a voice agent is an agent you configure once in the
[dashboard](https://playground.rumik.ai) — its prompt, greeting, voice and
language — and then start from your own product with a single API call. the
agent listens, thinks and speaks; you only carry audio.
there are two ways to connect a caller, and the right one depends on where the
audio lives.
your **browser** joins the call directly over webrtc. lowest latency, and the
media never passes through your servers.
a plain **websocket** carrying base64 pcm. no webrtc stack — good for
servers, native apps and anything that already has audio buffers.
## which one
| | web call | realtime socket |
| ----------------- | -------------------- | ---------------------------------- |
| transport | webrtc (livekit) | websocket |
| audio handled by | the caller's browser | you |
| client dependency | a livekit client SDK | none |
| audio format | negotiated for you | pcm s16le mono 24 kHz |
| best for | web apps | servers, mobile, telephony bridges |
the choice of transport does not affect what a call costs — see
[billing](#billing) for that, which depends on your plan rather than on how the
caller connected.
## authentication
every endpoint takes your api key as a bearer token:
```
Authorization: Bearer rk_live_•••••••••
```
the key needs the **`agent` scope**. keys created in the dashboard have it by
default; older keys were granted it automatically, so nothing you already ship
stops working.
never put a `rk_live_` key in a browser. for web calls, start the call from
your server and pass the returned join credentials to the page. for the
realtime socket, mint a short-lived token with
[`/v1/register-call`](/register-call) and hand *that* to the page.
## before you start
in the dashboard, open **agents** and build one. note its id (a UUID) or its
handle (`ua_…`) — either works everywhere an agent is named.
press **deploy**. saving records a draft; deploying is what puts a version
live. the API refuses an agent that has never been deployed with
`409 agent_not_deployed`, so it can never answer a caller with a
configuration you did not release.
**api keys** → new key. the full key is shown once.
[`GET /v1/agent/limits`](/agent-limits) tells you how many concurrent calls
your plan allows and how many are running right now.
## errors
all four endpoints answer with the same envelope the rest of the API uses:
```json theme={null}
{ "error": "human-readable message", "code": "machine_readable_code" }
```
| status | `code` | what happened |
| ------ | ---------------------------- | ----------------------------------------------- |
| 401 | `unauthorized` | key missing, unknown, revoked or expired |
| 403 | `forbidden_scope` | the key lacks the `agent` scope |
| 404 | `agent_not_found` | no such agent on your account |
| 409 | `agent_not_deployed` | the agent exists but has never been deployed |
| 402 | `insufficient_balance` | your balance can't fund a call |
| 402 | `access_blocked` | account paused after a failed payment |
| 429 | `concurrency_limit_exceeded` | all your concurrent slots are busy |
| 502 | `agent_start_failed` | the agent could not be started — safe to retry |
| 503 | `not_configured` | voice agents are unavailable on this deployment |
a `429` carries the numbers you need to back off intelligently:
```json theme={null}
{
"error": "Silk is already processing 4 requests for this account. Try again when one finishes, or increase your plan capacity.",
"code": "concurrency_limit_exceeded",
"active_requests": 4,
"limit": 4
}
```
## billing
how a call is paid for depends on which plan the account is on. the two work
differently enough that it is worth knowing which one you are testing against —
`plan` in [`/v1/agent/limits`](/agent-limits) tells you.
every second is billed from your balance, at the agent's per-minute rate.
a call is capped at what your balance can fund: if you can only afford 40
seconds, the call ends after 40 seconds rather than being refused up front.
if you cannot afford a usable call at all, the start returns
`402 insufficient_balance`.
new accounts also get a small grant of free agent seconds, which is spent
before your balance is touched.
your plan includes a fixed number of simultaneous calls. calls run on those
slots and cost **nothing per second** — there is no per-call charge to
compute and nothing is metered against duration.
when every slot is busy, the next call is refused with
`429 concurrency_limit_exceeded`. wait for one to finish, or move to a
larger plan.
### concurrency is used first, and credits are never a fallback
an account can hold a concurrency plan **and** a credit balance at the same
time. when it does:
while a slot is free, the call runs on the plan. your credit balance is not
touched — not partially, not for the overflow, not at all.
a call that arrives with every slot busy gets `429 concurrency_limit_exceeded`.
it does **not** fall through to your credits, even if the balance would
comfortably cover it.
credits are not an overflow buffer for a concurrency plan. if you are on a
concurrency plan, a balance sitting in the account will never be spent on
agent calls — the only thing that raises your ceiling is a bigger plan.
so a single account behaves like exactly one of these at a time:
| account is on… | a call costs | past the concurrency limit |
| ---------------------------------- | ---------------------------- | ------------------------------------------ |
| credits, no plan | per second, from the balance | `429`, and `402` once the balance runs out |
| a concurrency plan | nothing | `429` — credits are not used |
| a concurrency plan **and** credits | nothing | `429` — credits are still not used |
### where calls show up
every call appears in **conversations** in the dashboard with its transcript and
recording, and in your usage — attributed to the api key that started it.
calls that ran on a concurrency plan appear there too, at zero cost.
# web call
Source: https://docs.rumik.ai/web-call
start a call and let the browser join it over webrtc.
`POST /v1/webcall` starts a call and returns the credentials a browser needs to
join it. the media flows browser ↔ agent directly; it never touches your
servers.
your backend calls `/v1/webcall` with your api key and the agent to run.
send the `host`, `token` and `roomName` down to the browser. the token is
short-lived and scoped to this one room.
the page connects with a livekit client and enables the microphone. the
agent greets the caller as soon as it sees them join.
## request
the agent to run — its UUID or its `ua_…` handle. `agent_id` is accepted too.
```bash curl theme={null}
curl -X POST https://silk-api.rumik.ai/v1/webcall \
-H "Authorization: Bearer rk_live_•••••••••" \
-H "Content-Type: application/json" \
-d '{ "agentId": "ua_fe3277d8" }'
```
## response
short-lived livekit access token. give it to the browser client; do not reuse
it for a second call.
the livekit websocket URL to connect to.
the room that was created for this call.
the call's id. it is what shows up in **conversations** and in your usage, so
store it if you want to correlate a call with your own records later.
```json theme={null}
{
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9…",
"host": "wss://livekit.rumik.ai",
"roomName": "call-891e23fe503944c28dbf5e5a2a105190",
"callId": "01a05b4d-b1ad-7be0-8d29-04b2fd7e7dca"
}
```
## joining from the browser
install a livekit client (`npm i livekit-client`, or the CDN build below) and
connect with the `host` and `token` you were given.
```html browser theme={null}
```
```python your server theme={null}
import requests
API_KEY = "rk_live_•••••••••"
BASE = "https://silk-api.rumik.ai"
def start_call(agent: str) -> dict:
r = requests.post(
f"{BASE}/v1/webcall",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"agentId": agent},
timeout=30,
)
r.raise_for_status()
return r.json() # {token, host, roomName, callId} -> forward to the page
```
```javascript node theme={null}
const res = await fetch("https://silk-api.rumik.ai/v1/webcall", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.RUMIK_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ agentId: "ua_fe3277d8" }),
});
if (!res.ok) {
const { code, error } = await res.json();
throw new Error(`${res.status} ${code}: ${error}`);
}
const call = await res.json(); // {token, host, roomName, callId}
```
## ending a call
the call ends when the caller leaves the room — `room.disconnect()`, closing the
tab, or losing the network. it also ends on its own when it reaches the agent's
maximum session length, or the length your balance can fund, whichever is
shorter.
you are billed for the seconds the call actually ran. the transcript and
recording land in **conversations** shortly after it ends.
## errors
see the [error table](/voice-agents#errors). the three you will meet in normal
operation:
the agent was saved but never deployed, so no version is live to answer.
press **deploy** in the dashboard; nothing is started or billed until you
do.
the account cannot fund a usable call. top up, or enable auto top-up in the
dashboard so this does not interrupt live traffic.
every concurrent slot is in use. the body carries `active_requests` and
`limit` — queue the caller and retry when a slot frees, or raise your plan
capacity.