M6 + M0: hardening, deploy docs, PoC scripts
- Pipeline-down fallback: dialplan plays gogo-fallback after a failed AudioSocket (§15) + script to synthesize the Bosnian prompt via TTS - Voice heartbeat file → /health/voice endpoint + live status in admin panel - Retention job also purges stale call registrations; chat sessions metered; 100%-usage super-admin alert email (GOGO_ADMIN_ALERT_EMAIL) - .env.example, docs/DEPLOY.md (compose stack, GPU node, backups pg_dump, update procedure, security notes) - §16 Definition of Done as an automated test: softphone-style call → fake partner availability → request pushed → webhook confirm → console SMS - M0 PoC scripts: STT accuracy harness (CER/WER + spoken-digit check; dry-run verified with espeak-ng samples + faster-whisper small on CPU — phone number extracted exactly), TTS bake-off (Azure vs ElevenLabs, latency+cost), and end-to-end latency smoke test speaking real AudioSocket to the live pipeline Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
46
.env.example
Normal file
46
.env.example
Normal file
@@ -0,0 +1,46 @@
|
||||
# Gogo Telefon — copy to .env and fill in. All variables are prefixed GOGO_.
|
||||
|
||||
# Core
|
||||
GOGO_ENV=prod
|
||||
GOGO_SECRET_KEY=change-me-long-random-string # sessions, action links, secret encryption
|
||||
GOGO_BASE_URL=https://app.gogotelefon.ba # public URL (action links, transcripts, widget)
|
||||
GOGO_DATABASE_URL=postgresql+psycopg://gogo:gogo@db:5432/gogo
|
||||
GOGO_INTERNAL_TOKEN=change-me-dialplan-token # Asterisk dialplan ↔ backend
|
||||
|
||||
# LLM (agent)
|
||||
GOGO_ANTHROPIC_API_KEY=sk-ant-...
|
||||
GOGO_LLM_MODEL=claude-haiku-4-5-20251001
|
||||
|
||||
# TTS — one of the two (default decided by the Phase-0 bake-off)
|
||||
GOGO_TTS_PROVIDER=azure # azure | elevenlabs
|
||||
GOGO_AZURE_SPEECH_KEY=
|
||||
GOGO_AZURE_SPEECH_REGION=westeurope
|
||||
GOGO_ELEVENLABS_API_KEY=
|
||||
|
||||
# STT (voice service)
|
||||
GOGO_WHISPER_MODEL=large-v3 # small for CPU dev
|
||||
GOGO_WHISPER_DEVICE=cuda # cuda | cpu | auto
|
||||
|
||||
# Email (proposal emails to owners)
|
||||
GOGO_EMAIL_PROVIDER=smtp # console for dev
|
||||
GOGO_SMTP_HOST=mail.example.ba
|
||||
GOGO_SMTP_PORT=587
|
||||
GOGO_SMTP_USER=
|
||||
GOGO_SMTP_PASSWORD=
|
||||
GOGO_EMAIL_FROM="Gogo Telefon <noreply@gogotelefon.ba>"
|
||||
GOGO_ADMIN_ALERT_EMAIL= # super-admin alerts (100% usage)
|
||||
|
||||
# SMS (via GSM gateway, M5; console logs SMS in dev)
|
||||
GOGO_SMS_PROVIDER=console # console | gsm_gateway
|
||||
GOGO_GSM_GATEWAY_URL=http://192.168.1.50
|
||||
GOGO_GSM_GATEWAY_USER=admin
|
||||
GOGO_GSM_GATEWAY_PASSWORD=
|
||||
|
||||
# Google Calendar OAuth (free/busy read-only, §8.1)
|
||||
GOGO_GOOGLE_CLIENT_ID=
|
||||
GOGO_GOOGLE_CLIENT_SECRET=
|
||||
|
||||
# Voice pipeline
|
||||
GOGO_VOICE_PORT=9092
|
||||
GOGO_RECORDINGS_DIR=/data/recordings
|
||||
GOGO_AUDIO_RETENTION_DAYS=90
|
||||
19
README.md
19
README.md
@@ -53,3 +53,22 @@ the **reference implementation of Appendix B** for partner IT teams:
|
||||
| `tests/fake_partner.py` | Appendix B reference implementation |
|
||||
|
||||
All code/comments in English; all client- and owner-facing text in Bosnian (Latin).
|
||||
|
||||
## Status (software phase complete)
|
||||
|
||||
| Milestone | State |
|
||||
|---|---|
|
||||
| M0 PoC scripts (STT/TTS/latency) | ✅ written + dry-run; real numbers need founder recordings + API keys |
|
||||
| M1 backend core + proposal engine | ✅ |
|
||||
| M2 agent in text mode | ✅ (Appendix-A acceptance tests run when an Anthropic key is set) |
|
||||
| M3 widget + dashboard + super-admin | ✅ verified live in browser |
|
||||
| M4 voice pipeline (softphone path) | ✅ incl. real-Asterisk config validation |
|
||||
| M5 hardware layer (GSM/SMS/forwarding) | ✅ written + mock-tested; live runbook in `docs/HARDWARE_TESTING.md` |
|
||||
| M6 hardening + deploy | ✅ `docs/DEPLOY.md` |
|
||||
|
||||
The §16 Definition of Done is an automated test: `tests/test_definition_of_done.py`.
|
||||
|
||||
Still needed from the founder: Anthropic API key, TTS key (Azure/ElevenLabs
|
||||
bake-off via `scripts/poc/tts_bakeoff.py`), speech samples for
|
||||
`scripts/poc/stt_test.py`, GSM gateway hardware + SIMs, Google OAuth client
|
||||
(for Calendar tenants), SMTP account.
|
||||
|
||||
95
docs/DEPLOY.md
Normal file
95
docs/DEPLOY.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Deploy — founder's DC (Docker Compose, §3/§16 M6)
|
||||
|
||||
## Layout
|
||||
|
||||
| Service | What | Where |
|
||||
|---|---|---|
|
||||
| `db` | PostgreSQL 16 | any node |
|
||||
| `app` | FastAPI backend (dashboard, admin, widget, webhooks, action links) | any node |
|
||||
| `voice` | AudioSocket server + Whisper STT | **GPU node** |
|
||||
| `asterisk` | SIP server (softphones, GSM gateway) | host networking, reachable by the gateway |
|
||||
|
||||
## First install
|
||||
|
||||
```bash
|
||||
git clone <repo> && cd gogotelefon
|
||||
cp .env.example .env # fill everything in; strong GOGO_SECRET_KEY!
|
||||
docker compose build
|
||||
docker compose up -d db app # app runs alembic upgrade head on start
|
||||
docker compose --profile voice up -d voice
|
||||
|
||||
# Asterisk config comes from the DB:
|
||||
docker compose exec app python -m gogo.telephony.asterisk --out ./asterisk/conf \
|
||||
--backend http://app:8000 --voice voice:9092
|
||||
python scripts/generate_fallback_audio.py ./asterisk/sounds # needs a TTS key
|
||||
docker compose --profile voice up -d asterisk
|
||||
|
||||
# seed the first admin (or python -m gogo.seed for a full demo tenant):
|
||||
docker compose exec app python - <<'EOF'
|
||||
import asyncio
|
||||
from gogo.auth import hash_password
|
||||
from gogo.db import get_sessionmaker
|
||||
from gogo.models import Admin
|
||||
async def main():
|
||||
async with get_sessionmaker()() as s:
|
||||
s.add(Admin(email="you@gogotelefon.ba", password_hash=hash_password("CHANGE-ME")))
|
||||
await s.commit()
|
||||
asyncio.run(main())
|
||||
EOF
|
||||
```
|
||||
|
||||
Put a TLS reverse proxy (caddy/nginx) in front of `app:8000` for
|
||||
`GOGO_BASE_URL` — action links and the widget are public.
|
||||
|
||||
## GPU node for Whisper
|
||||
|
||||
`Dockerfile.voice` installs CPU wheels by default. On the GPU node use the
|
||||
NVIDIA runtime and set `GOGO_WHISPER_DEVICE=cuda`, `GOGO_WHISPER_MODEL=large-v3`
|
||||
(one RTX 4090/L4 handles 10–15 concurrent calls, §3). Scale by adding voice
|
||||
containers and pointing different tenants' Asterisk `GOGO_VOICE` at them.
|
||||
|
||||
## Operational jobs (built-in, APScheduler in `app`)
|
||||
|
||||
- proposal expiry + owner reminders — every 5 min
|
||||
- partner polling fallback — every 5 min
|
||||
- retention: delete call audio older than `GOGO_AUDIO_RETENTION_DAYS`
|
||||
(transcripts kept, §5.4), purge stale call registrations — daily 04:00
|
||||
|
||||
## Monitoring
|
||||
|
||||
- `GET /health` (app), `GET /health/db`, `GET /health/voice` (heartbeat age)
|
||||
- admin panel → *Sistem*: DB, LLM key, email/SMS mode, voice heartbeat,
|
||||
delivery errors from `sms_log`/`email_log`
|
||||
- per-call latency breakdown: `calls.trace.turns[].{stt_ms,llm_ms}`
|
||||
|
||||
## Backups
|
||||
|
||||
```bash
|
||||
# nightly dump (cron on the DB node) — keep 14 days
|
||||
docker compose exec -T db pg_dump -U gogo gogo | gzip > /backup/gogo-$(date +%F).sql.gz
|
||||
find /backup -name 'gogo-*.sql.gz' -mtime +14 -delete
|
||||
```
|
||||
|
||||
Recordings volume: rsync `/data/recordings` if call audio must survive node
|
||||
loss (transcripts are in the DB either way; audio expires after 90 days).
|
||||
|
||||
Restore: `gunzip -c dump.sql.gz | docker compose exec -T db psql -U gogo gogo`.
|
||||
|
||||
## Update procedure
|
||||
|
||||
```bash
|
||||
git pull
|
||||
docker compose build app voice
|
||||
docker compose up -d app voice # alembic migrates on app start
|
||||
# if workers/tenants changed the SIP layout:
|
||||
docker compose exec app python -m gogo.telephony.asterisk --out ./asterisk/conf ...
|
||||
docker compose exec asterisk asterisk -rx 'core reload'
|
||||
```
|
||||
|
||||
## Security notes
|
||||
|
||||
- OAuth tokens + partner API keys are encrypted at rest (Fernet from
|
||||
`GOGO_SECRET_KEY`) — changing the key orphans them (reconnect/re-enter).
|
||||
- `/internal/*` must not be exposed publicly (compose keeps it on the
|
||||
internal network; the reverse proxy should only forward what's needed).
|
||||
- Action links are signed + single-use; widget is rate-limited per IP.
|
||||
@@ -772,8 +772,20 @@ async def health_page(
|
||||
{"at": row.at, "kind": f"SMS/{row.kind}", "to": row.to_msisdn, "error": row.error}
|
||||
)
|
||||
|
||||
# voice pipeline heartbeat (written by the M4 pipeline service)
|
||||
voice_status, voice_badge = "nije pokrenut (M4)", "pending"
|
||||
# voice pipeline heartbeat (written by the voice service, M6)
|
||||
import os
|
||||
import time as time_mod
|
||||
|
||||
from gogo.voice.server import heartbeat_path
|
||||
|
||||
try:
|
||||
age = time_mod.time() - os.path.getmtime(heartbeat_path())
|
||||
if age <= 60:
|
||||
voice_status, voice_badge = f"radi (heartbeat prije {int(age)}s)", "confirmed"
|
||||
else:
|
||||
voice_status, voice_badge = f"NE RADI (heartbeat star {int(age)}s)", "rejected"
|
||||
except OSError:
|
||||
voice_status, voice_badge = "nije pokrenut", "pending"
|
||||
|
||||
return templates.TemplateResponse(
|
||||
request,
|
||||
|
||||
@@ -21,3 +21,21 @@ async def health():
|
||||
async def health_db(session: AsyncSession = Depends(get_session)):
|
||||
await session.execute(text("SELECT 1"))
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@router.get("/health/voice")
|
||||
async def health_voice():
|
||||
"""Voice pipeline liveness via its heartbeat file (shared volume)."""
|
||||
import os
|
||||
import time
|
||||
|
||||
from gogo.voice.server import heartbeat_path
|
||||
|
||||
path = heartbeat_path()
|
||||
try:
|
||||
age = time.time() - os.path.getmtime(path)
|
||||
except OSError:
|
||||
return {"status": "down", "detail": "no heartbeat file"}
|
||||
if age > 60:
|
||||
return {"status": "down", "detail": f"heartbeat {int(age)}s old"}
|
||||
return {"status": "ok", "heartbeat_age_s": int(age)}
|
||||
|
||||
@@ -191,5 +191,8 @@ async def _resume_or_create(
|
||||
outcome="abandoned", # upgraded as the conversation progresses
|
||||
)
|
||||
session.add(chat)
|
||||
from gogo.metering import record_chat_session
|
||||
|
||||
await record_chat_session(session, tenant.id)
|
||||
await session.commit()
|
||||
return chat, []
|
||||
|
||||
@@ -22,6 +22,7 @@ class Settings(BaseSettings):
|
||||
smtp_password: str = ""
|
||||
smtp_starttls: bool = True
|
||||
email_from: str = "Gogo Telefon <noreply@gogotelefon.ba>"
|
||||
admin_alert_email: str = "" # super-admin alerts (100% usage etc.), empty = off
|
||||
|
||||
# SMS. provider=console logs instead of sending (GSM gateway impl in M5).
|
||||
sms_provider: str = "console" # console | gsm_gateway
|
||||
|
||||
10
gogo/jobs.py
10
gogo/jobs.py
@@ -97,6 +97,16 @@ async def run_retention_job() -> None:
|
||||
log.exception("failed to delete recording %s", path)
|
||||
continue
|
||||
call.recording_path = None
|
||||
|
||||
# drop stale dialplan handoff rows (they are only needed for minutes)
|
||||
from sqlalchemy import delete
|
||||
|
||||
from gogo.models import CallRegistration
|
||||
|
||||
reg_cutoff = datetime.now(UTC) - timedelta(days=7)
|
||||
await session.execute(
|
||||
delete(CallRegistration).where(CallRegistration.created_at < reg_cutoff)
|
||||
)
|
||||
await session.commit()
|
||||
if rows:
|
||||
log.info("retention: cleared %d recordings", len(rows))
|
||||
|
||||
@@ -61,6 +61,7 @@ async def record_agent_call(
|
||||
tenant.slug, used // 60, tenant.included_minutes,
|
||||
)
|
||||
await _send_warning(session, tenant, used // 60, 100)
|
||||
await _alert_admin(tenant, used // 60)
|
||||
elif used >= included_s * 0.8 and not counter.warned_80:
|
||||
counter.warned_80 = True
|
||||
await _send_warning(session, tenant, used // 60, 80)
|
||||
@@ -75,6 +76,28 @@ async def _send_warning(session: AsyncSession, tenant: Tenant, used_minutes: int
|
||||
log.exception("usage warning email failed (tenant %s)", tenant.slug)
|
||||
|
||||
|
||||
async def _alert_admin(tenant: Tenant, used_minutes: int) -> None:
|
||||
"""100% soft-limit hit → alert the super-admin for an upsell conversation (§12)."""
|
||||
from gogo.config import get_settings
|
||||
from gogo.email import EmailMessage, send_email
|
||||
|
||||
to = get_settings().admin_alert_email
|
||||
if not to:
|
||||
return
|
||||
try:
|
||||
await send_email(
|
||||
EmailMessage(
|
||||
[to],
|
||||
f"[Gogo] {tenant.name} prešao 100% minuta ({used_minutes} min)",
|
||||
f"Salon {tenant.name} ({tenant.slug}) je prešao uključene minute "
|
||||
f"({used_minutes}/{tenant.included_minutes}). Asistent i dalje odgovara "
|
||||
"(hard cutoff je isključen) — vrijeme za razgovor o većem paketu.",
|
||||
)
|
||||
)
|
||||
except Exception: # noqa: BLE001
|
||||
log.exception("admin alert email failed")
|
||||
|
||||
|
||||
async def record_sms(session: AsyncSession, tenant_id: uuid.UUID) -> None:
|
||||
counter = await get_counter(session, tenant_id)
|
||||
counter.sms_sent += 1
|
||||
|
||||
@@ -156,6 +156,9 @@ exten => _[+0-9].,1,NoOp(Gogo inbound ${{EXTEN}} for {slug})
|
||||
same => n,Dial(${{TARGETS}},${{RINGTIME}})
|
||||
same => n,GotoIf($["${{DIALSTATUS}}"="ANSWER"]?answered)
|
||||
same => n(agent),AudioSocket(${{GOGO_UUID}},${{GOGO_VOICE}})
|
||||
; if the voice pipeline is down AudioSocket fails instantly → fallback (§15);
|
||||
; after a normal agent call the caller is gone and Playback is a no-op
|
||||
same => n,Playback(gogo-fallback)
|
||||
same => n,Hangup()
|
||||
same => n(answered),Set(RES=${{CURL(${{GOGO_BACKEND}}/internal/calls/${{GOGO_UUID}}/answered?token=${{GOGO_TOKEN}}&duration=${{CDR(billsec)}})}})
|
||||
same => n,Hangup()
|
||||
|
||||
@@ -94,14 +94,38 @@ async def handle_connection(reader: asyncio.StreamReader, writer: asyncio.Stream
|
||||
await session.run()
|
||||
|
||||
|
||||
async def _heartbeat() -> None:
|
||||
"""Touch a heartbeat file every 15 s; /health/voice + admin panel read its age (M6)."""
|
||||
import os
|
||||
|
||||
path = heartbeat_path()
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
while True:
|
||||
try:
|
||||
with open(path, "w") as f:
|
||||
f.write(str(int(asyncio.get_running_loop().time())))
|
||||
except OSError:
|
||||
log.exception("cannot write heartbeat %s", path)
|
||||
await asyncio.sleep(15)
|
||||
|
||||
|
||||
def heartbeat_path() -> str:
|
||||
import os
|
||||
|
||||
return os.path.join(get_settings().recordings_dir, ".voice-heartbeat")
|
||||
|
||||
|
||||
async def serve() -> None:
|
||||
s = get_settings()
|
||||
server = await asyncio.start_server(handle_connection, s.voice_host, s.voice_port)
|
||||
addrs = ", ".join(str(sock.getsockname()) for sock in server.sockets)
|
||||
log.info("AudioSocket server listening on %s", addrs)
|
||||
# health heartbeat file for the admin panel / monitoring (M6)
|
||||
async with server:
|
||||
await server.serve_forever()
|
||||
heartbeat = asyncio.create_task(_heartbeat())
|
||||
try:
|
||||
async with server:
|
||||
await server.serve_forever()
|
||||
finally:
|
||||
heartbeat.cancel()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
|
||||
36
scripts/generate_fallback_audio.py
Normal file
36
scripts/generate_fallback_audio.py
Normal file
@@ -0,0 +1,36 @@
|
||||
"""Generate the Asterisk fallback prompt 'gogo-fallback' (§15) — played when the
|
||||
voice pipeline is unreachable. Uses the configured TTS provider.
|
||||
|
||||
Run: python scripts/generate_fallback_audio.py [asterisk-sounds-dir]
|
||||
Then place gogo-fallback.wav into Asterisk's sounds dir (or mount it in compose).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
from gogo.voice.audio import pcm_to_wav # noqa: E402
|
||||
from gogo.voice.tts import make_tts # noqa: E402
|
||||
|
||||
TEXT = (
|
||||
"Poštovani, trenutno nismo u mogućnosti primiti vaš poziv. "
|
||||
"Molimo pozovite kasnije, ili posjetite našu web stranicu. Hvala i prijatno!"
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
out_dir = Path(sys.argv[1] if len(sys.argv) > 1 else "./asterisk/sounds")
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
tts, voice = make_tts()
|
||||
pcm = await tts.synthesize(TEXT, voice)
|
||||
path = out_dir / "gogo-fallback.wav"
|
||||
path.write_bytes(pcm_to_wav(pcm))
|
||||
print(f"wrote {path} ({len(pcm) // 16} ms)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
127
scripts/poc/latency_smoke.py
Normal file
127
scripts/poc/latency_smoke.py
Normal file
@@ -0,0 +1,127 @@
|
||||
"""M0 PoC #3 — end-to-end latency smoke test (§14 Phase 0).
|
||||
|
||||
Measures mouth-to-ear response time through the REAL pipeline: connects to the
|
||||
running AudioSocket voice server exactly like Asterisk would, plays a recorded
|
||||
utterance (or a tone with --fake providers), and measures the gap between
|
||||
end-of-utterance and the first agent audio frame of the reply.
|
||||
|
||||
Target (§15): < 1.5 s, worst case < 2.5 s. GATE: > 2.5 s → rethink providers.
|
||||
|
||||
Setup:
|
||||
1. backend + voice server running (see docs/VOICE_TESTING.md)
|
||||
2. a CallRegistration for the demo tenant is created automatically here
|
||||
|
||||
Run: python scripts/poc/latency_smoke.py [utterance.wav] [--host 127.0.0.1] [--port 9092]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
import time
|
||||
import wave
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from sqlalchemy import select # noqa: E402
|
||||
|
||||
from gogo.db import get_sessionmaker # noqa: E402
|
||||
from gogo.models import CallRegistration, Tenant # noqa: E402
|
||||
from gogo.voice.audio import FRAME_BYTES, FRAME_MS, resample, silence, tone # noqa: E402
|
||||
|
||||
KIND_TERMINATE, KIND_UUID, KIND_AUDIO = 0x00, 0x01, 0x10
|
||||
|
||||
|
||||
def load_utterance(path: str | None) -> bytes:
|
||||
if path:
|
||||
with wave.open(path, "rb") as w:
|
||||
pcm = w.readframes(w.getnframes())
|
||||
if w.getframerate() != 8000:
|
||||
pcm = resample(pcm, w.getframerate(), 8000)
|
||||
return pcm
|
||||
print("(no wav given — using a 1s tone; STT will hear nothing useful, "
|
||||
"use a real recording for a true number)")
|
||||
return tone(440, 1000)
|
||||
|
||||
|
||||
async def register_call() -> str:
|
||||
async with get_sessionmaker()() as session:
|
||||
tenant = (
|
||||
await session.execute(select(Tenant).where(Tenant.slug == "salon-merima"))
|
||||
).scalar_one_or_none()
|
||||
if tenant is None:
|
||||
print("run `python -m gogo.seed` first")
|
||||
sys.exit(1)
|
||||
reg = CallRegistration(tenant_id=tenant.id, caller_msisdn="+38765123456")
|
||||
session.add(reg)
|
||||
await session.commit()
|
||||
return str(reg.id)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("--")]
|
||||
host = sys.argv[sys.argv.index("--host") + 1] if "--host" in sys.argv else "127.0.0.1"
|
||||
port = int(sys.argv[sys.argv.index("--port") + 1]) if "--port" in sys.argv else 9092
|
||||
|
||||
import uuid as uuid_mod
|
||||
|
||||
reg_id = uuid_mod.UUID(await register_call())
|
||||
utterance = load_utterance(args[0] if args else None) + silence(800)
|
||||
|
||||
reader, writer = await asyncio.open_connection(host, port)
|
||||
writer.write(bytes([KIND_UUID, 0, 16]) + reg_id.bytes)
|
||||
await writer.drain()
|
||||
|
||||
async def read_msg():
|
||||
header = await reader.readexactly(3)
|
||||
length = int.from_bytes(header[1:3], "big")
|
||||
payload = await reader.readexactly(length) if length else b""
|
||||
return header[0], payload
|
||||
|
||||
# 1) greeting: time from handshake to first audio frame
|
||||
t_start = time.monotonic()
|
||||
kind, _ = await read_msg()
|
||||
greeting_first_audio = time.monotonic() - t_start
|
||||
print(f"greeting first audio after {greeting_first_audio * 1000:.0f} ms")
|
||||
|
||||
# drain the greeting while it plays (send silence like a real caller)
|
||||
async def drain_audio(quiet_ms_needed: int = 600) -> None:
|
||||
last_audio = time.monotonic()
|
||||
while (time.monotonic() - last_audio) * 1000 < quiet_ms_needed:
|
||||
try:
|
||||
kind, _ = await asyncio.wait_for(read_msg(), timeout=0.05)
|
||||
if kind == KIND_AUDIO:
|
||||
last_audio = time.monotonic()
|
||||
except TimeoutError:
|
||||
pass
|
||||
writer.write(bytes([KIND_AUDIO]) + FRAME_BYTES.to_bytes(2, "big") + silence(FRAME_MS))
|
||||
await writer.drain()
|
||||
await asyncio.sleep(FRAME_MS / 1000)
|
||||
|
||||
await drain_audio()
|
||||
|
||||
# 2) speak the utterance paced like a phone, then measure to first reply frame
|
||||
for i in range(0, len(utterance), FRAME_BYTES):
|
||||
frame = utterance[i : i + FRAME_BYTES].ljust(FRAME_BYTES, b"\x00")
|
||||
writer.write(bytes([KIND_AUDIO]) + len(frame).to_bytes(2, "big") + frame)
|
||||
await writer.drain()
|
||||
await asyncio.sleep(FRAME_MS / 1000)
|
||||
t_end_of_speech = time.monotonic()
|
||||
|
||||
while True:
|
||||
kind, payload = await read_msg()
|
||||
if kind == KIND_AUDIO and payload.strip(b"\x00"):
|
||||
break
|
||||
if kind == KIND_TERMINATE:
|
||||
print("server hung up before replying")
|
||||
return
|
||||
response_latency = time.monotonic() - t_end_of_speech
|
||||
print(f"\nmouth-to-ear response latency: {response_latency * 1000:.0f} ms")
|
||||
verdict = "OK ✓" if response_latency < 1.5 else ("worst-case ⚠" if response_latency < 2.5 else "FAIL ✗ — rethink providers (§14)")
|
||||
print(f"target <1500 ms, worst case <2500 ms → {verdict}")
|
||||
writer.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
112
scripts/poc/stt_test.py
Normal file
112
scripts/poc/stt_test.py
Normal file
@@ -0,0 +1,112 @@
|
||||
"""M0 PoC #1 — STT accuracy on local speech samples (§14 Phase 0).
|
||||
|
||||
Kill-the-risk test: can faster-whisper (language hint 'sr') reliably transcribe
|
||||
service names, personal names and PHONE NUMBERS SPOKEN ALOUD over phone-quality
|
||||
audio?
|
||||
|
||||
Prepare a directory of WAV/MP3/OGG samples recorded by the founder + a
|
||||
manifest.csv with two columns: filename,expected_text
|
||||
|
||||
Run:
|
||||
python scripts/poc/stt_test.py <samples_dir> [--model large-v3] [--device cuda]
|
||||
|
||||
Outputs per-file transcript vs expected, character error rate (CER), word error
|
||||
rate (WER), digit accuracy (for phone numbers) and timing.
|
||||
GATE (§14): if phone numbers/names are unusable → stop and rethink providers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def levenshtein(a: list, b: list) -> int:
|
||||
prev = list(range(len(b) + 1))
|
||||
for i, ca in enumerate(a, 1):
|
||||
cur = [i]
|
||||
for j, cb in enumerate(b, 1):
|
||||
cur.append(min(prev[j] + 1, cur[j - 1] + 1, prev[j - 1] + (ca != cb)))
|
||||
prev = cur
|
||||
return prev[-1]
|
||||
|
||||
|
||||
def normalize(s: str) -> str:
|
||||
return re.sub(r"[^\wšđčćž ]", "", s.lower()).strip()
|
||||
|
||||
|
||||
def digits(s: str) -> str:
|
||||
words = {
|
||||
"nula": "0", "jedan": "1", "dva": "2", "tri": "3", "četiri": "4",
|
||||
"pet": "5", "šest": "6", "sedam": "7", "osam": "8", "devet": "9",
|
||||
}
|
||||
out = []
|
||||
for token in normalize(s).split():
|
||||
if token.isdigit():
|
||||
out.append(token)
|
||||
elif token in words:
|
||||
out.append(words[token])
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) < 2:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
samples_dir = Path(sys.argv[1])
|
||||
model_name = sys.argv[sys.argv.index("--model") + 1] if "--model" in sys.argv else "large-v3"
|
||||
device = sys.argv[sys.argv.index("--device") + 1] if "--device" in sys.argv else "auto"
|
||||
|
||||
manifest = samples_dir / "manifest.csv"
|
||||
if not manifest.exists():
|
||||
print(f"missing {manifest} (columns: filename,expected_text)")
|
||||
sys.exit(1)
|
||||
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
print(f"loading {model_name} on {device}…")
|
||||
t0 = time.monotonic()
|
||||
model = WhisperModel(model_name, device=device,
|
||||
compute_type="float16" if device == "cuda" else "int8")
|
||||
print(f"model loaded in {time.monotonic() - t0:.1f}s\n")
|
||||
|
||||
rows = list(csv.DictReader(manifest.open()))
|
||||
total_cer = total_wer = 0.0
|
||||
digit_ok = digit_total = 0
|
||||
for row in rows:
|
||||
path = samples_dir / row["filename"]
|
||||
expected = row["expected_text"]
|
||||
t0 = time.monotonic()
|
||||
segments, info = model.transcribe(str(path), language="sr", beam_size=5, vad_filter=True)
|
||||
got = " ".join(s.text.strip() for s in segments).strip()
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
exp_n, got_n = normalize(expected), normalize(got)
|
||||
cer = levenshtein(list(exp_n), list(got_n)) / max(1, len(exp_n))
|
||||
wer = levenshtein(exp_n.split(), got_n.split()) / max(1, len(exp_n.split()))
|
||||
total_cer += cer
|
||||
total_wer += wer
|
||||
|
||||
exp_digits = digits(expected)
|
||||
if exp_digits:
|
||||
digit_total += 1
|
||||
digit_ok += exp_digits == digits(got)
|
||||
|
||||
flag = "✓" if wer < 0.2 else ("~" if wer < 0.5 else "✗")
|
||||
print(f"{flag} {row['filename']} ({elapsed:.1f}s, audio {info.duration:.1f}s)")
|
||||
print(f" očekivano: {expected}")
|
||||
print(f" dobijeno : {got}\n")
|
||||
|
||||
n = max(1, len(rows))
|
||||
print("=" * 60)
|
||||
print(f"samples: {len(rows)} avg CER: {total_cer / n:.1%} avg WER: {total_wer / n:.1%}")
|
||||
if digit_total:
|
||||
print(f"phone-number samples exactly right: {digit_ok}/{digit_total}")
|
||||
print("GATE: WER > 50% on names or wrong digits on most numbers → rethink STT (§14)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
82
scripts/poc/tts_bakeoff.py
Normal file
82
scripts/poc/tts_bakeoff.py
Normal file
@@ -0,0 +1,82 @@
|
||||
"""M0 PoC #2 — TTS bake-off: Azure Neural vs ElevenLabs Flash (§14 Phase 0).
|
||||
|
||||
Synthesizes typical agent sentences with every candidate voice, saves WAVs for
|
||||
the founder to listen to, and prints latency + rough per-character cost.
|
||||
|
||||
Env: GOGO_AZURE_SPEECH_KEY / GOGO_AZURE_SPEECH_REGION and/or GOGO_ELEVENLABS_API_KEY.
|
||||
Run: python scripts/poc/tts_bakeoff.py [out_dir]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from gogo.config import get_settings # noqa: E402
|
||||
from gogo.voice.audio import pcm_to_wav # noqa: E402
|
||||
from gogo.voice.tts import AzureTTS, ElevenLabsTTS # noqa: E402
|
||||
|
||||
SENTENCES = [
|
||||
"Dobar dan, dobili ste salon Merima. Ja sam Gogo, virtuelni asistent — razgovor se snima. Kako vam mogu pomoći?",
|
||||
"U srijedu poslijepodne slobodno je u dva i trideset ili u pet. Šta vam više odgovara?",
|
||||
"Farbanje cijele dužine je od šezdeset do devedeset maraka, zavisno od dužine kose.",
|
||||
"Važi. Na koje ime da zavedem zahtjev?",
|
||||
"Prosljeđujem salonu zahtjev: šišanje i feniranje, srijeda u pet. Kontaktiraće vas u najkraćem roku radi potvrde. Hvala na pozivu i prijatno!",
|
||||
"Izvinite, nisam vas dobro razumio. Možete li ponoviti?",
|
||||
]
|
||||
|
||||
AZURE_VOICES = ["sr-RS-SophieNeural", "sr-RS-NicholasNeural", "hr-HR-GabrijelaNeural", "hr-HR-SreckoNeural"]
|
||||
ELEVEN_VOICES = ["JBFqnCBsd6RMkjVDRZzb"] # replace with shortlisted hr/sr voices
|
||||
|
||||
# rough public prices (2026): Azure Neural ~$15/1M chars, ElevenLabs Flash ~$50/1M chars
|
||||
COST_PER_CHAR_USD = {"azure": 15 / 1_000_000, "elevenlabs": 50 / 1_000_000}
|
||||
|
||||
|
||||
async def bake(provider_name: str, provider, voices: list[str], out: Path) -> None:
|
||||
for voice in voices:
|
||||
total_chars = total_ms = 0
|
||||
for i, sentence in enumerate(SENTENCES, 1):
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
pcm = await provider.synthesize(sentence, voice)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f" {voice} #{i}: FAILED — {e}")
|
||||
continue
|
||||
ms = int((time.monotonic() - t0) * 1000)
|
||||
total_chars += len(sentence)
|
||||
total_ms += ms
|
||||
path = out / f"{provider_name}-{voice.replace(':', '_')}-{i}.wav"
|
||||
path.write_bytes(pcm_to_wav(pcm))
|
||||
print(f" {voice} #{i}: {ms} ms, {len(pcm) // 16} ms audio → {path.name}")
|
||||
if total_chars:
|
||||
cost = total_chars * COST_PER_CHAR_USD[provider_name]
|
||||
print(
|
||||
f" {voice}: avg latency {total_ms // len(SENTENCES)} ms, "
|
||||
f"~${cost:.4f} for all {total_chars} chars "
|
||||
f"(~${cost / len(SENTENCES) * 1000:.3f} per 1000-char call)\n"
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
out = Path(sys.argv[1] if len(sys.argv) > 1 else "./poc-tts-out")
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
s = get_settings()
|
||||
if s.azure_speech_key:
|
||||
print("== Azure Neural ==")
|
||||
await bake("azure", AzureTTS(), AZURE_VOICES, out)
|
||||
else:
|
||||
print("(GOGO_AZURE_SPEECH_KEY not set — skipping Azure)")
|
||||
if s.elevenlabs_api_key:
|
||||
print("== ElevenLabs Flash ==")
|
||||
await bake("elevenlabs", ElevenLabsTTS(), ELEVEN_VOICES, out)
|
||||
else:
|
||||
print("(GOGO_ELEVENLABS_API_KEY not set — skipping ElevenLabs)")
|
||||
print(f"\nListen to the WAVs in {out}/ and pick 2–3 voices for the dashboard (§6.1).")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
117
tests/test_definition_of_done.py
Normal file
117
tests/test_definition_of_done.py
Normal file
@@ -0,0 +1,117 @@
|
||||
"""§16 Definition of Done, software phase:
|
||||
|
||||
'a demo where a softphone call books a request end-to-end (agent conversation →
|
||||
availability from fake partner API → request pushed → webhook outcome →
|
||||
console-SMS logged) … all without any hardware present.'
|
||||
|
||||
(The chat-widget + mock-tenant half lives in tests/test_chat_widget.py.)
|
||||
"""
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from gogo.agent.llm import LLMResponse, ScriptedLLM, ToolUse, set_llm
|
||||
from gogo.domain import BookingStatus
|
||||
from gogo.models import BookingRequest, Call, CallRegistration, Service
|
||||
from gogo.voice.call import CallLimits, CallSession
|
||||
from tests.fake_partner import staff_resolve
|
||||
from tests.test_voice import UTTERANCE, FakeCallerTransport, frames, voice_fakes # noqa: F401
|
||||
|
||||
SLOT = {"start": "2026-07-15T17:00:00+02:00", "end": "2026-07-15T17:45:00+02:00"}
|
||||
|
||||
|
||||
async def test_definition_of_done_voice_partner_flow(
|
||||
session, partner_tenant, wire_fake_partner, gogo_client, emails, sms, voice_fakes # noqa: F811
|
||||
):
|
||||
stt, tts = voice_fakes
|
||||
wire_fake_partner.slots["SRV-12"] = [SLOT]
|
||||
|
||||
svc = (
|
||||
await session.execute(select(Service).where(Service.tenant_id == partner_tenant.id))
|
||||
).scalar_one()
|
||||
|
||||
# the "softphone caller" asks for a manicure; the agent checks availability
|
||||
# against the partner API and pushes the booking request into it
|
||||
stt.texts = ["Htjela bih manikir u srijedu, Amra Hodžić, broj s kojeg zovem."]
|
||||
set_llm(
|
||||
ScriptedLLM(
|
||||
[
|
||||
LLMResponse(
|
||||
text="",
|
||||
tool_uses=[
|
||||
ToolUse(
|
||||
id="t1",
|
||||
name="check_availability",
|
||||
input={
|
||||
"service_id": str(svc.id),
|
||||
"date_from": "2026-07-15",
|
||||
"date_to": "2026-07-15",
|
||||
},
|
||||
)
|
||||
],
|
||||
stop_reason="tool_use",
|
||||
),
|
||||
LLMResponse(
|
||||
text="",
|
||||
tool_uses=[
|
||||
ToolUse(
|
||||
id="t2",
|
||||
name="submit_booking_request",
|
||||
input={
|
||||
"service_id": str(svc.id),
|
||||
"service_name_raw": "manikir",
|
||||
"client_name": "Amra Hodžić",
|
||||
"client_phone": "+38765123456",
|
||||
"slots": [SLOT],
|
||||
"summary": "Manikir, srijeda u 17.",
|
||||
},
|
||||
)
|
||||
],
|
||||
stop_reason="tool_use",
|
||||
),
|
||||
LLMResponse(
|
||||
text="Prosljeđujem zahtjev salonu — kontaktiraće vas. Hvala i prijatno!"
|
||||
),
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
reg = CallRegistration(tenant_id=partner_tenant.id, caller_msisdn="+38765123456")
|
||||
session.add(reg)
|
||||
await session.commit()
|
||||
reg_id = reg.id
|
||||
|
||||
transport = FakeCallerTransport(UTTERANCE)
|
||||
await CallSession(
|
||||
transport, reg_id, pace=0, limits=CallLimits(inactivity_s=5, max_call_s=60)
|
||||
).run()
|
||||
set_llm(None)
|
||||
|
||||
# 1) availability came from the fake partner, request was PUSHED into it
|
||||
assert "REQ-0001" in wire_fake_partner.requests
|
||||
pushed = wire_fake_partner.requests["REQ-0001"]
|
||||
assert pushed["client"]["name"] == "Amra Hodžić"
|
||||
assert pushed["service_id"] == "SRV-12"
|
||||
assert pushed["source"] == "voice"
|
||||
# partner tenants get no owner email by default (§8.2)
|
||||
assert emails == []
|
||||
|
||||
# 2) salon staff confirm in their own software → webhook → Gogo closes + SMS
|
||||
resp = await staff_resolve(
|
||||
gogo_client, str(partner_tenant.id), "REQ-0001", "confirmed", confirmed_slot=SLOT
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
|
||||
session.expire_all()
|
||||
req = (await session.execute(select(BookingRequest))).scalar_one()
|
||||
assert req.status == BookingStatus.confirmed.value
|
||||
|
||||
# 3) console-SMS logged to the client
|
||||
assert len(sms) == 1
|
||||
to, body = sms[0]
|
||||
assert to == "+38765123456"
|
||||
assert "Potvrđen termin" in body and "srijeda" in body
|
||||
|
||||
# 4) the call itself is fully accounted for
|
||||
call = (await session.execute(select(Call))).scalar_one()
|
||||
assert call.outcome == "request_created"
|
||||
assert call.transcript and call.recording_path
|
||||
Reference in New Issue
Block a user