Files

Senad Uka 714aa1782b M4: voice pipeline (software-only telephony)

- AudioSocket server (asyncio TCP, Asterisk wire protocol) bridging calls
  into the same M2 agent used by chat
- Call session engine: greeting → energy-VAD utterance collection → STT →
  agent turn → TTS playback with barge-in (120ms of caller speech stops
  playback); inactivity + max-duration guards; unintelligible audio reaches
  the agent as '[nerazumljivo]' so the two-attempt rule stays in the prompt
- STTProvider (faster-whisper, lang hint 'sr', GPU/CPU via env) and
  TTSProvider (Azure Neural raw-8k PCM / ElevenLabs Flash) + test fakes
- Recording (mixed caller+agent WAV), transcripts, per-turn latency trace,
  metering via record_agent_call (§12)
- Internal dialplan API: /internal/calls/register (ring targets computed from
  working hours + ring settings §5.2), /answered (human outcome, no minutes),
  /hangup (abandoned → missed-call SMS §5.5); shared-token auth
- Asterisk config generator (pjsip.conf + extensions.conf from DB): worker
  endpoints, per-tenant test-caller endpoint, register→Dial→AudioSocket
  dialplan with h-extension reporting — validated against a real Asterisk 20
  container (modules, dialplan, endpoints all load)
- docker-compose 'voice' profile (voice server + Asterisk), Dockerfile.voice,
  docs/VOICE_TESTING.md softphone runbook
- 16 new tests incl. full fake-call e2e and a real-TCP AudioSocket wire test

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-11 11:05:56 +02:00

3.0 KiB

Raw Blame History

M4 — Software-only voice testing (no GSM hardware)

Everything here runs on a laptop/DC per §16: desktop/mobile softphones register directly to Asterisk over IP; the GSM leg is the only thing not exercised (that's M5 + the joint hardware session).

1. Prereqs

Postgres running, schema migrated, demo tenant seeded:

.venv/bin/alembic upgrade head
.venv/bin/python -m gogo.seed

.env with at least GOGO_ANTHROPIC_API_KEY (agent) and one TTS key (GOGO_AZURE_SPEECH_KEY + region, or GOGO_ELEVENLABS_API_KEY).
Whisper model downloads on first run (GOGO_WHISPER_MODEL=small is fine on CPU).

2. Start the stack

# backend API (internal dialplan endpoints live here)
.venv/bin/uvicorn gogo.main:app --host 0.0.0.0 --port 8000

# voice pipeline (AudioSocket server on :9092)
.venv/bin/python -m gogo.voice.server

# generate Asterisk config from the DB, then start Asterisk
.venv/bin/python -m gogo.telephony.asterisk --out ./asterisk/conf \
    --backend http://127.0.0.1:8000 --voice 127.0.0.1:9092
docker compose --profile voice up asterisk

3. Softphones

Two SIP accounts exist per tenant after config generation:

Role	Username	Password	Purpose
Worker	from dashboard → Telefonija → QR	generated	rings before the agent
Test caller	`test-caller-salon-merima`	`test-<first 10 chars of widget key>` (in pjsip.conf)	simulates the customer

Register the worker softphone (e.g. Linphone on a phone, scan the QR) and the test-caller softphone (e.g. Zoiper on the laptop) against the machine's IP.

4. Test scenarios

Ring-group first (§5.2): from the test-caller dial any number (e.g. 100). The worker softphone rings for ring_timeout_s; answer → normal human call, dashboard logs outcome javilo se osoblje, no agent minutes metered.
Agent fallback: dial again, don't answer → after the timeout the agent picks up, greets with the recording disclosure, books a request. Check: dashboard → Zahtjevi (new request), Pozivi (transcript + audio player), Potrošnja (minutes metered), owner email (proposal + .ics).
Out of hours (A.5): set working hours to closed (dashboard → Salon), dial → agent answers immediately with the out-of-hours greeting.
Barge-in: interrupt the agent mid-sentence — playback must stop and the agent must respond to what you said.
Abandoned call (§5.5): dial, hang up while the worker phone still rings → Pozivi shows prekinut and the missed-call SMS appears in the backend log (console SMS provider).
Latency (§15): calls.trace (call detail JSON) stores per-turn stt_ms/llm_ms — target < 1500 ms combined before TTS starts.

5. Automated coverage

tests/test_voice.py covers the same flows without Asterisk: VAD, the full call loop against fakes (booking, unclear speech, hangup, inactivity, barge-in), the AudioSocket wire protocol over real TCP, the dialplan internal API (ring targets, out-of-hours, abandoned + SMS) and config generation.

3.0 KiB Raw Blame History