Bilingual phone orders → checked orders (AssemblyAI)
Takes a Mandarin + English code-switched phone order, transcribes it, extracts the order, prices it from the menu and cross-checks it against the staff read-back, flagging anything a human should review
Singapore customers rarely order in one language. "两份海南鸡饭，一份要少辣, and one char kway teow, 加蛋" is a normal sentence. This demo turns a phone recording like that into an order the kitchen can use, and flags the ones a human should check.
How it works
- Transcription: AssemblyAI Universal-3.5 Pro, constrained to Mandarin and English, keeping mid-sentence code-switching as spoken; speaker labels for staff vs customer; menu names in both languages as keyterms
- Extraction: LLM Gateway with a JSON schema, so dishes can only come from the menu
- Validation: prices come from the menu, never from the model; checks for Singapore phone numbers, off-menu dishes and odd quantities
- Cross-check: a second pass extracts the staff read-back and compares it with the customer's order; any disagreement goes to review
What the test run showed
I tested it on a 37-second synthetic call. All results are in the README, including what didn't work:
- Transcription was the strong part: code-switching kept intact, dish names mostly right
- Free accounts only get a 4B model, which got quantities and modifiers wrong
- The cross-check catches errors the two passes don't share, but when both passes use the same model they make the same mistake, and it gets through
- Two places where the API and the docs disagree, written up as product feedback