Problem & Challenge
Interview practice tools usually grade only the text of an answer. A human interviewer also judges how the answer was delivered and how composed the candidate looked, and that feedback normally needs a person to review every recording.
The engineering challenge was to score content, speech delivery and facial composure from a single recorded answer, combine them into one fair score, and keep working when a camera or microphone is missing, without letting one signal inflate the result.
Technical Approach
MockAI has a React candidate platform, a separate admin platform and a FastAPI backend with JWT authentication and role-based access. Each recorded answer is stored and evaluated by a background evaluation worker, so the interview flow never waits on model inference.
Engineering Challenges & Solutions
1. Keeping a silent answer from scoring well
A candidate who says nothing but looks calm could score highly if missing modalities simply had their weight redistributed.
Solution: a verbal-primacy rule. If an answer has no speech, content and delivery score zero and composure stays capped at its 20% weight, so a silent answer cannot score above 20%.
2. Missing camera or microphone
Candidates do not always have both devices available, and failing the evaluation would end the interview.
Solution: graceful degradation with fixed weight tables. Audio only uses 62.5% content and 37.5% delivery; a typed answer uses content only; typed with camera uses 71.4% content and 28.6% composure.
3. Comparing answers of different difficulty
An average of raw question scores treats an easy question and a hard one as equal.
Solution: difficulty-weighted aggregation (easy 1.0×, medium 1.25×, hard 1.5×). Skipped questions score zero but keep their weight, so skipping cannot raise the overall score.
Explainable Feedback
- Score rationale: each report shows how the content, delivery and composure scores combined into the final score.
- Strengths and gaps: rubric concepts covered and missed, with STAR-method coaching suggestions.
- Confidence and stress: a combined reading from acoustic and facial signals, tracked across sessions.
Testing
An end-to-end acceptance run executed all three modalities on a real recorded answer, from FFmpeg normalisation through fused scoring. The repository includes automated tests for media normalisation, speech recognition, the NLP scorer, facial analysis, fusion rules, role-based access and the end-to-end pipeline.