Language RolePlay: QA
for an AI Speaking Tool on Moodle

Most speaking apps are easy to demo and hard to trust. They sound impressive once, then behave differently the next time you open them, on a different phone, with a different microphone. Language RolePlay could not afford that. It is an AI speaking-practice tool for English learners, wired into a Moodle portal, and the whole point is that a student’s spoken answer is assessed consistently and the result lands where the teacher can see it. This is the story of the software testing and quality assurance that made it dependable.

Language RolePlay is an AI speaking-practice tool integrated into Moodle. Higher-level students run AI roleplays; beginner and elementary students run listen-and-repeat pronunciation and intonation drills, where the tool reads a sentence at normal speed, then slowly with the sounds isolated, then at normal speed again, and the student repeats until it is right. Students reach an activity through a monthly link, enter their name and email, and follow the directions. Activities can be time-limited, and, above all, the admin and the students’ speaking-assignment reports are centralized in Moodle rather than on an external site.

Put simply, testing this tool is hard because it combines three things that are each awkward to test on their own: an AI that judges speech and is not naturally repeatable, audio recording and playback that changes with the browser, device and microphone, and a cross-system integration that has to land reports inside Moodle. The QA work made each of those parts testable instead of leaving them to chance.

The Starting Point: An AI Tool Wired Into a Learning Portal

An AI speaking tool bolted onto Moodle brings together several problems that each resist ordinary testing. Six of them shaped the QA plan:

• AI speech assessment is not repeatable. The tool listens and responds, and AI output varies from run to run, so the activities had to be made deterministic before they could be checked.
• Audio behaves differently on every device. Recording and playback depend on the browser, the device and the microphone, so speaking activities had to be exercised on the setups students actually use, not one ideal machine.
• Reports must reach Moodle. Centralized reporting is the reason the project exists, so a finished activity had to reliably produce the correct speaking-assignment report inside the portal.
• Two activity types, several levels. Roleplays serve higher levels and listen-and-repeat drills serve beginners, so each type and level had to be covered and given the right instructions.
• Time limits have to hold. Activities can be timed, so the timer and the fate of a partial attempt at the limit had to be tested at the boundary.
• The whole student journey has to work. From the monthly link and the name-and-email entry through the activity to the report, the entire path had to hold together end to end.

The QA Approach: Make the Hard Parts Repeatable

The move that unlocked everything was turning unpredictable behaviour into something a test can assert. Because AI speech assessment is not deterministic on its own, the speaking activities are driven from fixed audio samples with known expected outcomes, so the same input always produces a result a test can check. The listen-and-repeat drill is verified as an ordered loop rather than a single event, which matters because the loop, not any one prompt, is what the beginner actually experiences.

The pronunciation drill runs in a strict order, and the test follows it step for step:

1. The tool reads the target sentence at normal speed.
2. It repeats the sentence slowly, with the individual sounds isolated.
3. It reads the sentence at normal speed once more.
4. The student repeats it, and the loop continues, correcting, until the pronunciation is right.

On that base, the rest is proven rather than assumed. Audio recording and playback are tested across browsers and devices, including microphone permission prompts and outright failures, so the speaking activities work on real student hardware. Judging a spoken answer reliably is the core of the product, and that assessment layer is the kind of speech recognition work that only earns trust once it is pinned to fixed inputs and expected results. Roleplays for higher levels and drills for beginner and elementary levels are each covered with the right instructions, the activity time limit is tested at its edges including partial attempts, and end-to-end tests walk the student journey from the monthly link through the activity to the report.

Technical Architecture of the Test Suite

The testing is layered so a defect is caught at the cheapest level it can be caught. Unit tests cover the individual pieces, integration tests cover the touchpoints between the tool and Moodle, and end-to-end tests cover the whole student path, with the awkward inputs, speech and audio, made deterministic underneath. The parts worth calling out:

• Deterministic speech testing. Fixed audio samples with expected outcomes drive the AI assessment, and the drill loop is verified in sequence, so behaviour that is naturally variable becomes something a test can assert.
• Real-device audio coverage. Recording and playback are run across browsers, devices and microphone conditions, including permission handling and failure states, so no student is left on an untested setup.
• Moodle integration tests. A completed activity is checked to produce the correct speaking-assignment report in the portal, so the centralized reporting that replaces the old external flow is demonstrated, not hoped for.
• Activity types and levels. Roleplays and listen-and-repeat drills are each covered by test cases with the instructions each level should receive.
• Boundary tests on time limits. The timer is tested at its limit, including what happens to a partial attempt when time runs out, so the rule is enforced correctly and fairly.
• Automated in CI. The whole suite runs in continuous integration on every change, and test cases and bug reports keep the coverage traceable.

Pulling a tool this varied together, an AI layer, an audio layer, a Moodle integration and a timed student journey, is squarely the sort of end-to-end verification our software QA and testing team is built for, where the job is to make each moving part provable and then prove they work as one.

Challenges Solved

Making AI speech testable came down to fixed samples and expected outcomes plus a step-by-step check of the drill loop, so a behaviour that never repeats exactly could still be verified. Making audio dependable meant testing recording and playback on the browsers, devices and microphones students really have, permission failures included, rather than a single clean machine. Proving the reporting meant integration tests that a finished activity lands the right report in Moodle, so the project’s core promise is shown rather than assumed. Covering both activity types meant separate test cases for roleplays and drills at each level with the correct instructions. Holding the time limit meant boundary tests on the timer and on partial attempts. And keeping the whole thing honest over time meant automating the student journey in CI, so a regression across the link, the activity or the report is caught before a learner ever meets it.

The through-line was determinism. The tool’s most valuable features are also its least predictable, and the QA turned that unpredictability into fixed inputs, expected outputs and repeatable runs, which is what let the team change the product with confidence. This kind of tooling is common across education and e-learning platforms, where the software has to behave the same way for every student in every classroom.

The Result

Language RolePlay behaves consistently where it used to be unpredictable. Its speaking activities give the same verified result for the same input, its audio works on the setups students actually bring, its levels and time limits are correct, and its reports reliably appear in Moodle, all proven repeatably in continuous integration. The team can add and change activities without wondering whether last month’s behaviour still holds.

If you are building an AI or audio product that is hard to test, or an integration that has to land data in another system reliably, you can bring on full-stack developers who have tested exactly this kind of tool, or tell us what you are building and we will map out how to make its awkward parts provable.

Frequently Asked Questions

How do you test an AI feature that is not repeatable?

By removing the randomness from the test, not the product. Here the AI speech assessment is driven from fixed audio samples with known expected outcomes, so the same input always yields a result the test can check, and the listen-and-repeat drill loop is verified step by step. That makes AI-driven behaviour something you can assert against rather than eyeball once.

How do you test audio recording and playback across devices?

Recording and playback are exercised across real browsers, devices and microphone conditions, including permission prompts and failures, rather than a single ideal machine. That way the speaking activities are proven on the hardware students actually use, which is where audio behaviour tends to diverge.

How do you verify that reports reach Moodle?

With integration tests. A completed activity is checked to produce the correct speaking-assignment report inside the Moodle portal, so the centralized reporting that is the point of the project is demonstrated on every run rather than assumed to work.

Are time-limited activities tested?

Yes. The activity timer is tested at its boundary, including what happens to a partial attempt when time runs out, so the limit is enforced correctly and fairly instead of behaving unpredictably at the edge.

Does the whole student journey get tested end to end?

Yes. End-to-end tests follow the student path from the monthly link and the name-and-email entry, through the roleplay or drill, to the report in Moodle, and the suite runs in continuous integration on every change, so a break anywhere along that path is caught before it reaches a learner.