Education
Classifying admission documents with a language model
A training centre received a few thousand applications each intake and sorted them by hand. I put a model into the workflow with a quality harness and a cost ceiling.
91% of documents classified correctly without a human
An interface concept reconstructed for this use case. Brands and data are fictional; this is not a screenshot of a client system.
- Client
- Client asked not to be named
- Duration
- 2 months
- Engagement
- Fixed price
Context
Each intake, two staff spent about three weeks reading and sorting a few thousand applications into six categories. The work was repetitive, and the error rate climbed towards the end of each cycle through fatigue.
Management had tried an off-the-shelf automation tool and dropped it after two weeks: it could not explain why it had put an application in a category, so nobody trusted the output.
What was built
I started with the evaluation harness, not the model. Four hundred hand-sorted applications became the reference set before any model was tried against it.
The model returns a category together with its reasoning and a confidence score. Anything below the confidence threshold goes to a person instead of being guessed. Model spend has a hard monthly ceiling; past it the system routes everything to the manual queue rather than quietly spending money.
Outcome
91% of applications were classified correctly with no human involvement, measured on the 400-item reference set rather than on data the model had already seen.
The remaining 9% land in the manual queue, which is the intended behaviour: the system says "I am not sure" instead of guessing. Three weeks of seasonal work became about four days.