CV parsing: deterministic engine or machine learning?

In short: for a recruitment platform I split CV analysis in two. A trained classifier assigns the specialty domain, and a deterministic rule engine computes years of experience. The rule engine processed 3,254 CVs in 19.7 seconds and matched a reference implementation exactly on a 300-CV validation set; the classifier reached 0.795 micro-average F1 on 5,617 real CVs. This article explains why I did not use one model for both, and how to decide where machine learning is worth it.

The problem

A Moroccan recruitment firm ran its sourcing through an applicant tracking system (ATS). Two tasks stayed slow and unreliable by hand: classifying each CV by specialty domain and working out a candidate's real years of experience, then surfacing the right candidates from a pool of several thousand CVs for each open position. The constraint was to improve this without replacing the ATS or changing recruiters' habits, on shared hosting with no root or SSH access. The project is described in the Werin Sourcing case study.

Two different problems wearing one label

"Extract structured data from a CV" sounds like one task. It is two:

• Classification (which specialty is this person?) is fuzzy. Two recruiters can disagree, and the answer depends on context spread across the whole document. This is where a trained model earns its place.
• Experience calculation (how many years of work?) is arithmetic over dates. A person can write dates in many formats, jobs overlap, and gaps exist, but once the dates are parsed, the answer is a calculation. A model that "estimates" it can be wrong in ways nobody can explain.

Results

Component	Approach	Measured result
Domain classifier	Fine-tuned model, trained on 5,617 real CVs exported from the ATS, split into training, validation and test sets	0.795 micro-average F1 on the held-out test set, about 136 ms inference per CV
Experience engine	Deterministic rules	3,254 CVs in 19.7 seconds; exact 300-out-of-300 match against a Python reference implementation

One honest caveat: comparing the engine against the ATS's own legacy experience field was inconclusive, because that field itself turned out to be inconsistent. A reference you cannot trust cannot validate anything, which is why I validated against an independent implementation instead.

How to decide: model or rules?

1. Can you write the rule down? If an expert can state it precisely, write it as code. It will be fast, explainable and reproducible.
2. Is there a clear ground truth? Dates and sums have one. "Which specialty" does not, so you must measure agreement, not correctness.
3. What does a wrong answer cost? Where an error silently corrupts data, prefer the deterministic path, and put a human review step in front of any write-back. In this project nothing is written back to the ATS before an admin reviews it.
4. Can you reproduce it? A rules engine gives the same output every run. That matters when someone asks why a candidate was filtered out.

Run it safely on a large existing pool

Recomputing fields across thousands of existing candidates is a risky bulk operation. The design that worked: a background "bootstrap" job with pause and resume, cost tracking, and an admin review screen before anything is written back. Treat bulk back-fills as products with controls, not as one-off scripts.

What I would tell a team starting this

• Measure the baseline first, otherwise nobody can tell later whether the project worked.
• Use a held-out test set from real data, not examples you wrote.
• Report the metric you actually ran (micro-average F1 here), and say which parts are not built yet. Semantic search and outreach automation are on the roadmap of this project, not delivered.

If your team is deciding where AI belongs in a workflow, the AI-Augmented Delivery Enablement offer starts with a process review and a measured baseline. For a single architecture question, see the Architecture Power Hour.