All projects
AI Platforms

Calibre — AI Job-Matching Platform

AI job-matching platform combining quality-controlled job inventory, resume classification, semantic matching, and controlled AI scoring.

AI PlatformsValidated Pilot

Executive overview

Calibre combines job inventory controls, resume and job classification, semantic retrieval, compatibility rules, and selective LLM-assisted scoring.

The platform was designed to prevent poor-quality jobs, wrong-family recommendations, and uncontrolled paid scoring from flowing directly into user-facing results.

I led the product and solution direction, defined the major controls and validation approach, directed AI-assisted implementation, reviewed changes, and validated pilot readiness.

Architecture and examples on this page are sanitised for public presentation. No provider names, internal identifiers, prompts, or scoring weights are exposed.

Business problem

  • Imported jobs may be irrelevant, duplicated, expired or incomplete.
  • Semantic similarity can still surface jobs from the wrong profession family.
  • Sparse resumes are difficult to classify.
  • LLM scoring introduces cost and provider reliability risks.
  • Code tests alone do not prove end-to-end pilot readiness.
  • Operators need visibility into inventory, matching and paid execution.

Constraints

Variable job-source quality

Upstream feeds differ in completeness and reliability.

Sparse resume data

Short or partial resumes weaken automatic classification.

Cross-family false matches

Semantic similarity alone can cross professional families.

Missing embeddings

Active jobs may lack embeddings needed for retrieval.

Provider-cost exposure

Paid AI scoring must be tightly bounded.

Usage-recording dependency

Paid operations must record usage reliably.

Feature rollout safety

Changes must be gated behind flags until validated.

Need for repeatable validation

Pilot readiness must be defensible, not ad-hoc.

Architecture

Three flows converge on ranked recommendations, wrapped by a control layer.

Supply flow
  1. Job Sources
  2. Inventory Quality Gates
  3. Normalization & Classification
  4. Embedding Generation
  5. Vector Storage
Candidate flow
  1. Resume
  2. Parsing & Profile Classification
  3. Candidate Embedding
Matching flow
  1. Candidate + Job Embeddings
  2. Semantic Retrieval
  3. Family & Business Rules
  4. Selective LLM Scoring
  5. Ranked Recommendations
Control layer
  1. Feature Flags
  2. Usage Accounting
  3. Provider Circuit Breaker
  4. Admin Controls
  5. Validation Cohorts
StageResponsibilityInputOutputFailure riskControl
Inventory GatesReject weak or irrelevant jobs earlyRaw imported jobsFiltered inventoryWeak jobs consuming paid resourcesRule-based gates + review of exclusions
ClassificationAssign job & profile familiesJob text, resume textFamily labelsSparse resumes classified into wrong familyConfidence thresholds + manual sampling
Semantic RetrievalReturn broadly relevant candidatesEmbeddingsCandidate shortlistCross-family similarityFamily compatibility overlay
LLM ScoringRefine ranking on narrowed setShortlistScored matchesCost & provider failureFail-closed execution + feature flag

Key engineering decisions

Quality gates before paid processing

Reject weak or irrelevant jobs before consuming embedding, storage or LLM resources.

Classification before final ranking

Use job and candidate family classification to reduce professionally irrelevant matches.

Semantic retrieval plus compatibility controls

Use embeddings for broad similarity while deterministic rules prevent obvious family conflicts.

Selective LLM scoring

Use paid reasoning only after cheaper controls narrow the candidate set.

Fail-closed provider execution

Stop paid operations when authorisation, telemetry or safety requirements are unavailable.

Feature-gated rollout

Keep scanning, scoring and taxonomy changes disabled until validation evidence is complete.

Failure modes surfaced during validation

  • Management jobs incorrectly excluded by quality filters.
  • Short resumes classified into the wrong family.
  • Cross-family results ranking too highly.
  • Active jobs missing embeddings.
  • Background workers creating uncontrolled provider usage.
  • Passing unit tests without proving pilot readiness.
Each failure mode was addressed either through a control change, a classification fix, or by keeping the affected path behind a feature flag until further validation.

Validation approach

  1. Inventory validation
  2. Classification validation
  3. Semantic retrieval validation
  4. Cross-family leakage check
  5. Limited LLM scoring
  6. Usage review
  7. Pilot readiness decision
  • Controlled job cohorts across multiple families.
  • Expected positive and negative results defined up front.
  • Dry runs before paid scoring.
  • Limited scoring samples with usage review.
  • Feature flags disabled by default.
  • DEV validation before wider rollout.
  • Hard stops when evidence is weak.

My role

I led the product and solution direction, defined architecture and controls, prepared implementation briefs, directed AI-assisted development, reviewed and corrected changes, ran validation, and remained accountable for pilot-readiness decisions.

AI tools accelerated implementation, testing, documentation and review. I defined the direction, reviewed assumptions and changes, validated outcomes and remained accountable for readiness.

Outcome

  • Weak inventory was filtered before downstream matching.
  • Professional-family controls reduced obvious cross-family results during validation.
  • Paid scoring was placed behind eligibility, telemetry and feature controls.
  • Operators gained clearer control over inventory, scans and matching readiness.
  • The platform reached validated-pilot status.
  • Commercial outcomes remain unproven.

Limitations

  • Pilot validation is not long-term production evidence.
  • Source quality still affects every downstream stage.
  • LLM scoring requires continued calibration.
  • Sparse resumes may remain difficult to classify.
  • Commercial value has not yet been established.
  • Paid execution requires ongoing monitoring.

What I would improve next

  • Larger benchmark cohorts.
  • Automated regression datasets.
  • Better job freshness controls.
  • User-facing match explanations.
  • Taxonomy governance.
  • Cost-per-useful-match reporting.
  • Long-term quality monitoring.

Interview preparation

Interview preparation
How I explain this project in an interview
30-second summary

Calibre is a quality-gated job-matching platform. It filters weak inventory, classifies both jobs and resumes, uses semantic retrieval with family compatibility overlays, and only invokes paid LLM scoring on a narrowed set behind a feature flag. It reached validated-pilot status; commercial adoption is not claimed.

2-minute architecture explanation

Jobs flow through inventory gates, normalisation and classification, then embedding into a vector store. Resumes are parsed, classified into a profile family, and embedded. Matching does broad semantic retrieval, applies deterministic family and business rules to prevent obvious cross-family results, then runs selective LLM scoring on the narrowed set. A control layer — feature flags, usage accounting, a provider circuit breaker, admin controls and validation cohorts — wraps every paid operation so cost, telemetry and safety failures stop paid execution instead of silently proceeding.

Deep-dive questions
  • What problem did the platform solve?
  • What did I personally own?
  • What work was AI-assisted?
  • Why were deterministic gates needed before the LLM?
  • What failed during validation?
  • How was cross-family leakage tested?
  • What was validated, and what remains unproven?
  • What would change at larger scale?
Back to projects