Rethinking CV-to-Career Recommendation: Skill-Aware Hybrid Retrieval, Open-Set Calibration, and Fairness Auditing with Transformer Encoders

Abstract

Graduate underemployment in Indonesia is frequently attributed to a curriculum-to-industry skill mismatch. We present a content-based career-recommendation web system that maps a CV or a structured form to a database of 78 careers through sentence-transformer embeddings, and we evaluate it along four axes that closed-set benchmarks miss. First, a five-seed protocol shows that previously reported near-perfect retrieval (Hit@5 = 1.0) is a reproducible closed-set ceiling effect: ModernBERT attains MRR 0.871 and Hit@1 0.769 at n = 200. Second, an open-set rejection evaluation on resumes whose true role is absent from the catalogue reveals that the strongest retriever is also the worst-calibrated, accepting every out-of-scope resume at all operating thresholds. Third, a skill-aware hybrid retrieval score that fuses semantic cosine similarity with lexical skill overlap improves lightweight encoders by up to +5.7 MRR and +10 Hit@1 points and never hurts the strong encoder. Fourth, a counterfactual fairness audit via gender name and pronoun swaps shows the deployed ModernBERT is near-invariant (0.7% top-1 flips) whereas general-purpose encoders flip 14–16% of top recommendations. These results reframe the contribution from systems integration to an evaluation-and-formulation study for safe CV-to-career retrieval.