Boundary Conditions for Semantic Log Parsing in Systems Observability Pipelines
TL;DR
The semantic parser is treated as an existing component, and the contribution is evidence about when that component changes downstream detection behaviour enough to justify pilot testing, which supports cautious, workload-aware use of semantic parsing in observability pipelines.
Abstract
Parsing decisions in log anomaly detection pipelines directly shape downstream detection effectiveness. Despite growing work on semantic parsers and LLM-assisted parsing, the deployment conditions under which semantic enrichment is preferable to efficient syntactic parsing have not been empirically characterised. Semantic-enriched parsing can improve template distinguishability by using lightweight language-model representations. It also introduces extra representation complexity and embedding inference overhead. Systems operators therefore need evidence about when this additional complexity is useful, negligible, or unstable. This is a systems observability design study, not a parser-proposal paper: the semantic parser is treated as an existing component, and the contribution is evidence about when that component changes downstream detection behaviour enough to justify pilot testing. We empirically characterise these boundary conditions by evaluating a semantic-enriched parser against three syntactic baselines (Drain, IPLoM, and Spell) across five public LogHub datasets spanning HPC, distributed storage, and cloud logs. We use three anomaly detectors with different modelling assumptions: DeepLog, LogAnomaly, and CNN. We report mean and standard deviation over five random seeds, define semantic-enrichment benefit as ΔSE = F1(semantic) – F1(Drain), and also report ΔSE* relative to the best syntactic baseline. We introduce the Template Expansion Ratio (TER), a simple post-parsing diagnostic for how much additional template vocabulary the semantic parser creates relative to Drain. Material gains concentrate on BGL and Spirit, negligible gains occur on HDFS and OpenStack, and Thunderbird shows unstable behaviour under very low anomaly prevalence. The results support cautious, workload-aware use of semantic parsing in observability pipelines rather than treating semantic enrichment as a universally beneficial replacement for syntactic parsers.