InstaNovo v1.2.2 on the ProteoBench de novo benchmark

Abstract

Predictions, scores and figures for InstaNovo v1.2.2 across seven inference modes on the ProteoBench de novo DDA-HCD module (nine-species balanced, 779,879 spectra), together with the published datapoints of the seven tools on that leaderboard at the time of deposit. What is here Filenames are flat: Zenodo's upload API rejects / in a file key, so every file is named for what it is rather than placed in a folder. pattern contents predictions_instanovo_v1_2_2_ .csv.gz raw predictions, one file per inference mode — see Prediction files below datapoint_instanovo_v1_2_2_ .json the ProteoBench datapoint for that mode: every metric, every matching variant datapoint_published_ .json the seven published datapoints this is compared against figure_ .html four interactive figures, rendered with ProteoBench's own plot generator plot_instanovo.py regenerates the figures make_tables.py regenerates every table below il_analysis.py regenerates the isoleucine/leucine measurements below zenodo_upload.py, zenodo_metadata.json how this record was deposited Reproducing The scripts need ProteoBench itself, which supplies both the scoring definitions and the plot generator, so the figures match the live site rather than approximating it: pip install proteobench pandas python plot_instanovo.py # the four figures python make_tables.py # every table below python il_analysis.py # the isoleucine/leucine measurements Everything is read from the directory the scripts live in. il_analysis.py additionally downloads the ProteoBench ground truth into cache/ on first run — that file is not part of this deposit, for the reason given under Provenance. The four figures file evaluation allow_il allow_deamidation figure_mass.html mass n/a n/a figure_exact_strict.html exact false false figure_exact_il.html exact true false figure_exact_both.html exact true true exact_il is the live site's default view (plot_main_metric defaults to allow_il=True, allow_deamidation=False). The numbers Generated by make_tables.py, not typed. Bold marks the best value in each column — highest for precision, recall, AUC and spectra sequenced; fastest for runtime. Coverage is not bolded: it is the share of the benchmark a mode answered rather than a score to maximise, and two values exceed 1 because of a known upstream bug (caveat 4). exact both is identical to mass by construction — see below. Against the published tools Peptide precision, best InstaNovo mode (knapsack_beam10_refined) against every tool published on the ProteoBench de novo DDA-HCD leaderboard. Snapshot as of 11 October 2026. The leaderboard is a moving target: tools are added as submissions are merged, so this ranking is a point-in-time comparison and not a standing claim. Pinned to Proteobench/Results_denovo_lfq_DDA_HCD commit f16c90e of 10 August 2026, carrying 7 datapoints, which was still the repository head on 11 October 2026. InstaNovo itself is not among them -- it has not been submitted, so it is shown here for comparison only. The live leaderboard is at https://proteobench.cubimed.rub.de/denovo_DDA_HCD. tool mass exact strict exact I/L exact both InstaNovo v1.2.2 0.7321 0.4437 0.7142 0.7321 Pi-PrimeNovo 0.6824 0.2542 0.6605 0.6824 ContraNovo 0.6700 0.2558 0.6498 0.6700 Casanovo 0.6498 0.2490 0.6340 0.6498 Pi-HelixNovo 0.5949 0.2291 0.5787 0.5949 AdaNovo 0.5894 0.2272 0.5802 0.5894 PepNet 0.3995 0.1573 0.3844 0.3995 DeepNovo 0.3464 0.1448 0.3336 0.3464 Every mode, every metric Peptide level Precision mode mass exact strict exact I/L exact both greedy 0.6946 0.4045 0.6749 0.6946 greedy_refined 0.6964 0.4123 0.6776 0.6964 beam10 0.7278 0.4222 0.7068 0.7278 beam10_refined 0.7317 0.4438 0.7140 0.7317 knapsack_beam10 0.7283 0.4223 0.7072 0.7283 knapsack_beam10_refined 0.7321 0.4437 0.7142 0.7321 diffusion_only 0.7142 0.4205 0.6946 0.7142 Recall mode mass exact strict exact I/L exact both greedy 0.6946 0.4045 0.6749 0.6946 greedy_refined 0.6964 0.4122 0.6775 0.6964 beam10 0.7278 0.4222 0.7068 0.7278 beam10_refined 0.7317 0.4438 0.7140 0.7317 knapsack_beam10 0.7283 0.4223 0.7072 0.7283 knapsack_beam10_refined 0.7321 0.4437 0.7142 0.7321 diffusion_only 0.7142 0.4205 0.6946 0.7142 AUC (area under the precision-coverage curve) mode mass exact strict exact I/L exact both greedy 0.9177 0.6539 0.9025 0.9177 greedy_refined 0.8812 0.6164 0.8649 0.8812 beam10 0.9220 0.6563 0.9066 0.9220 beam10_refined 0.9257 0.6921 0.9127 0.9257 knapsack_beam10 0.9234 0.6570 0.9079 0.9234 knapsack_beam10_refined 0.9260 0.6921 0.9130 0.9260 diffusion_only 0.9109 0.6587 0.8973 0.9109 Amino-acid level Precision mode mass exact strict exact I/L exact both greedy 0.8335 0.8197 0.8556 0.8579 greedy_refined 0.8390 0.8259 0.8609 0.8630 beam10 0.8608 0.8435 0.8803 0.8826 beam10_refined 0.8639 0.8491 0.8832 0.8852 knapsack_beam10 0.8550 0.8408 0.8774 0.8797 knapsack_beam10_refined 0.8621 0.8480 0.8821 0.8841 diffusion_only 0.8658 0.8459 0.8814 0.8837 Recall mode mass exact strict exact I/L exact both greedy 0.8314 0.8177 0.8535 0.8557 greedy_refined 0.8368 0.8237 0.8586 0.8607 beam10 0.8603 0.8430 0.8798 0.8821 beam10_refined 0.8633 0.8484 0.8825 0.8846 knapsack_beam10 0.8552 0.8410 0.8776 0.8799 knapsack_beam10_refined 0.8622 0.8481 0.8822 0.8842 diffusion_only 0.8607 0.8408 0.8762 0.8784 AUC mode mass exact strict exact I/L exact both greedy 0.8931 0.8937 0.9029 0.9034 greedy_refined 0.9465 0.9228 0.9512 0.9528 beam10 0.9075 0.9073 0.9163 0.9169 beam10_refined 0.9669 0.9458 0.9697 0.9711 knapsack_beam10 0.9060 0.9079 0.9170 0.9176 knapsack_beam10_refined 0.9677 0.9467 0.9707 0.9720 diffusion_only 0.9584 0.9359 0.9613 0.9627 Coverage and totals Coverage -- the share of the benchmark each mode answered. mode peptide amino acid greedy 1.000000 0.997510 greedy_refined 0.999962 0.997383 beam10 1.000000 0.999380 beam10_refined 0.999969 0.999237 knapsack_beam10 1.000000 1.000239 (>1, see caveat 4) knapsack_beam10_refined 0.999965 1.000087 (>1, see caveat 4) diffusion_only 0.999964 0.994049 Spectra correctly sequenced, of 779,879. Runtime for each mode is tabulated separately below. mode mass exact strict exact I/L exact both runtime greedy 541,709 315,468 526,364 541,709 1h 40m greedy_refined 543,093 321,505 528,391 543,093 2h 57m beam10 567,610 329,226 551,195 567,610 10h 11m beam10_refined 570,636 346,117 556,814 570,636 20h 48m knapsack_beam10 567,962 329,326 551,519 567,962 25h 24m knapsack_beam10_refined 570,945 346,040 556,957 570,945 35h 50m diffusion_only 556,969 327,949 541,716 556,969 2h 48m Runtime Wall clock for the full benchmark, 779,879 spectra, one NVIDIA H100 per mode. Taken from the AIchor experiment records. mode wall clock hours spectra/hour vs fastest greedy 1h 40m 1.7 467,927 — greedy_refined 2h 57m 3.0 264,366 1.8× beam10 10h 11m 10.2 76,584 6.1× beam10_refined 20h 48m 20.8 37,494 12.5× knapsack_beam10 25h 24m 25.4 30,704 15.2× knapsack_beam10_refined 35h 50m 35.8 21,764 21.5× diffusion_only 2h 48m 2.8 278,528 1.7× Prediction files One file per inference mode, gzipped CSV, one row per spectrum (779,879 rows). checkpoint names the model weights; decoding_strategy and n_beams are the settings recorded on that mode's datapoint. file mode decoding strategy beams refined checkpoint(s) size predictions_instanovo_v1_2_2_greedy.csv.gz greedy greedy search 1 no instanovo-v1.2.0 136 MB predictions_instanovo_v1_2_2_greedy_refined.csv.gz greedy_refined greedy search 1 yes instanovo-v1.2.0; instanovoplus-v1.1.0 173 MB predictions_instanovo_v1_2_2_beam10.csv.gz beam10 beam search 10 no instanovo-v1.2.0 656 MB predictions_instanovo_v1_2_2_beam10_refined.csv.gz beam10_refined beam search 10 yes instanovo-v1.2.0; instanovoplus-v1.1.0 695 MB predictions_instanovo_v1_2_2_knapsack_beam10.csv.gz knapsack_beam10 knapsack beam search 10 no instanovo-v1.2.0 649 MB predictions_instanovo_v1_2_2_knapsack_beam10_refined.csv.gz knapsack_beam10_refined knapsack beam search 10 yes instanovo-v1.2.0; instanovoplus-v1.1.0 688 MB predictions_instanovo_v1_2_2_diffusion_only.csv.gz diffusion_only diffusion sampling — yes instanovoplus-v1.1.0 44 MB Each mode's full metrics are in the matching datapoint_instanovo_v1_2_2_ .json. Every file has the same 12 core columns -- spectrum_id, precursor_mz, precursor_charge, the emitted predictions with log_probs and token_log_probs, and delta_mass_ppm. The three beam-search modes carry 30 columns more: the sequence, log probability and per-token log probabilities of every one of the 10 beams, not only the one that was emitted. That is why they are roughly five times the size, and it is what makes re-ranking or oracle-beam analysis possible from this deposit without rerunning inference. What the four views mean A de novo prediction can be scored two ways. Mass-based matching accepts a residue whose mass matches the reference within tolerance. Exact matching additionally requires the residue identity to be the same. Exact matching then has to decide what to do with pairs mass spectrometry cannot physically separate: I and L are isomeric — identical mass. No instrument can tell them apart. Which one a tool emits is a convention, not a capability. Deamidated Q/N are isobaric with E/D — the same situation for those modifications. Hence the toggles, and hence four readings of one submission. Mass takes no variants The ambiguity toggles cannot change a mass comparison, because the pairs they concern are mass-degenerate by definition. Verified empirically: all four (allow_il, allow_deamidation) combinations produce an identical mass figure. That is why there is one mass plot, not four. exact (both) is identical to mass Note the first and last columns