NGS Bioinformatics Pipeline Validation
In clinical NGS the pipeline is part of the test. A change to an aligner version can change results as much as a change to a reagent.
SLC is an independent, nationwide clinical laboratory consulting network. We are not affiliated with, authorized by or endorsed by any sequencing platform maker, kit supplier or accrediting organization.
Pipeline components
Demultiplexing, read QC and trimming, alignment to a defined reference build, duplicate marking, variant calling by type, annotation, filtering and reporting tools.
Validation approach
Run reference truth sets and clinical samples through the locked pipeline; measure sensitivity and PPV by variant type and region; test edge cases (homopolymers, large indels, low VAF); validate in silico mutated data where appropriate.
Version control
Record every tool version, parameter, reference and database version. Use code repositories and containers so the exact pipeline can be reproduced.
Revalidation on change
Define change tiers: a database update may need a limited check, while a new caller requires broader revalidation. Document impact assessment, testing and director approval before release.
Security and access
Role-based access, audit trails, and backup and disaster recovery for clinical data.
Data storage and retention
42 CFR 493.1105 sets minimum retention periods. Quality control and patient test records, including instrument printouts and records of analytic systems activities, must be kept at least 2 years; records of performance specifications established or verified under 493.1253 must be kept for as long as the test system is in use and no less than 2 years; test procedures for 2 years after discontinuation; PT records for 2 years; and test reports for at least 2 years after reporting (pathology reports 10 years). Decide which NGS files (FASTQ, BAM/CRAM, VCF, reports) count as test records for your lab and how long you keep them, considering accreditation and state requirements, which may be longer, and your reanalysis policy.
What counts as the pipeline
For CLIA purposes the pipeline includes every computational step between raw sequencer output and the reported result: base calling and demultiplexing settings, read processing, alignment, variant calling, annotation, filtering, coverage calculation and report generation, plus reference genome build, annotation databases and any in-house scripts. Third-party or vendor software is still part of your test system and must be validated for your use.
Truth sets and test data
Publicly characterized reference genomes with high-confidence variant calls give a starting truth set for small variants. Supplement them with clinical samples previously tested by an orthogonal method, engineered reference materials containing variants of the types you claim, and in silico data where variants are inserted into real reads to test detection of rare or difficult events. Record versions of all truth sets used.
Measuring performance
Compare pipeline calls against truth by variant type and genomic context, calculating sensitivity and positive predictive value with confidence intervals. Examine false positives and false negatives individually to understand causes, and adjust filters only before the pipeline is locked. Confirm reproducibility by rerunning the same data and getting identical results.
Change tiers in practice
Infrastructure only (new server, same software and containers): reproducibility check on a sample set
Annotation database update: comparison of annotations on a sample set, review of changes
Tool version update: rerun validation data, compare calls, assess differences
New tool or algorithm: substantial revalidation for affected variant types
Reference genome change: full revalidation
Documentation for inspectors
Keep a pipeline description with each step, tool, version and parameter; the validation plan, data and summary; a version history linking each clinical run to the pipeline version used; change records with impact assessments and approvals; and procedures for access control, backup and data recovery.
Cloud and vendor pipelines
Cloud platforms and vendor analysis software can reduce infrastructure work, but the lab still needs to control versions, know when the vendor updates components, and validate changes before clinical use. Agree in writing how updates are communicated and scheduled, and confirm how data are secured and retained.
How SLC helps
SLC helps labs document pipelines, design validation studies, set up change control and versioning procedures, and prepare bioinformatics evidence for CAP inspection. Plans from $500.
Access control and audit trails
Limit who can change pipeline code, parameters and reference files. Use accounts tied to individuals, log changes, and keep audit trails that show who ran each analysis and when. Separate development and clinical environments so testing a new version never touches clinical results.
Reanalysis and reinterpretation
Exome and genome data can be reanalyzed later as knowledge grows. If you offer reanalysis, write a policy covering how it is requested, which pipeline version is used, whether reprocessing from raw data is needed, and how updated reports are issued. Retain the data needed to support that policy.
Personnel for bioinformatics
Staff who perform pipeline steps that affect patient results may be performing testing tasks. Document their qualifications, training and competency, and define who can approve pipeline changes.
Variant classes need separate validation
Single nucleotide variants, small insertions and deletions, larger indels, copy number variants, structural variants, gene fusions, repeat expansions and mitochondrial variants each use different algorithms or settings and fail in different ways. Validate each class you report with enough examples to estimate performance, and list classes the pipeline does not assess.
Reproducibility and determinism
Running the same input through the same pipeline version should give identical output. Pin software versions, random seeds where tools use them, reference files and databases, and test reproducibility as part of validation and after infrastructure changes.
Sample-level QC in the pipeline
Build automated checks into the pipeline: read counts, coverage of required regions, duplication, contamination estimates, sex and identity checks. Flag samples that fail rather than silently passing them on, and make the flags visible to the reviewer.
Common pipeline inspection findings
No documented list of tool versions and parameters
Clinical runs on a pipeline version different from the validated one
Database or software updates without impact assessment
No reproducibility testing after moving to new infrastructure
No procedure for backup, restore and disaster recovery
Unclear who may change the pipeline
Backup and disaster recovery
Define what is backed up, how often, where copies are kept, and how quickly you can restore. Test restoration at least yearly and document the result.
Software: compliance tracking with myLabCompliance
Paper binders and spreadsheets break down once a lab runs multiple instruments, shifts and testing personnel. We recommend myLabCompliance, a web-based CLIA compliance management platform for technical consultants, lab directors, lab owners and quality teams. Its public site describes tools for personnel management, validation, quality control, equipment, document control, scheduling, analytics and audit-ready workflows.
QC logs and quality control records kept in one place
Personnel records and competency tracking
Validation records for new instruments and methods
Equipment records for maintenance and calibration
Document control for SOPs and policies
Scheduling for recurring tasks such as PT events and competency
More NGS resources from SLC
Frequently asked questions
Does a pipeline update require revalidation?
Changes require documented assessment and validation proportional to the change, approved by the director.
How long must NGS data be retained?
At least the CLIA minimums in 42 CFR 493.1105, plus any accreditation or state rules.
Sound Lab Consultants (SLC), PO Box 2135, Chandler, AZ 85244. A nationwide clinical laboratory consulting network. Plans from $500.