Error during " snp:aggregate_pileup_variants (1) " process on canine model

[vlebars]

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v2.2.0

Workflow Execution

Command line (Cluster)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

#!/bin/bash
#SBATCH --cpus-per-task=32
#SBATCH --mem=128G

Activate nextflow environment

. /local/env/envconda.sh

conda activate /home/genouest/cnrs_umr6290/vlebars/env/env-nextflow

nextflow run epi2me-labs/wf-human-variation
--bam '/groups/igdrion/AKC_methylDog/primary/dog/chapouille_1_ADN11198_2_SH_BH/SQK-LSK114/flowcellprom/dorado7.4.13/chapouille_1_ADN11198_2_SH_BH.sorted.bam'
--basecaller_cfg '[email protected]'
--ref '/groups/dog/data/canFam4/sequence/UU_Cfam_GSD_1.0_canFam4/UU_Cfam_GSD_1.0_ROSY.fa'
--sample_name 'chapouille_1_ADN11198_2_SH_BH'
--snp
--sv
--out_dir '/groups/igdrion/AKC_methylDog/secondary/2-VARIANT_CALLING'
--cnv false
--str false
--annotation false
--include_all_ctgs true
-profile singularity

Workflow Execution - CLI Execution Profile

singularity

What happened?

Pipeline crashes at the snp:aggregate_pileup_variants process.

executor >  local (5)
[ed/3e2275] process > ingress:checkBamHeaders (1)    [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:sortBam                -
[-        ] process > ingress:mergeBams              -
[-        ] process > ingress:catSortBams            -
[e1/d339b9] process > ingress:validateIndex (1)      [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:samtools_index         -
[bb/591c8f] process > ingress:check_for_alignment... [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:minimap2_alignment     -
[f4/89239a] process > cram_cache (1)                 [100%] 1 of 1, cached: 1 ✔
[ae/7ddc13] process > getAllChromosomesBed (1)       [100%] 1 of 1, cached: 1 ✔
[b4/10db2b] process > mosdepth_input (1)             [100%] 1 of 1, cached: 1 ✔
[01/519dd5] process > getVersions                    [100%] 1 of 1, cached: 1 ✔
[6d/0cf500] process > getParams                      [100%] 1 of 1, cached: 1 ✔
[5f/0a8b15] process > readStats (1)                  [100%] 1 of 1, cached: 1 ✔
[d5/e89b95] process > makeAlignmentReport            [100%] 1 of 1, cached: 1 ✔
[-        ] process > failedQCReport                 -
[c3/f00c40] process > lookup_clair3_model (1)        [100%] 1 of 1, cached: 1 ✔
[af/67d451] process > snp:make_chunks (1)            [100%] 1 of 1, cached: 1 ✔
[6b/f8c289] process > snp:pileup_variants (2645)     [100%] 2652 of 2652, cac...
[-        ] process > snp:aggregate_pileup_variants  -
[-        ] process > snp:select_het_snps            -
[-        ] process > snp:phase_contig               -
[-        ] process > snp:get_qual_filter            -
[-        ] process > snp:create_candidates          -
[-        ] process > snp:evaluate_candidates        -
[-        ] process > snp:aggregate_full_align_va... -
[-        ] process > snp:merge_pileup_and_full_vars -
[-        ] process > snp:aggregate_all_variants     -
[bc/b097b0] process > sv:variantCall:sniffles2 (1)   [100%] 1 of 1, cached: 1 ✔
[74/f4f3c5] process > sv:variantCall:filterCalls (1) [100%] 1 of 1, cached: 1 ✔
[60/75e8b2] process > sv:variantCall:sortVCF (1)     [100%] 1 of 1, cached: 1 ✔
[c1/b071e4] process > sv:runReport:getVersions       [100%] 1 of 1, cached: 1 ✔
[0f/3b3054] process > sv:runReport:getParams         [100%] 1 of 1, cached: 1 ✔
[fb/e9ea31] process > sv:runReport:report (1)        [100%] 1 of 1, cached: 1 ✔
[d5/36690b] process > output_sv (4)                  [100%] 4 of 4 ✔
[-        ] process > refine_with_sv                 -
[-        ] process > concat_refined_snp             -
[-        ] process > vcfStats                       -
[fa/b57123] process > report_snp:getVersions         [100%] 1 of 1, cached: 1 ✔
[a0/332d01] process > report_snp:getParams           [100%] 1 of 1, cached: 1 ✔
[-        ] process > report_snp:makeReport          -
[-        ] process > output_snp                     -
[45/c9dd8d] process > configure_jbrowse (1)          [100%] 1 of 1, cached: 1 ✔
[-        ] process > combine_metrics_json           -
[6d/edc9c3] process > publish_artifact (8)           [100%] 8 of 8, cached: 7
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'snp:aggregate_pileup_variants (1)'

Caused by:
  Process `snp:aggregate_pileup_variants` input file name collision -- There are multiple input files for each of the following file names: input_vcfs/pileup_chr34_1.vcf, input_vcfs/pileup_chr32_1.vcf, input_vcfs/pileup_chr3_4.vcf, input_vcfs/pileup_chr3_2.vcf


Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

I tried a different input BAM (still canine sample) but still got the same bug (with the same 4 input files triggering the error)

Relevant log output

Nov-20 20:08:45.284 [Actor Thread 15284] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=snp:aggregate_pileup_variants (1); work-dir=null
  error [nextflow.exception.ProcessUnrecoverableException]: Process `snp:aggregate_pileup_variants` input file name collision -- There are multiple input files for each of the following file names: input_vcfs/pileup_chr34_1.vcf, input_vcfs/pileup_chr32_1.vcf, input_vcfs/pileup_chr3_4.vcf, input_vcfs/pileup_chr3_2.vcf
Nov-20 20:08:45.319 [Actor Thread 15284] ERROR nextflow.processor.TaskProcessor - Error executing process > 'snp:aggregate_pileup_variants (1)'

Caused by:
  Process `snp:aggregate_pileup_variants` input file name collision -- There are multiple input files for each of the following file names: input_vcfs/pileup_chr34_1.vcf, input_vcfs/pileup_chr32_1.vcf, input_vcfs/pileup_chr3_4.vcf, input_vcfs/pileup_chr3_2.vcf

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

[SamStudio8]

This looks a little curious - it looks as though there may be SNP chunks with the same name being passed to aggregate_pileup_variants. The log you have provided has cached results, is this the same error you had on the first run of the workflow? Or is this a new error on resuming a failed workflow?

[vlebars]

Hi, thank you for your response.
The error is almost the same on the first run, only the input_vcf files causing the error are different :

First run:

ERROR ~ Error executing process > 'snp:aggregate_pileup_variants (1)'

Caused by:
  Process `snp:aggregate_pileup_variants` input file name collision -- There are multiple input files for each of the following file names: input_vcfs/pileup_chr3_4.vcf, input_vcfs/pileup_chr3_2.vcf

Resumed runs:

ERROR ~ Error executing process > 'snp:aggregate_pileup_variants (1)'

Caused by:
  Process `snp:aggregate_pileup_variants` input file name collision -- There are multiple input files for each of the following file names: input_vcfs/pileup_chr34_1.vcf, input_vcfs/pileup_chr32_1.vcf, input_vcfs/pileup_chr3_4.vcf, input_vcfs/pileup_chr3_2.vcf

(even with different bam input the error and the chunks names causing the error (first or resumed runs) are always the same)

Thank you
Victor