Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

value error; index error #22

Open
kdelmore opened this issue Jun 15, 2021 · 4 comments
Open

value error; index error #22

kdelmore opened this issue Jun 15, 2021 · 4 comments

Comments

@kdelmore
Copy link

Wondering if anyone has run into the following two errors. I'm running the program on 79 individuals and 6 of them are giving me one of the errors. I'm using bam files generated by LR and the following command line:
linkedsv.py -i phased_possorted_bam.bam
-r genome.fa
-d ind_noblack
--black_region_bed empty_blacklist.bed
--gap_region_bed genome_N.bed
-t 4

Error 1 =
[06/15/2021 11:26:19 (469.017 MB)] cat /home/delmore/10x/linked_sv/T012832_noblack/assembly_raw_variants.0.txt /home/delmore/10x/linked_sv/T012832_noblack/assembly_raw_variants.1.txt /home/delmore/10x/linked_sv/T012832_noblack/assembly_raw_variants.2.txt /home/delmore/10x/linked_sv/T012832_noblack/assembly_raw_variants.3.txt > /home/delmore/10x/linked_sv/T012832_noblack/local_assembly_raw_variants.txt
Traceback (most recent call last):
File "/home/delmore/tools/LinkedSV/linkedsv.py", line 313, in
main()
File "/home/delmore/tools/LinkedSV/linkedsv.py", line 63, in main
detect_sv_from_short_reads.detect_sv_from_short_reads(args, dbo_args, endpoint_args)
File "/home/delmore/tools/LinkedSV/scripts/detect_sv_from_short_reads.py", line 80, in detect_sv_from_short_reads
local_assembly.small_deletion_dection_by_local_assembly(args.samtools, args.bedtools, args.fermikit_dir, args.input_bam, args.ref_fa, args.faidx_file, args.out_dir, local_assembly_out_file, args.n_thread, window_size, max_depth, rm_temp_files)
File "/home/delmore/tools/LinkedSV/scripts/local_assembly.py", line 102, in small_deletion_dection_by_local_assembly
extract_del_from_vcf_file(all_processes_out_combined_vcf_file, out_del_call_file)
File "/home/delmore/tools/LinkedSV/scripts/local_assembly.py", line 285, in extract_del_from_vcf_file
pos1 = int(items[1])
ValueError: invalid literal for int() with base 10: ''

Intermediate files generated by the program =
-rw-r--r-- 1 delmore 629M May 31 08:02 phased_possorted_bam.bam.hap_depth.txt
-rw-r--r-- 1 delmore 592 Jun 14 19:23 phased_possorted_bam.bam.barcode_statistics
-rw-r--r-- 1 delmore 12G Jun 14 19:23 phased_possorted_bam.bam.bcd21.gz
-rw-r--r-- 1 delmore 1.2G Jun 14 19:45 phased_possorted_bam.bam.low_mapq.bcd21.gz
-rw-r--r-- 1 delmore 3.5G Jun 14 19:57 phased_possorted_bam.bam.bcd22.tmp
-rw-r--r-- 1 delmore 4.5G Jun 14 20:12 phased_possorted_bam.bam.bcd22
-rw-r--r-- 1 delmore 241M Jun 14 20:12 phased_possorted_bam.bam.weird_reads.txt
-rw-r--r-- 1 delmore 220M Jun 14 20:16 phased_possorted_bam.bam.barcode_cov.bed
-rw-r--r-- 1 delmore 4.6K Jun 14 20:16 phased_possorted_bam.bam.high_cov.bed
-rw-r--r-- 1 delmore 736 Jun 14 20:17 phased_possorted_bam.bam.fragment_statistics
-rw-r--r-- 1 delmore 9.0K Jun 14 20:17 phased_possorted_bam.bam.arguments
-rw-r--r-- 1 delmore 147M Jun 14 20:19 phased_possorted_bam.bam.node33
-rw-r--r-- 1 delmore 135K Jun 14 20:22 phased_possorted_bam.bam.node_cluster33
-rw-r--r-- 1 delmore 48K Jun 14 20:22 phased_possorted_bam.bam.node_cluster55
-rw-r--r-- 1 delmore 60K Jun 14 20:22 phased_possorted_bam.bam.node_cluster53
-rw-r--r-- 1 delmore 143K Jun 14 20:22 phased_possorted_bam.bam.node_cluster35
-rw-r--r-- 1 delmore 241K Jun 14 20:24 phased_possorted_bam.bam.bk_cand_pairs
-rw-r--r-- 1 delmore 307M Jun 14 20:32 phased_possorted_bam.bam.read_depth.txt
-rw-r--r-- 1 delmore 307M Jun 14 20:45 phased_possorted_bam.bam.bcd11
-rw-r--r-- 1 delmore 747M Jun 14 20:46 phased_possorted_bam.bam.bcd12
-rw-r--r-- 1 delmore 480M Jun 14 21:04 phased_possorted_bam.bam.bcd13
-rw-r--r-- 1 delmore 362K Jun 14 21:06 phased_possorted_bam.bam.qbkpair.bedpe
-rw-r--r-- 1 delmore 109K Jun 14 21:06 phased_possorted_bam.bam.raw_large_svcalls.bedpe
-rw-r--r-- 1 delmore 11K Jun 14 21:09 phased_possorted_bam.bam.filtered_large_svcalls.bedpe
-rw-r--r-- 1 delmore 0 Jun 14 21:09 phased_possorted_bam.bam.large_cnv.bedpe
-rw-r--r-- 1 delmore 5.4M Jun 14 21:14 phased_possorted_bam.bam.weird_reads.clusters.txt
-rw-r--r-- 1 delmore 79K Jun 14 21:15 discordant_read_pairs.del.bedpe
-rw-r--r-- 1 delmore 178M Jun 15 08:51 assembly_raw_variants.1.txt
-rw-r--r-- 1 delmore 178M Jun 15 09:34 assembly_raw_variants.3.txt
-rw-r--r-- 1 delmore 176M Jun 15 09:54 assembly_raw_variants.0.txt
-rw-r--r-- 1 delmore 179M Jun 15 11:26 assembly_raw_variants.2.txt
-rw-r--r-- 1 delmore 709M Jun 15 11:26 local_assembly_raw_variants.txt
-rw-r--r-- 1 delmore 186K Jun 15 11:27 local_assembly.del.bedpe

Error 2 =
inner size cut-off = -2147483648
gap distance cut-off = -2147483648
started round 2 clustering
finished round 2 clustering
time used is 19.09 seconds
total number of reads is: 0
total number of weird reads is: 0
[06/14/2021 16:26:00 (116.593 MB)] sorting bam file by barcode
[06/14/2021 16:26:00 (116.593 MB)] running command: /home/delmore/tools/LinkedSV/scripts/../bin/output_bam_coreinfo /home/delmore/10x/alignments/APC6476/outs/phased_possorted_bam.bam | samtools sort -l 1 -m 1G -@ 4 -t BX -o /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.sortbx.bam -
[06/14/2021 18:16:56 (116.224 MB)] extracting barcode info from bam file
[06/14/2021 18:16:56 (116.269 MB)] running command: /home/delmore/tools/LinkedSV/scripts/../bin/extract_barcode_info /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.sortbx.bam STDOUT /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.barcode_statistics | /home/delmore/tools/LinkedSV/scripts/../bin/pigz --fast --processes 3 - > /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.bcd21.gz
[06/14/2021 18:16:57 (116.290 MB)] extracting low mapq bcd21
[06/14/2021 18:16:58 (116.453 MB)] clustering reads
[06/14/2021 18:16:58 (116.453 MB)] /home/delmore/tools/LinkedSV/scripts/../bin/cluster_reads /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.bcd21.gz /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.bcd22 /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.weird_reads.txt 1 -1 20 4
[06/14/2021 18:17:31 (116.470 MB)] searching for extremely high coverage region
[06/14/2021 18:17:34 (116.998 MB)] calculating distribution parameters
[06/14/2021 18:17:35 (117.002 MB)] total number of reads in the genome is: 320402874
[06/14/2021 18:17:35 (117.002 MB)] calculating fragment parameters from file: /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.bcd22
[06/14/2021 18:17:35 (117.002 MB)] N95_fragment_length is: -1
Traceback (most recent call last):
File "/home/delmore/tools/LinkedSV/linkedsv.py", line 313, in
main()
File "/home/delmore/tools/LinkedSV/linkedsv.py", line 47, in main
detect_increased_fragment_ends(args, dbo_args, endpoint_args)
File "/home/delmore/tools/LinkedSV/linkedsv.py", line 208, in detect_increased_fragment_ends
global_distribution.estimate_global_distribution(args, dbo_args, endpoint_args, endpoint_args.bcd22_file)
File "/home/delmore/tools/LinkedSV/scripts/global_distribution.py", line 133, in estimate_global_distribution
get_fragment_parameters(args, dbo_args, endpoint_args, global_dist_fp, target_bcd22_file, is_fast_mode)
File "/home/delmore/tools/LinkedSV/scripts/global_distribution.py", line 213, in get_fragment_parameters
args.fragment_length_lmda = fit_geometric_distribution(frm_length_list, readpair = False)
File "/home/delmore/tools/LinkedSV/scripts/global_distribution.py", line 171, in fit_geometric_distribution
k = np.percentile(length_list, cdf2)
File "<array_function internals>", line 6, in percentile
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/lib/function_base.py", line 3706, in percentile
a, q, axis, out, overwrite_input, interpolation, keepdims)
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/lib/function_base.py", line 3826, in _quantile_unchecked
interpolation=interpolation)
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/lib/function_base.py", line 3403, in _ureduce
r = func(a, **kwargs)
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/lib/function_base.py", line 3941, in _quantile_ureduce_func
x1 = take(ap, indices_below, axis=axis) * weights_below
File "<array_function internals>", line 6, in take
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 194, in take
return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
File "/data/modules/python/3.7.1/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 61, in _wrapfunc
return bound(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.

Intermediate files generated by the program =
-rw-r--r-- 1 delmore 592 May 30 16:15 phased_possorted_bam.bam.barcode_statistics
-rw-r--r-- 1 delmore 9.0K May 30 19:39 phased_possorted_bam.bam.arguments
-rw-r--r-- 1 delmore 141M May 30 19:45 phased_possorted_bam.bam.node33
-rw-r--r-- 1 delmore 116K May 30 19:55 phased_possorted_bam.bam.node_cluster33
-rw-r--r-- 1 delmore 52K May 30 19:56 phased_possorted_bam.bam.node_cluster55
-rw-r--r-- 1 delmore 56K May 30 19:56 phased_possorted_bam.bam.node_cluster53
-rw-r--r-- 1 delmore 121K May 30 19:58 phased_possorted_bam.bam.node_cluster35
-rw-r--r-- 1 delmore 203K May 30 20:01 phased_possorted_bam.bam.bk_cand_pairs
-rw-r--r-- 1 delmore 307M May 30 20:12 phased_possorted_bam.bam.read_depth.txt
-rw-r--r-- 1 delmore 307M May 30 20:32 phased_possorted_bam.bam.bcd11
-rw-r--r-- 1 delmore 747M May 30 20:42 phased_possorted_bam.bam.bcd12
-rw-r--r-- 1 delmore 186M Jun 1 04:15 phased_possorted_bam.bam.sortbx.bam.tmp.0003.bam
-rw-r--r-- 1 delmore 169M Jun 1 04:15 phased_possorted_bam.bam.sortbx.bam.tmp.0001.bam
-rw-r--r-- 1 delmore 169M Jun 1 04:15 phased_possorted_bam.bam.sortbx.bam.tmp.0000.bam
-rw-r--r-- 1 delmore 183M Jun 1 04:16 phased_possorted_bam.bam.sortbx.bam.tmp.0002.bam
-rw-r--r-- 1 delmore 186M Jun 1 04:28 phased_possorted_bam.bam.sortbx.bam.tmp.0007.bam
-rw-r--r-- 1 delmore 167M Jun 1 04:28 phased_possorted_bam.bam.sortbx.bam.tmp.0005.bam
-rw-r--r-- 1 delmore 179M Jun 1 04:28 phased_possorted_bam.bam.sortbx.bam.tmp.0006.bam
-rw-r--r-- 1 delmore 189M Jun 1 04:28 phased_possorted_bam.bam.sortbx.bam.tmp.0004.bam
-rw-r--r-- 1 delmore 183M Jun 1 04:35 phased_possorted_bam.bam.sortbx.bam.tmp.0011.bam
-rw-r--r-- 1 delmore 196M Jun 1 04:38 phased_possorted_bam.bam.sortbx.bam.tmp.0010.bam
-rw-r--r-- 1 delmore 195M Jun 1 04:41 phased_possorted_bam.bam.sortbx.bam.tmp.0008.bam
-rw-r--r-- 1 delmore 198M Jun 1 04:45 phased_possorted_bam.bam.sortbx.bam.tmp.0009.bam
-rw-r--r-- 1 delmore 155M Jun 1 04:52 phased_possorted_bam.bam.sortbx.bam.tmp.0012.bam
-rw-r--r-- 1 delmore 176M Jun 1 04:54 phased_possorted_bam.bam.sortbx.bam.tmp.0014.bam
-rw-r--r-- 1 delmore 173M Jun 1 04:56 phased_possorted_bam.bam.sortbx.bam.tmp.0015.bam
-rw-r--r-- 1 delmore 172M Jun 1 04:59 phased_possorted_bam.bam.sortbx.bam.tmp.0013.bam
-rw-r--r-- 1 delmore 167M Jun 1 05:03 phased_possorted_bam.bam.sortbx.bam.tmp.0017.bam
-rw-r--r-- 1 delmore 169M Jun 1 05:05 phased_possorted_bam.bam.sortbx.bam.tmp.0018.bam
-rw-r--r-- 1 delmore 171M Jun 1 05:08 phased_possorted_bam.bam.sortbx.bam.tmp.0019.bam
-rw-r--r-- 1 delmore 170M Jun 1 05:11 phased_possorted_bam.bam.sortbx.bam.tmp.0016.bam
-rw-r--r-- 1 delmore 164M Jun 1 05:32 phased_possorted_bam.bam.sortbx.bam.tmp.0023.bam
-rw-r--r-- 1 delmore 170M Jun 1 05:32 phased_possorted_bam.bam.sortbx.bam.tmp.0021.bam
-rw-r--r-- 1 delmore 181M Jun 1 05:32 phased_possorted_bam.bam.sortbx.bam.tmp.0022.bam
-rw-r--r-- 1 delmore 183M Jun 1 05:33 phased_possorted_bam.bam.sortbx.bam.tmp.0020.bam
-rw-r--r-- 1 delmore 189M Jun 1 05:42 phased_possorted_bam.bam.sortbx.bam.tmp.0025.bam
-rw-r--r-- 1 delmore 161M Jun 1 05:42 phased_possorted_bam.bam.sortbx.bam.tmp.0026.bam
-rw-r--r-- 1 delmore 173M Jun 1 05:42 phased_possorted_bam.bam.sortbx.bam.tmp.0024.bam
-rw-r--r-- 1 delmore 165M Jun 1 05:42 phased_possorted_bam.bam.sortbx.bam.tmp.0027.bam
-rw-r--r-- 1 delmore 166M Jun 1 05:45 phased_possorted_bam.bam.sortbx.bam.tmp.0029.bam
-rw-r--r-- 1 delmore 171M Jun 1 05:46 phased_possorted_bam.bam.sortbx.bam.tmp.0031.bam
-rw-r--r-- 1 delmore 169M Jun 1 05:46 phased_possorted_bam.bam.sortbx.bam.tmp.0028.bam
-rw-r--r-- 1 delmore 168M Jun 1 05:47 phased_possorted_bam.bam.sortbx.bam.tmp.0030.bam
-rw-r--r-- 1 delmore 165M Jun 1 05:51 phased_possorted_bam.bam.sortbx.bam.tmp.0035.bam
-rw-r--r-- 1 delmore 178M Jun 1 05:51 phased_possorted_bam.bam.sortbx.bam.tmp.0033.bam
-rw-r--r-- 1 delmore 178M Jun 1 05:51 phased_possorted_bam.bam.sortbx.bam.tmp.0034.bam
-rw-r--r-- 1 delmore 169M Jun 1 05:51 phased_possorted_bam.bam.sortbx.bam.tmp.0032.bam
-rw-r--r-- 1 delmore 172M Jun 1 05:56 phased_possorted_bam.bam.sortbx.bam.tmp.0039.bam
-rw-r--r-- 1 delmore 183M Jun 1 05:56 phased_possorted_bam.bam.sortbx.bam.tmp.0036.bam
-rw-r--r-- 1 delmore 187M Jun 1 05:56 phased_possorted_bam.bam.sortbx.bam.tmp.0038.bam
-rw-r--r-- 1 delmore 182M Jun 1 05:56 phased_possorted_bam.bam.sortbx.bam.tmp.0037.bam
-rw-r--r-- 1 delmore 173M Jun 1 06:00 phased_possorted_bam.bam.sortbx.bam.tmp.0041.bam
-rw-r--r-- 1 delmore 171M Jun 1 06:00 phased_possorted_bam.bam.sortbx.bam.tmp.0043.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:00 phased_possorted_bam.bam.sortbx.bam.tmp.0042.bam
-rw-r--r-- 1 delmore 172M Jun 1 06:00 phased_possorted_bam.bam.sortbx.bam.tmp.0040.bam
-rw-r--r-- 1 delmore 169M Jun 1 06:05 phased_possorted_bam.bam.sortbx.bam.tmp.0045.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:05 phased_possorted_bam.bam.sortbx.bam.tmp.0047.bam
-rw-r--r-- 1 delmore 164M Jun 1 06:05 phased_possorted_bam.bam.sortbx.bam.tmp.0044.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:05 phased_possorted_bam.bam.sortbx.bam.tmp.0046.bam
-rw-r--r-- 1 delmore 188M Jun 1 06:09 phased_possorted_bam.bam.sortbx.bam.tmp.0048.bam
-rw-r--r-- 1 delmore 173M Jun 1 06:09 phased_possorted_bam.bam.sortbx.bam.tmp.0049.bam
-rw-r--r-- 1 delmore 179M Jun 1 06:09 phased_possorted_bam.bam.sortbx.bam.tmp.0050.bam
-rw-r--r-- 1 delmore 184M Jun 1 06:09 phased_possorted_bam.bam.sortbx.bam.tmp.0051.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:13 phased_possorted_bam.bam.sortbx.bam.tmp.0054.bam
-rw-r--r-- 1 delmore 174M Jun 1 06:13 phased_possorted_bam.bam.sortbx.bam.tmp.0055.bam
-rw-r--r-- 1 delmore 171M Jun 1 06:14 phased_possorted_bam.bam.sortbx.bam.tmp.0052.bam
-rw-r--r-- 1 delmore 174M Jun 1 06:14 phased_possorted_bam.bam.sortbx.bam.tmp.0053.bam
-rw-r--r-- 1 delmore 165M Jun 1 06:16 phased_possorted_bam.bam.sortbx.bam.tmp.0059.bam
-rw-r--r-- 1 delmore 179M Jun 1 06:17 phased_possorted_bam.bam.sortbx.bam.tmp.0058.bam
-rw-r--r-- 1 delmore 173M Jun 1 06:18 phased_possorted_bam.bam.sortbx.bam.tmp.0057.bam
-rw-r--r-- 1 delmore 164M Jun 1 06:18 phased_possorted_bam.bam.sortbx.bam.tmp.0056.bam
-rw-r--r-- 1 delmore 170M Jun 1 06:22 phased_possorted_bam.bam.sortbx.bam.tmp.0063.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:22 phased_possorted_bam.bam.sortbx.bam.tmp.0062.bam
-rw-r--r-- 1 delmore 179M Jun 1 06:22 phased_possorted_bam.bam.sortbx.bam.tmp.0061.bam
-rw-r--r-- 1 delmore 171M Jun 1 06:22 phased_possorted_bam.bam.sortbx.bam.tmp.0060.bam
-rw-r--r-- 1 delmore 167M Jun 1 06:27 phased_possorted_bam.bam.sortbx.bam.tmp.0064.bam
-rw-r--r-- 1 delmore 174M Jun 1 06:27 phased_possorted_bam.bam.sortbx.bam.tmp.0065.bam
-rw-r--r-- 1 delmore 175M Jun 1 06:27 phased_possorted_bam.bam.sortbx.bam.tmp.0067.bam
-rw-r--r-- 1 delmore 187M Jun 1 06:27 phased_possorted_bam.bam.sortbx.bam.tmp.0066.bam
-rw-r--r-- 1 delmore 169M Jun 1 06:31 phased_possorted_bam.bam.sortbx.bam.tmp.0071.bam
-rw-r--r-- 1 delmore 182M Jun 1 06:32 phased_possorted_bam.bam.sortbx.bam.tmp.0069.bam
-rw-r--r-- 1 delmore 180M Jun 1 06:32 phased_possorted_bam.bam.sortbx.bam.tmp.0068.bam
-rw-r--r-- 1 delmore 198M Jun 1 06:34 phased_possorted_bam.bam.sortbx.bam.tmp.0070.bam
-rw-r--r-- 1 delmore 168M Jun 1 06:37 phased_possorted_bam.bam.sortbx.bam.tmp.0074.bam
-rw-r--r-- 1 delmore 163M Jun 1 06:37 phased_possorted_bam.bam.sortbx.bam.tmp.0072.bam
-rw-r--r-- 1 delmore 176M Jun 1 06:38 phased_possorted_bam.bam.sortbx.bam.tmp.0075.bam
-rw-r--r-- 1 delmore 166M Jun 1 06:40 phased_possorted_bam.bam.sortbx.bam.tmp.0073.bam
-rw-r--r-- 1 delmore 192M Jun 1 06:44 phased_possorted_bam.bam.sortbx.bam.tmp.0076.bam
-rw-r--r-- 1 delmore 184M Jun 1 06:45 phased_possorted_bam.bam.sortbx.bam.tmp.0078.bam
-rw-r--r-- 1 delmore 195M Jun 1 06:46 phased_possorted_bam.bam.sortbx.bam.tmp.0077.bam
-rw-r--r-- 1 delmore 196M Jun 1 06:47 phased_possorted_bam.bam.sortbx.bam.tmp.0079.bam
-rw-r--r-- 1 delmore 186M Jun 1 06:53 phased_possorted_bam.bam.sortbx.bam.tmp.0082.bam
-rw-r--r-- 1 delmore 166M Jun 1 06:53 phased_possorted_bam.bam.sortbx.bam.tmp.0083.bam
-rw-r--r-- 1 delmore 184M Jun 1 06:53 phased_possorted_bam.bam.sortbx.bam.tmp.0081.bam
-rw-r--r-- 1 delmore 183M Jun 1 06:53 phased_possorted_bam.bam.sortbx.bam.tmp.0080.bam
-rw-r--r-- 1 delmore 186M Jun 1 07:03 phased_possorted_bam.bam.sortbx.bam.tmp.0085.bam
-rw-r--r-- 1 delmore 178M Jun 1 07:03 phased_possorted_bam.bam.sortbx.bam.tmp.0086.bam
-rw-r--r-- 1 delmore 179M Jun 1 07:03 phased_possorted_bam.bam.sortbx.bam.tmp.0087.bam
-rw-r--r-- 1 delmore 177M Jun 1 07:03 phased_possorted_bam.bam.sortbx.bam.tmp.0084.bam
-rw-r--r-- 1 delmore 20 Jun 14 18:16 phased_possorted_bam.bam.bcd21.gz
-rw-r--r-- 1 delmore 65 Jun 14 18:16 phased_possorted_bam.bam.low_mapq.bcd21.gz
-rw-r--r-- 1 delmore 248 Jun 14 18:17 phased_possorted_bam.bam.bcd22.tmp
-rw-r--r-- 1 delmore 285 Jun 14 18:17 phased_possorted_bam.bam.bcd22
-rw-r--r-- 1 delmore 0 Jun 14 18:17 phased_possorted_bam.bam.weird_reads.txt
-rw-r--r-- 1 delmore 0 Jun 14 18:17 phased_possorted_bam.bam.barcode_cov.bed
-rw-r--r-- 1 delmore 0 Jun 14 18:17 phased_possorted_bam.bam.high_cov.bed
-rw-r--r-- 1 delmore 140 Jun 14 18:17 phased_possorted_bam.bam.fragment_statistics

Thanks! Kira
@fangli80
Copy link
Collaborator

Hi,
For the first error, please clone the latest version. It should be fixed.
The second error seems due to a failure in sorting the bam file. (maybe out of memory or no enough space in disk). If you upload the full stderr and stdout files, maybe I can find the problem.

You can also run the command manually and see what the error is.

/home/delmore/tools/LinkedSV/scripts/../bin/output_bam_coreinfo /home/delmore/10x/alignments/APC6476/outs/phased_possorted_bam.bam | samtools sort -l 1 -m 1G -@ 4 -t BX -o /home/delmore/10x/linked_sv/APC6476_noblack/phased_possorted_bam.bam.sortbx.bam -

Best,
Li

@kdelmore
Copy link
Author

job.4496443.APC6732.linkedsv.txt

Thanks Li. I'll try increasing memory for the second error and have attached the log here in case there are any clues in there. One final relatively consistent error I'm getting is below. Could this be memory as well?

[06/15/2021 19:43:06 (116.134 MB)] searching for extremely high coverage region
[06/15/2021 19:43:07 (116.642 MB)] calculating distribution parameters
[06/15/2021 19:43:07 (116.642 MB)] ERROR! statistics file (/home/delmore/10x/linked_sv/BE81584_noblack/phased_possorted_bam.bam.barcode_statistics) does not exist!

intermediate files =

-rw-r--r-- 1 delmore 156M May 29 18:29 phased_possorted_bam.bam.sortbx.bam.tmp.0000.bam
-rw-r--r-- 1 delmore 165M May 29 18:30 phased_possorted_bam.bam.sortbx.bam.tmp.0001.bam
-rw-r--r-- 1 delmore 170M May 29 18:30 phased_possorted_bam.bam.sortbx.bam.tmp.0002.bam
-rw-r--r-- 1 delmore 181M May 29 18:31 phased_possorted_bam.bam.sortbx.bam.tmp.0003.bam
-rw-r--r-- 1 delmore 161M May 29 18:44 phased_possorted_bam.bam.sortbx.bam.tmp.0006.bam
-rw-r--r-- 1 delmore 159M May 29 18:46 phased_possorted_bam.bam.sortbx.bam.tmp.0005.bam
-rw-r--r-- 1 delmore 183M May 29 18:47 phased_possorted_bam.bam.sortbx.bam.tmp.0004.bam
-rw-r--r-- 1 delmore 182M May 29 18:49 phased_possorted_bam.bam.sortbx.bam.tmp.0007.bam
-rw-r--r-- 1 delmore 154M May 29 19:03 phased_possorted_bam.bam.sortbx.bam.tmp.0011.bam
-rw-r--r-- 1 delmore 175M May 29 19:04 phased_possorted_bam.bam.sortbx.bam.tmp.0010.bam
-rw-r--r-- 1 delmore 176M May 29 19:05 phased_possorted_bam.bam.sortbx.bam.tmp.0008.bam
-rw-r--r-- 1 delmore 179M May 29 19:05 phased_possorted_bam.bam.sortbx.bam.tmp.0009.bam
-rw-r--r-- 1 delmore 165M May 29 19:12 phased_possorted_bam.bam.sortbx.bam.tmp.0013.bam
-rw-r--r-- 1 delmore 160M May 29 19:12 phased_possorted_bam.bam.sortbx.bam.tmp.0014.bam
-rw-r--r-- 1 delmore 164M May 29 19:13 phased_possorted_bam.bam.sortbx.bam.tmp.0015.bam
-rw-r--r-- 1 delmore 157M May 29 19:13 phased_possorted_bam.bam.sortbx.bam.tmp.0012.bam
-rw-r--r-- 1 delmore 158M May 29 19:19 phased_possorted_bam.bam.sortbx.bam.tmp.0016.bam
-rw-r--r-- 1 delmore 163M May 29 19:19 phased_possorted_bam.bam.sortbx.bam.tmp.0017.bam
-rw-r--r-- 1 delmore 161M May 29 19:20 phased_possorted_bam.bam.sortbx.bam.tmp.0018.bam
-rw-r--r-- 1 delmore 174M May 29 19:20 phased_possorted_bam.bam.sortbx.bam.tmp.0019.bam
-rw-r--r-- 1 delmore 164M May 29 19:34 phased_possorted_bam.bam.sortbx.bam.tmp.0022.bam
-rw-r--r-- 1 delmore 172M May 29 19:35 phased_possorted_bam.bam.sortbx.bam.tmp.0021.bam
-rw-r--r-- 1 delmore 156M May 29 19:35 phased_possorted_bam.bam.sortbx.bam.tmp.0023.bam
-rw-r--r-- 1 delmore 170M May 29 19:36 phased_possorted_bam.bam.sortbx.bam.tmp.0020.bam
-rw-r--r-- 1 delmore 131M May 29 19:39 phased_possorted_bam.bam.sortbx.bam.tmp.0026.bam
-rw-r--r-- 1 delmore 171M May 29 19:40 phased_possorted_bam.bam.sortbx.bam.tmp.0024.bam
-rw-r--r-- 1 delmore 181M May 29 19:40 phased_possorted_bam.bam.sortbx.bam.tmp.0025.bam
-rw-r--r-- 1 delmore 172M May 29 19:40 phased_possorted_bam.bam.sortbx.bam.tmp.0027.bam
-rw-r--r-- 1 delmore 155M May 29 19:48 phased_possorted_bam.bam.sortbx.bam.tmp.0028.bam
-rw-r--r-- 1 delmore 162M May 29 19:49 phased_possorted_bam.bam.sortbx.bam.tmp.0031.bam
-rw-r--r-- 1 delmore 158M May 29 19:50 phased_possorted_bam.bam.sortbx.bam.tmp.0029.bam
-rw-r--r-- 1 delmore 164M May 29 19:51 phased_possorted_bam.bam.sortbx.bam.tmp.0030.bam
-rw-r--r-- 1 delmore 156M May 29 19:53 phased_possorted_bam.bam.sortbx.bam.tmp.0032.bam
-rw-r--r-- 1 delmore 158M May 29 19:54 phased_possorted_bam.bam.sortbx.bam.tmp.0033.bam
-rw-r--r-- 1 delmore 159M May 29 19:55 phased_possorted_bam.bam.sortbx.bam.tmp.0034.bam
-rw-r--r-- 1 delmore 162M May 29 19:56 phased_possorted_bam.bam.sortbx.bam.tmp.0035.bam
-rw-r--r-- 1 delmore 162M May 29 20:17 phased_possorted_bam.bam.sortbx.bam.tmp.0038.bam
-rw-r--r-- 1 delmore 161M May 29 20:18 phased_possorted_bam.bam.sortbx.bam.tmp.0036.bam
-rw-r--r-- 1 delmore 162M May 29 20:18 phased_possorted_bam.bam.sortbx.bam.tmp.0039.bam
-rw-r--r-- 1 delmore 172M May 29 20:18 phased_possorted_bam.bam.sortbx.bam.tmp.0037.bam
-rw-r--r-- 1 delmore 171M May 29 20:24 phased_possorted_bam.bam.sortbx.bam.tmp.0043.bam
-rw-r--r-- 1 delmore 167M May 29 20:25 phased_possorted_bam.bam.sortbx.bam.tmp.0040.bam
-rw-r--r-- 1 delmore 172M May 29 20:25 phased_possorted_bam.bam.sortbx.bam.tmp.0041.bam
-rw-r--r-- 1 delmore 183M May 29 20:25 phased_possorted_bam.bam.sortbx.bam.tmp.0042.bam
-rw-r--r-- 1 delmore 156M May 29 20:34 phased_possorted_bam.bam.sortbx.bam.tmp.0046.bam
-rw-r--r-- 1 delmore 170M May 29 20:34 phased_possorted_bam.bam.sortbx.bam.tmp.0047.bam
-rw-r--r-- 1 delmore 163M May 29 20:34 phased_possorted_bam.bam.sortbx.bam.tmp.0044.bam
-rw-r--r-- 1 delmore 166M May 29 20:34 phased_possorted_bam.bam.sortbx.bam.tmp.0045.bam
-rw-r--r-- 1 delmore 163M May 29 20:49 phased_possorted_bam.bam.sortbx.bam.tmp.0051.bam
-rw-r--r-- 1 delmore 157M May 29 20:49 phased_possorted_bam.bam.sortbx.bam.tmp.0050.bam
-rw-r--r-- 1 delmore 159M May 29 20:50 phased_possorted_bam.bam.sortbx.bam.tmp.0048.bam
-rw-r--r-- 1 delmore 161M May 29 20:50 phased_possorted_bam.bam.sortbx.bam.tmp.0049.bam
-rw-r--r-- 1 delmore 158M May 29 21:00 phased_possorted_bam.bam.sortbx.bam.tmp.0053.bam
-rw-r--r-- 1 delmore 159M May 29 21:00 phased_possorted_bam.bam.sortbx.bam.tmp.0052.bam
-rw-r--r-- 1 delmore 167M May 29 21:01 phased_possorted_bam.bam.sortbx.bam.tmp.0055.bam
-rw-r--r-- 1 delmore 158M May 29 21:02 phased_possorted_bam.bam.sortbx.bam.tmp.0054.bam
-rw-r--r-- 1 delmore 179M May 29 21:18 phased_possorted_bam.bam.sortbx.bam.tmp.0056.bam
-rw-r--r-- 1 delmore 163M May 29 21:19 phased_possorted_bam.bam.sortbx.bam.tmp.0057.bam
-rw-r--r-- 1 delmore 164M May 29 21:20 phased_possorted_bam.bam.sortbx.bam.tmp.0058.bam
-rw-r--r-- 1 delmore 166M May 29 21:21 phased_possorted_bam.bam.sortbx.bam.tmp.0059.bam
-rw-r--r-- 1 delmore 161M May 29 21:35 phased_possorted_bam.bam.sortbx.bam.tmp.0062.bam
-rw-r--r-- 1 delmore 164M May 29 21:35 phased_possorted_bam.bam.sortbx.bam.tmp.0060.bam
-rw-r--r-- 1 delmore 162M May 29 21:36 phased_possorted_bam.bam.sortbx.bam.tmp.0063.bam
-rw-r--r-- 1 delmore 157M May 29 21:36 phased_possorted_bam.bam.sortbx.bam.tmp.0061.bam
-rw-r--r-- 1 delmore 164M May 29 21:50 phased_possorted_bam.bam.sortbx.bam.tmp.0067.bam
-rw-r--r-- 1 delmore 157M May 29 21:50 phased_possorted_bam.bam.sortbx.bam.tmp.0064.bam
-rw-r--r-- 1 delmore 165M May 29 21:50 phased_possorted_bam.bam.sortbx.bam.tmp.0066.bam
-rw-r--r-- 1 delmore 160M May 29 21:51 phased_possorted_bam.bam.sortbx.bam.tmp.0065.bam
-rw-r--r-- 1 delmore 163M May 29 22:02 phased_possorted_bam.bam.sortbx.bam.tmp.0069.bam
-rw-r--r-- 1 delmore 170M May 29 22:03 phased_possorted_bam.bam.sortbx.bam.tmp.0070.bam
-rw-r--r-- 1 delmore 162M May 29 22:03 phased_possorted_bam.bam.sortbx.bam.tmp.0071.bam
-rw-r--r-- 1 delmore 158M May 29 22:04 phased_possorted_bam.bam.sortbx.bam.tmp.0068.bam
-rw-r--r-- 1 delmore 156M May 29 22:09 phased_possorted_bam.bam.sortbx.bam.tmp.0072.bam
-rw-r--r-- 1 delmore 159M May 29 22:11 phased_possorted_bam.bam.sortbx.bam.tmp.0074.bam
-rw-r--r-- 1 delmore 161M May 29 22:12 phased_possorted_bam.bam.sortbx.bam.tmp.0073.bam
-rw-r--r-- 1 delmore 157M May 29 22:12 phased_possorted_bam.bam.sortbx.bam.tmp.0075.bam
-rw-r--r-- 1 delmore 169M May 29 22:27 phased_possorted_bam.bam.sortbx.bam.tmp.0078.bam
-rw-r--r-- 1 delmore 176M May 29 22:27 phased_possorted_bam.bam.sortbx.bam.tmp.0077.bam
-rw-r--r-- 1 delmore 171M May 29 22:27 phased_possorted_bam.bam.sortbx.bam.tmp.0079.bam
-rw-r--r-- 1 delmore 179M May 29 22:27 phased_possorted_bam.bam.sortbx.bam.tmp.0076.bam
-rw-r--r-- 1 delmore 162M May 29 22:37 phased_possorted_bam.bam.sortbx.bam.tmp.0082.bam
-rw-r--r-- 1 delmore 158M May 29 22:38 phased_possorted_bam.bam.sortbx.bam.tmp.0083.bam
-rw-r--r-- 1 delmore 190M May 29 22:39 phased_possorted_bam.bam.sortbx.bam.tmp.0081.bam
-rw-r--r-- 1 delmore 164M May 29 22:41 phased_possorted_bam.bam.sortbx.bam.tmp.0080.bam
-rw-r--r-- 1 delmore 158M May 29 22:48 phased_possorted_bam.bam.sortbx.bam.tmp.0087.bam
-rw-r--r-- 1 delmore 161M May 29 22:49 phased_possorted_bam.bam.sortbx.bam.tmp.0086.bam
-rw-r--r-- 1 delmore 155M May 29 22:49 phased_possorted_bam.bam.sortbx.bam.tmp.0084.bam
-rw-r--r-- 1 delmore 161M May 29 22:49 phased_possorted_bam.bam.sortbx.bam.tmp.0085.bam
-rw-r--r-- 1 delmore 176M May 29 23:06 phased_possorted_bam.bam.sortbx.bam.tmp.0091.bam
-rw-r--r-- 1 delmore 185M May 29 23:06 phased_possorted_bam.bam.sortbx.bam.tmp.0088.bam
-rw-r--r-- 1 delmore 188M May 29 23:06 phased_possorted_bam.bam.sortbx.bam.tmp.0089.bam
-rw-r--r-- 1 delmore 185M May 29 23:06 phased_possorted_bam.bam.sortbx.bam.tmp.0090.bam
-rw-r--r-- 1 delmore 184M May 29 23:28 phased_possorted_bam.bam.sortbx.bam.tmp.0094.bam
-rw-r--r-- 1 delmore 189M May 29 23:28 phased_possorted_bam.bam.sortbx.bam.tmp.0092.bam
-rw-r--r-- 1 delmore 159M May 29 23:28 phased_possorted_bam.bam.sortbx.bam.tmp.0095.bam
-rw-r--r-- 1 delmore 176M May 29 23:29 phased_possorted_bam.bam.sortbx.bam.tmp.0093.bam
-rw-r--r-- 1 delmore 169M May 29 23:52 phased_possorted_bam.bam.sortbx.bam.tmp.0099.bam
-rw-r--r-- 1 delmore 168M May 29 23:52 phased_possorted_bam.bam.sortbx.bam.tmp.0097.bam
-rw-r--r-- 1 delmore 158M May 29 23:52 phased_possorted_bam.bam.sortbx.bam.tmp.0096.bam
-rw-r--r-- 1 delmore 177M May 29 23:52 phased_possorted_bam.bam.sortbx.bam.tmp.0098.bam
-rw-r--r-- 1 delmore 167M May 29 23:58 phased_possorted_bam.bam.sortbx.bam.tmp.0102.bam
-rw-r--r-- 1 delmore 179M May 29 23:58 phased_possorted_bam.bam.sortbx.bam.tmp.0101.bam
-rw-r--r-- 1 delmore 161M May 29 23:58 phased_possorted_bam.bam.sortbx.bam.tmp.0100.bam
-rw-r--r-- 1 delmore 166M May 29 23:58 phased_possorted_bam.bam.sortbx.bam.tmp.0103.bam
-rw-r--r-- 1 delmore 155M May 30 00:11 phased_possorted_bam.bam.sortbx.bam.tmp.0105.bam
-rw-r--r-- 1 delmore 157M May 30 00:12 phased_possorted_bam.bam.sortbx.bam.tmp.0104.bam
-rw-r--r-- 1 delmore 170M May 30 00:13 phased_possorted_bam.bam.sortbx.bam.tmp.0106.bam
-rw-r--r-- 1 delmore 176M May 30 00:14 phased_possorted_bam.bam.sortbx.bam.tmp.0107.bam
-rw-r--r-- 1 delmore 167M May 30 00:24 phased_possorted_bam.bam.sortbx.bam.tmp.0111.bam
-rw-r--r-- 1 delmore 171M May 30 00:25 phased_possorted_bam.bam.sortbx.bam.tmp.0110.bam
-rw-r--r-- 1 delmore 184M May 30 00:26 phased_possorted_bam.bam.sortbx.bam.tmp.0108.bam
-rw-r--r-- 1 delmore 179M May 30 00:26 phased_possorted_bam.bam.sortbx.bam.tmp.0109.bam
-rw-r--r-- 1 delmore 158M May 30 00:34 phased_possorted_bam.bam.sortbx.bam.tmp.0113.bam
-rw-r--r-- 1 delmore 162M May 30 00:34 phased_possorted_bam.bam.sortbx.bam.tmp.0114.bam
-rw-r--r-- 1 delmore 172M May 30 00:34 phased_possorted_bam.bam.sortbx.bam.tmp.0115.bam
-rw-r--r-- 1 delmore 161M May 30 00:34 phased_possorted_bam.bam.sortbx.bam.tmp.0112.bam
-rw-r--r-- 1 delmore 161M May 30 00:40 phased_possorted_bam.bam.sortbx.bam.tmp.0118.bam
-rw-r--r-- 1 delmore 156M May 30 00:41 phased_possorted_bam.bam.sortbx.bam.tmp.0119.bam
-rw-r--r-- 1 delmore 165M May 30 00:42 phased_possorted_bam.bam.sortbx.bam.tmp.0117.bam
-rw-r--r-- 1 delmore 165M May 30 00:42 phased_possorted_bam.bam.sortbx.bam.tmp.0116.bam
-rw-r--r-- 1 delmore 162M May 30 00:56 phased_possorted_bam.bam.sortbx.bam.tmp.0123.bam
-rw-r--r-- 1 delmore 160M May 30 00:56 phased_possorted_bam.bam.sortbx.bam.tmp.0120.bam
-rw-r--r-- 1 delmore 181M May 30 00:56 phased_possorted_bam.bam.sortbx.bam.tmp.0122.bam
-rw-r--r-- 1 delmore 169M May 30 00:56 phased_possorted_bam.bam.sortbx.bam.tmp.0121.bam
-rw-r--r-- 1 delmore 163M May 30 01:09 phased_possorted_bam.bam.sortbx.bam.tmp.0124.bam
-rw-r--r-- 1 delmore 164M May 30 01:09 phased_possorted_bam.bam.sortbx.bam.tmp.0126.bam
-rw-r--r-- 1 delmore 180M May 30 01:09 phased_possorted_bam.bam.sortbx.bam.tmp.0125.bam
-rw-r--r-- 1 delmore 169M May 30 01:09 phased_possorted_bam.bam.sortbx.bam.tmp.0127.bam
-rw-r--r-- 1 delmore 168M May 30 01:22 phased_possorted_bam.bam.sortbx.bam.tmp.0130.bam
-rw-r--r-- 1 delmore 166M May 30 01:23 phased_possorted_bam.bam.sortbx.bam.tmp.0129.bam
-rw-r--r-- 1 delmore 173M May 30 01:23 phased_possorted_bam.bam.sortbx.bam.tmp.0131.bam
-rw-r--r-- 1 delmore 171M May 30 01:24 phased_possorted_bam.bam.sortbx.bam.tmp.0128.bam
-rw-r--r-- 1 delmore 20 Jun 15 19:43 phased_possorted_bam.bam.bcd21.gz
-rw-r--r-- 1 delmore 65 Jun 15 19:43 phased_possorted_bam.bam.low_mapq.bcd21.gz
-rw-r--r-- 1 delmore 248 Jun 15 19:43 phased_possorted_bam.bam.bcd22.tmp
-rw-r--r-- 1 delmore 285 Jun 15 19:43 phased_possorted_bam.bam.bcd22
-rw-r--r-- 1 delmore 0 Jun 15 19:43 phased_possorted_bam.bam.weird_reads.txt
-rw-r--r-- 1 delmore 0 Jun 15 19:43 phased_possorted_bam.bam.barcode_cov.bed
-rw-r--r-- 1 delmore 0 Jun 15 19:43 phased_possorted_bam.bam.high_cov.bed
-rw-r--r-- 1 delmore 0 Jun 15 19:43 phased_possorted_bam.bam.fragment_statistics

@fangli80
Copy link
Collaborator

From the log file, I can see that the sorting was failed due to existing temporary files.

samtools sort: failed to create temporary file "/home/delmore/10x/linked_sv/APC6732_noblack/phased_possorted_bam.bam.sortbx.bam.tmp.0000.bam": File exists

Could you please delete the intermediate files in the folder and run it again?
Thanks,
Li

@mw-qggp
Copy link

mw-qggp commented Sep 8, 2021

I got the same error when I used linkedSV. In my case, I assume it is because of the huge number of weird reads. I simulated the reads with LRSIM and used longranger wgs for mapping. Can anyone help me?
Thanks in advance,
Marius

drwxr-xr-x 25  user 256K Sep  8 08:49 ../
-rw-r--r--  1  user  139 Sep  8 08:39 phased_possorted_bam.bam.fragment_statistics
drwxr-xr-x  2 user 4.0K Sep  8 08:39 ./
-rw-r--r--  1  user    0 Sep  8 08:39 phased_possorted_bam.bam.high_cov.bed
-rw-r--r--  1  user    0 Sep  8 08:39 phased_possorted_bam.bam.barcode_cov.bed
-rw-r--r--  1  user   0 Sep  8 08:39 phased_possorted_bam.bam.weird_reads.txt
-rw-r--r--  1 user 285 Sep  8 08:39 phased_possorted_bam.bam.bcd22
-rw-r--r--  1  user  248 Sep  8 08:35 phased_possorted_bam.bam.bcd22.tmp
-rw-r--r--  1  user  21M Sep  8 08:33 phased_possorted_bam.bam.low_mapq.bcd21.gz
-rw-r--r--  1 user 1.3G Sep  8 08:31 phased_possorted_bam.bam.bcd21.gz
-rw-r--r--  1 user  588 Sep  8 08:31 phased_possorted_bam.bam.barcode_statistics


[bam_sort_core] merging from 12 files and 4 in-memory blocks...
started round 1 clustering
finished round 1 clustering
time used is 150.58 seconds
mean and sd of inner size: -1.79769e+308, 0
quantiles of inner size:
0.1% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
1% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
5% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
10% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
25% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
50% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
75% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
90% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
95% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
99% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
99.9% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000

quantiles of gap distance:
0.1% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
1% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
5% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
10% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
25% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
50% => -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
0090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
0090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
0090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
0090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000
490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000

inner size cut-off = -2147483648
gap distance cut-off = -2147483648
started round 2 clustering
finished round 2 clustering
time used is 199.94 seconds
total number of reads is: 41782350
total number of weird reads is: 41171745
[09/08/2021 08:24:34 (99.811 MB)] sorting bam file by barcode
am.sortbx.bam -
[09/08/2021 08:29:23 (99.811 MB)] extracting barcode info from bam file
rc/LinkedSV/scripts/../bin/pigz --fast --processes 3 - > /gpfs/project/mawei111/10X_simulations_PotatoTools/test_linkedSV/phased_possorted_bam.bam.bcd21.gz
[09/08/2021 08:31:06 (99.811 MB)] extracting low mapq bcd21
[09/08/2021 08:33:27 (117.019 MB)] clustering reads
d_bam.bam.weird_reads.txt 1 -1 20 4
[09/08/2021 08:39:19 (117.019 MB)] searching for extremely high coverage region
[09/08/2021 08:39:19 (117.023 MB)] calculating distribution parameters
[09/08/2021 08:39:19 (117.023 MB)] total number of reads in the genome is: 43999610
[09/08/2021 08:39:19 (117.023 MB)] calculating fragment parameters from file: /gpfs/project/mawei111/10X_simulations_PotatoTools/test_linkedSV/phased_possorted_bam.bam.bcd22
[09/08/2021 08:39:19 (117.023 MB)] N95_fragment_length is: -1
Traceback (most recent call last):
  File "/gpfs/project/projects/qggp/src/LinkedSV/linkedsv.py", line 313, in <module>
    main()
  File "/gpfs/project/projects/qggp/src/LinkedSV/linkedsv.py", line 47, in main
    detect_increased_fragment_ends(args, dbo_args, endpoint_args)
  File "/gpfs/project/projects/qggp/src/LinkedSV/linkedsv.py", line 208, in detect_increased_fragment_ends
    global_distribution.estimate_global_distribution(args, dbo_args, endpoint_args, endpoint_args.bcd22_file)
  File "/gpfs/project/projects/qggp/src/LinkedSV/scripts/global_distribution.py", line 133, in estimate_global_distribution
    get_fragment_parameters(args, dbo_args, endpoint_args, global_dist_fp, target_bcd22_file, is_fast_mode)
  File "/gpfs/project/projects/qggp/src/LinkedSV/scripts/global_distribution.py", line 213, in get_fragment_parameters
    args.fragment_length_lmda = fit_geometric_distribution(frm_length_list, readpair = False) 
  File "/gpfs/project/projects/qggp/src/LinkedSV/scripts/global_distribution.py", line 171, in fit_geometric_distribution
    k = np.percentile(length_list, cdf2)
  File "/software/python/3.6.5/ivybridge/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4291, in percentile
    interpolation=interpolation)
  File "/software/python/3.6.5/ivybridge/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4033, in _ureduce
    r = func(a, **kwargs)
  File "/software/python/3.6.5/ivybridge/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4405, in _percentile
    x1 = take(ap, indices_below, axis=axis) * weights_below
  File "/software/python/3.6.5/ivybridge/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 159, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/software/python/3.6.5/ivybridge/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 52, in _wrapfunc
    return getattr(obj, method)(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kdelmore @fangli80 @mw-qggp