grep-related CRITICAL error: Unhandled exception occurred
I am having a CRITICAL error, but unlike for other issues, this seems to involve the command call grep
. Interestingly, for the samples I am running busco
so far, the error only appears when the genome has been polished by two or more rounds of gcpp
(arrow algorithm). Wonder if that could relate to the error.
Anyway, the output returned is the following:
INFO: ***** Start a BUSCO v5.0.0 analysis, current time: 02/20/2021 02:38:06 *****
INFO: Configuring BUSCO with local environment
INFO: Mode is genome
INFO: Input file is /home/elcortegano/busco_home/CC2931.arrow2.fa
INFO: Downloading information on latest versions of BUSCO data...
INFO: Downloading file 'https://busco-data.ezlab.org/v5/data/lineages/chlorophyta_odb10.2020-08-05.tar.gz'
INFO: Decompressing file '/home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10.tar.gz'
INFO: Running BUSCO using lineage dataset chlorophyta_odb10 (eukaryota, 2020-08-05)
INFO: Running 1 job(s) on metaeuk, starting at 02/20/2021 02:38:18
INFO: [metaeuk] 1 of 1 task(s) completed
INFO: ***** Run HMMER on gene sequences *****
INFO: Running 1519 job(s) on hmmsearch, starting at 02/20/2021 02:48:44
INFO: [hmmsearch] 152 of 1519 task(s) completed
INFO: [hmmsearch] 304 of 1519 task(s) completed
INFO: [hmmsearch] 456 of 1519 task(s) completed
INFO: [hmmsearch] 608 of 1519 task(s) completed
INFO: [hmmsearch] 760 of 1519 task(s) completed
INFO: [hmmsearch] 912 of 1519 task(s) completed
INFO: [hmmsearch] 1064 of 1519 task(s) completed
INFO: [hmmsearch] 1216 of 1519 task(s) completed
INFO: [hmmsearch] 1368 of 1519 task(s) completed
INFO: [hmmsearch] 1519 of 1519 task(s) completed
INFO: Extracting missing and fragmented buscos from the file refseq_db.faa...
INFO: Running 1 job(s) on metaeuk, starting at 02/20/2021 02:50:35
INFO: [metaeuk] 1 of 1 task(s) completed
INFO: ***** Run HMMER on gene sequences *****
INFO: Running 57 job(s) on hmmsearch, starting at 02/20/2021 03:20:47
INFO: [hmmsearch] 6 of 57 task(s) completed
INFO: [hmmsearch] 12 of 57 task(s) completed
INFO: [hmmsearch] 18 of 57 task(s) completed
INFO: [hmmsearch] 23 of 57 task(s) completed
INFO: [hmmsearch] 29 of 57 task(s) completed
INFO: [hmmsearch] 35 of 57 task(s) completed
INFO: [hmmsearch] 40 of 57 task(s) completed
INFO: [hmmsearch] 46 of 57 task(s) completed
INFO: [hmmsearch] 52 of 57 task(s) completed
INFO: [hmmsearch] 57 of 57 task(s) completed
CRITICAL: Unhandled exception occurred:
Traceback (most recent call last):
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/run_BUSCO.py", line 366, in run_BUSCO
busco_run.run_analysis()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/BuscoRunner.py", line 72, in run_analysis
self.analysis.run_analysis()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 388, in run_analysis
self.run_hmmer(
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/BuscoLogger.py", line 65, in wrapped_func
self.retval = func(*args, **kwargs)
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/BuscoAnalysis.py", line 128, in run_hmmer
self.validate_output()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 439, in validate_output
exon_records = self.get_exon_records(hmmer_results)
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 507, in get_exon_records
matches = subprocess.check_output(
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['grep', 'contig_164_1|arrow|arrow|arrow|.*|1908811|1909964|', '/home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/initial_results/CC2931.arrow2.fa.headersMap.tsv']' returned non-zero exit status 1.
And the complete log is (skipping hmmsearch lines with increasing numbers):
cat busco_2421827449.log
INFO:busco.run_BUSCO ***** Start a BUSCO v5.0.0 analysis, current time: 02/20/2021 02:38:06 *****
DEBUG:busco.ConfigManager Getting config file
INFO:busco.ConfigManager Configuring BUSCO with local environment
INFO:busco.BuscoConfig Mode is genome
INFO:busco.BuscoConfig Input file is /home/elcortegano/busco_home/CC2931.arrow2.fa
INFO:busco.BuscoDownloadManager Downloading information on latest versions of BUSCO data...
DEBUG:busco.BuscoConfig State of BUSCO config before run:
DEBUG:busco.BuscoConfig {'_allow_no_value': False,
'_comment_prefixes': ('#', ';'),
'_converters': <configparser.ConverterMapping object at 0x7fe3c5e9eca0>,
'_defaults': {},
'_delimiters': ('=', ':'),
'_dict': <class 'dict'>,
'_empty_lines_in_values': True,
'_inline_comment_prefixes': (),
'_input_filepath': '/home/elcortegano/busco_home/CC2931.arrow2.fa',
'_interpolation': <configparser.BasicInterpolation object at 0x7fe406f25610>,
'_mode': 'genome',
'_optcre': re.compile('\n (?P<option>.*?) # very permissive!\n \\s*(?P<vi>=|:)\\s* # any number of space/tab,\n # followed by any of t, re.VERBOSE),
'_proxies': {'DEFAULT': <Section: DEFAULT>,
'augustus': <Section: augustus>,
'busco_run': <Section: busco_run>,
'etraining': <Section: etraining>,
'gff2gbSmallDNA.pl': <Section: gff2gbSmallDNA.pl>,
'hmmsearch': <Section: hmmsearch>,
'makeblastdb': <Section: makeblastdb>,
'metaeuk': <Section: metaeuk>,
'new_species.pl': <Section: new_species.pl>,
'optimize_augustus.pl': <Section: optimize_augustus.pl>,
'prodigal': <Section: prodigal>,
'sepp': <Section: sepp>,
'tblastn': <Section: tblastn>},
'_sections': {'augustus': {'command': '', 'path': ''},
'busco_run': {'auto-lineage': 'False',
'auto-lineage-euk': 'False',
'auto-lineage-prok': 'False',
'cpu': '1',
'datasets_version': 'odb10',
'download_base_url': 'https://busco-data.ezlab.org/v5/data/',
'download_path': '/home/elcortegano/busco_home/busco_downloads',
'evalue': '0.001',
'force': 'False',
'in': '/home/elcortegano/busco_home/CC2931.arrow2.fa',
'limit': '3',
'lineage_dataset': 'chlorophyta_odb10',
'long': 'False',
'main_out': '/home/elcortegano/busco_home/arrow2',
'mode': 'genome',
'offline': 'False',
'out': 'arrow2',
'out_path': '/home/elcortegano/busco_home',
'quiet': 'False',
'restart': 'False',
'update-data': 'False',
'use_augustus': 'False'},
'etraining': {'command': '', 'path': ''},
'gff2gbSmallDNA.pl': {'command': '', 'path': ''},
'hmmsearch': {'command': '', 'path': ''},
'makeblastdb': {'command': '', 'path': ''},
'metaeuk': {'command': '', 'path': ''},
'new_species.pl': {'command': '', 'path': ''},
'optimize_augustus.pl': {'command': '', 'path': ''},
'prodigal': {'command': '', 'path': ''},
'sepp': {'command': '', 'path': ''},
'tblastn': {'command': '', 'path': ''}},
'_strict': True,
'conf_file': 'local environment',
'default_section': 'DEFAULT',
'downloader': <busco.BuscoDownloadManager.BuscoDownloadManager object at 0x7fe3c5e9efd0>,
'main_out': '/home/elcortegano/busco_home/arrow2',
'params': {'augustus_parameters': None,
'augustus_species': None,
'auto-lineage': False,
'auto-lineage-euk': False,
'auto-lineage-prok': False,
'config_file': None,
'cpu': None,
'datasets_version': None,
'download_base_url': None,
'download_path': None,
'evalue': None,
'force': False,
'help': '==SUPPRESS==',
'in': 'CC2931.arrow2.fa',
'limit': None,
'lineage_dataset': 'chlorophyta_odb10',
'list_datasets': '==SUPPRESS==',
'long': False,
'metaeuk_parameters': None,
'metaeuk_rerun_parameters': None,
'mode': 'genome',
'offline': False,
'out': 'arrow2',
'out_path': None,
'quiet': False,
'restart': False,
'update-data': False,
'use_augustus': False,
'version': '==SUPPRESS=='}}
INFO:busco.BuscoDownloadManager Downloading file 'https://busco-data.ezlab.org/v5/data/lineages/chlorophyta_odb10.2020-08-05.tar.gz'
DEBUG:busco.BuscoDownloadManager md5 hash is 1e7eb8376e0d6b08451b8bdb33f58776
INFO:busco.BuscoDownloadManager Decompressing file '/home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10.tar.gz'
DEBUG:busco.analysis.BuscoAnalysis Check all required tools are accessible...
DEBUG:busco.analysis.BuscoAnalysis Checking dataset for HMM profiles
INFO:busco.analysis.BuscoAnalysis Running BUSCO using lineage dataset chlorophyta_odb10 (eukaryota, 2020-08-05)
DEBUG:busco.busco_tools.base Tool: metaeuk
DEBUG:busco.busco_tools.base Version: 4.a0f584d
INFO:busco.busco_tools.Toolset Running 1 job(s) on metaeuk, starting at 02/20/2021 02:38:18
DEBUG:busco.busco_tools.Toolset cmd call: metaeuk easy-predict --threads 1 /home/elcortegano/busco_home/CC2931.arrow2.fa /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/refseq_db.faa /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/initial_results/CC2931.arrow2.fa /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/tmp --max-intron 90000 --max-seq-len 120000 --min-exon-aa 15 --max-overlap 15 --min-intron 5 --overlap 1
INFO:busco.busco_tools.Toolset [metaeuk] 1 of 1 task(s) completed
INFO:busco.analysis.BuscoAnalysis ***** Run HMMER on gene sequences *****
DEBUG:busco.busco_tools.base Tool: hmmsearch
DEBUG:busco.busco_tools.base Version: 3.1
INFO:busco.busco_tools.Toolset Running 1519 job(s) on hmmsearch, starting at 02/20/2021 02:48:44
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/initial_run_results/10012at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/10012at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/initial_results/CC2931.arrow2.fa.modified.fas
...
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/initial_run_results/99at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/99at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/initial_results/CC2931.arrow2.fa.modified.fas
INFO:busco.busco_tools.Toolset [hmmsearch] 1519 of 1519 task(s) completed
DEBUG:busco.busco_tools.hmmer ***** Results: *****
C:96.3%[S:95.9%,D:0.4%],F:1.9%,M:1.8%,n:1519
1462 Complete BUSCOs (C)
1456 Complete and single-copy BUSCOs (S)
6 Complete and duplicated BUSCOs (D)
29 Fragmented BUSCOs (F)
28 Missing BUSCOs (M)
1519 Total BUSCO groups searched
INFO:busco.busco_tools.metaeuk Extracting missing and fragmented buscos from the file refseq_db.faa...
DEBUG:busco.busco_tools.base Tool: metaeuk
DEBUG:busco.busco_tools.base Version: 4.a0f584d
INFO:busco.busco_tools.Toolset Running 1 job(s) on metaeuk, starting at 02/20/2021 02:50:35
DEBUG:busco.busco_tools.Toolset cmd call: metaeuk easy-predict --threads 1 /home/elcortegano/busco_home/CC2931.arrow2.fa /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/refseq_db_rerun.faa /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/tmp --max-intron 90000 --max-seq-len 120000 --min-exon-aa 5 --max-overlap 5 --min-intron 1 --overlap 1 -s 6
INFO:busco.busco_tools.Toolset [metaeuk] 1 of 1 task(s) completed
INFO:busco.analysis.BuscoAnalysis ***** Run HMMER on gene sequences *****
DEBUG:busco.busco_tools.base Tool: hmmsearch
DEBUG:busco.busco_tools.base Version: 3.1
INFO:busco.busco_tools.Toolset Running 57 job(s) on hmmsearch, starting at 02/20/2021 03:20:47
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/3069at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/3069at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/8939at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/8939at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/2403at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/2403at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/10500at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/10500at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/5531at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/5531at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/8781at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/8781at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
INFO:busco.busco_tools.Toolset [hmmsearch] 6 of 57 task(s) completed
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/5735at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/5735at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
...
DEBUG:busco.busco_tools.Toolset cmd call: hmmsearch --domtblout /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/hmmer_output/rerun_results/9991at3041.out --cpu 1 /home/elcortegano/busco_home/busco_downloads/lineages/chlorophyta_odb10/hmms/9991at3041.hmm /home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/rerun_results/CC2931.arrow2.fa.modified.fas
INFO:busco.busco_tools.Toolset [hmmsearch] 57 of 57 task(s) completed
CRITICAL:busco.run_BUSCO Unhandled exception occurred:
Traceback (most recent call last):
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/run_BUSCO.py", line 366, in run_BUSCO
busco_run.run_analysis()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/BuscoRunner.py", line 72, in run_analysis
self.analysis.run_analysis()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 388, in run_analysis
self.run_hmmer(
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/BuscoLogger.py", line 65, in wrapped_func
self.retval = func(*args, **kwargs)
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/BuscoAnalysis.py", line 128, in run_hmmer
self.validate_output()
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 439, in validate_output
exon_records = self.get_exon_records(hmmer_results)
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/site-packages/busco/analysis/GenomeAnalysis.py", line 507, in get_exon_records
matches = subprocess.check_output(
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/home/elcortegano/anaconda3/envs/busco/lib/python3.9/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['grep', 'contig_164_1|arrow|arrow|arrow|.*|1908811|1909964|', '/home/elcortegano/busco_home/arrow2/run_chlorophyta_odb10/metaeuk_output/initial_results/CC2931.arrow2.fa.headersMap.tsv']' returned non-zero exit status 1.