Kalamari database fails createtaxdb

[Changwanseo]

Expected Behavior

mmseqs databases Kalamari ./Kalamari tmp
expected to well download database

Current Behavior

double free or corruption (out)
Aborted (core dumped)
Error: createbintaxonomy failed
Error: createtaxdb died

Steps to Reproduce (for bugs)

Status of the directory:
ls returns tmp (no other things in the directory)

Run mmseqs databases Kalamari ./Kalamari tmp to reproduce

MMseqs Output (for bugs)

Please make sure to also post the complete output of MMseqs. You can use gist.github.com for large output.

(mmseqs) XXXX@XXXX:XXXX/Kalamari$ mmseqs databases Kalamari ./Kalamari tmp
databases Kalamari ./Kalamari tmp

MMseqs Version:                 16.747c6
Tsv                             false
Force restart with latest tmp   false
Remove temporary files          false
Compressed                      0
Threads                         64
Verbosity                       3


01/16 16:06:44 [NOTICE] Downloading 1 item(s)

01/16 16:06:44 [NOTICE] Download complete: tmp/1057546075966508143/kalamari.tsv.aria2

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
5c50b7|OK  |   5.1MiB/s|tmp/1057546075966508143/kalamari.tsv.aria2

Status Legend:
(OK):download completed.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  633M    0  633M    0     0  1625k      0 --:--:--  0:06:38 --:--:-- 3169k
createdb tmp/1057546075966508143/kalamari.fasta ./Kalamari --compressed 0 -v 3

Converting sequences
[203] 1s 538ms
Time for merging to Kalamari_h: 0h 0m 0s 1ms
Time for merging to Kalamari: 0h 0m 2s 123ms
Database type: Nucleotide
Time for processing: 0h 0m 5s 555ms
prefixid ./Kalamari_h tmp/1057546075966508143/header_pref.tsv --tsv --threads 64 -v 3

[=================================================================] 100.00% 225 0s 21ms
Time for merging to header_pref.tsv: 0h 0m 0s 8ms
Time for processing: 0h 0m 0s 194ms
Create directory tmp/1057546075966508143/taxonomy
createtaxdb ./Kalamari tmp/1057546075966508143/taxonomy --threads 64 -v 3

Download taxdump.tar.gz

01/16 16:13:44 [NOTICE] Downloading 1 item(s)
[#a04c79 64MiB/64MiB(98%) CN:1 DL:4.0MiB]
01/16 16:13:59 [NOTICE] Download complete: tmp/1057546075966508143/taxonomy/taxdump.tar.gz.aria2

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
a04c79|OK  |   4.4MiB/s|tmp/1057546075966508143/taxonomy/taxdump.tar.gz.aria2

Status Legend:
(OK):download completed.
Loading nodes file ... Done, got 2631776 nodes
Loading merged file ... Done, added 86929 merged nodes.
Loading names file ... Done
double free or corruption (out)
Aborted (core dumped)
Error: createbintaxonomy failed
Error: createtaxdb died

Context

Providing context helps us come up with a solution and improve our documentation for the future.

Your Environment

Include as many relevant details about the environment you experienced the bug in.

• Git commit used (The string after "MMseqs Version:" when you execute MMseqs without any parameters): 16.747c6
• Which MMseqs version was used (Statically-compiled, self-compiled, Homebrew, etc.): conda
• For self-compiled and Homebrew: Compiler and Cmake versions used and their invocation: X
• Server specifications (especially CPU support for AVX2/SSE and amount of system memory): 5975WX
• Operating system and version: Ubuntu 20.04

[jasmezz]

We at nf-core experience the same error since yesterday. It was working the days before, so probably something upstream changed (e.g. taxonomy download from NCBI)?

[milot-mirdita]

I can't reproduce the issue anymore. Maybe NCBI fixed the taxdump already?

[Changwanseo]

Still happens on me..

[milot-mirdita]

Can you upload the names.dmp nodes.dmp and merged.dmp (or the taxdump.tar.gz) somewhere in the tmp folder? I cant make it crash :/

[milot-mirdita]

I also don't get the same numbers, i think it was already updated and you might still have the files cached in tmp

Loading nodes file ... Done, got 2631723 nodes
Loading merged file ... Done, added 86940 merged nodes.

[Changwanseo]

Oh removing tmp and retrying works now. Thanks a lot!

[milot-mirdita]

I still would have liked to see what the ncbi broke :D

[milot-mirdita]

Thanks a lot for sharing the files. I reproduced the root cause and fixed it. This crash shouldn't happen again in future releases