Not reproducible

[jnmaloof]

Version 0.23.4

I get a different number of reads and bases after filtering every time I run fastp

(base) exouser@julin-2:$ for i in $(seq 1 10); do echo "run $i";  fastp --in1 GH.lane67.fastq.gz --out1 GH.lane67.trimmed.fastq.gz --length_required 50 -q 20 -u 70 --phred64 --cut_right --cut_right_window_size 4 --cut_right_mean_quality 20 2>&1 >/dev/null | grep -E "(too short)|(total bases: 9)" ; done
run 1
total bases: 99875000
reads failed due to too short: 593
run 2
total bases: 99844193
reads failed due to too short: 705
run 3
total bases: 99868071
reads failed due to too short: 595
run 4
total bases: 99864238
reads failed due to too short: 608
run 5
total bases: 99862997
reads failed due to too short: 625
run 6
total bases: 99900970
reads failed due to too short: 451
run 7
total bases: 99894738
reads failed due to too short: 454
run 8
total bases: 99879032
reads failed due to too short: 538
run 9
total bases: 99844285
reads failed due to too short: 711
run 10
total bases: 99875189
reads failed due to too short: 575

[jnmaloof]

And even worse, if I change the number of threads to 1 I get very different results:

~/Assignments/assignment-09-jnmaloof/input/Brapa_fastq$ for i in $(seq 1 10); do echo "run $i";  fastp --in1 GH.lane67.fastq.gz --out1 GH.lane67.trimmed.fastq.gz --length_required 50 -q 20 -u 70 --phred64 --thread 1 --cut_right --cut_right_window_size 4 --cut_right_mean_quality 20 2>&1 >/dev/null | grep -E "(too short)|(total bases: 9)" ; done
run 1
total bases: 95838782
reads failed due to too short: 18579
run 2
total bases: 95252057
reads failed due to too short: 21209
run 3
total bases: 95333601
reads failed due to too short: 20843
run 4
total bases: 95233893
reads failed due to too short: 21348
run 5
total bases: 95322517
reads failed due to too short: 20845
run 6
total bases: 95282512
reads failed due to too short: 21062
run 7
total bases: 95392059
reads failed due to too short: 20542
run 8
total bases: 95425813
reads failed due to too short: 20426
run 9
total bases: 95422163
reads failed due to too short: 20373
run 10
total bases: 95362468
reads failed due to too short: 20601

[jnmaloof]

Examining thread dependency a bit more:

for t in {1..8}
    do
        for i in {1..2}
            do
                echo "threads $t rep $i"
                fastp --in1 GH.lane67.fastq.gz --out1 GH.lane67.trimmed.fastq.gz --length_required 50 -q 20 -u 70 --phred64 --thread $t  --cut_right --cut_right_window_size 4 --cut_right_mean_quality 20 2>&1 >/dev/null | grep -E "(too short)|(total bases: 9)"
            done
    done
threads 1 rep 1
total bases: 95469777
reads failed due to too short: 20233
threads 1 rep 2
total bases: 95242300
reads failed due to too short: 21336
threads 2 rep 1
total bases: 97828605
reads failed due to too short: 9742
threads 2 rep 2
total bases: 97857064
reads failed due to too short: 9565
threads 3 rep 1
total bases: 99872436
reads failed due to too short: 581
threads 3 rep 2
total bases: 99899052
reads failed due to too short: 451
threads 4 rep 1
total bases: 99939418
reads failed due to too short: 284
threads 4 rep 2
total bases: 99930726
reads failed due to too short: 318
threads 5 rep 1
total bases: 99926647
reads failed due to too short: 338
threads 5 rep 2
total bases: 99926602
reads failed due to too short: 345
threads 6 rep 1
total bases: 99923806
reads failed due to too short: 346
threads 6 rep 2
total bases: 99917299
reads failed due to too short: 384
threads 7 rep 1
total bases: 99894695
reads failed due to too short: 490
threads 7 rep 2
total bases: 99904110
reads failed due to too short: 441
threads 8 rep 1
total bases: 99888522
reads failed due to too short: 510
threads 8 rep 2
total bases: 99892127
reads failed due to too short: 514

[jnmaloof]

If I revert to version 0.20.1 then things are repreoducible (and give me a different result than any of those above: 42341 reads are removed for being too short). That is on par with what trimmomatic returns and is probably the correct result.