Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more benchmark_xl output #3589

Merged
merged 1 commit into from
May 30, 2024
Merged

Conversation

jonsneyers
Copy link
Member

Fixes #3585

Also adds more details when using the flag --print_distance_percentiles.

Also fixes the aggregate value for ssimulacra2: it was ignoring negative values and taking the geomean of just the positive ones; now it just takes the arithmetic mean of all ssimulacra2 values.

Before:

Encoding      kPixels    Bytes          BPP  E MP/s  D MP/s     Max norm  SSIMULACRA2   PSNR        pnorm       BPP*pnorm   QABPP   Bugs
----------------------------------------------------------------------------------------------------------------------------------------
50th/90th percentile distance: 0.74641287  0.89263189
jxl:d0.5        13562  4936672    2.9119624   2.323  34.301   0.76519037  91.52246513  46.23   0.34268783  0.997894059218   2.913      0
50th/90th percentile distance: 1.30287242  1.55675101
jxl:d1          13562  3045965    1.7967034   2.239  34.252   1.35021809  86.95225346  42.34   0.58427450  1.049767991430   2.440      0
50th/90th percentile distance: 2.29967427  2.89556241
jxl:d2          13562  1835797    1.0828695   2.292  32.952   2.41013471  78.73319052  38.83   0.97862361  1.059721707947   2.693      0
50th/90th percentile distance: 3.09834218  3.91219139
jxl:d3          13562  1334015    0.7868867   2.248  33.389   3.23678298  71.66090226  36.88   1.31249357  1.032783716883   2.611      0
50th/90th percentile distance: 15.69093132  17.99648857
jxl:d24         13562   270716    0.1596855   0.605  38.327  16.06720416  -2.15884859  29.27   5.74953159  0.918116671120   2.607      0
50th/90th percentile distance: 16.13576889  18.36326790
jxl:d25         13562   261793    0.1544221   0.606  35.376  16.68272962  -3.80875468  29.17   5.89834294  0.910834629742   2.593      0
Aggregate:      13562  1173373    0.6921298   1.463  34.723   3.59547238  81.85819581  36.57   1.43471342  0.993007961883   2.639      0

After:

46 images
Encoding      kPixels    Bytes          BPP  E MP/s  D MP/s     Max norm  SSIMULACRA2   PSNR        pnorm       BPP*pnorm   QABPP   Bugs
----------------------------------------------------------------------------------------------------------------------------------------
   _________________________________________| median:         0.74641287  91.96166229          0.33922365
  /                                         | p10 (worst):    0.89263189  89.77494049          0.38702947
 /                                          | p5 (worst):     0.93385208  89.31037903          0.39110589
jxl:d0.5        13562  4936672    2.9119624   2.315  31.096   0.76519037  91.52246533  46.23   0.34268782  0.997894047755   2.913      0
   _________________________________________| median:         1.30287242  88.15885162          0.57385707
  /                                         | p10 (worst):    1.55675101  82.56882477          0.70896733
 /                                          | p5 (worst):     1.61269927  82.47370911          0.72702777
jxl:d1          13562  3045965    1.7967034   2.205  34.431   1.35021809  86.95225222  42.34   0.58427450  1.049767990116   2.440      0
   _________________________________________| median:         2.29967427  80.39791870          0.95846683
  /                                         | p10 (worst):    2.89556241  70.31759644          1.26363790
 /                                          | p5 (worst):     2.96536636  68.60237885          1.32267296
jxl:d2          13562  1835797    1.0828695   2.306  34.101   2.41013471  78.73319006  38.83   0.97862363  1.059721729400   2.693      0
   _________________________________________| median:         3.09834218  73.34270477          1.27836907
  /                                         | p10 (worst):    3.91219139  60.53897095          1.67121387
 /                                          | p5 (worst):     4.07968855  55.49774170          1.81831229
jxl:d3          13562  1334015    0.7868867   2.316  32.388   3.23678298  71.66090188  36.88   1.31249357  1.032783719159   2.611      0
   _________________________________________| median:        15.69093132   5.88187265          6.02538586
  /                                         | p10 (worst):   17.99648857 -26.42949295          6.62558889
 /                                          | p5 (worst):    18.10667038 -35.92488098          6.94895744
jxl:d24         13562   270716    0.1596855   0.606  34.924  16.06720416  -2.15884861  29.27   5.74953159  0.918116671543   2.607      0
   _________________________________________| median:        16.13576889   4.86366034          6.11648321
  /                                         | p10 (worst):   18.36326790 -28.11663246          6.71656990
 /                                          | p5 (worst):    18.51855659 -36.31728745          7.22389078
jxl:d25         13562   261793    0.1544221   0.609  37.065  16.68272962  -3.80875474  29.17   5.89834299  0.910834637497   2.593      0
Aggregate:      13562  1173373    0.6921298   1.469  33.948   3.59547238  53.81686769  36.57   1.43471343  0.993007964975   2.639      0

tools/benchmark/benchmark_xl.cc Fixed Show fixed Hide fixed
tools/benchmark/benchmark_xl.cc Fixed Show fixed Hide fixed
@jonsneyers jonsneyers added this pull request to the merge queue May 30, 2024
Merged via the queue into libjxl:main with commit 784e43e May 30, 2024
101 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants