integrate with OnnxSlim #12683

Kayzwer · 2024-05-14T13:42:05Z

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

This PR is a placeholder without actual code changes. 🚫

📊 Key Changes

There are no code changes in this Pull Request.

🎯 Purpose & Impact

Since there are no changes, this PR might serve a different purpose, such as documentation, discussion, or project management. 📚
Users and developers should not expect any updates to features, performance, or bug fixes from this PR. No impact on current usage or functionality! 🔄

codecov · 2024-05-14T13:45:13Z

Codecov Report

Attention: Patch coverage is 22.22222% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 70.57%. Comparing base (a2ecb24) to head (21034ab).

Files	Patch %	Lines
ultralytics/engine/exporter.py	22.22%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #12683      +/-   ##
==========================================
+ Coverage   70.15%   70.57%   +0.41%     
==========================================
  Files         122      122              
  Lines       15636    15645       +9     
==========================================
+ Hits        10970    11041      +71     
+ Misses       4666     4604      -62

Flag	Coverage Δ
Benchmarks	`35.49% <22.22%> (-0.05%)`	⬇️
GPU	`37.27% <0.00%> (-0.03%)`	⬇️
Tests	`66.72% <22.22%> (+0.45%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Ilyabasharov · 2024-05-16T11:43:22Z

@Kayzwer Hello, thx for your effort! What do you think, what is better to use - onnx-simplifier or onnx-slim?

Kayzwer · 2024-05-16T12:13:14Z

@Kayzwer Hello, thx for your effort! What do you think, what is better to use - onnx-simplifier or onnx-slim?

try both

Kayzwer · 2024-05-16T12:13:58Z

@Kayzwer Hello, thx for your effort! What do you think, what is better to use - onnx-simplifier or onnx-slim?

also thanks to @inisis for developing the tool

inisis · 2024-05-17T02:38:04Z

@Kayzwer Hello, thx for your effort! What do you think, what is better to use - onnx-simplifier or onnx-slim?

also thanks to @inisis for developing the tool

Thanks for your pr, and onnxslim can have a better performance.

glenn-jocher · 2024-05-17T09:25:11Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?

Are the number of layers, operations or file size different?

Kayzwer · 2024-05-17T11:18:46Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?

Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

inisis · 2024-05-17T14:44:51Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?
Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

Thanks a lot, I have tested myself.

initialencounter · 2024-05-18T03:18:05Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?
Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

Thanks a lot, I have tested myself.

we have tested all the onnx models generated by tests/test_exports.py, and each model is processd by onnxsim and onnxslim and log to a file, you can find it here.

here is what chatgpt generates:

Final Recommendation

Use onnxsim if you prefer a conservative approach that simplifies the model structure without changing the model size. It ensures the model retains its original constants and is less likely to introduce changes that might affect the model's behavior.

Use onnxslim if you aim for more aggressive optimization, reducing the number of operations more significantly and are willing to accept a slight increase in model size. This approach might yield better performance in terms of computational efficiency at runtime.

Kayzwer · 2024-05-18T03:20:35Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?
Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

Thanks a lot, I have tested myself.

we have tested all the onnx models generated by tests/test_exports.py, and each model is processd by onnxsim and onnxslim and log to a file, you can find it here.

here is what chatgpt generates:
Final Recommendation

Use onnxsim if you prefer a conservative approach that simplifies the model structure without changing the model size. It ensures the model retains its original constants and is less likely to introduce changes that might affect the model's behavior.

Use onnxslim if you aim for more aggressive optimization, reducing the number of operations more significantly and are willing to accept a slight increase in model size. This approach might yield better performance in terms of computational efficiency at runtime.

but it is possible to apply two at the same time right? (onnxsim first, onnxslim second)

inisis · 2024-05-18T06:11:45Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?
Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

Thanks a lot, I have tested myself.

we have tested all the onnx models generated by tests/test_exports.py, and each model is processd by onnxsim and onnxslim and log to a file, you can find it here.

here is what chatgpt generates:
Final Recommendation

Use onnxsim if you prefer a conservative approach that simplifies the model structure without changing the model size. It ensures the model retains its original constants and is less likely to introduce changes that might affect the model's behavior.

Use onnxslim if you aim for more aggressive optimization, reducing the number of operations more significantly and are willing to accept a slight increase in model size. This approach might yield better performance in terms of computational efficiency at runtime.

the slimmed model is slightly larger because it contains more detailed tensor value info for the intermediate layer.

initialencounter · 2024-05-18T06:22:30Z

@inisis @Kayzwer thanks guys! Do you have benchmarks or info on i.e. YOLOv8n models exported with both onnx-simplifier and onnx-slim?
Are the number of layers, operations or file size different?

@glenn-jocher I will do some experiments with yolov8n.

Thanks a lot, I have tested myself.

we have tested all the onnx models generated by tests/test_exports.py, and each model is processd by onnxsim and onnxslim and log to a file, you can find it here.
here is what chatgpt generates:
Final Recommendation

Use onnxsim if you prefer a conservative approach that simplifies the model structure without changing the model size. It ensures the model retains its original constants and is less likely to introduce changes that might affect the model's behavior.

Use onnxslim if you aim for more aggressive optimization, reducing the number of operations more significantly and are willing to accept a slight increase in model size. This approach might yield better performance in terms of computational efficiency at runtime.
but it is possible to apply two at the same time right? (onnxsim first, onnxslim second)

I have conducted an experiment, using yolov8n_False_1.onnx which are non dynamic shape, using sim and slim to process it, and convert them to tensorrt, and here are the result.

from the result we can see that using slimmed model, it's 1647.92 qps, and simmed it 1635.28, and eigne file size is 18345652 vs 20483532

glenn-jocher · 2024-05-18T13:22:28Z

@initialencounter hey, thanks for adding to the discussion! Regarding applying onnx-simplifier and onnx-slim sequentially, you can definitely try using both tools on a model to maximize optimization. Each tool removes different redundancies which potentially could complement each other -- applying onnx-simplifier first and then onnx-slim might yield an even more optimized model. However, it's important to thoroughly test the final model to ensure that these optimizations don't impede its accuracy or expected behavior! 🧪🚀

initialencounter · 2024-05-19T03:26:47Z

@initialencounter hey, thanks for adding to the discussion! Regarding applying onnx-simplifier and onnx-slim sequentially, you can definitely try using both tools on a model to maximize optimization. Each tool removes different redundancies which potentially could complement each other -- applying onnx-simplifier first and then onnx-slim might yield an even more optimized model. However, it's important to thoroughly test the final model to ensure that these optimizations don't impede its accuracy or expected behavior! 🧪🚀

@glenn-jocher , all my experiments are done seperately with onnxslim and onnxsim alone and I have tested the accuracy of the exported engine, here are the result, using the script from tests/test_cuda.py.

from the result we can see, that the slimmed engine produces the right result

Kayzwer · 2024-05-19T04:22:48Z

@initialencounter hey, thanks for adding to the discussion! Regarding applying onnx-simplifier and onnx-slim sequentially, you can definitely try using both tools on a model to maximize optimization. Each tool removes different redundancies which potentially could complement each other -- applying onnx-simplifier first and then onnx-slim might yield an even more optimized model. However, it's important to thoroughly test the final model to ensure that these optimizations don't impede its accuracy or expected behavior! 🧪🚀

@glenn-jocher , all my experiments are done seperately with onnxslim and onnxsim alone and I have tested the accuracy of the exported engine, here are the result, using the script from tests/test_cuda.py.

from the result we can see, that the slimmed engine produces the right result

so using only onnxslim can get optimal performance?

glenn-jocher · 2024-05-19T06:55:38Z

@Kayzwer thanks for sharing your findings! Yes, based on your experiments, it seems using only onnxslim has provided satisfactory performance and accuracy. If the slimmed model meets your requirements, you might not need additional optimization with onnx-simplifier. However, different models and applications can benefit variably, so a combined approach could be explored if further optimization is necessary. Keep up the great experimentation! 🚀

inisis · 2024-05-19T07:27:33Z

@glenn-jocher Hi, what else do we need to provide, in order to merge onnxslim into ultralytics

glenn-jocher · 2024-05-19T14:43:30Z

@inisis the best way to validate both speed and accuracy of the exported models is to export both and then validate on COCO (for both speed and mAP):

yolo export model=yolov8n.pt format=onnx
yolo val model=yolov8n.onnx data=coco.yaml

inisis · 2024-05-20T02:22:04Z

(py3.10) root@ubuntu20:~# yolo val model=slim_yolov8n.onnx data=coco.yaml
WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.
Ultralytics YOLOv8.2.18 🚀 Python-3.10.14 torch-2.2.1+cu121 CUDA:0 (NVIDIA GeForce RTX 4090, 24217MiB)
Loading slim_yolov8n.onnx for ONNX Runtime inference...
Forcing batch=1 square inference (1,3,640,640) for non-PyTorch models
val: Scanning /root/yolov5/datasets/coco/labels/val2017.cache... 4952 images, 48 backgrounds, 0 corrupt: 100%|██████████| 5000/5000 [00:00<?, ?it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 5000/5000 [01:08<00:00, 72.92it/s]
                   all       5000      36335      0.629      0.476      0.521       0.37
                person       5000      10777      0.751      0.678      0.745      0.515
               bicycle       5000        314      0.693      0.414      0.466      0.269
                   car       5000       1918      0.656      0.527      0.566      0.363
            motorcycle       5000        367       0.71      0.573      0.654      0.412
              airplane       5000        143      0.748      0.769      0.839      0.654
                   bus       5000        283       0.73      0.664      0.739      0.621
                 train       5000        190      0.795      0.774      0.833      0.648
                 truck       5000        414      0.519      0.384       0.45      0.301
                  boat       5000        424      0.561      0.297      0.373      0.209
         traffic light       5000        634       0.64      0.352      0.415      0.213
          fire hydrant       5000        101      0.859      0.693       0.78      0.627
             stop sign       5000         75      0.678      0.627      0.677      0.615
         parking meter       5000         60      0.687      0.511      0.578      0.449
                 bench       5000        411      0.554      0.275      0.297      0.198
                  bird       5000        427      0.662      0.365      0.426      0.281
                   cat       5000        202      0.767      0.832      0.846      0.647
                   dog       5000        218      0.685      0.693      0.727      0.588
                 horse       5000        272      0.688      0.658       0.69      0.521
                 sheep       5000        354      0.616      0.669      0.669      0.466
                   cow       5000        372      0.714      0.604      0.674       0.48
              elephant       5000        252      0.698      0.843      0.815      0.628
                  bear       5000         71      0.815      0.747      0.833      0.672
                 zebra       5000        266      0.805      0.808      0.879       0.66
               giraffe       5000        232      0.856      0.836      0.884      0.686
              backpack       5000        371      0.494      0.164        0.2      0.105
              umbrella       5000        407       0.61      0.521      0.538      0.359
               handbag       5000        540      0.475      0.122      0.161     0.0815
                   tie       5000        252      0.636      0.377      0.428      0.267
              suitcase       5000        299      0.558      0.425      0.488      0.334
               frisbee       5000        115      0.727      0.757      0.763      0.578
                  skis       5000        241      0.631       0.34      0.377      0.194
             snowboard       5000         69      0.536      0.348      0.381      0.267
           sports ball       5000        260      0.702      0.442      0.481      0.331
                  kite       5000        327      0.612      0.526      0.556      0.379
          baseball bat       5000        145      0.555      0.372      0.411      0.215
        baseball glove       5000        148      0.649      0.486      0.516      0.304
            skateboard       5000        179      0.659      0.592      0.645      0.456
             surfboard       5000        267      0.599      0.476        0.5      0.312
         tennis racket       5000        225      0.677      0.596      0.661      0.403
                bottle       5000       1013      0.603      0.382      0.454      0.298
            wine glass       5000        341      0.667      0.328      0.407      0.263
                   cup       5000        895      0.572      0.438      0.486      0.346
                  fork       5000        215      0.596      0.312      0.375      0.257
                 knife       5000        325      0.448       0.16      0.166     0.0965
                 spoon       5000        253      0.438      0.129      0.162     0.0973
                  bowl       5000        623      0.587      0.485      0.526      0.393
                banana       5000        370      0.555      0.319      0.374      0.228
                 apple       5000        236      0.427       0.23      0.221      0.151
              sandwich       5000        177      0.564      0.467      0.475      0.359
                orange       5000        285      0.471      0.421      0.361      0.274
              broccoli       5000        312      0.507      0.359      0.367       0.21
                carrot       5000        365      0.457      0.285      0.307      0.192
               hot dog       5000        125      0.717      0.406      0.489       0.36
                 pizza       5000        284      0.654      0.616      0.658      0.502
                 donut       5000        328       0.61      0.491      0.516      0.412
                  cake       5000        310       0.56      0.406       0.45        0.3
                 chair       5000       1771      0.577      0.344      0.404      0.259
                 couch       5000        261       0.61      0.567      0.588      0.429
          potted plant       5000        342      0.507      0.374      0.377      0.223
                   bed       5000        163      0.555      0.558        0.6      0.443
          dining table       5000        695      0.524       0.43      0.428      0.287
                toilet       5000        179       0.73      0.725       0.78      0.645
                    tv       5000        288      0.738      0.628      0.724      0.551
                laptop       5000        231       0.69      0.662      0.699      0.578
                 mouse       5000        106      0.662      0.647      0.704      0.521
                remote       5000        283      0.427      0.212      0.283      0.165
              keyboard       5000        153      0.592      0.569       0.65       0.49
            cell phone       5000        262      0.542       0.37      0.405      0.275
             microwave       5000         55      0.661      0.564      0.624      0.499
                  oven       5000        143      0.642      0.497       0.54       0.36
               toaster       5000          9      0.593      0.222      0.422      0.304
                  sink       5000        225      0.578      0.449      0.504      0.328
          refrigerator       5000        126      0.684      0.595      0.659      0.506
                  book       5000       1129      0.457      0.108      0.191     0.0945
                 clock       5000        267      0.727       0.61      0.672      0.459
                  vase       5000        274      0.574      0.474      0.471      0.331
              scissors       5000         36       0.74      0.333      0.342      0.277
            teddy bear       5000        190       0.64      0.574      0.605      0.414
            hair drier       5000         11          1          0    0.00606    0.00425
            toothbrush       5000         57      0.434      0.211      0.218      0.137
Speed: 0.3ms preprocess, 6.4ms inference, 0.0ms loss, 1.6ms postprocess per image
Saving runs/detect/val/predictions.json...

Evaluating pycocotools mAP using runs/detect/val/predictions.json and /root/yolov5/datasets/coco/annotations/instances_val2017.json...
loading annotations into memory...
Done (t=0.43s)
creating index...
index created!
Loading and preparing results...
DONE (t=5.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=55.12s).
Accumulating evaluation results...
DONE (t=13.29s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.373
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.525
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.405
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.187
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.410
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.533
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.320
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.536
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.592
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.362
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.657
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.772
Results saved to runs/detect/val
💡 Learn more at https://docs.ultralytics.com/modes/val

(py3.10) root@ubuntu20:~# yolo val model=sim_yolov8n.onnx data=coco.yaml
WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.
Ultralytics YOLOv8.2.18 🚀 Python-3.10.14 torch-2.2.1+cu121 CUDA:0 (NVIDIA GeForce RTX 4090, 24217MiB)
Loading sim_yolov8n.onnx for ONNX Runtime inference...
Forcing batch=1 square inference (1,3,640,640) for non-PyTorch models
val: Scanning /root/yolov5/datasets/coco/labels/val2017.cache... 4952 images, 48 backgrounds, 0 corrupt: 100%|██████████| 5000/5000 [00:00<?, ?it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 5000/5000 [01:11<00:00, 70.10it/s]
                   all       5000      36335      0.628      0.477      0.521       0.37
                person       5000      10777      0.749      0.679      0.745      0.515
               bicycle       5000        314      0.692      0.414      0.466      0.269
                   car       5000       1918      0.656      0.527      0.566      0.363
            motorcycle       5000        367       0.71      0.575      0.654      0.412
              airplane       5000        143      0.748      0.769      0.839      0.654
                   bus       5000        283      0.729      0.664      0.739      0.621
                 train       5000        190       0.79      0.774      0.833      0.648
                 truck       5000        414      0.518      0.384       0.45      0.301
                  boat       5000        424      0.561      0.298      0.373      0.209
         traffic light       5000        634      0.637      0.352      0.415      0.213
          fire hydrant       5000        101      0.859      0.693       0.78      0.627
             stop sign       5000         75      0.678      0.627      0.677      0.615
         parking meter       5000         60      0.687      0.512      0.578      0.449
                 bench       5000        411      0.551      0.275      0.297      0.197
                  bird       5000        427      0.661      0.365      0.426      0.281
                   cat       5000        202      0.766      0.832      0.846      0.647
                   dog       5000        218      0.684      0.693      0.727      0.588
                 horse       5000        272      0.687      0.658       0.69      0.521
                 sheep       5000        354      0.612      0.669      0.669      0.466
                   cow       5000        372      0.713      0.605      0.674       0.48
              elephant       5000        252      0.698      0.844      0.815      0.628
                  bear       5000         71      0.816      0.748      0.833      0.672
                 zebra       5000        266      0.806       0.81      0.879      0.661
               giraffe       5000        232      0.856      0.836      0.884      0.686
              backpack       5000        371      0.489      0.165        0.2      0.105
              umbrella       5000        407      0.609      0.521      0.538      0.359
               handbag       5000        540      0.473      0.123      0.161     0.0815
                   tie       5000        252      0.635      0.377      0.428      0.267
              suitcase       5000        299      0.558      0.425      0.488      0.334
               frisbee       5000        115      0.727      0.757      0.763      0.578
                  skis       5000        241       0.63       0.34      0.377      0.194
             snowboard       5000         69      0.534      0.348      0.381      0.267
           sports ball       5000        260      0.702      0.446      0.481      0.331
                  kite       5000        327      0.613      0.529      0.556      0.379
          baseball bat       5000        145      0.552      0.372      0.411      0.215
        baseball glove       5000        148      0.648      0.486      0.516      0.304
            skateboard       5000        179      0.659      0.592      0.645      0.456
             surfboard       5000        267      0.598      0.476        0.5      0.312
         tennis racket       5000        225      0.676      0.596      0.661      0.403
                bottle       5000       1013      0.603      0.384      0.454      0.298
            wine glass       5000        341      0.666      0.328      0.407      0.263
                   cup       5000        895      0.571      0.439      0.486      0.346
                  fork       5000        215      0.596      0.312      0.375      0.257
                 knife       5000        325      0.446       0.16      0.166     0.0965
                 spoon       5000        253      0.439       0.13      0.162     0.0973
                  bowl       5000        623      0.586      0.486      0.526      0.393
                banana       5000        370      0.552      0.319      0.374      0.228
                 apple       5000        236      0.428      0.231      0.221      0.151
              sandwich       5000        177      0.564      0.468      0.475      0.359
                orange       5000        285       0.47      0.421      0.361      0.274
              broccoli       5000        312      0.506      0.359      0.367       0.21
                carrot       5000        365      0.455      0.285      0.307      0.192
               hot dog       5000        125      0.718      0.407      0.489       0.36
                 pizza       5000        284      0.653      0.616      0.658      0.502
                 donut       5000        328      0.609      0.491      0.516      0.412
                  cake       5000        310      0.559      0.406       0.45        0.3
                 chair       5000       1771      0.576      0.344      0.404      0.259
                 couch       5000        261      0.607      0.567      0.588      0.429
          potted plant       5000        342      0.507      0.374      0.377      0.223
                   bed       5000        163      0.554      0.558        0.6      0.443
          dining table       5000        695      0.522       0.43      0.428      0.287
                toilet       5000        179       0.73      0.725       0.78      0.645
                    tv       5000        288      0.738      0.628      0.724      0.551
                laptop       5000        231      0.689      0.662      0.699      0.578
                 mouse       5000        106      0.663      0.648      0.704      0.521
                remote       5000        283      0.426      0.212      0.282      0.165
              keyboard       5000        153       0.59      0.569       0.65       0.49
            cell phone       5000        262      0.543      0.372      0.405      0.275
             microwave       5000         55      0.661      0.564      0.624      0.499
                  oven       5000        143       0.64      0.497       0.54       0.36
               toaster       5000          9      0.592      0.222      0.422      0.304
                  sink       5000        225      0.577      0.449      0.504      0.327
          refrigerator       5000        126      0.683      0.595      0.659      0.506
                  book       5000       1129      0.454      0.109      0.191     0.0945
                 clock       5000        267      0.726       0.61      0.672      0.459
                  vase       5000        274      0.574      0.476      0.471      0.331
              scissors       5000         36      0.739      0.333      0.342      0.277
            teddy bear       5000        190       0.64      0.574      0.605      0.414
            hair drier       5000         11          1          0    0.00606    0.00425
            toothbrush       5000         57      0.433      0.211      0.218      0.137
Speed: 0.3ms preprocess, 6.9ms inference, 0.0ms loss, 1.6ms postprocess per image
Saving runs/detect/val2/predictions.json...

Evaluating pycocotools mAP using runs/detect/val2/predictions.json and /root/yolov5/datasets/coco/annotations/instances_val2017.json...
loading annotations into memory...
Done (t=0.42s)
creating index...
index created!
Loading and preparing results...
DONE (t=5.11s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=55.43s).
Accumulating evaluation results...
DONE (t=13.47s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.373
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.525
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.405
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.187
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.410
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.533
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.320
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.536
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.592
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.362
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.657
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.772
Results saved to runs/detect/val2
💡 Learn more at https://docs.ultralytics.com/modes/val

@glenn-jocher Hi, here are the results, the average inference speed for slim model is 6.4ms while accuracy stay the same.

glenn-jocher · 2024-05-21T03:19:35Z

Hey @inisis, thanks for sharing the results! It looks like the slim model maintains accuracy while offering a slight improvement in inference speed. This could be beneficial for applications requiring faster processing without a compromise on performance. If there's anything specific you need for integrating onnxslim into Ultralytics, such as further testing or documentation updates, just let me know! 😊🚀

inisis · 2024-05-21T03:26:58Z

Hey @inisis, thanks for sharing the results! It looks like the slim model maintains accuracy while offering a slight improvement in inference speed. This could be beneficial for applications requiring faster processing without a compromise on performance. If there's anything specific you need for integrating onnxslim into Ultralytics, such as further testing or documentation updates, just let me know! 😊🚀

@glenn-jocher Thanks for your quick response, I'm wondering if there are performance tests for Ultralytics on CI, and we can make a pr which utilize onnxslim to do model optimization and see the speed and accuracy results. Onnxslim is a very useful tool and easy to maintain and cross-platform, users don't need to make much effort to get a speedup. It's very honored to be merged into Ultralytics. 😊🚀😊🚀😊🚀

glenn-jocher · 2024-05-21T09:47:13Z

Hey @inisis, absolutely, we do have CI performance tests set up for Ultralytics. You're welcome to submit a PR with onnxslim integration! We can then review the changes together, especially focusing on the speed and accuracy benchmarks. It sounds like a great addition to enhance model performance with minimal user effort. Looking forward to your contribution! 😊🚀

inisis · 2024-05-21T09:50:47Z

Hi, @Kayzwer there is a conflict here, can you help fix it, and I want to make onnxslim as default option, because it's faster.

Kayzwer · 2024-05-21T09:56:19Z

Hi, @Kayzwer there is a conflict here, can you help fix it, and I want to make onnxslim as default option, because it's faster.

just make slim to true in default.yaml file

inisis · 2024-05-21T11:21:05Z

Hi, @Kayzwer there is a conflict here, can you help fix it, and I want to make onnxslim as default option, because it's faster.

just make slim to true in default.yaml file

@Kayzwer Can you help fix the merge conflict or should I create another pull request, current workflow is a little bit complicated for me. _

Kayzwer · 2024-05-21T11:23:28Z

Hi, @Kayzwer there is a conflict here, can you help fix it, and I want to make onnxslim as default option, because it's faster.

just make slim to true in default.yaml file

@Kayzwer Can you help fix the merge conflict or should I create another pull request, current workflow is a little bit complicated for me. _

alright, I will fix it later

glenn-jocher · 2024-05-21T18:39:32Z

Hi @Kayzwer, thanks for offering to help! If you could resolve the merge conflict, that would be great. Setting slim=true in the default.yaml as the default sounds like a good plan to enhance performance. Let me know if you need any more details from my side! 😊🚀

Kayzwer mentioned this pull request May 14, 2024

OnnxSlim support #12681

Open

2 tasks

Kayzwer closed this May 21, 2024

Kayzwer force-pushed the main branch from 1f56813 to c8b6a8b Compare May 21, 2024 13:10

initialencounter mentioned this pull request May 30, 2024

ultralytics 8.2.27 replace onnxsim with onnxslim #12989

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

integrate with OnnxSlim #12683

integrate with OnnxSlim #12683

Kayzwer commented May 14, 2024 •

edited by github-actions bot

codecov bot commented May 14, 2024 •

edited

Ilyabasharov commented May 16, 2024

Kayzwer commented May 16, 2024

Kayzwer commented May 16, 2024

inisis commented May 17, 2024

glenn-jocher commented May 17, 2024 •

edited

Kayzwer commented May 17, 2024 •

edited

inisis commented May 17, 2024

initialencounter commented May 18, 2024

Kayzwer commented May 18, 2024

inisis commented May 18, 2024

initialencounter commented May 18, 2024

glenn-jocher commented May 18, 2024

initialencounter commented May 19, 2024

Kayzwer commented May 19, 2024

glenn-jocher commented May 19, 2024

inisis commented May 19, 2024

glenn-jocher commented May 19, 2024

inisis commented May 20, 2024

glenn-jocher commented May 21, 2024

inisis commented May 21, 2024

glenn-jocher commented May 21, 2024

inisis commented May 21, 2024

Kayzwer commented May 21, 2024

inisis commented May 21, 2024

Kayzwer commented May 21, 2024

glenn-jocher commented May 21, 2024

integrate with OnnxSlim #12683

integrate with OnnxSlim #12683

Conversation

Kayzwer commented May 14, 2024 • edited by github-actions bot

🛠️ PR Summary

🌟 Summary

📊 Key Changes

🎯 Purpose & Impact

codecov bot commented May 14, 2024 • edited

Codecov Report

Ilyabasharov commented May 16, 2024

Kayzwer commented May 16, 2024

Kayzwer commented May 16, 2024

inisis commented May 17, 2024

glenn-jocher commented May 17, 2024 • edited

Kayzwer commented May 17, 2024 • edited

inisis commented May 17, 2024

initialencounter commented May 18, 2024

Kayzwer commented May 18, 2024

inisis commented May 18, 2024

initialencounter commented May 18, 2024

glenn-jocher commented May 18, 2024

initialencounter commented May 19, 2024

Kayzwer commented May 19, 2024

glenn-jocher commented May 19, 2024

inisis commented May 19, 2024

glenn-jocher commented May 19, 2024

inisis commented May 20, 2024

glenn-jocher commented May 21, 2024

inisis commented May 21, 2024

glenn-jocher commented May 21, 2024

inisis commented May 21, 2024

Kayzwer commented May 21, 2024

inisis commented May 21, 2024

Kayzwer commented May 21, 2024

glenn-jocher commented May 21, 2024

Kayzwer commented May 14, 2024 •

edited by github-actions bot

codecov bot commented May 14, 2024 •

edited

glenn-jocher commented May 17, 2024 •

edited

Kayzwer commented May 17, 2024 •

edited