-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to export a YOLOv8-det model to onnx with fp16. #12721
Comments
👋 Hello @LorenzoSun-V, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered. If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it. If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results. Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users. InstallPip install the pip install ultralytics EnvironmentsYOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
StatusIf this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit. |
Hello! Thanks for reaching out. It appears that you encountered an issue where the exported ONNX model remains in FP32 despite specifying FP16. This can happen since some model components might not fully support FP16, or implementation may default back to FP32 for stability. It's essential to verify that all components of your model are compatible with FP16. For now, please ensure you're using the latest version of the export tool and YOLOv8, as updates might contain fixes or enhancements for such features. If all else seems correct, please consider opening an issue with more detailed environment information so we can help diagnose the problem more effectively! |
@LorenzoSun-V hello! Thank you for sharing the version details of your ONNX packages. From the versions you've listed, it seems your setup should be compatible with exporting ONNX models in FP16. However, if the export still doesn’t reflect FP16 precision, you might want to ensure that all the operations in your model are supported in FP16 on ONNX. Sometimes, specific operations can cause the model to revert to FP32. Additionally, please double-check the export command for any typos, especially around the If issues persist, this might be specific to how ONNX handles FP16 conversion internally for certain operations. Upgrading to the latest version of the ONNX library might also help if not already using it. Hope this helps! 😊 |
I use this example to infer image by FP16 onnx model successfully on CPU. But failed on CUDA: |
Hello! It's great to hear that you've successfully run the FP16 ONNX model on the CPU. Regarding the issue with CUDA, it seems like there might be a compatibility issue with FP16 and your CUDA setup. Could you please ensure that your GPU supports FP16 operations and that the latest drivers and CUDA versions are installed? Sometimes, updating these can resolve such issues. If the problem persists, consider using the FP32 model for CUDA inference, as it generally has broader support across different GPU architectures. Here's a quick example of how to force FP32 during the export if needed: yolo export model=/path/to/my/best.pt format=onnx imgsz=640 half=false device=0 Hope this helps! 😊 Let us know how it goes! |
Hi, the code in example doesn't include |
Hello! Thanks for pointing this out. You're absolutely right; including Here's a quick example of how you might modify the preprocessing function to include def preprocess(img_path):
img = Image.open(img_path)
img = letterbox(img, new_shape=(640, 640))[0]
img = img.transpose((2, 0, 1)) # HWC to CHW
img = np.ascontiguousarray(img)
return img This should help standardize the input and improve the model's performance consistency across different formats. Thanks for your suggestion! 😊 |
Thanks for your patient reply! Everything is running correctly now. |
That's great to hear! If you have any more questions or need further assistance in the future, feel free to reach out. Happy coding! 😊 |
Search before asking
YOLOv8 Component
No response
Bug
I used the command to export onnx with fp16:
But it seems to be fp32 when I open it by Netron:
In addition, the size of exported onnx model doesn't get smaller.
Do you have any ideas about this phenomenon?
Environment
No response
Minimal Reproducible Example
No response
Additional
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: