Inference speed is None after transferring results to CPU

[Rick-v-E]

Search before asking

• I have searched the YOLOv8 issues and found no similar bug report.

YOLOv8 Component

Predict

Bug

I use the speed attribute to check the inference time of the network. However, I noticed that the speeds are becoming None after transferring the detection results to the CPU.

Environment

Ultralytics YOLOv8.2.5 🚀 Python-3.10.13 torch-2.1.1+cu121 CUDA:0 (NVIDIA GeForce RTX 3090, 24257MiB)
Setup complete ✅ (32 CPUs, 62.7 GB RAM, 345.5/915.3 GB disk)

OS Linux-6.5.0-28-generic-x86_64-with-glibc2.35
Environment Linux
Python 3.10.13
Install pip
RAM 62.71 GB
CPU AMD Ryzen 9 5950X 16-Core Processor
CUDA 12.1

matplotlib ✅ 3.8.2>=3.3.0
opencv-python ✅ 4.8.1.78>=4.6.0
pillow ✅ 10.1.0>=7.1.2
pyyaml ✅ 6.0>=5.3.1
requests ✅ 2.31.0>=2.23.0
scipy ✅ 1.11.4>=1.4.1
torch ✅ 2.1.1>=1.8.0
torchvision ✅ 0.16.1>=0.9.0
tqdm ✅ 4.66.1>=4.64.0
psutil ✅ 5.9.1
py-cpuinfo ✅ 9.0.0
thop ✅ 0.1.1-2209072238>=0.1.1
pandas ✅ 1.4.2>=1.1.4
seaborn ✅ 0.13.0>=0.11.0

Minimal Reproducible Example

from ultralytics import YOLO

path_to_weights = ...
path_to_image = ...

model = YOLO(path_to_weights)

results = model.predict(path_to_image)[0]

print(results.speed)

results = results.cpu()

print(results.speed)

This results in:

{'preprocess': 21.38376235961914, 'inference': 49.85451698303223, 'postprocess': 200.82473754882812}
{'preprocess': None, 'inference': None, 'postprocess': None}

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @Rick-v-E, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hi there! It looks like the issue you're encountering with the speed attribute becoming None after transferring results to the CPU is related to how the cpu() method is handling the speed tracking. The .cpu() method typically moves data from GPU memory to CPU memory but does not carry over the speed metrics, which explains why they are set to None.

To retain the speed information after moving your results to CPU, you can simply copy the speed dictionary before calling .cpu(). Here's how you can modify your script:

from ultralytics import YOLO

path_to_weights = ...
path_to_image = ...

model = YOLO(path_to_weights)

results = model.predict(path_to_image)[0]

# Store speed data before moving to CPU
speed_data = results.speed.copy()

results = results.cpu()

# Restoring the speed data after the transfer
results.speed = speed_data

print(results.speed)

This modification ensures that the inference speed data is preserved and accessible after the results have been transferred to the CPU. Hope this helps! 😊

[Rick-v-E]

Hi, thanks for your answer! :) That' s indeed what I am doing now and works fine, however, I was a bit surprised that the speed attribute was missing after moving the GPU data to the CPU.

I initially understood that calling the .cpu() function transfers all tensors to the CPU, however when looking at the code, it creates a new copy of the Results object. Every attribute is copied except for the speed attribute. Does it make sense to copy that attribute as well?

[glenn-jocher]

Hello! I'm glad to hear that the solution worked for you! 😊 Indeed, the .cpu() function transfers tensor data to the CPU by creating a new Results object and currently, it does not automatically include the speed attribute. Copying the speed attribute as well could be beneficial for maintaining a complete record of the inference performance even after the data transfer. This could potentially be a useful enhancement for the library!

If you think this feature would be beneficial, you may consider opening a feature request on GitHub or if you're inclined to contribute, submitting a pull request to include this functionality. The community contributions help in making the tool more robust and user-friendly! Thank you for your feedback and suggestions! 🚀

[Harly-1506]

Hello everyone,

Check out my new GitHub repository for running YOLOv8 object detection and segmentation inference using OpenVINO and NumPy only. This implementation is faster than the Torch version, offering improved performance and efficiency. Visit the repository here: Faster Inference YOLOv8.

Your feedback and contributions are welcome!

Thanks

[glenn-jocher]

Hello!

Great work on creating a repository that enhances YOLOv8 inference with OpenVINO and NumPy! 🚀 It's exciting to see community-driven efforts that push for performance improvements. I'll definitely check it out and encourage others to do the same. Keep up the fantastic work, and I look forward to seeing how your project evolves with community feedback and contributions.

Thanks for sharing! 😊