How to Train model with different dataset contain different classes

[Vinaygoudasp7]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hi
i am new to Ai/Ml i am working on yolov8, i have to train this model from scratch so i have selected 3 dataset of different classes. my doubt if i train model from scratch by using yolov8n.yaml by using one of dataset then i train model which i got best.pt (from previous train) with next dataset can i do like this or if any alternate ways to solve this problem

Additional

No response

[github-actions]

👋 Hello @Vinaygoudasp7, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[Vinaygoudasp7]

i have searched but i didnt get similar question so can you provide ans to clarify my doubt

[glenn-jocher]

Hello! Thanks for reaching out with your question.

Yes, you can sequentially train your model on different datasets, but this may not be the most effective approach if the datasets involve distinctly different classes. Each subsequent training might overwrite or forget the previously learned classes (a phenomenon known as catastrophic forgetting).

A better approach would be to merge all datasets into a single one and train your model from scratch on this combined set. This helps the model learn features from all classes at once, improving its overall generalization.

Here’s a simple way to combine datasets:

1. Ensure that all datasets use the same annotation format.
2. Merge the image and annotation files into common directories.
3. Update your dataset YAML file to point to these directories.

Example YAML setup might look like this:

# combined_dataset.yaml
train: ./data/combined_train/
val: ./data/combined_val/

nc: 10  # Total number of classes
names: ['class1', 'class2', ..., 'class10']

Then, you can start training:

yolo detect train data=combined_dataset.yaml model=yolov8n.yaml

I hope this helps! Let me know if you have any more questions.

[Vinaygoudasp7]

thank you for given solution and think it should be effective for few number of dataset but is there any different approach to train model with different dataset and is it work if i transfer weight of previous model

[glenn-jocher]

Hello! Glad the initial solution was helpful. 😊

For training on multiple datasets with different classes, another effective method is using incremental learning or transfer learning where you transfer weights from a previously trained model.

Here’s a concise approach:

1. Start by training a base model on one dataset.
2. For each subsequent dataset, use the weights from the last trained model as a starting point. You'll update the YAML to reflect the new dataset and possibly increase the nc (number of classes) if the new dataset includes additional classes.

yolo detect train data=new_dataset.yaml model=previous_best.pt

Just be cautious with the class labels to ensure consistency across datasets. Each training phase should ideally include samples (even if few) from the previous classes to prevent forgetting.

This method can help the model learn new classes while retaining knowledge about the old ones. Keep in touch if you need more clarifications!

[y824165978]

Maybe this will help you! Extending YOLOv8 COCO Model With New Classes Without Affecting Old Weights

[Vinaygoudasp7]

@glenn-jocher thank you for solution if possible you can mention how can i add class in data.yaml file and it helpfull

@y824165978 thank you also for sloution

[glenn-jocher]

Hello! I'm glad you found the solutions helpful! 😊 To add a class in your data.yaml file, you just need to update the names field to include your new class. For example:

nc: 4  # number of classes
names: ['class1', 'class2', 'class3', 'new_class']

Make sure that the annotations in your dataset correspond to these class labels by index. Let me know if you need further assistance!

[Vinaygoudasp7]

@glenn-jocher thanks a lot for giving detailed solution for my problem , its help to me

[Vinaygoudasp7]

@glenn-jocher c

Hello! Glad the initial solution was helpful. 😊

For training on multiple datasets with different classes, another effective method is using incremental learning or transfer learning where you transfer weights from a previously trained model.

Here’s a concise approach:

1. Start by training a base model on one dataset.
2. For each subsequent dataset, use the weights from the last trained model as a starting point. You'll update the YAML to reflect the new dataset and possibly increase the nc (number of classes) if the new dataset includes additional classes.

yolo detect train data=new_dataset.yaml model=previous_best.pt

Just be cautious with the class labels to ensure consistency across datasets. Each training phase should ideally include samples (even if few) from the previous classes to prevent forgetting.

This method can help the model learn new classes while retaining knowledge about the old ones. Keep in touch if you need more clarifications!

you told to update data.yaml file, data.yaml of new dataset

[glenn-jocher]

Hello! Yes, you should update the data.yaml file for each new dataset you train on. This file should reflect the specific classes and paths relevant to the current dataset. Here's a quick example of how you might adjust it:

train: path/to/new/train/images
val: path/to/new/val/images

nc: number_of_classes_including_new_ones
names: ['class1', 'class2', ..., 'new_class']

Just replace the paths and class details as needed for your new dataset. This setup ensures that your model learns from the new data while leveraging the knowledge gained from previous training. 😊

[Vinaygoudasp7]

@glenn-jocher ok thank you for previous answer it works and can i train yolo-world model with custom dataset and how

[Vinaygoudasp7]

Load a pretrained YOLOv8s-worldv2 model and train it on the COCO8 example dataset for 100 epochs

!yolo train model=yolov8s-worldv2.pt data=D:/AIML/YOLO_Object_detection/Pen_book_detection-1/data.yaml epochs=30 imgsz=640

data.yaml
names:

• Book
• Pen
• person
• bicycle
• car
• motorcycle
• airplane
• bus
• train
• truck
• boat
• traffic light
• fire hydrant
• stop sign
• parking meter
• bench
• bird
• cat
• dog
• horse
• sheep
• cow
• elephant
• bear
• zebra
• giraffe
• backpack
• umbrella
• handbag
• tie
• suitcase
• frisbee
• skis
• snowboard
• sports ball
• kite
• baseball bat
• baseball glove
• skateboard
• surfboard
• tennis racket
• bottle
• wine glass
• cup
• fork
• knife
• spoon
• bowl
• banana
• apple
• sandwich
• orange
• broccoli
• carrot
• hot dog
• pizza
• donut
• cake
• chair
• couch
• potted plant
• bed
• dining table
• toilet
• tv
• laptop
• mouse
• remote
• keyboard
• cell phone
• microwave
• oven
• toaster
• sink
• refrigerator
• book
• clock
• vase
• scissors
• teddy bear
• hair drier
• toothbrush

nc: 82

roboflow:

license: CC BY 4.0

project: pen_book_detection-bhnvg

url: https://universe.roboflow.com/vinay-yeadc/pen_book_detection-bhnvg/dataset/1

version: 1

workspace: vinay-yeadc

test: ./test/images
train: ./train/images
val: ./valid/images

is it correct for traing with new dataset by including old class name and it not working can you tell me this
import torch
from ultralytics import YOLOWorld
from imutils.video import VideoStream
import cv2

video_stream=VideoStream(0).start()

video_stream=cv2.VideoCapture('./images_videos/7648337-hd_1920_1080_30fps.mp4')

device='cuda' if torch.cuda.is_available() else 'cpu'
print(device)
model_path='D:/AIML/YOLO_Object_detection/runs/detect/train33/weights/best.pt'
model=YOLOWorld(model='yolov8s-worldv2.pt').to(device=device)

model.set_classes(['cup','bottel'])

while True:
frame= video_stream.read()
# ret,frame=video_stream.read()
original_width = frame.shape[1]
original_height = frame.shape[0]
resized_frame=cv2.resize(frame,(640,640))
cv2.namedWindow('object detection by YOLO-world',cv2.WINDOW_NORMAL)
results=model(source=resized_frame)
print(results)
for result in results:
boxes=result.boxes.cpu().numpy()
bboxs=boxes.xyxy
confs=boxes.conf
clses=boxes.cls
classes_names=result.names
print(classes_names)
print("boxes",bboxs)
print("conf",confs)
print("class_value",clses)

    for bbox,conf,cls in zip(bboxs,confs,clses):
        if conf>0.4:
            x1,y1,x2,y2=bbox
            scaleW=original_width/640
            scaleH=original_height/640
            x1=int(x1*scaleW)
            y1=int(y1*scaleH)
            x2=int(x2*scaleW)
            y2=int(y2*scaleH)
            print(x1,y1,x2,y2)
            # # Ensure the bounding box stays within the image boundaries
            # x1 = max(0, min(x1, original_width))
            # y1 = max(0, min(y1, original_height))
            # x2 = max(0, min(x2, original_width))
            # y2 = max(0, min(y2, original_height))
            text=f'{classes_names[cls]} {round(conf,2)*100}'
            cv2.putText(frame,text,(x1,y1-10),1,1.5,(0,255,25),2)
            cv2.rectangle(frame,(x1,y1),(x2,y2),(0,255,25),2)
    
    cv2.imshow('object detection by YOLO-world',frame)
if cv2.waitKey(1) & 0xFF==ord('q'):
    break

cv2.destroyAllWindows()
video_stream.stop()

[glenn-jocher]

Hello! It looks like you're on the right track with your training setup and the use of the YOLO-World model. However, if you're experiencing issues, here are a few things to check:

1. Data Paths: Ensure that the paths specified in your data.yaml for train, val, and test are correct and accessible.
2. Class Names and Count: Verify that the nc (number of classes) in your data.yaml matches the length of the names list. It looks correct in your provided YAML.
3. Model Loading: When loading the model for inference, make sure the path model_path points to the correct .pt file. It seems you're loading a different model for inference (yolov8s-worldv2.pt) than the one you might have trained (best.pt from a custom training run). Ensure consistency to reflect the trained weights.

For the video stream processing code, it seems mostly correct. If the model isn't performing as expected, it might be due to how the model was trained or possibly an issue with how the frames are being processed or resized.

If you continue to face issues, please provide any error messages or specific behaviors that are not working as expected. This will help in diagnosing the problem more effectively. 😊

[Vinaygoudasp7]

i am not geting any error i am training by using cmd !yolo train model=yolov8s-worldv2.pt data=D:/AIML/YOLO_Object_detection/Pen_book_detection-1/data.yaml epochs=30 imgsz=640 it gives best.pt file then i want to use to predict object but it not working as accepted

[glenn-jocher]

@Vinaygoudasp7 hello! It sounds like you've successfully trained your model but are having issues with the predictions. To help you better, could you please clarify what you mean by "not working as expected"? Are you seeing incorrect detections, or is there another issue during prediction?

For predictions, ensure you're loading the best.pt model correctly and using the correct image sizes and classes. Here's a quick example on how to load your trained model and perform a prediction:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/best.pt')

# Run prediction
results = model.predict('path/to/your/image.jpg')
results.show()  # To display the image with predictions

Make sure the path to best.pt and the image is correct. If there are still issues, providing more details will help diagnose the problem better! 😊

[Vinaygoudasp7]

model.set_classes( ['Book' 'Pen' 'person' 'bicycle' 'car' 'motorcycle' 'airplane' 'bus' 'train' 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet',
'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush'])

frame= video_stream.read()
# ret,frame=video_stream.read()
original_width = frame.shape[1]
original_height = frame.shape[0]
resized_frame=cv2.resize(frame,(640,640))
cv2.namedWindow('object detection by YOLO-world',cv2.WINDOW_NORMAL)
results=model.predict(source=resized_frame,device=device,stream=True)
for result in results:
print(result)
boxes=result.boxes.cpu().numpy()
bboxs=boxes.xyxy
confs=boxes.conf
clses=boxes.cls
classes_names=result.names
print(classes_names)
print("boxes",bboxs)
print("conf",confs)
print("class_value",clses)

    for bbox,conf,cls in zip(bboxs,confs,clses):
        if conf>0.4:
            x1,y1,x2,y2=bbox
            scaleW=original_width/640
            scaleH=original_height/640
            x1=int(x1*scaleW)
            y1=int(y1*scaleH)
            x2=int(x2*scaleW)
            y2=int(y2*scaleH)
            print(x1,y1,x2,y2)
            # # Ensure the bounding box stays within the image boundaries
            # x1 = max(0, min(x1, original_width))
            # y1 = max(0, min(y1, original_height))
            # x2 = max(0, min(x2, original_width))
            # y2 = max(0, min(y2, original_height))
            text=f'{classes_names[cls]} {round(conf,2)*100}'
            cv2.putText(frame,text,(x1,y1-10),1,1.5,(0,255,25),2)
            cv2.rectangle(frame,(x1,y1),(x2,y2),(0,255,25),2)

{0: 'Book', 1: 'Pen', 2: 'person', 3: 'bicycle', 4: 'car', 5: 'motorcycle', 6: 'airplane', 7: 'bus', 8: 'train', 9: 'truck', 10: 'boat', 11: 'traffic light', 12: 'fire hydrant', 13: 'stop sign', 14: 'parking meter', 15: 'bench', 16: 'bird', 17: 'cat', 18: 'dog', 19: 'horse', 20: 'sheep', 21: 'cow', 22: 'elephant', 23: 'bear', 24: 'zebra', 25: 'giraffe', 26: 'backpack', 27: 'umbrella', 28: 'handbag', 29: 'tie', 30: 'suitcase', 31: 'frisbee', 32: 'skis', 33: 'snowboard', 34: 'sports ball', 35: 'kite', 36: 'baseball bat', 37: 'baseball glove', 38: 'skateboard', 39: 'surfboard', 40: 'tennis racket', 41: 'bottle', 42: 'wine glass', 43: 'cup', 44: 'fork', 45: 'knife', 46: 'spoon', 47: 'bowl', 48: 'banana', 49: 'apple', 50: 'sandwich', 51: 'orange', 52: 'broccoli', 53: 'carrot', 54: 'hot dog',
55: 'pizza', 56: 'donut', 57: 'cake', 58: 'chair', 59: 'couch', 60: 'potted plant', 61: 'bed', 62: 'dining table', 63: 'toilet', 64:
'tv', 65: 'laptop', 66: 'mouse', 67: 'remote', 68: 'keyboard', 69: 'cell phone', 70: 'microwave', 71: 'oven', 72: 'toaster', 73: 'sink', 74: 'refrigerator', 75: 'book', 76: 'clock', 77: 'vase', 78: 'scissors', 79: 'teddy bear', 80: 'hair drier', 81: 'toothbrush'} these are cls which i have trained, the first two cls are custom dataset and rest of are coco dataset means the class of yolov8s_world.pt it is base model and using this only i have trined our custom dataset trained dataset which means coco dataset
and result get is boxes []
conf []
class_value []

why the result not geting and i used base model as

[Vinaygoudasp7]

can i use pretrained yolov8s_worldv2.pt for training the custom dataset and also while training yolo world by using custom dataset it creates some images of train_batch.jpg in runs directory it labeled with different classes other than like the pen image it labeled with other classes in data.yaml classes as per previous data.yaml file or as per previous prompt why like this as shown below

and also other classes are not reflected in new trained model best.pt

after training i get like this
60 epochs completed in 0.770 hours.
Optimizer stripped from runs\detect\train36\weights\last.pt, 25.9MB
Optimizer stripped from runs\detect\train36\weights\best.pt, 25.9MB

Validating runs\detect\train36\weights\best.pt...
Ultralytics YOLOv8.2.14 🚀 Python-3.10.14 torch-2.3.0+cu118 CUDA:0 (NVIDIA GeForce RTX 3050, 8191MiB)
YOLOv8s-worldv2 summary (fused): 195 layers, 12749288 parameters, 0 gradients, 51.2 GFLOPs
all 116 388 0.513 0.537 0.531 0.404
Book 116 295 0.391 0.268 0.277 0.188
Pen 116 93 0.636 0.806 0.786 0.621
Speed: 2.6ms preprocess, 11.9ms inference, 0.0ms loss, 4.9ms postprocess per image
Results saved to runs\detect\train36
💡 Learn more at https://docs.ultralytics.com/modes/train

[glenn-jocher]

Hello! Yes, you can use the pretrained yolov8s_worldv2.pt for training on your custom dataset. It's a great way to leverage the learned features from the model and adapt them to your specific classes.

Regarding the train_batch.jpg images showing incorrect labels, this could be due to several reasons:

• Ensure that your data.yaml file is correctly configured with the right paths and class names for your new dataset.
• Make sure that the class indices in your annotation files match the order of classes listed in your data.yaml.

For the issue of other classes not being reflected in the new trained model, it's important to include a representative sample of all desired classes during training to avoid class imbalance and ensure the model learns to recognize all of them effectively.

If you continue to see unexpected results, consider reviewing your dataset and annotations for consistency and accuracy. Happy training! 😊

[Vinaygoudasp7]

names:

• Book
• Pen
• person
• bicycle
• car
• motorcycle
• airplane
• bus
• train
• truck
• boat
• traffic light
• fire hydrant
• stop sign
• parking meter
• bench
• bird
• cat
• dog
• horse
• sheep
• cow
• elephant
• bear
• zebra
• giraffe
• backpack
• umbrella
• handbag
• tie
• suitcase
• frisbee
• skis
• snowboard
• sports ball
• kite
• baseball bat
• baseball glove
• skateboard
• surfboard
• tennis racket
• bottle
• wine glass
• cup
• fork
• knife
• spoon
• bowl
• banana
• apple
• sandwich
• orange
• broccoli
• carrot
• hot dog
• pizza
• donut
• cake
• chair
• couch
• potted plant
• bed
• dining table
• toilet
• tv
• laptop
• mouse
• remote
• keyboard
• cell phone
• microwave
• oven
• toaster
• sink
• refrigerator
• book
• clock
• vase
• scissors
• teddy bear
• hair drier
• toothbrush

nc: 82
roboflow:
license: CC BY 4.0
project: pen_book_detection-bhnvg
url: https://universe.roboflow.com/vinay-yeadc/pen_book_detection-bhnvg/dataset/1
version: 1
workspace: vinay-yeadc
test: ./test/images
train: ./train/images
val: ./valid/images
this data.yaml file can you review this bcz the names contain some of pretrained classes(coco dataset) and new custom dataset classes(first two classes)

[glenn-jocher]

@Vinaygoudasp7 hello! Your data.yaml looks well-organized and it's great to see you're combining COCO classes with your custom classes. Just ensure that the indices in your annotation files correctly match the order of classes listed here. If you're using both pretrained and new classes, make sure that your training data includes enough examples of each class to avoid class imbalance. This setup should work well for training a model that recognizes a diverse set of objects. Happy training! 😊

[Vinaygoudasp7]

hi i get anther doubt can i use this command to train !yolo detect train data=D:/AIML/Traffic_signal_and_violation_detections/trafficlights21/data.yaml model=yolov8m-worldv2.pt epochs=10 imgsz=640 freeze=20 it is like in the yolov5 by freezeing some layer of trained previous dataset

[glenn-jocher]

Hello! Yes, you can use the freeze=20 argument in your training command to freeze the first 20 layers of the yolov8m-worldv2.pt model during training. This is similar to what you might do with YOLOv5 and can be effective for fine-tuning on a new dataset while preserving learned features from the base model. Your command looks good! Just ensure that the path to your data.yaml is correct and that it's properly configured. Happy training! 😊