Ultralytics
datasets/segment/ #8528
giscus
bot
Replies: 9 comments 29 replies
-
|
That's great but how do we then check the auto annotated labels and correct them? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🌟 To review and correct auto-annotated labels, you can use visualization tools like CVAT or LabelImg. These tools allow you to load your images and corresponding annotations to visually inspect and edit them as needed. For example, after converting your dataset to the Ultralytics YOLO format, you can load an image and its annotations in LabelImg to make corrections: # Assuming you've installed LabelImg and your dataset is in the correct format
# Navigate to your dataset directory and launch LabelImg
labelImgThen, open your image and its corresponding This process helps ensure your model trains on accurate data, improving its performance. Happy annotating! 🛠️ |
Beta Was this translation helpful? Give feedback.
-
|
hello there,Did you succeed? Are there any specific steps you can share to verify the auto annotation result? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🙌 To verify your auto annotated results, you can visually inspect the generated labels with label visualization tools like CVAT or LabelImg. This allows you to quickly see if the annotations match up with the actual objects in the images. For a quick check, you could also load an image and its annotation in Python and plot them together: import matplotlib.pyplot as plt
import matplotlib.patches as patches
from PIL import Image
def plot_image_with_annotations(image_path, label_path):
# Load image
im = Image.open(image_path)
fig, ax = plt.subplots(1)
ax.imshow(im)
# Read the annotations
with open(label_path) as f:
for line in f.readlines():
class_index, *coords = map(float, line.split(' '))
x1, y1, x2, y2 = coords[:4] # Assuming box format for simplicity
rect = patches.Rectangle((x1, y1), x2-x1, y2-y1, linewidth=2, edgecolor='r', facecolor='none')
ax.add_patch(rect)
plt.show()
# Replace 'path_to_image' and 'path_to_label' with your actual file paths
plot_image_with_annotations('path_to_image.jpg', 'path_to_label.txt')This script will display the image with the annotated bounding boxes, allowing you to easily spot any inconsistencies or errors in the annotations. Let me know if this helps or if you need more assistance! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hey! I am still struggeling with verifying the auto annotation results. I had no success in getting CVAT or LabelImg to run with Yolov8 segmentation labels. I managed it with Roboflow but Roboflow is not free to use after the free trial. The code you provided helps with verifying box format but not really with more complex shapes. Do you have any tips or references to help me with this issue? Best regards, |
Beta Was this translation helpful? Give feedback.
-
|
Hi Simon! 🌟 I understand the difficulties with handling complex shapes for segmentation in YOLOv8, especially when tools like CVAT or LabelImg might not fully meet your needs for segmentation labels. For complex shapes, VGG Image Annotator (VIA) is another excellent free tool that supports polygon annotations, which might better suit segmentation tasks. VIA is quite straightforward to use for annotating images with complex shapes and can export annotations in a format that you may convert to the Ultralytics YOLO format. Here’s a quick guide on how to do that:
Here is a very simplified version of how you might handle the conversion in Python: import json
# Load VIA annotations
with open('via_annotations.json') as f:
via_data = json.load(f)
# Example conversion (simplified)
for key, item in via_data['_via_img_metadata'].items():
filename = item['filename'][:-4] + '.txt' # Change extension
with open(filename, 'w') as file:
for region in item['regions']:
shape_attributes = region['shape_attributes']
all_points_x = shape_attributes['all_points_x']
all_points_y = shape_attributes['all_points_y']
points = [str(pt) for pair in zip(all_points_x, all_points_y) for pt in pair]
line = ' '.join(['0'] + points) # Assuming class index 0
file.write(line + '\n')This script is simplified and assumes all objects belong to class '0'. Adjust according to your dataset's class indices and structure. For larger datasets or more complex needs, consider developing or adapting tools specifically for segmentation in YOLOv8. Hope this helps, Simon! If you need further assistance, feel free to ask. 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, Should i train the model with negative samples? if It is yes, how should it be to labeling? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! 😊 Yes, training your model with negative samples (images without any objects of interest) can be beneficial, especially for reducing false positives. For labeling negative samples, simply provide an image without any corresponding label file. This way, the model learns that not every image contains objects of interest. For instance, if you have an image named Here's a quick example of how your dataset directory might look: And in your path: ../datasets/your_dataset # dataset root dir
train: images/train # train images
val: images/val # val images
# Classes
names:
0: class1
1: class2This approach helps the model understand the context better and improves its ability to distinguish between backgrounds and actual objects. Happy training! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
the file paths for custom data are still the same with detection, right? i mean i must create 2 folders that 1 for image and the other for label and inside image and label folder i continue creating train and test folder for both |
Beta Was this translation helpful? Give feedback.
-
|
@TranThanhTuan2509 yes, the file paths for custom data in segmentation tasks follow a similar structure to detection tasks. You'll indeed need two main folders: one for images and another for labels. Inside each, you should create separate subfolders for training and testing data ( For segmentation tasks, ensure your label files match the Ultralytics YOLO format for segmentation, with each row in a label file representing an object instance as If you're setting up your dataset YAML file, it would look something like this: path: /path/to/dataset # dataset root dir
train: images/train # train images
val: images/val # val images
test: images/test # test images (optional)
# Classes
names:
0: person
1: bicycle
# ... add as many classes as you haveRemember, the |
Beta Was this translation helpful? Give feedback.
-
|
Hi! Thanks in advance |
Beta Was this translation helpful? Give feedback.
-
|
@TaisiaKozharina hi there! The order of So, it's essential to maintain consistent ordering to ensure the model trains correctly on the right class labels. If you find discrepancies, it's best to realign them before proceeding with training to avoid any confusion or errors in model performance. 🚗🚲 Hope this helps! Let me know if you have any more questions. |
Beta Was this translation helpful? Give feedback.
-
|
Can I segment people in depth images acquired by LiDAR? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you can segment people in depth images acquired by LiDAR using Ultralytics YOLO models, including YOLOv8, designed for instance segmentation tasks. However, it's essential to ensure that your depth images are converted into a format that the model can process, typically RGB or grayscale images. Depth information can be encoded as intensity values in such images. For instance segmentation with YOLO, your dataset should follow the Ultralytics YOLO format for segmentation, where each object instance in an image has a corresponding row in a Ensure your LiDAR depth images and annotations are correctly formatted. If you're starting with a dataset in a different format, you might need to convert it to the Ultralytics YOLO format. For segmentation tasks, remember to set up your dataset YAML file correctly, specifying paths to your training, validation, and (optionally) test sets, along with class names. Here's a simple example to start training a segmentation model on your dataset: from ultralytics import YOLO
# Load a pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Train the model on your dataset
results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640)Remember, the success of segmenting people in depth images will largely depend on the quality of your dataset and how well the depth information represents the scene. Happy segmenting! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I see segmentation format for YOLOv8 as, say, |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Converting brush labelled images into our specified point coordinates format for segmentation tasks can indeed be a bit tricky, but here's a short guide to help you out. 😊 To convert from a brush labelled mask (png) to our segmentation format, you'll need to extract polygon points from the mask edges. You can use tools like OpenCV's import cv2
import numpy as np
# Load your brush labelled mask
mask = cv2.imread('your_mask.png', 0) # 0 for grayscale
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Now, select the contour with max area or iterate through all contours
# For example:
contour = max(contours, key=cv2.contourArea)
# Simplify contour
epsilon = 0.001 * cv2.arcLength(contour, True)
approx = cv2.approxPolyDP(contour, epsilon, True)
# Convert to normalized coordinates
height, width = mask.shape
normalized_contour = approx.reshape(-1, 2) / [width, height]For the direction of the points, it doesn't matter as long as you are consistent across your dataset. However, the counterclockwise direction is typically used. The points should be as closely spaced as required to capture the shape of your object accurately but avoiding too many points for efficiency. Each object in your image needs at least 3 xy points, so ensure your approximation doesn't reduce the points below this threshold. Feel free to adjust the Hope this helps! Let me know if you have more questions. |
Beta Was this translation helpful? Give feedback.
-
|
I have a follow up question about the dataset format. My dataset is organized as shown below. Inside some of my annotation folders, (say for example folder_1/1.txt) I have several label for different objects that belong to the same class. Here's an example 199 0.131608 0.000000 1.000000 0.869366 1.000000 1.000000 0.000000 0.000000 0.131608 0.000000 The problem is , the training code is considering those unique annotations as duplicate and removing them I get the following error message train: WARNING dataset/ What should I do in this case, and is my data structured in folder, and formatted in text correctly? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@pderrenger, thank you for your quick response, Paula! the dataset is for segmentation indeed and I cut the polygons short before I asked the question here for clarity. I don't have bounding boxes and I exclusively have polygons that I plotted on the images and verified they're correct. Here's an actual data file I have millions of annotations so modifying them isn't an option :) I wonder if there's an option for "is crowded" like Mask-RCNN? I'll look it up. |
Beta Was this translation helpful? Give feedback.
-
|
Upon a closer inspection, I found out that the annotations aren't even close to each other. The additional annotations from all the files are incorrectly marked as duplicates. So modifying the train code won't help, |
Beta Was this translation helpful? Give feedback.
-
|
@AloshkaD hey there! 👋 It sounds like there's a little mix-up with the annotation handling. If your annotations are correctly formatted and not duplicates but are still being flagged, it could be due to how the detection script processes the data. Each annotation must be unique for a given image, especially for complex segmentation tasks where objects might be densely packed or overlap. Given that your dataset is for segmentation and based on your detailed annotations, it might be helpful to revisit the dataset loading and parsing logic to ensure it accommodates the specific nuances of your data, like closely situated or overlapping polygons. Sometimes the devil is in the details, and a small tweak in the data preprocessing or loader can make a big difference. 🛠️ For instance, if "is crowded" flags akin to those used in COCO dataset formats could help distinguish between legitimate close annotations and duplicates, implementing a similar strategy in your dataset parsing logic might be beneficial. However, if you're working directly with YOLO format or another custom format without such flags, ensuring that your data loader script correctly interprets closely adjacent annotations without mistakenly pruning them could be key. # Example pseudocode adjustment for loading segmentation data
def load_data(annotation_path):
annotations = read_annotations(annotation_path)
objects = []
for anno in annotations:
obj = parse_annotation(anno)
# Logic to handle close or overlapping annotations
# For example, check spatial relationship or 'is crowded' flag here
objects.append(obj)
return objectsFinally, if you're still encountering issues, diving deeper into the training script where duplicates are flagged and adjusting the logic or threshold for considering annotations as duplicates might be necessary, as you've noted. Remember, the Ultralytics team and the community are always here to help. If specific adjustments don't clear things up, sharing a snippet of your loading and preprocessing logic could provide more insights for a more targeted assist. Keep up the great work! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Once again, I appreciate your prompt response, Paula! I've addressed the problem and would like to share the solution here to help others who might encounter the same issue. I developed code to preprocess my dataset and convert it from Mask R-CNN format from the ground up. Although there is an existing data converter code available, I opted not to use it due to some errors encountered with the normalization scale. An example error I encountered when I used Ultralytics' converter is Consequently, I implemented my own converter and a normalization function as follows: Then after you mentioned the close polygons as a culprit, it clicked to me and I tested the following function instead In the latter function, I used the image coordinates for the normalization. Obvious mistake costs me two days of scrambling around :) Thanks for the hint! |
Beta Was this translation helpful? Give feedback.
-
|
@AloshkaD hey there! 🎉 It's fantastic to see you found a solution and are willing to share it with the community. Thanks for the details on how you resolved the annotation normalization issue! For those facing similar normalization challenges with their datasets, the key takeaway is to use the image's width and height to normalize the polygon coordinates properly. This ensures that the coordinates are correctly scaled relative to the image size, avoiding the issue of "non-normalized or out of bounds coordinates" warnings. Here's a quick recap of the correct approach: def normalize_polygon_to_yolov8_format(polygon, image_width, image_height):
normalized_polygon = []
for point in polygon:
normalized_x = point[0] / image_width
normalized_y = point[1] / image_height
normalized_polygon.extend([normalized_x, normalized_y])
return normalized_polygonJust remember to replace Thanks again for sharing your insights, and happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Does this segmentation code also output a bounding box for target detection? If I have 20 classes, 10 of which are object detection and 10 of which are semantic segmentation, can I do it with this code? What should I do? Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
You can create a bounding box from any network or code that returns a segmentation mask/polygon. Simply use something like this |
Beta Was this translation helpful? Give feedback.
-
|
Absolutely, you can indeed perform both object detection and semantic segmentation with Ultralytics YOLOv8 by preparing your dataset accordingly. For the mixed task of detecting 10 classes and segmenting another 10, you'd need to structure your dataset and possibly tweak your model configuration to handle both types of tasks. Your dataset should include both bounding box annotations for object detection classes and segmentation mask coordinates for segmentation classes, adhering to the Ultralytics YOLO format. Here’s a simplified approach:
In this example, the first row represents a detection with class index For mixed tasks like the one you described, careful dataset preparation and model configuration tweaking are key. Don't hesitate to experiment with different architectures and training setups to find what works best for your specific case. Good luck! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hello! I'm working on a project where I'm working with chest x-ray images belonging to 3 classes: Pneumonia, COVID-19, and Healthy. I wish to perform instance segmentation on my dataset so that I can identify the lungs using bounding boxes and highlight pixels WITHIN the lungs which belong to a particular class for added precision. However, the YOLO Dataset guide for Segmentation tasks specifies that it is very similar (near identical) to the Detection task. I believe the formatting looks something like this: Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]path: ../datasets/coco8-seg # dataset root dir Classes (80 COCO classes)names: ...77: teddy bear I have adapted this to my own dataset. The result is that the bounding boxes are pretty accurate but the segmentation mask highlights all the pixels within the box rather than highlighting the lungs (my intended motive for added precision). Is there a way we can include binary segmentation mask images in which the lungs are highlighted in white and the background is highlighted in black to guide the segmentation process and make it more accurate? I am using the yolov8n-seg.pt model for the task at hand. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Hello! 👋 Thanks for reaching out with your question. It seems you're looking to enhance the precision of your instance segmentation tasks, specifically targeting lung areas within chest x-rays. To achieve this with YOLOv8 segmentation models (such as For each of your x-ray images, you would create corresponding binary mask images where lungs are highlighted in white against a black background. These mask images are then directly used during model training to improve segmentation accuracy, focusing on the lung regions. While the YOLO formatting mainly handles bounding box annotations, to include segmentation masks, you'd typically need to ensure your dataset annotations include pixel-wise mask information alongside the object classes and bounding box annotations, following the outlined format for instance segmentation: With this format, your dataset should have one text file per image, with each line in the text file representing a mask for an object instance in the respective image, described by points around the object's outline. Given your specific use case, it might involve a preprocessing step where your binary lung masks are converted into point coordinates to match the format expected by YOLOv8n-seg. This conversion process might require custom scripting depending on your dataset's current format. An example approach could be to:
This way, you enrich the training data to guide the model more effectively towards lung regions, potentially enhancing the accuracy of your instance segmentation efforts for Pneumonia, COVID-19, and Healthy classes within chest x-rays. However, I'd recommend digging into any additional documentation available on the YOLOv8 segmentation model and dataset preparation to ensure alignment with your project's goals. Hope this helps, and best of luck with your project! If you have any more questions, feel free to ask. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you so much for your prompt response. Just to clarify, if I need to include segmentation mask coordinates, the YOLO annotation formatting would look something like this: ... where the first four coordinates after the class index would correspond to the bounding box coordinates and the remaining would be coordinates describing the lung contours (which I would extract from the binary segmentation masks) ? Also, I would need to use the binary mask images for training the model rather than the chest x-ray images, or would the model work with either of the two? |
Beta Was this translation helpful? Give feedback.
-
|
@Varun10-hub hello! You're absolutely correct with your understanding of the formatting for segmentation in the YOLO model. The formatting Regarding your second question, you will still use the chest x-ray images for training the YOLO model. The binary mask images are used to generate the segmentation coordinates (like those lung contours), which are provided as annotations to train the model. You don't feed the binary masks directly as training images. Instead, these masks help define where the model should focus within the training images, boosting the accuracy of segmenting the specific regions of interest within the lungs. This setup should provide a powerful approach to precisely segment lung regions affected by conditions such as Pneumonia or COVID-19 in your x-ray images! Keep up the great work! 😊👍 |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you very much for your prompt response. I tried implementing the solution. I used OpenCV's find_contours function and then approximated using PolyDP. The contour points are perfectly identified by the code (I checked that by plotting them against the binary mask image). I extracted the points and prepared the text files as per the described YOLO segmentation format. Here is one example of a text file I used as a label: 0 0.017094017094017096 0.017094017094017096 0.9786324786324786 0.017094017094017096 0.9786324786324786 0.9786324786324786 0.017094017094017096 0.9786324786324786 0.7143 0.0 0.7143 0.0312 0.7098 0.0357 0.6473 0.0402 0.6429 0.0536 0.625 0.0714 0.6116 0.0714 0.6071 0.0759 0.6071 0.1473 0.5938 0.1607 0.5938 0.2455 0.5848 0.2545 0.5759 0.2545 0.5759 0.2723 0.5848 0.2812 0.5893 0.2946 0.5893 0.3705 0.5714 0.4062 0.5714 0.4375 0.5804 0.442 0.5804 0.4955 0.5893 0.5 0.5893 0.5312 0.5759 0.5402 0.5759 0.567 0.5848 0.5759 0.5804 0.6339 0.5893 0.6429 0.5893 0.7366 0.5848 0.7411 0.5848 0.7679 0.5759 0.7768 0.5714 0.7902 0.5714 0.808 0.5848 0.8259 0.5848 0.8393 0.6027 0.8482 0.6071 0.8616 0.6071 0.8884 0.6116 0.8929 0.6116 0.9241 0.625 0.9286 0.6339 0.9375 0.6696 0.9375 0.683 0.9554 0.7098 0.9554 0.7098 0.9062 0.7188 0.8973 0.817 0.8973 0.8214 0.9018 0.8214 0.9241 0.8304 0.9286 0.8482 0.9286 0.8571 0.942 0.8571 0.9554 0.875 0.9554 0.8839 0.9643 0.8839 0.9955 0.9955 0.9955 0.9955 0.317 0.9777 0.317 0.9732 0.3125 0.9732 0.2857 0.9643 0.2768 0.9643 0.2589 0.9598 0.2545 0.9598 0.1875 0.9554 0.183 0.9554 0.1607 0.9286 0.1429 0.9286 0.1116 0.9241 0.1071 ... I encountered an error saying that the labels were not found, and all the labels were either classified as 'corrupt' or 'background' when the script checks if the labels are valid or not. Is it because the label file is really long, or is there some problem with the numbers describing the contour points? Also, the training process works if I just take the object detection coordinates, but not when I take the segmentation mask coordinates along with the bounding box coordinates. Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you for your detailed update. It sounds like you're on the right track with using OpenCV's The issue you're encountering might be due to the format or length of the coordinates in your label files. Ensure that each row in your label file starts with the class index followed by the segmentation coordinates, and that each segmentation label has at least three points. Here's a quick checklist:
If the labels are still not being recognized, consider simplifying the contours to reduce the number of points and see if that resolves the issue. For further details, you can refer to the Ultralytics documentation on segmentation datasets. Best of luck, and feel free to reach out if you have more questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, is it possible to train the segmentation model for objects which consist of more than 1 piece? If yes, how to construct a dataset for such a model? For my task, I need to show the model that two separated pieces are actually a part of a whole. Thanks in advance |
Beta Was this translation helpful? Give feedback.
-
|
@mykhailo, you can segment as many pieces, as you want. N annotated pieces will give N lines in .txt file. The first number in each line is class id. If these pieces belong to same class (instance segmentation), just provide thin margins between the pieces during annotating, otherwise they would merge in single piece. |
Beta Was this translation helpful? Give feedback.
-
|
@andyrey but what I need is completely opposite. There are parts of objects separated by some distance which contain other objects or backgrounds. But I still want to train model to recognized as a single object. These parts of object are completely different in nature but there are some properties that indicate that they are linked together. |
Beta Was this translation helpful? Give feedback.
-
|
@mykhailo: the most direct approach is hierarchical- you train one model for whole object segmentation and another model for inner parts segmentation which work inside mask after the first model. Is it possible to do within single model in YOLOv8 -say, we mark and label whole part of the object as class 0, subpart1 as class 1, subpart2 as class 2, would be possible, if the net is non-exclusive, I am not sure, should be tested. |
Beta Was this translation helpful? Give feedback.
-
|
@andyrey my question is: how to represent whole object in the dataset? Because in the label file in each row, I should provide a boundary for each object. But if an object can only be represented as two separate boundaries? For example, if whole object is partially covered by other objects in a way it is splited into several pieces. But I still want a model to correctly identify them as whole. |
Beta Was this translation helpful? Give feedback.
-
|
@MykhailoPoleshchukLitslink: Simply bound the whole object, as if it is not occluded by aside object. YOLOv8 doesn't process inner holes on the target object. |
Beta Was this translation helpful? Give feedback.
-
datasets/segment/
Learn how Ultralytics YOLO supports various dataset formats for instance segmentation. This guide includes information on data conversions, auto-annotations, and dataset usage.
https://docs.ultralytics.com/datasets/segment/
Beta Was this translation helpful? Give feedback.
All reactions