New: Ultralytics YOLO-Human

[Laughing-q]

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Introducing new YOLOHuman model for human attribute detection! 🚀

📊 Key Changes

• Added YOLOHuman class as part of the model imports.
• Introduced a new YAML configuration for the YOLOv8 human detection model.
• Implemented additional augmentations to include human attribute data handling.
• Established a new dataset class, HumanDataset, for loading and processing human-related datasets.
• Included Human object in results to encapsulate detected human attributes.
• Enriched model __init__.py to include YOLOHuman.
• Formulated HumanPredictor, HumanTrainer, and HumanValidator under the new YOLO human module for prediction, training, and validation.

🎯 Purpose & Impact

• Enhances Model Catalog: Expands Ultralytics' model offerings to include human-specific attribute detection.
• Improves Dataset Handling: Offers streamlined process for datasets involving human features.
• Facilitates Human-centric Applications: Paves the way for more sophisticated applications such as demographic analysis, security enhancements, and personalized customer experiences.

[glenn-jocher]

@Laughing-q @ambitious-octopus val test is also failing because our save_txt functionality is not adapted correctly (should output txt files in the same format as the labels).

yolo val human model=weights/yolov8n-human.pt data=human8.yaml imgsz=32 save_txt

Save JSON is working correct:

yolo val human model=weights/yolov8n-human.pt data=human8.yaml imgsz=32 save_json

[ambitious-octopus]

@glenn-jocher Updated https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8n-human.pt with a freshly trained model.

[Laughing-q]

@glenn-jocher @ambitious-octopus Guys I removed YOLOHuman class since it's not needed while we treat human as a new task of YOLO.
Also I've fixed the save_one_txt issue for human task. There's actually another update made by me, is that I figured we can directly use the save_txt method in Results instead of recreating something similar/redundant for val mode of each task.

ultralytics/ultralytics/engine/results.py

Lines 332 to 366 in bac868c

	def save_txt(self, txt_file, save_conf=False):
	"""
	Save predictions into txt file.
	Args:
	txt_file (str): txt file path.
	save_conf (bool): save confidence score or not.
	"""
	is_obb = self.obb is not None
	boxes = self.obb if is_obb else self.boxes
	masks = self.masks
	probs = self.probs
	kpts = self.keypoints
	texts = []
	if probs is not None:
	# Classify
	[texts.append(f"{probs.data[j]:.2f} {self.names[j]}") for j in probs.top5]
	elif boxes:
	# Detect/segment/pose
	for j, d in enumerate(boxes):
	c, conf, id = int(d.cls), float(d.conf), None if d.id is None else int(d.id.item())
	line = (c, *(d.xyxyxyxyn.view(-1) if is_obb else d.xywhn.view(-1)))
	if masks:
	seg = masks[j].xyn[0].copy().reshape(-1) # reversed mask.xyn, (n,2) to (n*2)
	line = (c, *seg)
	if kpts is not None:
	kpt = torch.cat((kpts[j].xyn, kpts[j].conf[..., None]), 2) if kpts[j].has_visible else kpts[j].xyn
	line += (*kpt.reshape(-1).tolist(),)
	line += (conf,) * save_conf + (() if id is None else (id,))
	texts.append(("%g " * len(line)).rstrip() % line)
	if texts:
	Path(txt_file).parent.mkdir(parents=True, exist_ok=True) # make directory
	with open(txt_file, "a") as f:
	f.writelines(text + "\n" for text in texts)

ultralytics/ultralytics/models/yolo/detect/val.py

Lines 256 to 263 in 7593b4a

	def save_one_txt(self, predn, save_conf, shape, file):
	"""Save YOLO detections to a txt file in normalized coordinates in a specific format."""
	gn = torch.tensor(shape)[[1, 0, 1, 0]] # normalization gain whwh
	for *xyxy, conf, cls in predn.tolist():
	xywh = (ops.xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist() # normalized xywh
	line = (cls, *xywh, conf) if save_conf else (cls, *xywh) # label format
	with open(file, "a") as f:
	f.write(("%g " * len(line)).rstrip() % line + "\n")

So I updated our detect/obb/human tasks with Results.save_txt. Other tasks i.e segment/pose actually have the save_one_txt part commented so I left it there for now and let's develop these two in another PR later.

ultralytics/ultralytics/models/yolo/segment/val.py

Lines 157 to 158 in 7593b4a

	# if self.args.save_txt:
	# save_one_txt(predn, save_conf, shape, file=save_dir / 'labels' / f'{path.stem}.txt')

[ambitious-octopus]

@ambitious-octopus test_data_utils() is failing because we need to upload a HUB-format dataset to https://github.com/ultralytics/hub/tree/main/example_datasets to join the datasets for other tasks.

@glenn-jocher Uploaded HUB-format datataset PR.

[Laughing-q]

@glenn-jocher Meanwhile I noticed that we have a lot duplicated code in Validator.update_metrics across different tasks.

ultralytics/ultralytics/models/yolo/segment/val.py

Line 100 in 7593b4a

def update_metrics(self, preds, batch):

Looking into this PR: #12645, the author had to update each val.py to add the feature because we have multiple modified versions of this method. And that's why @ambitious-octopus encountered the target_img key missing issue after the PR merged(because I had to create another modified version for human task).
I actually tried to refactor this part a bit in the OBB PR so we don't have a modified version of update_metrics lying in obb/val.py. But there's still a lot duplication with other tasks.
I think I'll revisit this part of code and try to eliminate the duplicated part as much as possible later some day in another PR.

[Laughing-q]

@ambitious-octopus the CI tests are failing, seems to related to new updated yolov8n-human.pt

And that's because the model was trained withtask=detect but now since we're updating the logic here to treat human as a new task so the tests are failing with training from the new yolov8n-human.pt

[Laughing-q]

@ambitious-octopus let's reset the weight back, and later today I'll launch new training on our server with task=human to get all sizes of model. :)

[Laughing-q]

@glenn-jocher @ambitious-octopus ok I've re-uploaded the weight and now everything works properly in tests except the hub dataset, which I guess it'll be good when the PR that @ambitious-octopus opened merged. :)

And now there's several gpus freed on our server, I'll launch several training right now.

[ambitious-octopus]

docs image

[Burhan-Q]

I did not serve the docs locally, just quickly reviewed the raw markdown on GitHub. A few notes and suggestions, but overall looks excellent!

[Burhan-Q]

Fill in TODO sections for predict.md and train.md

[Burhan-Q]

Do we want the header to be a link like this? I haven't looked at how it renders, but I don't think this is normally something we do.

[Burhan-Q]

Instead of referencing the YAML files, probably better to reference the docs pages for the relevant datasets.

[Burhan-Q]

Not sure if Models here needs to be a link.

[Burhan-Q]

A few notes:

• I think that "artificially annotated" should be bolded and underlined artificially annotated for emphasis
• I don't think a reason ("Due to proprietary reasons") for why the dataset isn't available should be given, just that we've decided to not make it publicly available.
• I think this section should be removed "ensures that the models possess a competitive edge, offering advanced capabilities and superior performance" as it's fairly baseless and seems (to me) like it's asking for trouble.

[Burhan-Q]

- Height (Cm)
+ Height (cm)

[Burhan-Q]

Flocks of Humans

[Burhan-Q]

See comments in docs/en/tasks/human.md on this text.

[Burhan-Q]

I would include data="coco8-human" even tho it will work without. Trust me, it will avoid confusion later.

[Burhan-Q]

Same comment from earlier