__add__ for Dataset, IterableDataset

[oh-gnues-iohc]

It's too cumbersome to write this command every time we perform a dataset merging operation. pythonfrom datasets import concatenate_datasets We have added a simple __add__ magic method to each class using concatenate_datasets.

from datasets import load_dataset

bookcorpus = load_dataset("bookcorpus", split="train")
wiki = load_dataset("wikimedia/wikipedia", "20231101.ab", split="train")
wiki = wiki.remove_columns([col for col in wiki.column_names if col != "text"])  # only keep the 'text' column

bookcorpus + wiki

#Dataset({
#   features: ['text'],
#   num_rows: 74004228
#})
#Dataset({
#    features: ['text'],
#    num_rows: 6152
#})
#Dataset({
#   features: ['text'],
#    num_rows: 74010380
#})

[mariosasko]

Hi! You can find a reason why we are against this feature in #3449.

It's too cumbersome to write this command every time we perform a dataset merging operation

Explicit is better than implicit, so this isn't a good enough reason.

Thanks for the effort nonetheless :)!