is raw text fine tuning possible?

[genesiz7]

i am using this platform for finetune of llama2, mistral 7b model, thanks.
i know it is possible that QA set for fine tuning for those models.
however i could not find the method for fine tune for raw text.
how can i fine tune llama2 or mistral2 with raw text in this platform?
please assist for me.

[hiyouga]

Refer to this example: https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/lora_single_gpu/pretrain.sh

[Nihilentropy-117]

make a HF Dataset with one column, 'text' then in dataset_info.json add
"$yourDatasetName": { "hf_hub_url": "$user/$dataset", "columns": { "prompt": "text" } },
then this will work to run in jupyter
run_exp(dict( stage="pt", do_train=True, model_name_or_path="unsloth/llama-3-8b-Instruct-bnb-4bit", dataset="$yourDatasetName", template="empty", finetuning_type="lora", lora_target="all", output_dir="llama3_lora", per_device_train_batch_size=2, gradient_accumulation_steps=4, lr_scheduler_type="cosine", logging_steps=10, warmup_ratio=0.1, save_steps=1000, learning_rate=5e-5, num_train_epochs=3.0, max_samples=1500, max_grad_norm=1.0, quantization_bit=4, loraplus_lr_ratio=16.0, use_unsloth=True, fp16=True, ))