HuggingFace Weight & Biases Matplotlib Openpyxl
This project is practice code for 한국소프트웨어종합학술대회(KSC) 2023 질문 생성 성능 향상을 위한 대규모 언어 모델 Post-training 적용 방법 [Paper].
asahi417/lm-question-generation
LMQG
LoRA
Hyperparamter Tuning Plan (09/07 ~ 0913)
| Sever | GPU | Model | Lora R | Estimated Runtime | Estimated Date |
|---|---|---|---|---|---|
| 01 | device=0 | google/flan-t5-xl | 64 | 3 Day | 09/10 |
| 01 | device=1 | google/flan-t5-xl | 128 | 3 Day | 09/10 |
| 02 | device=0 | google/flan-t5-xl | 4 | 5 Day | 09/12 |
| 02 | device=1 | google/flan-t5-xl | 8 | 5 Day | 09/12 |
| 03 | device=0 | google/flan-t5-xl | 16 | 6 Day | 09/13 |
| 04 | device=0 | google/flan-t5-xl | 32 | 6 Day | 09/13 |
bigscience-workshop/promptsource
allenai/natural-instructions
Folder Information
lmqg:Offcial Github Folder
lmqg_collate_fn: Instruction-Tuning + Collate_fn
lmqg_collate_fn_inference: Instruction-Tuning Inference + Collate_fn
lmqg_post_non_lora: Fine-Tuning
lmqg_original: Fine-Tuning Evaluation
lmqg_inference: Fine-Tuning Inference
A Survey of Large Language Models
Generative Language Models for Paragraph-Level Question Generation
An Empirical Comparison of LM-based Question and Answer Generation Methods
A Practical Toolkit for Multilingual Question and Answer Generation
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Finetuned Language Models Are Zero-Shot Learners
LoRA: Low-Rank Adaptation of Large Language Models
Alpaca: Intermittent Execution without Checkpoints
Instruction Tuning with GPT-4