Skip to content

Awesomes for Open Source Large Language Models and Datasets.

Notifications You must be signed in to change notification settings

chenyaofo/OpenLM-Awesome

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 

Repository files navigation

OpenLM Awesomes

Awesome

An up-to-date awesome list of open-source large language models, instruction & RLHF datasets and high-performance codebases.

Open Large Language Models

Model Stars Organization Language1 Checkpoints
LLaMa2 Meta AI Multi-Lang 7B & 13B & 30B & 65B
OPT Meta AI Eng. 125M & 350M & 1.3B & 2.7B & 6.7B & 13B & 30B & 66B & 175B3
BLOOM -- BigScience Multi-Lang 560m & 1.1B & 1.7B & 3B & 7.1B & 176B
GLM4 Tsinghua CHI. & Eng. 2B & 10B & 130B5

Open Instruction-following Large Language Models

Model Stars Organization Based Model Checkpoints
BLOOMZ -- BigScience BLOOM 560m & 1.1B & 1.7B & 3B & 7.1B & 176B
Alpaca Stanford LLaMa 7B
Vicuna lm-sys LLaMa 7B & 13B
Chinese-Vicuna Facico LLaMa 7B

Open Instruction Tuning and RLHF Datasets

Dataset Language #Samples Annotation Type Online Link
alpaca_chinese_dataset Chinese 52K GPT-generated & Human-annotated hikariming/alpaca_chinese_dataset
BELLE/data Chinese 1.5M GPT-generated BELLE/data/1.5M
pCLUE Chinese 1.2M Formated from Existing Datasets CLUEbenchmark/pCLUE
Med-ChatGLM/data Chinese 7K GPT-generated SCIR-HI/Med-ChatGLM
COIG Chinese 181K Follow this paper BAAI/COIG

High-performance Open Instruction Tuning Codebase

  • DeepSpeed Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
  • ColossalChat is a project to implement LLM with RLHF, powered by the Colossal-AI project.
  • LMFlow: An extensible, convenient, and efficient toolbox for finetuning large machine learning models, designed to be user-friendly, speedy and reliable, and accessible to the entire community.

Footnotes

  1. Languages used in pretraining texts. CHI is short for Chinese. Eng is short for English.

  2. Direct access to the LLaMa models is currently not possible due to Meta AI's policies. However, interested individuals can apply to access the models by filling out a form, which can be found at here. Alternatively, the checkpoints for the LLaMa models can be accessed via the following link on the Decapoda Research page of Huggingface.

  3. Direct access to the OPT models with 176B params is currently not possible due to Meta AI's policies. However, interested individuals can apply to access the models by filling out a form, which can be found at here.

  4. There are other GLM models (with different pretraining corpus) that are available at the THUDM page of Huggingface. GLM models include different size: 10B/2B/515M/410M/335M/110M (English), 10B/335M (Chinese). In above Table, GLM models with Engilish are provided.

  5. Direct access to the GLM model with 130B params is currently not possible due to THUDM's policies. However, interested individuals can apply to access the models by filling out a form, which can be found at here.

Releases

No releases published

Packages

No packages published