-
Notifications
You must be signed in to change notification settings - Fork 2
List of Large Language Models and APIs
toncho11 edited this page Apr 27, 2023
·
52 revisions
-
cohere API - trial / paid
- it provides straight forward context stuffing / few shot learning
- no model download
- OpneAI provides Python API for GPT3 - trail / paid
- example code for fine-tuning
- no model download
- GODEL - downloadable and usable with Hugging Face
- LLaMA 65 billion model from Meta AI
- LLaMA is a series of large language models from 7B to 65B parameters, trained by Meta AI. They train for longer on more data and show that something like gpt-3 can be outperformed by significantly smaller models when trained like this.
- downloadable
- an auto-regressive language model
-
how to use with Hugging Face
- download the model weights and tokenizer
- then you need to convert them to Huggign Face's own format with a script
-
chat bot example
- LLaMA model
- download the model weights and tokenizer
- Using C++ code
- you need the original LLaMA model weights
- LLaMA Alpaca - uses the LLAMA model but fine tuned with different weights
- Good video
- Stanford Alpaca team managed to come up with a state-of-the-art instruct model by fine-tuning LLaMA on a fairly small dataset (that was actually produced programatically by GPT-3)
- Stanford Alpaca - code and documentation to train Stanford's Alpaca models, and generate the data
- C++ code
- model and weights for Alpaca
- runs on CPU only, requires 4GB of RAM and it compiles on Windows as well
- Alpaca-LoRA - is a 7B-parameter LLaMA model finetuned to follow instructions. It is trained on the Stanford Alpaca dataset and makes use of the Huggingface LLaMA implementation.
- article LORA: LOW-RANK ADAPTATION OF LARGE LANGUAGE MODELS
- Chat bot service with Alpaca-LoRA
- GPT-Neo 2.7B - a transformer model designed using EleutherAI's replication of the GPT-3 architecture. The model is available on HuggingFace. Although it can be used for different tasks, the model is best at what it was pretrained for, which is generating texts from a prompt.
-
GPT-J - a good example of a very capable model that only works correctly with few-shot learning. GPT-J 6B was trained on the Pile, a large-scale curated dataset created by EleutherAI.
- Chat bot with GPT-J - although it runs on your local machine it still requires an API key from MessengerX.io
- An Instruct Version Of GPT-J Using Stanford Alpaca's Dataset has been produced by NLP Cloud. The model is heavy and standard laptop might not be enough to run it.
- DOLLY - Databricks’ Dolly is a large language model that fine-tuned the (GPT-J) on a focused corpus of 50k records (Stanford Alpaca) and produced high quality instruction following behavior not characteristic of the foundation model on which it is based.
- GPT4ALL
-
Vicuna-13B - an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT
- description
- code github
- requires you to first get the original weights from Facebook of LLaMA and then you need to apply a delta with weights provided by Vicuna. They provided a script that does apply the delta and converts the weights to the latest Hugging Face format.
- Current version is 13B, 7B is not available. It requires either 60 GB of RAM or 28 GB of GPU.
- Baize
- It uses ChatGPT to engage in a conversation with itself in order to automatically generate a high-quality multi-turn chat corpus. It fine-tuned Meta’s open-source large language model, LLaMA, resulting in a new model called Baize. This open-source chat model has exceptional performance and can function with just one GPU
- paper, github, demo
-
Koala - a chatbot trained by fine-tuning Meta’s LLaMA on dialogue data gathered from the web
- 13B
- again based on LLaMA
- dialog fine tuned
- trained with EasyLM
-
ChatGLM
- 6B
- Hugging Chat
- Models can be fine-tuned with xTuring
- openplayground - An LLM playground you can run on your laptop.