jadeGPT

This is a conversion of nanoGPT to Jupyter Notebook and Gradio web ui app. Big thanks to Andrej Karpathy. You should watch his video here: https://www.youtube.com/watch?v=kCc8FmEb1nY

Gradio web app

Installation

1. Install git
2. Install python 3.10
3. Install Jupyter Notebook or Gradio
4. Dependencies:

• pytorch
• numpy
• pip install transformers
• pip install datasets
• pip install tiktoken

5. In terminal: git clone https://github.com/phatdatnguyen/jadeGPT

Quick guide

For Gradio

• Open jadeGPT folder
• In terminal: python jadegpt_ui.py
• Use the Train tab for training a new GPT model using a custom dataset
• Use the Fine-tune tab for fine-tuning a checkpoint or a pretrained GPT2 model
• If you only want to generate text from a checkpoint or a pretrained GPT2 model, use the Fine-tune tab to load a model and start generating text

For Jupyter Notebook

• Open Jupyter Notebook and navigate to the jadeGPT folder
• Use train-gpt.ipynb to train a new GPT model using a custom dataset
• Use finetune-gpt.ipynb to finetune a checkpoint or a pretrained GPT2 model
• Use sample-gpt.ipynb to generate text from a checkpoint
• Use sample-gpt2.ipynb to generate text from a pretrained GPT2 model