Skip to content

Latest commit

 

History

History
16 lines (14 loc) · 1.29 KB

llm-notebook-setup.md

File metadata and controls

16 lines (14 loc) · 1.29 KB

LLM Notebook Setup

A quick guide for getting set for fine-tuning or inference using a jupyter notebook.

You have a few options for running fine-tuning notebooks:

  1. Hosted service (Recommended), e.g. Runpod or Vast.ai:
  • Runpod one-click template here - easier setup.
    • To support the Trelis Research YouTube channel, you can first sign up for an account with this link.
  • Vast.ai one-click template here - offers smaller GPUs (which are cheaper to run).
    • To support the Trelis Research YouTube channel, you can first sign up for an account with this affiliate link.
  1. Google Colab (free and good for 7B models or smaller):
  • Upload the .ipynb notebook
  • Select a T4 GPU from Runtime -> Change Runtime Type.
  • make sure to comment out flash attention when loading the model.
  1. Your own computer (assuming you have an AMD or Nvidia GPU) - ADVANCED:
  • Set up jupyter lab and a virtual environment using the instructions in the 'jupyter-lab-setup.md' file of this repo. see here.