Superb-tuning A Tiny-Llama Mannequin with Unsloth

Introduction

After the Llama and Mistral fashions have been launched, the open-source LLMs took the limelight out of OpenAI. Since then, a number of fashions have been launched primarily based on Llama and Mistral structure, acting on par with proprietary fashions like GPT-3.5 Turbo, Claude, Gemini, and so on. Nevertheless, these fashions are too giant for use in shopper {hardware}.

However these days, there was an emergence of a brand new class of LLMs. These are the LLMs within the sub-7B parameter class. Fewer parameters make them compact sufficient to be run in shopper {hardware} whereas retaining effectivity akin to the 7B fashions. Fashions like Tiny-Llama-1B, Microsoft’s Phi-2, and Alibaba’s Qwen-3b might be nice substitutes for bigger fashions to run regionally or deploy on edge. On the identical time, fine-tuning is essential to deliver the very best out of any base mannequin for any downstream duties.
Right here, we are going to discover the right way to Superb-tune a base Tiny-Llama mannequin on a cleaned Alpaca dataset.

Studying Aims

• Perceive fine-tuning and completely different strategies of it.
• Study instruments and strategies for environment friendly fine-tuning.
• Study WandB for logging coaching logs.
• Superb-tune Tiny-Llama on the Alpaca dataset in Colab.

This text was revealed as part of the Knowledge Science Blogathon.

What’s LLM Superb-Tuning?

Superb-tuning is the method of creating a pre-trained mannequin be taught new information. The pre-trained mannequin is a general-purpose mannequin skilled on a considerable amount of knowledge. Nevertheless, usually, they fail to carry out as supposed, and fine-tuning is the best technique to make the mannequin adapt to particular use circumstances. For instance, base LLMs do effectively at textual content technology on single-turn QA however wrestle with multi-turn conversations like chat fashions.

The bottom fashions have to be skilled on transcripts of dialogues to have the ability to maintain multi-turn conversations. Superb-tuning is crucial to mildew pre-trained fashions into completely different avatars. The standard of Superb-tuned fashions is determined by the standard of knowledge and base mannequin capabilities. There are a number of methods to mannequin fine-tuning, like LoRA, QLoRA, and so on.

Let’s briefly undergo these ideas.

LoRA

LoRA stands for Low-rank Adaptation, a preferred fine-tuning method through which we choose just a few trainable parameters as an alternative of updating all of the parameters through a low-rank approximation of unique weight matrices. The LoRA mannequin might be Superb-tuned quicker on much less compute-intensive {hardware}.

QLoRA

QLoRA or Quantized LoRA is a step additional than the LoRA. As a substitute of a full-precision mannequin, it quantizes the mannequin weights to decrease floating level precision earlier than making use of LoRA. Quantization is the method of downcasting greater bit values to decrease values. A 4-bit quantization course of entails quantizing the 16-bit weights to 4-bit float values.

Quantizing the mannequin results in a considerable discount in mannequin dimension with comparable accuracy to the unique mannequin. In QLoRA, we take a quantized mannequin and apply LoRA to it. The fashions might be quantized in a number of methods, corresponding to by way of llama.cpp, AWQ, bitsandbytes, and so on. 

Superb-Tuning with Unsloth

Unsloth is an open-source platform for fine-tuning common Massive Language Fashions quicker. It helps common LLMs, together with Llama-2 and Mistral, and their derivatives like Yi, Open-hermes, and so on. It implements customized triton kernels and a handbook back-prop engine to enhance the pace of the mannequin coaching.

Right here, we are going to use the Unsloth to Superb-tune a base 4-bit quantized Tiny-Llama mannequin on the Alpaca dataset. The mannequin is quantized with bits and bytes, and kernels are optimized with OpenAI’s Triton.

Logging with WandB

In Machine studying, it’s essential to log coaching and analysis metrics. This offers us a whole image of the prepare run. Weights and Biases (WandB) is an open-source library for visualizing and monitoring machine studying experiments. It has a devoted net app for visualizing coaching metrics in real-time. It additionally lets us handle manufacturing fashions centrally. We are going to use WandB solely to trace our Tiny-Llama fine-tuning run. 

To make use of WandB, join a free account and create an API key.

Now, let’s begin fine-tuning our mannequin.

Easy methods to Superb-tune Tiny-Llama?

Superb-tuning is a compute-heavy process. It requires a machine with 10-15 GB of VRAM, or you should utilize Colab’s free Tesla T4 GPU runtime.

Now set up Unsloth and WandB

%%seize
import torch
major_version, minor_version = torch.cuda.get_device_capability()
!pip set up wandb
if major_version >= 8:
    # Use this for brand new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40)
    !pip set up "unsloth[colab_ampere] @ git+https://github.com/unslothai/unsloth.git"
else:
    # Use this for older GPUs (V100, Tesla T4, RTX 20xx)
    !pip set up "unsloth[colab] @ git+https://github.com/unslothai/unsloth.git"
go

The following factor is to load the 4-bit quantized pre-trained mannequin with Unsloth.

from unsloth import FastLanguageModel
import torch
max_seq_length = 4096 # Select any! We auto help RoPE Scaling internally!
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to cut back reminiscence utilization. Could be False.

mannequin, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/tinyllama-bnb-4bit", # "unsloth/tinyllama" for 16bit loading
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

This can set up the mannequin regionally. The 4-bit mannequin dimension can be round 760 MBs.

Now apply PEFT to the 4-bit Tiny-Llama mannequin.

mannequin = FastLanguageModel.get_peft_model(
    mannequin,
    r = 32, # Select any quantity > 0 ! Steered 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 32,
    lora_dropout = 0, # At present solely helps dropout = 0
    bias = "none",    # At present solely helps bias = "none"
    use_gradient_checkpointing = True, # @@@ IF YOU GET OUT OF MEMORY - set to True @@@
    random_state = 3407,
    use_rslora = False,  # We help rank stabilized LoRA
    loftq_config = None, # And LoftQ
)

Put together Knowledge

The following step is to arrange the dataset for fine-tuning. As I discussed earlier, we are going to use a cleaned Alpaca dataset. This can be a cleaned model of the unique Alpaca dataset. It follows the instruction-input-response format. Right here is an instance of Alpaca knowledge

Now, let’s put together our knowledge.

@title put together knowledge

#alpaca_prompt = """Under is an instruction that describes a process, paired with an enter that
 offers additional context.
 Write a response that appropriately completes the request.

### Instruction:
{}

### Enter:
{}

### Response:
{}"""

EOS_TOKEN = tokenizer.eos_token
def formatting_prompts_func(examples):
    directions = examples["instruction"]
    inputs       = examples["input"]
    outputs      = examples["output"]
    texts = []
    for instruction, enter, output in zip(directions, inputs, outputs):
        # Should add EOS_TOKEN, in any other case your technology will go on eternally!
        textual content = alpaca_prompt.format(instruction, enter, output) + EOS_TOKEN
        texts.append(textual content)
    return { "textual content" : texts, }
go

from datasets import load_dataset
dataset = load_dataset("yahma/alpaca-cleaned", cut up = "prepare")
dataset = dataset.map(formatting_prompts_func, batched = True,)

Now, cut up the information into prepare and eval knowledge. I’ve taken small eval knowledge as bigger eval knowledge slows down the coaching.

dataset_dict = dataset.train_test_split(test_size=0.004)

Configure WandB

Now, configure Weights and Biases in your present runtime.

# @title wandb init
import wandb
wandb.login()

Present API key to log in to WandB when prompted.

Arrange surroundings variables.

%env WANDB_WATCH=all
%env WANDB_SILENT=true

Practice Mannequin

Thus far, we now have loaded the 4-bit mannequin, created the LoRA configuration, ready the dataset, and configured WandB. The following step is to coach the mannequin on the information. For that, we have to outline a coach from the Trl library. We are going to use the SFTrainer from Trl. However earlier than that, initialize WandB and outline acceptable coaching arguments.

import os

from trl import SFTTrainer
from transformers import TrainingArguments
from transformers.utils import logging
import wandb

logging.set_verbosity_info()
project_name = "tiny-llama" 
entity = "wandb"
# os.environ["WANDB_LOG_MODEL"] = "checkpoint"

wandb.init(venture=project_name, identify = "tiny-llama-unsloth-sft")

Coaching Arguments

args = TrainingArguments(
        per_device_train_batch_size = 2,
        per_device_eval_batch_size=2,
        gradient_accumulation_steps = 4,
        evaluation_strategy="steps",
        warmup_ratio = 0.1,
        num_train_epochs = 1,
        learning_rate = 2e-5,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        optim = "adamw_8bit",
        weight_decay = 0.1,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
        report_to="wandb",  # allow logging to W&B
        # run_name="tiny-llama-alpaca-run",  # identify of the W&B run (non-compulsory)
        logging_steps=1,  # how typically to log to W&B
        logging_strategy = 'steps',
        save_total_limit=2,
    )

That is necessary for coaching. To maintain GPU utilization low, hold the prepare, eval batch, and gradient accumulating steps low. The logging_steps is the variety of steps earlier than metrics are logged to WandB.

Now, initialize the SFTTrainer.

coach = SFTTrainer(
    mannequin = mannequin,
    tokenizer = tokenizer,
    train_dataset = dataset_dict["train"],
    eval_dataset=dataset_dict["test"],
    dataset_text_field = "textual content",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = True, # Packs quick sequences collectively to save lots of time!
    args = args,
)

Now, begin the coaching.

trainer_stats = coach.prepare()
wandb.end()

Throughout the coaching run, WandB will monitor the coaching and eval metrics. You go to the given dashboard hyperlink and see it in real-time.

This can be a screenshot from my run on a Colab pocket book.

The coaching pace will depend upon a number of elements, together with the coaching and eval knowledge sizes, prepare and eval batch dimension, and the variety of epochs. When you encounter GPU utilization points, strive lowering batch and gradient accumulation step sizes. The prepare batch dimension = batch_size_per_device * gradient_accumulation_steps. And the variety of optimization steps = whole coaching knowledge/batch dimension. You’ll be able to play with the parameters and see which works higher.

You’ll be able to visualize the coaching and analysis lack of your coaching on the WandB dashboard.

Practice Loss

Eval Loss

Inferencing

It can save you the LoRA adapters regionally or push them to the HuggingFace Repository.

mannequin.save_pretrained("lora_model") # Native saving
# mannequin.push_to_hub("your_name/lora_model", token = "...") # On-line saving

You may as well load the saved mannequin from the disk and use it for inferencing.

if False:
    from unsloth import FastLanguageModel
    mannequin, tokenizer = FastLanguageModel.from_pretrained(
        model_name = "lora_model", # YOUR MODEL YOU USED FOR TRAINING
        max_seq_length = max_seq_length,
        dtype = dtype,
        load_in_4bit = load_in_4bit,
    )

inputs = tokenizer(
[
    alpaca_prompt.format(
        "capital of France?", # instruction
        "", # input
        "", # output - leave this blank for a generation!
    )
]*1, return_tensors = "pt").to("cuda")

outputs = mannequin.generate(**inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

For streaming Mannequin responses.

from transformers import TextStreamer

text_streamer = TextStreamer(tokenizer)
_ = mannequin.generate(**inputs, streamer = text_streamer, max_new_tokens = 64)

So, this was all about fine-tuning a Tiny-Llama mannequin with WandB logging.

Right here is the Colab Pocket book for a similar.

Conclusion

Small LLMs might be helpful for deploying on compute-restricted {hardware}, corresponding to private computer systems, cell phones, and different wearables, and so on. Superb-tuning permits these fashions to carry out higher on downstream duties. On this article, we discovered the right way to Superb-tune a base language mannequin on a dataset.

Key Takeaways

• Superb-tuning is the method of creating a pre-trained mannequin adapt to a selected new process.
• Tiny-Llama is an LLM with only one.1 billion parameters and is skilled on 3 trillion tokens.
• There are other ways to Superb-tune LLMs, like LoRA and QLoRA.
• Unsloth is an open-source platform that gives CUDA-optimized LLMs to hurry up fine-tuning LLMs.
• Weights and Biases (WandB) is a device for monitoring and storing ML experiments.

Often Requested Questions

The media proven on this article just isn’t owned by Analytics Vidhya and is used on the Creator’s discretion.

Associated