Text-to-LoRA: Hypernetwork that generates task-specific LLM adapters (LoRAs)

Summary

Text-to-LoRA (T2L): Instant Transformer Adaption 🛠️ Installation Install uv if you don't have uv (see https://docs.astral.sh/uv/getting-started/installation/) With uv installed, run the following to install the dependencies. git clone https://github.com/SakanaAI/text-to-lora.git cd text-to-lora # make sure you have `uv` installed # (see https://docs.astral.sh/uv/getting-started/installation/) uv self update uv venv --python 3.10 --seed uv sync # we use the following wheel for installation # you might have to change the wheel to be compatible with your hardware uv pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.6.3/flash_attn-2.6.3+cu123torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl uv pip install src/fishfarm 🚀 Demo Downloading trained T2L ⚠️ You need to download the checkpoints before running any of the demos. ⚠️ ⚠️ You need a >16GB GPU to handle both models simultaneously to run any of these demos. ⚠️ uv run huggingface-cli login uv run huggingface-cli download SakanaAI/text-to-lora --local-dir . --include " trained_t2l/* " Web UI This demo runs Mistral-7B-Instruct-v0.2 locally alongside a T2L model. # start webui locally uv run python webui/app.py LoRA generation from CLI This script allows us to generate a LoRA based on a task description. Running for the first time would take longer as the base model will be downloaded and cached. # uv run generate_lora.py {T2l_DIRECTORY} {TASK_DESCRIPTION} # e.g., uv run python scripts/generate_lora.py \ trained_t2l/llama_8b_t2l \ " This task challenges your problem-solving abilities through mathematical reasoning. You must carefully read each scenario and systematically work through the data to compute the final outcome. " # You might be able to run T2L w/ `gemma-2-2b-it` with a smaller GPU. uv run python scripts/generate_lora.py \ trained_t2l/gemma_2b_t2l \ " This task challenges your problem-solving abilities through mathematical reasoning. You must carefully read each scenario and systemat...