InstructLab Deep Dive Part 3

Index:
InstructLab Deep Dive Part 1
What is InstructLab and what problem does it solve?

InstructLab Deep Dive Part 2
Installing InstructLab on Fedora 40 with enabled CUDA.

InstructLab Deep Dive Part 3
Installing InstructLab on Fedora 40 with Ansible

InstructLab Deep Dive Part 4
Building InstructLab Podman/Docker container and running it on runpod.io

InstructLab Deep Dive Part 5
Advanced techniques with InstructLab, AI Agents and Function calling

Installing InstructLab on Fedora 40 with Ansible

Below is an Ansible Playbook that automates the installation of NVIDIA drivers, CUDA, necessary packages, and InstructLab on Fedora 39/40. The playbook is structured to follow the steps outlined in a Deep Dive Part 2.

---
- name: Setup InstructLab on Fedora 39/40 VM
  hosts: all
  become: yes

  tasks:

############# Install and Update packages #############

    - name: Install and update Fedora 39/40
      dnf:
        name: '*'
        state: latest

    - name: Ensure Development Tools and necessary packages are installed
      ansible.builtin.dnf:
        name:
          - "@Development Tools"
          - gcc
          - gcc-c++
          - clang17
          - make
          - git
          - wget
          - vim
          - curl
          - podman
        state: present
      notify: start_enable_podman


############# Install NVidia Driver and CUDA packages #############

    - name: Blacklist nouveau driver
      lineinfile:
        path: /etc/modprobe.d/blacklist.conf
        line: 'blacklist nouveau'
      notify: "reboot_now"

    - name: Add NVIDIA CUDA repository
      dnf_repository:
        name: nvidia-cuda
        description: NVIDIA CUDA repo
        baseurl: https://developer.download.nvidia.com/compute/cuda/repos/fedora39/x86_64/
        enabled: yes
        gpgcheck: no

    - name: Install NVIDIA driver and utilities
      ansible.builtin.dnf:
        name:
          - nvidia-driver
          - nvidia-modprobe
          - dnf-plugin-nvidia
        state: present
      notify: "reboot_now"

    - name: Install NVIDIA CUDA support packages and CUDA Toolkit 12.4
      ansible.builtin.dnf:
        name:
          - nvidia-driver-cuda
          - nvidia-driver-cuda-libs
          - nvidia-driver-NVML
          - cuda-toolkit-12-4
          - nvtop
        state: present

    - name: Configure environment variables for CUDA
      blockinfile:
        path: ~/.bashrc
        block: |
          export CUDA_HOME=/usr/local/cuda
          export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
          export PATH=$PATH:/usr/local/cuda/bin
          export CUDACXX=/usr/local/cuda-12/bin/nvcc
          export CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=all-major"
          export FORCE_CMAKE=1
        marker: "# {mark} ANSIBLE MANAGED BLOCK"
        notify: "reboot_now"


############# Install NVidia Container Toolkit packages #############

    - name: Configure NVIDIA Container Toolkit repository
      ansible.builtin.shell: |
        curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
        sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

    - name: Install NVIDIA Container Toolkit
      dnf:
        name:
          - nvidia-container-toolkit
        state: present

    - name: Generate CDI specification for NVIDIA container
      ansible.builtin.command:
        cmd: "sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml"
        creates: /etc/cdi/nvidia.yaml


############# Install mini Conda #############

    - name: Setup mini conda for Python version management
      block:
        - command: wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
        - command: bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
        - command: rm ~/miniconda3/miniconda.sh
        - command: ~/miniconda3/bin/conda init bash

    - name: Create Python 3.11 environment for InstructLab
      command: conda create -n instructlab python=3.11
      environment:
        PATH: "/root/miniconda3/bin:{{ ansible_env.PATH }}"


############# Install InstructLab #############

    - name: Install InstructLab with CUDA support
      command: CUDAHOSTCXX=$(which clang++-17) pip install instructlab[cuda] -C cmake.args="-DLLAMA_CUDA=on"
      environment:
        PATH: "/root/miniconda3/envs/instructlab/bin:{{ ansible_env.PATH }}"


############# Handlers #############

  handlers:
    - name: reboot machine
      reboot:
        reboot_timeout: 240
        connect_timeout: 120
        pre_reboot_delay: 5
        post_reboot_delay: 60
      listen: "reboot_now"

    - name: start_enable_podman
      ansible.builtin.systemd:
        name: podman
        state: started
        enabled: true

Explanation:

• Development Tools and Packages: The playbook starts by installing the necessary development tools and additional packages such as gcc, clang17, and make.
• NVIDIA Driver Blacklisting: It blacklists the nouveau driver to ensure it doesn’t interfere with the NVIDIA driver installation.
• NVIDIA Repository and Driver Installation: It then adds the NVIDIA CUDA repository, installs the NVIDIA driver, and sets up the NVIDIA container toolkit.
• CUDA Paths Setup: Finally, it adds necessary CUDA paths to the user’s bash profile, ensuring the environment is configured correctly for CUDA usage.
• Miniconda Installation: The playbook downloads and installs Miniconda, then initializes it for bash shell.
• Python Environment for InstructLab: It creates a Python 3.11 environment specifically for InstructLab, ensuring compatibility with the required Python version.
• InstructLab Installation: The playbook installs InstructLab with CUDA support using the Clang 17 compiler.

This playbook automates the entire installation process, setting up everything needed for InstructLab on Fedora 40, minus the steps for initializing InstructLab and downloading or serving models.