Running Runpod on SkyPilot

Use SkyPilot to run LLMs, AI, and batch jobs on Runpod Pods and Serverless endpoints.

SkyPilot is a framework for running LLMs, AI, and batch jobs on any cloud.

This integration uses the Runpod CLI infrastructure to spin up on-demand Pods and deploy Serverless endpoints with SkyPilot.

Get started

1. Obtain your API key

   Get your API key from the Runpod Settings page.

2. Install Runpod

   Install the latest version of Runpod:

   pip install "runpod>=1.6"
   
3. Configure Runpod

   Run runpod config and paste your API key when prompted.

4. Install SkyPilot Runpod Cloud

   Install the SkyPilot Runpod cloud:

   pip install "skypilot-nightly[runpod]"
   
5. Verify your setup

   Run sky check to verify your credentials are set up correctly.

Run a project

1. Create a new project directory

   Create a new directory for your project:

   mkdir hello-sky
   cd hello-sky
   
2. Create a configuration file

   Create a file named hello_sky.yaml with the following content:

   resources:
     cloud: runpod
   
   # Working directory (optional) containing the project codebase.
   # Its contents are synced to ~/sky_workdir/ on the cluster.
   workdir: .
   
   # Setup commands (optional).
   # Typical use: pip install -r requirements.txt
   # Invoked under the workdir (i.e., can use its files).
   setup: |
     echo "Running setup."
   
   # Run commands.
   # Typical use: make use of resources, such as running training.
   # Invoked under the workdir (i.e., can use its files).
   run: |
     echo "Hello, SkyPilot!"
     conda env list
   
3. Launch your project

   Launch your project on the cluster:

   sky launch -c mycluster hello_sky.yaml
   
4. Confirm your GPU type

   You’ll see the available GPU options. Confirm your GPU type and the cluster will start spinning up.