buster/cli/README.md

Buster CLI

A powerful command-line interface for managing semantic models in Buster. Deploy and manage your data models with ease, whether they're standalone or part of a dbt project.

Installation

Choose the installation command for your operating system:

macOS (x86_64)

mkdir -p ~/.local/bin && curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-darwin-x86_64.tar.gz | tar xz && mv buster-cli ~/.local/bin/buster && chmod +x ~/.local/bin/buster

macOS (ARM/Apple Silicon)

mkdir -p ~/.local/bin && curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-darwin-arm64.tar.gz | tar xz && mv buster-cli ~/.local/bin/buster && chmod +x ~/.local/bin/buster

Linux (x86_64)

mkdir -p ~/.local/bin && curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-linux-x86_64.tar.gz | tar xz && mv buster-cli ~/.local/bin/buster && chmod +x ~/.local/bin/buster

Note: After installation, make sure ~/.local/bin is in your PATH. Add this to your shell's config file (.bashrc, .zshrc, etc.):

export PATH="$HOME/.local/bin:$PATH"

Windows (x86_64)

1. Download the Windows binary:

Invoke-WebRequest -Uri https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-windows-x86_64.zip -OutFile buster.zip

2. Extract and install:

Expand-Archive -Path buster.zip -DestinationPath $env:USERPROFILE\buster
Move-Item -Path $env:USERPROFILE\buster\buster-cli.exe -Destination $env:LOCALAPPDATA\Microsoft\WindowsApps\buster.exe

Quick Start Guide

1. Authentication

First, authenticate with Buster using your API key:

buster auth

This will prompt you for:

• API Key (required) - Get this from the Buster Platform
• Host (optional) - Defaults to production if not specified

You can also configure authentication using environment variables:

# Set API key via environment variable
export BUSTER_API_KEY=your_api_key_here

# Optional: Set custom host. For self-hosted instances.
export BUSTER_HOST=your_custom_host

The CLI will check for these environment variables in the following order:

1. Command line arguments
2. Environment variables
3. Interactive prompt

This is particularly useful for:

• CI/CD environments
• Automated scripts
• Development workflows where you don't want to enter credentials repeatedly

2. Generate Models

Generate Buster YAML models from your existing SQL files:

buster generate

Key flags for generation:

• --source-path: Directory containing your SQL files (defaults to current directory)
• --destination-path: Where to output the generated YAML files (defaults to current directory)
• --data-source-name: Name of the data source to use in the models
• --schema: Database schema name
• --database: Database name

The generate command will:

• Scan the source directory for SQL files
• Create corresponding YAML model files
• Create a buster.yml configuration file if it doesn't exist
• Preserve any existing model customizations

Example with all options:

buster generate \
  --source-path ./sql \
  --destination-path ./models \
  --data-source-name my_warehouse \
  --schema analytics \
  --database prod

3. Deploy Models

Deploy your models to Buster:

buster deploy

Deploy options:

• --path: Specific path to deploy (defaults to current directory)
• --dry-run: Validate the deployment without actually deploying (defaults to false)

Examples:

# Deploy all models in current directory
buster deploy

# Deploy a specific model or directory
buster deploy --path ./models/customers.yml

# Validate deployment without applying changes
buster deploy --dry-run

Project Structure

A typical Buster project structure:

your-project/
├── buster.yml          # Global configuration
├── models/            # Your semantic model definitions
│   ├── customers.yml
│   ├── orders.yml
│   └── products.yml
└── sql/              # SQL definitions
    ├── customers.sql
    ├── orders.sql
    └── products.sql

Configuration (buster.yml)

# buster.yml
data_source_name: "my_warehouse"  # Your default data source
schema: "analytics"               # Default schema for models
database: "prod"                  # Optional database name
exclude_files:                    # Optional list of files to exclude from generation
  - "temp_*.sql"                 # Exclude all SQL files starting with temp_
  - "test/**/*.sql"             # Exclude all SQL files in test directories
  - "customers.sql"         # Exclude a specific file

The configuration supports the following fields:

• data_source_name: (Required) Default data source for your models
• schema: (Required) Default schema for your models
• database: (Optional) Default database name
• exclude_files: (Optional) List of glob patterns for files to exclude from generation
  • Supports standard glob patterns (*, **, ?, etc.)
  • Matches against relative paths from source directory
  • Common use cases:
    • Excluding temporary files: temp_*.sql
    • Excluding test files: test/**/*.sql
    • Excluding specific files: customers.sql
    • Excluding files in directories: archive/**/*.sql

Model Definition Example

# models/customers.yml
version: 1
models:
  - name: customers
    description: "Core customer data model"
    data_source_name: "my_warehouse"
    schema: "analytics"
    
    entities:
      - name: customer_id
        expr: "id"
        type: "primary"
        description: "Primary customer identifier"
    
    dimensions:
      - name: email
        expr: "email"
        type: "string"
        description: "Customer email address"
    
    measures:
      - name: total_customers
        expr: "customer_id"
        agg: "count_distinct"
        description: "Total number of unique customers"

Best Practices

1. Organization

   • Keep YAML files in models/
   • Keep SQL files in sql/
   • Use buster.yml for shared settings

2. Model Generation

   • Start with clean SQL files
   • Generate models first before customizing
   • Review generated models before deployment

3. Deployment

   • Use --dry-run to validate changes
   • Deploy frequently to catch issues early
   • Keep model and SQL files in sync

Troubleshooting

Common issues and solutions:

1. Authentication Issues

   • Verify your API key is correct
   • Check if the host is properly specified (if using non-production)
   • Ensure network connectivity to Buster

2. Generation Issues

   • Verify SQL files are in the correct location
   • Check file permissions
   • Ensure SQL syntax is valid

3. Deployment Issues

   • Validate YAML syntax
   • Check for missing dependencies
   • Verify data source connectivity

License

MIT License - see LICENSE for details.