diff --git a/cli/README.md b/cli/README.md index 131e7f3e2..6f2a0b2ef 100644 --- a/cli/README.md +++ b/cli/README.md @@ -2,52 +2,105 @@ A powerful command-line interface for managing semantic models in Buster. Deploy and manage your data models with ease, whether they're standalone or part of a dbt project. -## Features - -- 🚀 Deploy semantic models directly from YAML files -- 🔄 Two-way compatibility with dbt projects -- 🎯 Simple configuration with smart defaults -- 📊 Support for dimensions, measures, and relationships -- 🛠️ Automatic SQL view generation -- 📝 Clear error reporting and validation - ## Installation +Choose the installation command for your operating system: + +### macOS (x86_64) ```bash -cargo install buster-cli +curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-darwin-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster ``` -## Quick Start +### Linux (x86_64) +```bash +curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-linux-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster +``` -1. **Authentication** +### Windows (x86_64) +1. Download the Windows binary: +```powershell +Invoke-WebRequest -Uri https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-windows-x86_64.zip -OutFile buster.zip +``` - Get your API key from [Buster Platform](https://platform.buster.so/app/settings/api-keys) and authenticate: +2. Extract and install: +```powershell +Expand-Archive -Path buster.zip -DestinationPath $env:USERPROFILE\buster +Move-Item -Path $env:USERPROFILE\buster\buster-cli.exe -Destination $env:LOCALAPPDATA\Microsoft\WindowsApps\buster.exe +``` - ```bash - buster auth - ``` +## Quick Start Guide -2. **Project Setup** +### 1. Authentication - Initialize a new project: +First, authenticate with Buster using your API key: - ```bash - buster init - ``` +```bash +buster auth +``` - This will: - - Create a `buster.yml` configuration file - - Set up the recommended directory structure - - Configure your data source connection +This will prompt you for: +- API Key (required) - Get this from the Buster Platform +- Host (optional) - Defaults to production if not specified -3. **Deploy Models** +### 2. Generate Models - ```bash - buster deploy - ``` +Generate Buster YAML models from your existing SQL files: + +```bash +buster generate +``` + +Key flags for generation: +- `--source-path`: Directory containing your SQL files (defaults to current directory) +- `--destination-path`: Where to output the generated YAML files (defaults to current directory) +- `--data-source-name`: Name of the data source to use in the models +- `--schema`: Database schema name +- `--database`: Database name + +The generate command will: +- Scan the source directory for SQL files +- Create corresponding YAML model files +- Create a `buster.yml` configuration file if it doesn't exist +- Preserve any existing model customizations + +Example with all options: +```bash +buster generate \ + --source-path ./sql \ + --destination-path ./models \ + --data-source-name my_warehouse \ + --schema analytics \ + --database prod +``` + +### 3. Deploy Models + +Deploy your models to Buster: + +```bash +buster deploy +``` + +Deploy options: +- `--path`: Specific path to deploy (defaults to current directory) +- `--dry-run`: Validate the deployment without actually deploying (defaults to false) + +Examples: +```bash +# Deploy all models in current directory +buster deploy + +# Deploy a specific model or directory +buster deploy --path ./models/customers.yml + +# Validate deployment without applying changes +buster deploy --dry-run +``` ## Project Structure +A typical Buster project structure: + ``` your-project/ ├── buster.yml # Global configuration @@ -55,15 +108,13 @@ your-project/ │ ├── customers.yml │ ├── orders.yml │ └── products.yml -└── sql/ # SQL definitions (optional) +└── sql/ # SQL definitions ├── customers.sql ├── orders.sql └── products.sql ``` -## Configuration - -### Global Configuration (buster.yml) +### Configuration (buster.yml) ```yaml # buster.yml @@ -71,7 +122,7 @@ data_source_name: "my_warehouse" # Your default data source schema: "analytics" # Default schema for models ``` -### Model Definition (models/*.yml) +### Model Definition Example ```yaml # models/customers.yml @@ -79,29 +130,21 @@ version: 1 models: - name: customers description: "Core customer data model" - data_source_name: "my_warehouse" # Optional, overrides global - schema: "analytics" # Optional, overrides global + data_source_name: "my_warehouse" + schema: "analytics" - # Define entities (for relationships) entities: - name: customer_id expr: "id" type: "primary" description: "Primary customer identifier" - # Define dimensions dimensions: - name: email expr: "email" type: "string" description: "Customer email address" - - name: signup_date - expr: "created_at::date" - type: "date" - description: "Date when customer signed up" - searchable: true # Enable value caching - # Define measures measures: - name: total_customers expr: "customer_id" @@ -109,121 +152,6 @@ models: description: "Total number of unique customers" ``` -### SQL Definition (sql/*.sql) - -```sql --- sql/customers.sql -SELECT - id as customer_id, - email, - created_at -FROM raw.customers -WHERE deleted_at IS NULL -``` - -## Commands - -### `buster deploy` - -Deploy your semantic models to Buster. - -```bash -# Deploy all models in the current directory -buster deploy - -# Deploy a specific model -buster deploy models/customers.yml - -# Deploy models in a specific directory -buster deploy models/ -``` - -The deploy command will: -1. Find and validate all model files -2. Locate corresponding SQL files -3. Generate default SQL if none exists -4. Deploy to Buster with proper error handling - -### `buster auth` - -Manage your Buster authentication. - -```bash -# Set up authentication -buster auth - -# View current auth status -buster auth status - -# Clear authentication -buster auth clear -``` - -### `buster init` - -Initialize a new Buster project. - -```bash -# Initialize in current directory -buster init - -# Initialize in a specific directory -buster init my-project/ -``` - -## Model Features - -### Entity Relationships - -Link models together using entity relationships: - -```yaml -# models/orders.yml -models: - - name: orders - entities: - - name: customers # References customers.yml - expr: "customer_id" - type: "foreign" - description: "Link to customer" -``` - -### Stored Values - -Enable value caching for better performance: - -```yaml -dimensions: - - name: country - expr: "country_code" - type: "string" - searchable: true # Values will be cached -``` - -### Default SQL Generation - -If no SQL file exists, Buster generates a default SELECT statement: - -```sql -SELECT * FROM schema.model_name -``` - -## Error Handling - -The CLI provides clear error messages for common issues: - -- Missing required fields -- Invalid relationships -- SQL syntax errors -- API communication issues -- Authentication problems - -Example error output: -``` -❌ Error processing customers.yml: data_source_name is required -⚠️ Warning: No SQL file found for 'customers', using default SELECT -``` - ## Best Practices 1. **Organization** @@ -231,50 +159,34 @@ Example error output: - Keep SQL files in `sql/` - Use `buster.yml` for shared settings -2. **Naming** - - Use descriptive model names - - Match SQL and YAML file names - - Use lowercase with underscores +2. **Model Generation** + - Start with clean SQL files + - Generate models first before customizing + - Review generated models before deployment -3. **Documentation** - - Add descriptions to all models - - Document dimensions and measures - - Explain relationships - -4. **SQL** - - Keep SQL simple and focused - - Use CTEs for complex logic - - Add comments for clarity - -## Known Limitations - -- SQL files must be one directory up from YAML files -- Environment is fixed to "dev" -- No automatic relationship creation -- Simple SQL fallback may not suit all cases +3. **Deployment** + - Use `--dry-run` to validate changes + - Deploy frequently to catch issues early + - Keep model and SQL files in sync ## Troubleshooting -### Common Issues +Common issues and solutions: -1. **"Data source not found"** - - Verify data source exists in Buster - - Check data_source_name in config - - Ensure env='dev' is set +1. **Authentication Issues** + - Verify your API key is correct + - Check if the host is properly specified (if using non-production) + - Ensure network connectivity to Buster -2. **"SQL file not found"** - - Check SQL file location - - Verify file naming matches - - Consider using default SQL +2. **Generation Issues** + - Verify SQL files are in the correct location + - Check file permissions + - Ensure SQL syntax is valid -3. **"Invalid relationship"** - - Verify referenced model exists - - Check entity name matches - - Ensure proper file structure - -## Contributing - -We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. +3. **Deployment Issues** + - Validate YAML syntax + - Check for missing dependencies + - Verify data source connectivity ## License