mirror of https://github.com/buster-so/buster.git
readme for cli
This commit is contained in:
dal
parent
946121fad4
commit
5b6ff85af8
304
cli/README.md
304
cli/README.md
|
|
@ -2,52 +2,105 @@
|
|||
|
||||
A powerful command-line interface for managing semantic models in Buster. Deploy and manage your data models with ease, whether they're standalone or part of a dbt project.
|
||||
|
||||
## Features
|
||||
|
||||
- 🚀 Deploy semantic models directly from YAML files
|
||||
- 🔄 Two-way compatibility with dbt projects
|
||||
- 🎯 Simple configuration with smart defaults
|
||||
- 📊 Support for dimensions, measures, and relationships
|
||||
- 🛠️ Automatic SQL view generation
|
||||
- 📝 Clear error reporting and validation
|
||||
|
||||
## Installation
|
||||
|
||||
Choose the installation command for your operating system:
|
||||
|
||||
### macOS (x86_64)
|
||||
```bash
|
||||
cargo install buster-cli
|
||||
curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-darwin-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
### Linux (x86_64)
|
||||
```bash
|
||||
curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-linux-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster
|
||||
```
|
||||
|
||||
1. **Authentication**
|
||||
### Windows (x86_64)
|
||||
1. Download the Windows binary:
|
||||
```powershell
|
||||
Invoke-WebRequest -Uri https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-windows-x86_64.zip -OutFile buster.zip
|
||||
```
|
||||
|
||||
Get your API key from [Buster Platform](https://platform.buster.so/app/settings/api-keys) and authenticate:
|
||||
2. Extract and install:
|
||||
```powershell
|
||||
Expand-Archive -Path buster.zip -DestinationPath $env:USERPROFILE\buster
|
||||
Move-Item -Path $env:USERPROFILE\buster\buster-cli.exe -Destination $env:LOCALAPPDATA\Microsoft\WindowsApps\buster.exe
|
||||
```
|
||||
|
||||
```bash
|
||||
buster auth
|
||||
```
|
||||
## Quick Start Guide
|
||||
|
||||
2. **Project Setup**
|
||||
### 1. Authentication
|
||||
|
||||
Initialize a new project:
|
||||
First, authenticate with Buster using your API key:
|
||||
|
||||
```bash
|
||||
buster init
|
||||
```
|
||||
```bash
|
||||
buster auth
|
||||
```
|
||||
|
||||
This will:
|
||||
- Create a `buster.yml` configuration file
|
||||
- Set up the recommended directory structure
|
||||
- Configure your data source connection
|
||||
This will prompt you for:
|
||||
- API Key (required) - Get this from the Buster Platform
|
||||
- Host (optional) - Defaults to production if not specified
|
||||
|
||||
3. **Deploy Models**
|
||||
### 2. Generate Models
|
||||
|
||||
```bash
|
||||
buster deploy
|
||||
```
|
||||
Generate Buster YAML models from your existing SQL files:
|
||||
|
||||
```bash
|
||||
buster generate
|
||||
```
|
||||
|
||||
Key flags for generation:
|
||||
- `--source-path`: Directory containing your SQL files (defaults to current directory)
|
||||
- `--destination-path`: Where to output the generated YAML files (defaults to current directory)
|
||||
- `--data-source-name`: Name of the data source to use in the models
|
||||
- `--schema`: Database schema name
|
||||
- `--database`: Database name
|
||||
|
||||
The generate command will:
|
||||
- Scan the source directory for SQL files
|
||||
- Create corresponding YAML model files
|
||||
- Create a `buster.yml` configuration file if it doesn't exist
|
||||
- Preserve any existing model customizations
|
||||
|
||||
Example with all options:
|
||||
```bash
|
||||
buster generate \
|
||||
--source-path ./sql \
|
||||
--destination-path ./models \
|
||||
--data-source-name my_warehouse \
|
||||
--schema analytics \
|
||||
--database prod
|
||||
```
|
||||
|
||||
### 3. Deploy Models
|
||||
|
||||
Deploy your models to Buster:
|
||||
|
||||
```bash
|
||||
buster deploy
|
||||
```
|
||||
|
||||
Deploy options:
|
||||
- `--path`: Specific path to deploy (defaults to current directory)
|
||||
- `--dry-run`: Validate the deployment without actually deploying (defaults to false)
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
# Deploy all models in current directory
|
||||
buster deploy
|
||||
|
||||
# Deploy a specific model or directory
|
||||
buster deploy --path ./models/customers.yml
|
||||
|
||||
# Validate deployment without applying changes
|
||||
buster deploy --dry-run
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
A typical Buster project structure:
|
||||
|
||||
```
|
||||
your-project/
|
||||
├── buster.yml # Global configuration
|
||||
|
|
@ -55,15 +108,13 @@ your-project/
|
|||
│ ├── customers.yml
|
||||
│ ├── orders.yml
|
||||
│ └── products.yml
|
||||
└── sql/ # SQL definitions (optional)
|
||||
└── sql/ # SQL definitions
|
||||
├── customers.sql
|
||||
├── orders.sql
|
||||
└── products.sql
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Global Configuration (buster.yml)
|
||||
### Configuration (buster.yml)
|
||||
|
||||
```yaml
|
||||
# buster.yml
|
||||
|
|
@ -71,7 +122,7 @@ data_source_name: "my_warehouse" # Your default data source
|
|||
schema: "analytics" # Default schema for models
|
||||
```
|
||||
|
||||
### Model Definition (models/*.yml)
|
||||
### Model Definition Example
|
||||
|
||||
```yaml
|
||||
# models/customers.yml
|
||||
|
|
@ -79,29 +130,21 @@ version: 1
|
|||
models:
|
||||
- name: customers
|
||||
description: "Core customer data model"
|
||||
data_source_name: "my_warehouse" # Optional, overrides global
|
||||
schema: "analytics" # Optional, overrides global
|
||||
data_source_name: "my_warehouse"
|
||||
schema: "analytics"
|
||||
|
||||
# Define entities (for relationships)
|
||||
entities:
|
||||
- name: customer_id
|
||||
expr: "id"
|
||||
type: "primary"
|
||||
description: "Primary customer identifier"
|
||||
|
||||
# Define dimensions
|
||||
dimensions:
|
||||
- name: email
|
||||
expr: "email"
|
||||
type: "string"
|
||||
description: "Customer email address"
|
||||
- name: signup_date
|
||||
expr: "created_at::date"
|
||||
type: "date"
|
||||
description: "Date when customer signed up"
|
||||
searchable: true # Enable value caching
|
||||
|
||||
# Define measures
|
||||
measures:
|
||||
- name: total_customers
|
||||
expr: "customer_id"
|
||||
|
|
@ -109,121 +152,6 @@ models:
|
|||
description: "Total number of unique customers"
|
||||
```
|
||||
|
||||
### SQL Definition (sql/*.sql)
|
||||
|
||||
```sql
|
||||
-- sql/customers.sql
|
||||
SELECT
|
||||
id as customer_id,
|
||||
email,
|
||||
created_at
|
||||
FROM raw.customers
|
||||
WHERE deleted_at IS NULL
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### `buster deploy`
|
||||
|
||||
Deploy your semantic models to Buster.
|
||||
|
||||
```bash
|
||||
# Deploy all models in the current directory
|
||||
buster deploy
|
||||
|
||||
# Deploy a specific model
|
||||
buster deploy models/customers.yml
|
||||
|
||||
# Deploy models in a specific directory
|
||||
buster deploy models/
|
||||
```
|
||||
|
||||
The deploy command will:
|
||||
1. Find and validate all model files
|
||||
2. Locate corresponding SQL files
|
||||
3. Generate default SQL if none exists
|
||||
4. Deploy to Buster with proper error handling
|
||||
|
||||
### `buster auth`
|
||||
|
||||
Manage your Buster authentication.
|
||||
|
||||
```bash
|
||||
# Set up authentication
|
||||
buster auth
|
||||
|
||||
# View current auth status
|
||||
buster auth status
|
||||
|
||||
# Clear authentication
|
||||
buster auth clear
|
||||
```
|
||||
|
||||
### `buster init`
|
||||
|
||||
Initialize a new Buster project.
|
||||
|
||||
```bash
|
||||
# Initialize in current directory
|
||||
buster init
|
||||
|
||||
# Initialize in a specific directory
|
||||
buster init my-project/
|
||||
```
|
||||
|
||||
## Model Features
|
||||
|
||||
### Entity Relationships
|
||||
|
||||
Link models together using entity relationships:
|
||||
|
||||
```yaml
|
||||
# models/orders.yml
|
||||
models:
|
||||
- name: orders
|
||||
entities:
|
||||
- name: customers # References customers.yml
|
||||
expr: "customer_id"
|
||||
type: "foreign"
|
||||
description: "Link to customer"
|
||||
```
|
||||
|
||||
### Stored Values
|
||||
|
||||
Enable value caching for better performance:
|
||||
|
||||
```yaml
|
||||
dimensions:
|
||||
- name: country
|
||||
expr: "country_code"
|
||||
type: "string"
|
||||
searchable: true # Values will be cached
|
||||
```
|
||||
|
||||
### Default SQL Generation
|
||||
|
||||
If no SQL file exists, Buster generates a default SELECT statement:
|
||||
|
||||
```sql
|
||||
SELECT * FROM schema.model_name
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The CLI provides clear error messages for common issues:
|
||||
|
||||
- Missing required fields
|
||||
- Invalid relationships
|
||||
- SQL syntax errors
|
||||
- API communication issues
|
||||
- Authentication problems
|
||||
|
||||
Example error output:
|
||||
```
|
||||
❌ Error processing customers.yml: data_source_name is required
|
||||
⚠️ Warning: No SQL file found for 'customers', using default SELECT
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Organization**
|
||||
|
|
@ -231,50 +159,34 @@ Example error output:
|
|||
- Keep SQL files in `sql/`
|
||||
- Use `buster.yml` for shared settings
|
||||
|
||||
2. **Naming**
|
||||
- Use descriptive model names
|
||||
- Match SQL and YAML file names
|
||||
- Use lowercase with underscores
|
||||
2. **Model Generation**
|
||||
- Start with clean SQL files
|
||||
- Generate models first before customizing
|
||||
- Review generated models before deployment
|
||||
|
||||
3. **Documentation**
|
||||
- Add descriptions to all models
|
||||
- Document dimensions and measures
|
||||
- Explain relationships
|
||||
|
||||
4. **SQL**
|
||||
- Keep SQL simple and focused
|
||||
- Use CTEs for complex logic
|
||||
- Add comments for clarity
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- SQL files must be one directory up from YAML files
|
||||
- Environment is fixed to "dev"
|
||||
- No automatic relationship creation
|
||||
- Simple SQL fallback may not suit all cases
|
||||
3. **Deployment**
|
||||
- Use `--dry-run` to validate changes
|
||||
- Deploy frequently to catch issues early
|
||||
- Keep model and SQL files in sync
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
Common issues and solutions:
|
||||
|
||||
1. **"Data source not found"**
|
||||
- Verify data source exists in Buster
|
||||
- Check data_source_name in config
|
||||
- Ensure env='dev' is set
|
||||
1. **Authentication Issues**
|
||||
- Verify your API key is correct
|
||||
- Check if the host is properly specified (if using non-production)
|
||||
- Ensure network connectivity to Buster
|
||||
|
||||
2. **"SQL file not found"**
|
||||
- Check SQL file location
|
||||
- Verify file naming matches
|
||||
- Consider using default SQL
|
||||
2. **Generation Issues**
|
||||
- Verify SQL files are in the correct location
|
||||
- Check file permissions
|
||||
- Ensure SQL syntax is valid
|
||||
|
||||
3. **"Invalid relationship"**
|
||||
- Verify referenced model exists
|
||||
- Check entity name matches
|
||||
- Ensure proper file structure
|
||||
|
||||
## Contributing
|
||||
|
||||
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
|
||||
3. **Deployment Issues**
|
||||
- Validate YAML syntax
|
||||
- Check for missing dependencies
|
||||
- Verify data source connectivity
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue