Ling0925
/
buster
mirror of https://github.com/buster-so/buster.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

readme for cli

This commit is contained in:
dal 2025-02-12 10:00:26 -07:00
parent 946121fad4
commit 5b6ff85af8
No known key found for this signature in database
GPG Key ID: 16F4B0E1E9F61122
1 changed files with 108 additions and 196 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

304
cli/README.md
View File

@ -2,52 +2,105 @@
A powerful command-line interface for managing semantic models in Buster. Deploy and manage your data models with ease, whether they're standalone or part of a dbt project. A powerful command-line interface for managing semantic models in Buster. Deploy and manage your data models with ease, whether they're standalone or part of a dbt project.
## Features
- 🚀 Deploy semantic models directly from YAML files
- 🔄 Two-way compatibility with dbt projects
- 🎯 Simple configuration with smart defaults
- 📊 Support for dimensions, measures, and relationships
- 🛠️ Automatic SQL view generation
- 📝 Clear error reporting and validation
## Installation ## Installation
Choose the installation command for your operating system:
### macOS (x86_64)
```bash ```bash
cargo install buster-cli curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-darwin-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster
``` ```
## Quick Start ### Linux (x86_64)
```bash
curl -L https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-linux-x86_64.tar.gz | tar xz && sudo mv buster-cli /usr/local/bin/buster && sudo chmod +x /usr/local/bin/buster
```
1. **Authentication** ### Windows (x86_64)
1. Download the Windows binary:
```powershell
Invoke-WebRequest -Uri https://github.com/buster-so/buster/releases/download/v0.0.1/buster-cli-windows-x86_64.zip -OutFile buster.zip
```
Get your API key from [Buster Platform](https://platform.buster.so/app/settings/api-keys) and authenticate: 2. Extract and install:
```powershell
Expand-Archive -Path buster.zip -DestinationPath $env:USERPROFILE\buster
Move-Item -Path $env:USERPROFILE\buster\buster-cli.exe -Destination $env:LOCALAPPDATA\Microsoft\WindowsApps\buster.exe
```
```bash ## Quick Start Guide
buster auth
```
2. **Project Setup** ### 1. Authentication
Initialize a new project: First, authenticate with Buster using your API key:
```bash ```bash
buster init buster auth
``` ```
This will: This will prompt you for:
- Create a `buster.yml` configuration file - API Key (required) - Get this from the Buster Platform
- Set up the recommended directory structure - Host (optional) - Defaults to production if not specified
- Configure your data source connection
3. **Deploy Models** ### 2. Generate Models
```bash Generate Buster YAML models from your existing SQL files:
buster deploy
``` ```bash
buster generate
```
Key flags for generation:
- `--source-path`: Directory containing your SQL files (defaults to current directory)
- `--destination-path`: Where to output the generated YAML files (defaults to current directory)
- `--data-source-name`: Name of the data source to use in the models
- `--schema`: Database schema name
- `--database`: Database name
The generate command will:
- Scan the source directory for SQL files
- Create corresponding YAML model files
- Create a `buster.yml` configuration file if it doesn't exist
- Preserve any existing model customizations
Example with all options:
```bash
buster generate \
--source-path ./sql \
--destination-path ./models \
--data-source-name my_warehouse \
--schema analytics \
--database prod
```
### 3. Deploy Models
Deploy your models to Buster:
```bash
buster deploy
```
Deploy options:
- `--path`: Specific path to deploy (defaults to current directory)
- `--dry-run`: Validate the deployment without actually deploying (defaults to false)
Examples:
```bash
# Deploy all models in current directory
buster deploy
# Deploy a specific model or directory
buster deploy --path ./models/customers.yml
# Validate deployment without applying changes
buster deploy --dry-run
```
## Project Structure ## Project Structure
A typical Buster project structure:
``` ```
your-project/ your-project/
├── buster.yml # Global configuration ├── buster.yml # Global configuration
@ -55,15 +108,13 @@ your-project/
│ ├── customers.yml │ ├── customers.yml
│ ├── orders.yml │ ├── orders.yml
│ └── products.yml │ └── products.yml
└── sql/ # SQL definitions (optional) └── sql/ # SQL definitions
├── customers.sql ├── customers.sql
├── orders.sql ├── orders.sql
└── products.sql └── products.sql
``` ```
## Configuration ### Configuration (buster.yml)
### Global Configuration (buster.yml)
```yaml ```yaml
# buster.yml # buster.yml
@ -71,7 +122,7 @@ data_source_name: "my_warehouse" # Your default data source
schema: "analytics" # Default schema for models schema: "analytics" # Default schema for models
``` ```
### Model Definition (models/*.yml) ### Model Definition Example
```yaml ```yaml
# models/customers.yml # models/customers.yml
@ -79,29 +130,21 @@ version: 1
models: models:
- name: customers - name: customers
description: "Core customer data model" description: "Core customer data model"
data_source_name: "my_warehouse" # Optional, overrides global data_source_name: "my_warehouse"
schema: "analytics" # Optional, overrides global schema: "analytics"
# Define entities (for relationships)
entities: entities:
- name: customer_id - name: customer_id
expr: "id" expr: "id"
type: "primary" type: "primary"
description: "Primary customer identifier" description: "Primary customer identifier"
# Define dimensions
dimensions: dimensions:
- name: email - name: email
expr: "email" expr: "email"
type: "string" type: "string"
description: "Customer email address" description: "Customer email address"
- name: signup_date
expr: "created_at::date"
type: "date"
description: "Date when customer signed up"
searchable: true # Enable value caching
# Define measures
measures: measures:
- name: total_customers - name: total_customers
expr: "customer_id" expr: "customer_id"
@ -109,121 +152,6 @@ models:
description: "Total number of unique customers" description: "Total number of unique customers"
``` ```
### SQL Definition (sql/*.sql)
```sql
-- sql/customers.sql
SELECT
id as customer_id,
email,
created_at
FROM raw.customers
WHERE deleted_at IS NULL
```
## Commands
### `buster deploy`
Deploy your semantic models to Buster.
```bash
# Deploy all models in the current directory
buster deploy
# Deploy a specific model
buster deploy models/customers.yml
# Deploy models in a specific directory
buster deploy models/
```
The deploy command will:
1. Find and validate all model files
2. Locate corresponding SQL files
3. Generate default SQL if none exists
4. Deploy to Buster with proper error handling
### `buster auth`
Manage your Buster authentication.
```bash
# Set up authentication
buster auth
# View current auth status
buster auth status
# Clear authentication
buster auth clear
```
### `buster init`
Initialize a new Buster project.
```bash
# Initialize in current directory
buster init
# Initialize in a specific directory
buster init my-project/
```
## Model Features
### Entity Relationships
Link models together using entity relationships:
```yaml
# models/orders.yml
models:
- name: orders
entities:
- name: customers # References customers.yml
expr: "customer_id"
type: "foreign"
description: "Link to customer"
```
### Stored Values
Enable value caching for better performance:
```yaml
dimensions:
- name: country
expr: "country_code"
type: "string"
searchable: true # Values will be cached
```
### Default SQL Generation
If no SQL file exists, Buster generates a default SELECT statement:
```sql
SELECT * FROM schema.model_name
```
## Error Handling
The CLI provides clear error messages for common issues:
- Missing required fields
- Invalid relationships
- SQL syntax errors
- API communication issues
- Authentication problems
Example error output:
```
❌ Error processing customers.yml: data_source_name is required
⚠️ Warning: No SQL file found for 'customers', using default SELECT
```
## Best Practices ## Best Practices
1. **Organization** 1. **Organization**
@ -231,50 +159,34 @@ Example error output:
- Keep SQL files in `sql/` - Keep SQL files in `sql/`
- Use `buster.yml` for shared settings - Use `buster.yml` for shared settings
2. **Naming** 2. **Model Generation**
- Use descriptive model names - Start with clean SQL files
- Match SQL and YAML file names - Generate models first before customizing
- Use lowercase with underscores - Review generated models before deployment
3. **Documentation** 3. **Deployment**
- Add descriptions to all models - Use `--dry-run` to validate changes
- Document dimensions and measures - Deploy frequently to catch issues early
- Explain relationships - Keep model and SQL files in sync
4. **SQL**
- Keep SQL simple and focused
- Use CTEs for complex logic
- Add comments for clarity
## Known Limitations
- SQL files must be one directory up from YAML files
- Environment is fixed to "dev"
- No automatic relationship creation
- Simple SQL fallback may not suit all cases
## Troubleshooting ## Troubleshooting
### Common Issues Common issues and solutions:
1. **"Data source not found"** 1. **Authentication Issues**
- Verify data source exists in Buster - Verify your API key is correct
- Check data_source_name in config - Check if the host is properly specified (if using non-production)
- Ensure env='dev' is set - Ensure network connectivity to Buster
2. **"SQL file not found"** 2. **Generation Issues**
- Check SQL file location - Verify SQL files are in the correct location
- Verify file naming matches - Check file permissions
- Consider using default SQL - Ensure SQL syntax is valid
3. **"Invalid relationship"** 3. **Deployment Issues**
- Verify referenced model exists - Validate YAML syntax
- Check entity name matches - Check for missing dependencies
- Ensure proper file structure - Verify data source connectivity
## Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
## License ## License