Ling0925
/
buster
mirror of https://github.com/buster-so/buster.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,346 Commits 151 Branches 17 Tags 116 MiB
Commit Graph

153 Commits

Author SHA1 Message Date
dal 79a2d7cb04
ok some changes to locations for semantic models 2025-05-06 12:15:34 -06:00
dal d33c798f63
closer 2025-05-06 12:01:34 -06:00
dal 3402a322dd
refactor the cli commands for local guide 2025-05-06 11:39:42 -06:00
dal cb430074a7
migrate the deploy logic into its own handler and simplify, init auto semantic creation, map semantic models from buster config 2025-05-06 11:12:19 -06:00
dal 76348e1c59
deploy now using the new refactored version 2025-05-06 10:49:57 -06:00
dal ded67c4fc8
cli deployment logic 2025-05-06 09:47:04 -06:00
dal 0b51ef4edf
cli migration and cleanup 2025-05-06 09:36:21 -06:00
dal ff348745c3
Implement CLI configuration and model discovery for projects 
- Updated BusterConfig structure to better support project contexts
- Added get_context_for_path method to determine project context for a file
- Enhanced resolve_model_paths to respect the project hierarchy
- Updated deploy.rs to use the revised model discovery and configuration inheritance
- Added unit tests and integration tests for project config functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-05-06 08:50:56 -06:00
dal c3c8100f3c
moved semantic layer to server libs for now 2025-05-05 18:41:16 -06:00
dal 1987aeb45a
auth, init 2025-05-05 18:24:05 -06:00
dal ef07010ff5
auth clear option 2025-05-05 18:16:54 -06:00
dal 2412a4ac96
init with errors 2025-05-05 18:06:56 -06:00
dal af3262d0c1
refactor for modular and testable code 2025-05-05 18:05:51 -06:00
dal fee9096eb5
deploy and in it work on cli 2025-05-05 15:22:55 -06:00
dal 2ce680cf1f
moved around the init objects for easier use 2025-05-05 15:12:40 -06:00
dal 771f37d244
project context skip yml config if none 2025-05-05 15:08:11 -06:00
dal 70cdb1882f
Refactor secret management and command structure 
- Removed the `SecretIdResult` struct from `vault.rs` and updated `create_secret` to return `Result<()>` instead of `Result<Uuid>`.
- Adjusted the SQL query in `create_secret` to directly call the vault function without fetching the ID.
- Updated error handling in `update_secret` and `delete_secret` for consistency.
- Refactored `create_data_source_handler` to use the data source ID for storing credentials in the vault.
- Deleted the `generate.rs` command file and removed its references in the command module.
- Introduced a new `ProjectContext` struct in `exclusion.rs` to support multi-project configurations in `BusterConfig`.
- Updated the `init` command to create a new configuration structure using `ProjectContext` and sort database types alphabetically.
 2025-05-05 15:03:14 -06:00
dal 82c415e80f
fix on stored v alues functions 2025-05-05 13:50:15 -06:00
dal 016ea352cd
implement all fo the connections with new default format 2025-05-05 11:38:59 -06:00
dal 56ed9d1775
changes to cli 2025-04-15 10:03:53 -06:00
dal 41baefac6e
search data catalog agent work 2025-04-15 07:40:02 -06:00
dal 6135bf904b
cli update and data catalog search mode 2025-04-15 07:06:42 -06:00
dal 34dabe8122
lol this is the actual commit 2025-04-11 09:54:49 -06:00
dal 43defba30d
clie assistant and finished reasoning 2025-04-11 08:18:23 -06:00
dal d9deefb18d
time tracking, docs for cli chat 2025-04-11 07:34:57 -06:00
dal 445e153ed6
cli coding agent feeling *real* nice. I'm excited 2025-04-10 12:37:06 -06:00
dal c23d682514
git 2025-04-10 11:52:12 -06:00
dal 7265067537
scroll. 2025-04-10 11:46:40 -06:00
dal 7eca5c9362
show tool calls 2025-04-10 11:32:20 -06:00
dal 3cd42527ec
string formatting 2025-04-10 11:21:00 -06:00
dal 9202a31780
refactor chat structure 2025-04-10 11:17:46 -06:00
dal a31be9d742
changes 2025-04-10 11:07:31 -06:00
dal 82f526ae46
buster cli agent 2025-04-10 11:01:00 -06:00
dal 603fcf2ede
cli working 2025-04-10 10:49:04 -06:00
dal 542b9941be
small buster chat poc 2025-04-10 09:57:34 -06:00
dal a9649b38a9
nailed down model file 2025-04-09 17:58:20 -06:00
dal 222da4b879
cli 2025-04-09 17:09:00 -06:00
dal 4e2246e91d
auth improvements for cli 2025-04-09 16:48:40 -06:00
dal c6d0c6b4b1
buster auto update check 2025-04-09 16:24:42 -06:00
dal 9563d4287f
moved the cli subdir around a bit 2025-04-09 16:15:47 -06:00
dal 98284cdc9a
some docs and stuff 2025-04-09 16:08:42 -06:00
dal 71f26f48ab
update to cli 2025-04-09 15:56:45 -06:00
dal 5162af9040
fix on deploy datasets 2025-03-28 15:50:59 -06:00
Nate Kelley 813b05c18a
Merge branch 'staging' into big-nate/bus-939-create-new-structure-for-chats 2025-03-04 10:53:16 -07:00
dal bdcbe7a301
input of api key 2025-02-26 10:15:33 -07:00
dal b2ff856e14
add in the model path feature. 2025-02-26 10:07:36 -07:00
dal 89e92cf81f
ok better version handling 2025-02-26 07:51:04 -07:00
dal a70389b4e8
bigquery support 2025-02-26 07:45:22 -07:00
dal 3bfe77339a
Enhance dataset and API error handling 
- Improve column deployment logic in dataset routes
- Add more detailed error reporting in CLI generate command
- Support additional Postgres credential parsing
- Add debug logging for Buster API interactions
 2025-02-25 21:11:06 -07:00
dal efca65932e
init added in the postgres onboarding 2025-02-25 20:32:34 -07:00
dal e7588c1d12
ok we got in the init 2025-02-25 20:17:00 -07:00
dal 9c7e217077
some good logging and config tracking 2025-02-25 20:11:35 -07:00
dal 1c55f5ae1c
prds rules 2025-02-25 19:40:36 -07:00
dal d5f56a86e1
updated readme 2025-02-25 12:34:51 -07:00
dal f78d93d37d
ok all ready for release and exluding tags. 2025-02-25 12:31:53 -07:00
dal 71a500cacf
recursive directory structure works greate 2025-02-25 12:13:23 -07:00
dal 5e947b7c55
ok deployment working appropriately 2025-02-25 11:59:42 -07:00
dal c1f7c0d95c
Add Redshift support for credentials and column retrieval 2025-02-25 11:46:38 -07:00
dal 3d77316fe6
ok init looking good. 2025-02-25 11:22:53 -07:00
dal 0d45439173
successfully init on redshift 2025-02-25 11:17:28 -07:00
dal 116677c0ab
bugfix - cli missing types and deploy 2025-02-18 13:35:07 -07:00
dal 751a2da463
Merge branch 'staging' into big-nate/bus-939-create-new-structure-for-chats 2025-02-14 15:22:58 -08:00
dal 5126af0d35
readme fix 2025-02-12 14:52:17 -07:00
dal d453d5a92b
just get rid of weird version caching 2025-02-12 14:35:01 -07:00
dal 533889a54f
cli exclusion and readme updates 2025-02-12 14:08:01 -07:00
dal 24edcf2968
update read me for env 2025-02-12 14:02:20 -07:00
dal af3cc188aa
exclusion pattern 2025-02-12 14:01:11 -07:00
dal ffbcedb8d5
lol upgrade version 2025-02-12 13:32:23 -07:00
dal a787c9ebaf
Update cli/src/commands/update.rs 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-02-12 12:13:51 -08:00
dal 6e51e417b0
Update cli/src/commands/update.rs 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-02-12 12:13:28 -08:00
dal 21296dba15
Update cli/src/commands/auth.rs 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-02-12 12:12:58 -08:00
dal 08134b3ba3
Update cli/src/commands/auth.rs 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-02-12 12:12:22 -08:00
dal caec707450
Add macOS ARM64 support and enhance CLI authentication and version management 2025-02-12 13:00:29 -07:00
dal 224cbe4d84
tls 1.3 forced 2025-02-12 10:36:25 -07:00
dal 5b6ff85af8
readme for cli 2025-02-12 10:00:26 -07:00
dal 946121fad4
change default url 2025-02-12 09:43:03 -07:00
dal 09ab45bbb5
cli release code 2025-02-12 09:00:02 -07:00
dal bfe61d52e2
nice 👍🏼 2025-02-12 08:27:02 -07:00
dal 59054cfefa
tweak to the diffing bug 2025-02-12 08:14:38 -07:00
dal 616da11b05
Add generate datasets API endpoint and implementation 2025-02-12 06:58:51 -07:00
dal b7659ab7b0
initial gathering 2025-02-12 06:48:29 -07:00
dal d95ea3a99d
ok fixed the local validation issue 2025-02-12 05:45:41 -07:00
dal 61b6cfeffc
database mapping correct 2025-02-12 05:22:36 -07:00
dal c7a631de40
tweak on database from modelfiles 2025-02-12 05:04:34 -07:00
dal e7fedd0a59
working except for bulk in the validation 2025-02-11 15:21:53 -07:00
dal 32efa01b51
validation done. 2025-02-11 12:26:07 -07:00
dal aee795b07b
add in ref 2025-02-11 12:11:05 -07:00
dal 61a545384f
dry run and local validation 2025-02-11 11:36:03 -07:00
dal bf05c7f06b
Add new thread events for chat generation progress tracking 2025-02-11 11:21:57 -07:00
dal efd044c6c2
yml files 2025-02-11 10:40:41 -07:00
dal 85c4470eab
adjust the models and lock them down for now 2025-02-11 10:29:57 -07:00
dal 2618166bf8
feat: Expand validation error types for more comprehensive deployment checks 
- Add new validation error types: ModelNotFound, InvalidRelationship, and ExpressionError
- Update deploy_v2.rs to categorize and handle these new error types
- Enhance error handling and reporting in dataset deployment process
 2025-02-05 23:56:41 -07:00
dal 6e5c299389
feat: improve dataset column validation and deployment process 
- Add comprehensive column validation before dataset deployment
- Validate existence of all required columns in source database
- Simplify column type and nullability retrieval
- Enhance error reporting for missing columns
- Update deployment logic to use pre-validated column information
 2025-02-05 17:20:11 -07:00
dal f081f3e16e
feat: enhance dataset validation and deployment error handling 
- Add detailed validation error logging in CLI
- Improve type compatibility checks in dataset validation
- Modify deployment process to handle and report validation errors more comprehensively
- Add Hash derive for Verification enum
- Update API and CLI to support more informative validation results
 2025-02-05 17:04:13 -07:00
dal 66d9528619
refactor: deploy datasets much simpler now 2025-02-05 14:05:12 -07:00
dal 05d613c9ad
chore: update .gitignore to exclude /prds directory 2025-02-04 16:52:54 -07:00
dal 59049b5604
refactor(stored_values): improve background processing and error handling for stored column values (#85) 
- Refactor stored values processing in dataset deployment to use background task
- Add `StoredValueColumn` struct to encapsulate column processing details
- Implement `process_stored_values_background` for parallel and resilient value storage
- Add logging for successful and failed stored value processing
- Update CLI to handle optional SQL definitions and improve file processing
 2025-02-04 11:30:45 -08:00
dal 42c6080598
refactor(stored_values): improve background processing and error handling for stored column values 
- Refactor stored values processing in dataset deployment to use background task
- Add `StoredValueColumn` struct to encapsulate column processing details
- Implement `process_stored_values_background` for parallel and resilient value storage
- Add logging for successful and failed stored value processing
- Update CLI to handle optional SQL definitions and improve file processing
 2025-02-04 12:24:06 -07:00
dal 5202438fa8
Dal/cli-updates-skip-dbt (#67) 
* feat: enhance deploy command with skip_dbt option

- Updated the Deploy command to accept a `skip_dbt` boolean argument, allowing users to bypass the dbt run during deployment.
- Refactored the deploy function to conditionally execute the dbt command based on the `skip_dbt` flag, improving deployment flexibility.

* Refactor query engine and CLI commands for improved functionality and error handling

- Updated `get_bigquery_columns` and `get_snowflake_columns` functions to enhance column name handling and ensure proper error reporting.
- Modified `get_snowflake_client` to accept a database ID for better connection management.
- Enhanced the `deploy` command in the CLI to include additional parameters (`path`, `data_source_name`, `schema`, `env`) for more flexible deployments.
- Improved error handling and reporting in the `deploy` function, including detailed summaries of deployment errors and successful file processing.
- Updated `get_model_files` to accept a directory path and added checks for file existence, enhancing robustness.
- Adjusted model file structures to include schema information and refined the upload process to handle optional parameters more effectively.

These changes collectively improve the usability and reliability of the query engine and deployment process.

* Update dataset DDL generation to include optional YML file content

- Modified `generate_sql_agent` to append optional YML file content to dataset DDL
- Ensures more comprehensive dataset representation during SQL agent generation
- Handles cases where YML file might be present or absent gracefully
 2025-01-24 16:00:38 -08:00
dal 1f1df4a7bb
Refactor dataset deployment logic and enhance request structure 
- Introduced a new `is_simple` flag in the `deploy_datasets` function to differentiate between full and simple dataset deployments.
- Updated the `deploy_datasets_handler` to accept the `is_simple` parameter, allowing for conditional processing of inserted datasets.
- Modified the `DeployDatasetsRequest` struct to include an optional `id` and `type_` field, enhancing the request's flexibility.
- Adjusted the handling of the `yml_file` field to be optional in the `DeployDatasetsRequest` struct.
- Updated the `process_batch` function to handle "USER-DEFINED" data types in addition to existing types.

These changes improve the dataset deployment process by allowing for more granular control and flexibility in handling different dataset types.
 2025-01-11 15:17:01 -07:00