Commit Graph

803 Commits

Author SHA1 Message Date
Nate Kelley bd66820792
Update packages/database/package.json
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-16 11:25:10 -06:00
dal 0d217714cc
added slack tracking on chats 2025-07-16 11:15:16 -06:00
Nate Kelley 1a8d6e1240
Merge pull request #520 from buster-so/big-nate/bus-1420-add-searchable-menu-to-share-dropdowns
Big nate/bus 1420 add searchable menu to share dropdowns
2025-07-16 10:37:20 -06:00
Nate Kelley 8fe5e9f906
working tests 2025-07-16 10:36:24 -06:00
Nate Kelley 0f7d1c313d
update rules for database folder 2025-07-16 10:29:10 -06:00
Nate Kelley f11afec05a
query params update for cursor rules 2025-07-16 10:13:30 -06:00
dal 2bc11900b9
Slack User auth and domain validation 2025-07-16 10:09:57 -06:00
Nate Kelley a032a966f8
make generic handler 2025-07-16 09:59:30 -06:00
Nate Kelley 0d01dfcb4c
array parter 2025-07-16 09:54:28 -06:00
Nate Kelley 5dc31814d8
Simplify the get user endpoint 2025-07-16 09:32:48 -06:00
Nate Kelley d9ed8b1423
finalize pagination endpoint 2025-07-16 09:25:51 -06:00
Jacob Anderson a4551e661b assumption prompt changes for first person voice and join logic 2025-07-16 09:09:37 -06:00
Nate Kelley b637bf356a
paginated queries 2025-07-15 23:05:12 -06:00
Nate Kelley fdf70abfb8
update database dev 2025-07-15 22:26:13 -06:00
Nate Kelley 094fc4c251
add pagination type responses 2025-07-15 17:23:32 -06:00
Nate Kelley fb0e79065f
udpate endpoint to accept more params 2025-07-15 16:54:18 -06:00
Nate Kelley a9b8acbb80
user to organization method 2025-07-15 16:36:14 -06:00
Nate Kelley 2e73d97ffc
create basic user to organization db 2025-07-15 16:20:40 -06:00
Nate Kelley 74993ca556
change folder name 2025-07-15 16:08:01 -06:00
Nate Kelley f23d933cc1
initial commit to move files 2025-07-15 15:44:26 -06:00
dal a751d22baa
initial slack challenge verification and receive messages 2025-07-14 16:08:01 -06:00
dal 4ab3ae97a2
env vars 2025-07-14 15:12:46 -06:00
Nate Kelley 727e66a630
Fix files on lint 2025-07-14 14:30:07 -06:00
dal 3b48e51969
format 2025-07-14 14:28:06 -06:00
dal bd34e72b93
bugfix on data metdata 2025-07-14 14:23:37 -06:00
Cursor Agent 2cf56e895c Change file type from 'agent-action' to 'todo' in chunk processor
Co-authored-by: dallin <dallin@buster.so>
2025-07-14 18:49:58 +00:00
dal d42fdd489d
Merge pull request #503 from buster-so/cursor/improve-529-error-recovery-process-6ba8
Handle 529 and clean up
2025-07-14 11:43:45 -07:00
dal fd901b40af
Update packages/ai/tests/utils/retry/cleanup-incomplete-tool-calls.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-14 11:42:55 -07:00
dal b7fc3eaa09
unit tests are passing 2025-07-14 12:40:53 -06:00
dal 69da316f6c
Merge remote-tracking branch 'origin/staging' into dallin/bus-1380-fix-unit-tests 2025-07-14 12:37:15 -06:00
dal a518f93f76
rest of them 2025-07-14 12:34:00 -06:00
dal 75cfb9d733
data source tests 2025-07-14 12:33:54 -06:00
Cursor Agent 7988ccbb58 Add 529 error handling with tool call cleanup and backoff strategy
Co-authored-by: dallin <dallin@buster.so>
2025-07-14 17:09:54 +00:00
dal 5a971dd8a1
fix on access-control tests 2025-07-14 11:01:53 -06:00
Nate Kelley 5352ae8f10
currency response 2025-07-14 09:41:25 -06:00
dal 2be9784516
Add unit and integration test scripts to package.json files across multiple packages 2025-07-14 09:11:20 -06:00
Nate Kelley 906986b51e
Update inlcude 2025-07-12 23:46:09 -06:00
Nate Kelley 963bf6b2f2
fix all of the bugs 2025-07-12 22:14:08 -06:00
Nate Kelley 4810daef9f
Update packages/server-shared/src/type-utilities/index.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-12 16:33:53 -06:00
Nate Kelley 4197cf799e
Update packages/database/src/connection.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-12 16:33:33 -06:00
Nate Kelley 5448af614a
lint fixes 👀 2025-07-12 16:32:47 -06:00
Nate Kelley 139a0f52c9
organization camelcase update 2025-07-12 16:31:35 -06:00
Nate Kelley aeaf7dbb57
type updates 2025-07-12 16:19:08 -06:00
Nate Kelley 2c2bf74475
Add prebuild script to server-shared 2025-07-12 16:07:55 -06:00
Nate Kelley d8e0146978
create a db utilities 2025-07-12 15:48:42 -06:00
Nate Kelley 9de6e8227e
Merge branch 'staging' into big-nate/bus-1355-make-it-so-regular-users-can-invite-others 2025-07-11 16:30:59 -06:00
dal 10431a81d3
add has_more_records to the server shared type 2025-07-11 15:30:53 -06:00
dal 550aba77e6
execute sql fixes 2025-07-11 15:28:01 -06:00
dal 7fe7ecb00b
Refactor SQL execution logic to limit results at the adapter level instead of modifying SQL queries. Removed the wrapQueryWithLimit function and updated related code to enhance performance and preserve caching in Snowflake. Updated documentation to reflect changes in query handling. 2025-07-11 14:54:34 -06:00
dal a36bba3b40
Add '@buster/ai' dependency and enhance chat cancellation logic
- Added '@buster/ai' as a workspace dependency in pnpm-lock.yaml and package.json.
- Updated database-migrations.yml to trigger on changes in the database package.
- Refined the cancelChatHandler function to include detailed message cleanup and trigger cancellation logic.
- Improved response handling in the chat cancellation endpoint to return a success message.
- Enhanced updateMessageFields to support marking messages as completed.
2025-07-11 09:51:51 -06:00
dal 87f3853ce8
stubbed out the cancel endpoint and added the trigger run col 2025-07-10 16:26:30 -06:00
Nate Kelley e7cfafa95e
Update pooler 2025-07-10 16:06:13 -06:00
dal 6d75c30834
Merge remote-tracking branch 'origin/staging' into dallin/bus-1345-figure-out-why-snowflake-is-so-delayed-here 2025-07-10 15:21:49 -06:00
dal 319f6baa4f
snowflake serverless optimization 2025-07-10 15:21:25 -06:00
Nate Kelley 252a892cae
Merge pull request #475 from buster-so/nate/hot-fix-server-build
Server build update
2025-07-10 12:50:47 -06:00
Nate Kelley e6307838b7
Update base.json 2025-07-10 12:22:18 -06:00
Nate Kelley e7ee256072
move shared to be zod v4 2025-07-10 12:16:34 -06:00
dal 315cfa7a8f
Merge pull request #472 from buster-so/dallin/bus-1346-access-paths-for-datasets-need-to-go-through-defaults-as
Dallin/bus 1346 access paths for datasets need to go through defaults as
2025-07-10 10:15:13 -07:00
dal 687cf6d072
lock down datasets to orgs 2025-07-10 11:14:02 -06:00
dal 0e0b14a47f
access controls listing datasets with new default datasets access pattern through security settings. 2025-07-10 11:03:45 -06:00
Nate Kelley 7b9892129c
update imports and type checks 2025-07-10 10:52:21 -06:00
Nate Kelley b28f6e1e35
additional types fixes 2025-07-10 10:14:16 -06:00
dal 4d7f1353ff
clean up tyeps 2025-07-10 07:52:41 -06:00
dal 3ca54c7f28
Refactor integration tests for domain management and workspace settings. Update error messages for permission checks to be more specific. Enhance domain handling logic for consistency and improve test cleanup procedures. 2025-07-10 07:45:04 -06:00
dal f22c03874c
tests are passing 2025-07-10 07:27:16 -06:00
dal 29e6a6ae7f
Workspace Settings Done with auto-add 2025-07-10 07:01:23 -06:00
dal c047d7dd40
Refactor security API handlers to improve organization validation and domain management. Consolidate permission checks and domain operations into utility functions for better code reuse. Update endpoint routes for workspace settings and approved domains, ensuring consistent response formatting. 2025-07-10 05:30:10 -06:00
dal 7f1cbe7488
Merge branch 'staging' into big-nate/bus-1260-set-a-default-permissionpermission-role-for-newly-added 2025-07-09 16:18:43 -06:00
dal e1f7f63270
add conversation history to the 2025-07-09 15:46:15 -06:00
dal e4387c7523
new prompts 2025-07-09 14:40:58 -06:00
dal b062694a2c
fixed types on slack handlers 2025-07-09 14:34:02 -06:00
dal 8e9e9a087e
claude rules and schema tweak 2025-07-09 14:15:46 -06:00
Nate Kelley b179a7dd9d
Merge pull request #458 from buster-so/dallin/bus-1212-wire-up-slack-alerts-for-flagchat-and-major-assumptions
Dallin/bus 1212 wire up slack alerts for flagchat and major assumptions
2025-07-09 14:10:44 -06:00
Nate Kelley c223309df2
Add slack integration endpoints 2025-07-09 14:08:29 -06:00
dal 5941d6975d
schema migration and handlers stubbed out 2025-07-09 14:07:55 -06:00
dal 30125e342e
Merge pull request #457 from buster-so/cursor/validate-chart-axis-configurations-6704
Validate chart axis configurations
2025-07-09 11:59:47 -07:00
dal 98190539b8
biome and test fix 2025-07-09 12:59:27 -06:00
dal 0f30e344ce
Update packages/ai/src/tools/visualization-tools/bar-line-axis-validator.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-09 11:55:23 -07:00
dal 0fe31e1059
Merge pull request #456 from buster-so/cursor/update-message-type-to-enum-a472
get rid of old message type
2025-07-09 11:46:00 -07:00
dal 4a7b87104d
get rid of old message type 2025-07-09 12:42:56 -06:00
dal 812910624c
revert schema changes 2025-07-09 12:39:12 -06:00
dal e2fe53261e
Enhance message post-processing to include Slack message existence check. Updated workflow to differentiate between follow-up messages based on Slack context. Added tests to validate new functionality and ensure proper handling of Slack-related follow-ups. 2025-07-09 12:08:23 -06:00
Cursor Agent 8ef482310d Add axis validation for bar and line charts with auto-correction
Co-authored-by: dallin <dallin@buster.so>
2025-07-09 17:58:16 +00:00
Nate Kelley 821ed90083
initial slack build out (no apis) 2025-07-09 11:52:33 -06:00
Cursor Agent 95787d7b46 Update Slack message tracking with new message source enum
Co-authored-by: dallin <dallin@buster.so>
2025-07-09 17:49:27 +00:00
Nate Kelley 12fa022cfe
Merge pull request #454 from buster-so/big-nate/bus-1260-set-a-default-permissionpermission-role-for-newly-added
Big nate/bus 1260 set a default permissionpermission role for newly added
2025-07-09 11:08:14 -06:00
Nate Kelley e22a795326
Add additinal fixes 2025-07-09 11:06:25 -06:00
Nate Kelley 26d77fef53
create workspace settings 2025-07-09 10:58:51 -06:00
dal 4bdddd179b
small change 2025-07-09 10:41:14 -06:00
Nate Kelley 59e6aacd9d
part 1 of making workspace toggles 2025-07-09 09:42:16 -06:00
dal cba5e99223
Refactor follow-up and initial message formatting to handle major assumptions more effectively. Added checks for major assumptions and updated message structure for clarity in output. 2025-07-09 09:13:58 -06:00
dal 0c2e3e2e95
greptile fixes 2025-07-09 08:53:14 -06:00
dal 897ac4ce83
added in the slack message tracking 2025-07-09 07:44:10 -06:00
dal 59c19354b9
Update flag-chat and format-initial-message steps to clarify the AI agent's name as "Buster" and enhance summary message guidelines for better communication with the data team. 2025-07-08 17:50:08 -06:00
dal f19b3a0b91
new prompt for follow up 2025-07-08 17:23:10 -06:00
Nate Kelley 46c3fccb9b
invite link updates 2025-07-08 17:20:31 -06:00
dal 1ec4b17143
added title to the update-message tool and routed to the 'summaryTitle' 2025-07-08 17:11:17 -06:00
dal 2367e81f66
Merge pull request #448 from buster-so/dal/retry-agent-logic
Dal/retry-agent-logic
2025-07-08 16:01:50 -07:00
dal 9f009e970d
Refactor healing strategy logic for improved clarity and consistency. Adjust error handling messages and ensure proper formatting. Enhance type definitions for better type safety. 2025-07-08 17:01:09 -06:00
dal a1f9de2168
retry final touches 2025-07-08 16:56:57 -06:00
Nate Kelley 9bd7824586
create security endpoints strucutre 2025-07-08 16:48:13 -06:00
dal 3b5d981ecd
abstract the error handler and retry into a different helper 2025-07-08 16:25:15 -06:00
Nate Kelley 7d22c42141
Move organization user to a server type 2025-07-08 15:06:17 -06:00
dal bf35c9b09c
abstract the error handler and retry into a different helper 2025-07-08 14:49:37 -06:00
dal a620d74d82
Merge branch 'staging' into dal/retry-agent-logic 2025-07-08 14:22:02 -06:00
dal 6af278cd9e
get rid of unused retry functions and logic 2025-07-08 13:35:25 -06:00
Nate Kelley 86aa9234b8
Merge branch 'staging' into big-nate/bus-1273-add-assumptions-and-confidence-score-to-web-app 2025-07-08 12:57:07 -06:00
Nate Kelley 8850b1ffd8
Assumption cards 2025-07-08 12:54:18 -06:00
Cursor Agent 0f97741e52 Convert column names to lowercase in metrics file creation tools
Co-authored-by: dallin <dallin@buster.so>
2025-07-08 17:35:50 +00:00
dal f21690acc0
added in the server shared types for post-processing and moved the type into shared from trigger. 2025-07-08 11:14:30 -06:00
dal 7fa5e496ff
Merge branch 'staging' into big-nate/bus-1273-add-assumptions-and-confidence-score-to-web-app 2025-07-08 10:59:16 -06:00
dal fe4d2c9a83
message post processing formatting 2025-07-08 10:55:19 -06:00
Nate Kelley 6b437d8737
post processing messages 2025-07-08 10:12:51 -06:00
Nate Kelley f4c2515f25
truncate metric headers and values 2025-07-08 09:46:51 -06:00
Nate Kelley 75e5c1ec4b
Merge branch 'staging' into big-nate/bus-1273-add-assumptions-and-confidence-score-to-web-app 2025-07-08 09:35:38 -06:00
dal 4246a14f4d
more retry healing logic 2025-07-08 09:10:03 -06:00
Nate Kelley aabb6908d7
move types to be on server 2025-07-07 22:26:42 -06:00
dal f1f73463ce
- Increased retryable errors
- modified issue with slack auth
2025-07-07 17:40:17 -06:00
dal 19fe3748d2
more logging 2025-07-07 17:31:47 -06:00
dal 73f652081a
increase timeouts 2025-07-07 17:14:12 -06:00
dal bb9ca0acbb
Merge branch 'staging' into more-fixes-for-horizontal-bars 2025-07-07 15:42:04 -07:00
Jacob Anderson 2212b69585 even more fixes for horizontal bar charts 2025-07-07 16:38:43 -06:00
dal d470609a10
Merge pull request #428 from buster-so/fixes-for-horizontal-charts
additional fixes for horizontal charts, hopefully fixing them
2025-07-07 15:21:11 -07:00
Nate Kelley 4e888f9506
Merge pull request #427 from buster-so/big-nate/bus-1181-point-to-google-workspace-profile-pics
Big nate/bus 1181 point to google workspace profile pics
2025-07-07 16:17:54 -06:00
Nate Kelley 2a6bd2f5a8
avatar passthrough 2025-07-07 16:15:10 -06:00
Jacob Anderson d9358d6450 additional fixes for horizontal charts, hopefully fixing them 2025-07-07 16:09:22 -06:00
Nate Kelley 0331be1cda
verification pass through avatar url 2025-07-07 15:53:18 -06:00
Nate Kelley 6d02d8900f
move types to server 2025-07-07 15:43:47 -06:00
dal 35761e9dd0
Merge pull request #426 from buster-so/cursor/ensure-correct-types-for-integration-handler-fac1
Ensure correct types for integration handler
2025-07-07 14:37:32 -07:00
dal b5b9da222c
Merge pull request #425 from buster-so/jacob-prompt-changes-v2
Jacob prompt changes v2
2025-07-07 13:39:52 -07:00
Jacob Anderson aa24e709fe changes to precomputed metric and vertical/horziontal bar chart logic 2025-07-07 14:36:42 -06:00
Nate Kelley b2f9e7c119
lint hot fix 2025-07-07 14:04:10 -06:00
Cursor Agent 76b18c8765 Convert Slack response types to Zod schemas for runtime validation
Co-authored-by: dallin <dallin@buster.so>
2025-07-07 20:02:29 +00:00
Nate Kelley ebff254da0
Update some zod types 2025-07-07 13:19:53 -06:00
Nate Kelley ca0c434e76
Merge branch 'staging' into nate/dashboard-links 2025-07-07 12:58:58 -06:00
Nate Kelley 0827499db0
update vitest 2025-07-07 12:58:29 -06:00
dal bc8793cd04
switch migration 2025-07-07 12:46:22 -06:00
dal b68d9365bb
update migraiton to create if not exists 2025-07-07 12:22:48 -06:00
dal 44b3627929
Merge branch 'staging' into nate/dashboard-links 2025-07-07 11:15:20 -07:00
dal 982bb50202
Merge branch 'staging' into cursor/add-metadata-to-braintrust-logs-b94f 2025-07-07 11:12:28 -07:00
dal 03536b81c0
dupli describe in schema and other build errors 2025-07-07 11:37:07 -06:00
dal 1f91ba14d5
Update packages/ai/src/steps/post-processing/schemas.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-07 10:19:40 -07:00
Jacob Anderson d0aad7c6af Fixes for horizontal bar chart and precomputed metric logic 2025-07-07 10:54:11 -06:00
Nate Kelley 80221a2837
Update access controls dev script 2025-07-07 10:44:43 -06:00
Nate Kelley ba6d1fe043
Fix broken unit tests 2025-07-07 10:28:32 -06:00
Nate Kelley 584bbbaefe
Merge branch 'staging' into nate/dashboard-links 2025-07-07 10:22:37 -06:00
Nate Kelley c432f7d1f5
Fix access control linting 2025-07-07 10:14:13 -06:00
Nate Kelley 5d532a2957
additional lint fixes 2025-07-07 10:12:31 -06:00
dal 3b01e5727b
added in the postProcessingMessage to the db 2025-07-07 10:10:47 -06:00
dal 0977195995
moved over processing logic 2025-07-07 09:57:48 -06:00
Nate Kelley 7e7aeb3480
lint fixes for ai packages 2025-07-07 09:57:04 -06:00
dal bc4ce29fc6
moving over slack processing work 2025-07-07 09:52:26 -06:00
Nate Kelley cf1b9b60ae
update some biome rules 2025-07-07 09:46:24 -06:00
Nate Kelley 1bf8418ab8
Remove a few any types from database 2025-07-07 09:40:06 -06:00
Nate Kelley 9637fbad86
Fix additional linting errors 2025-07-07 09:18:31 -06:00
Cursor Agent cdeafda0da Add Braintrust metadata retrieval for enhanced logging and tracing
Co-authored-by: dallin <dallin@buster.so>
2025-07-07 15:18:07 +00:00
Nate Kelley 0b5c8cb2e1
Merge pull request #418 from buster-so/cursor/create-unit-tests-for-schema-defaults-b3ac
Add comprehensive test suites for shared server type schemas
2025-07-07 08:30:12 -06:00
Nate Kelley 02a0986393
Update metric.types.test.ts 2025-07-07 08:29:49 -06:00
dal 21e08b9ac9
Merge pull request #416 from buster-so/cursor/optimize-unit-test-timeouts-6e59
Optimize unit test timeouts
2025-07-07 07:23:34 -07:00
dal 3091686823
package upgrades 2025-07-07 08:19:49 -06:00
Nate Kelley 90461c15b5
Merge branch 'staging' into nate/dashboard-links 2025-07-07 08:17:37 -06:00
Cursor Agent aa5449a4fa Reduce timeout duration in adapter timeout tests from 60s to 5s
Co-authored-by: dallin <dallin@buster.so>
2025-07-07 14:04:01 +00:00
dal 42268375f2
unit tests and integration with turbo, chats bug fix 2025-07-07 07:52:15 -06:00
Cursor Agent 964d107bb9 Refactor chat permission checks and update database helper functions
Co-authored-by: dallin <dallin@buster.so>
2025-07-04 15:21:36 +00:00
dal 79e7cb4b95
partial instead of any
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-04 08:13:59 -07:00
Cursor Agent 6fb546a4ad Refactor chat permission check and remove unused organization role check
Co-authored-by: dallin <dallin@buster.so>
2025-07-04 14:55:08 +00:00
Cursor Agent 7275f242ee Enhance chat permission checks with organization role validation
Co-authored-by: dallin <dallin@buster.so>
2025-07-04 14:41:29 +00:00
Cursor Agent 43873df729 Add comprehensive test suites for shared server type schemas
Co-authored-by: natemkelley <natemkelley@gmail.com>
2025-07-04 13:52:26 +00:00
Nate Kelley c99f95c542
Update chatConfig.defaults.test.ts 2025-07-04 07:34:11 -06:00
Nate Kelley 0d114ee850
Add some unit tests 2025-07-04 01:38:58 -06:00
Nate Kelley 043d2654b0
Move default to individual packages 2025-07-04 01:26:25 -06:00
Nate Kelley 847ea87dde
Export issues resolved 2025-07-04 01:17:35 -06:00
dal 9e5c52c9b8
Enhance think-and-prep instructions for clarity and detail. Updated tool usage guidance for `executeSql` and refined steps for addressing TODO items. 2025-07-03 17:19:22 -06:00
dal 3de899976e
Merge pull request #412 from buster-so/cursor/organize-slack-request-and-response-types-495c
Refactor Slack handler to utilize shared types and remove local schema definitions
2025-07-03 16:10:19 -07:00
Nate Kelley db66177c62
dashboard type changes 2025-07-03 17:10:08 -06:00
dal 0aa711f599
switch to snake_case 2025-07-03 17:07:13 -06:00
dal 6e2b72c101
migration for auth public user sync 2025-07-03 17:01:15 -06:00
Nate Kelley 2a8db894a3
additional fixes to chart types 2025-07-03 16:52:53 -06:00
dal f5eb9496c5
Refactor Slack handler to utilize shared types and remove local schema definitions 2025-07-03 16:23:59 -06:00
Cursor Agent 3a3836761f Add Slack integration request and response type definitions
Co-authored-by: dallin <dallin@buster.so>
2025-07-03 22:04:11 +00:00
Nate Kelley a4f255d19b
Merge pull request #411 from buster-so/nate/dashboard-links
Shared asset link helper + shared types in server shared 🤓
2025-07-03 15:55:25 -06:00
Nate Kelley 4cdde9203f
update server shared to include new types 2025-07-03 15:54:31 -06:00
dal 2ef68e2a1b
Refactor Slack integration handling and update tests for new default channel functionality
- Introduced `updateDefaultChannel` method in Slack helpers to manage default channel settings.
- Updated SlackHandler to utilize the new method for updating integration settings.
- Enhanced integration tests to cover new functionality and ensure proper handling of active integrations.
- Removed obsolete SQL migration for default channel as it is now managed in code.
2025-07-03 15:18:28 -06:00
Cursor Agent ba01e881e6 Add default channel support for Slack integration
Co-authored-by: dallin <dallin@buster.so>
2025-07-03 20:47:18 +00:00
Nate Kelley 7364f328d4
added additional server shared types 2025-07-03 14:33:55 -06:00
dal 144faeb802
extract values up to 100 2025-07-03 11:39:09 -06:00
dal 2da197697d
Merge branch 'main' into staging 2025-07-03 09:48:04 -06:00
Nate Kelley a64a7b93f6
Update broken types 2025-07-03 09:25:29 -06:00
dal 3db2cf72b2
Deployment (#397)
* Mastra braintrust (#391)

* type fixes

* biome clean on ai

* add user to flag chat

* attempt to get vercel deployed

* Update tsup.config.ts

* Update pnpm-lock.yaml

* Add @buster/server2 Hono API app with Vercel deployment configuration

* slack oauth integration

* mainly some clean up and biome formatting

* slack oauth

* slack migration + snapshot

* remove unused files

* finalized docker image for porter

* Create porter_app_buster-server_3155.yml file

* Add integration tests for Slack handler and refactor Slack OAuth service

- Introduced integration tests for the Slack handler, covering OAuth initiation, callback handling, and integration status retrieval.
- Refactored Slack OAuth service to improve error handling and ensure proper integration state management.
- Updated token storage implementation to use a database vault instead of Supabase.
- Enhanced existing tests for better coverage and reliability, including cleanup of test data.
- Added new utility functions for managing vault secrets in the database.

* docker image update

* new prompts

* individual tests and a schema fix

* server build

* final working dockerfile

* Update Dockerfile

* new messages to slack messages (#369)

* Update dockerfile

* Update validate-env.js

* update build pipeline

* Update the dockerfile flow

* finalize logging for pino

* stable base

* Update cors middleware logger

* Update cors.ts

* update docker to be more imformative

* Update index.ts

* Update auth.ts

* Update cors.ts

* Update cors.ts

* Update logger.ts

* remove logs

* more cors updates

* build server shared

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies. (#370)

* tons of file parsing errors (#371)

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies.

* tons of file parsing errors

* Dev mode updates

* more stable electric handler

* Dal/agent-self-healing-fixes (#372)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* Dal/agent-self-healing-fixes (#373)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* change porter staging deploy to mastra-braintrust.

* new path for porter deploy

* deploy to staging fix

* Create porter_app_mastra-braintrust-api_3155.yml file (#375)

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>

* Update sizing and opacity

* supe up the instance for mastra

* environment staging

* ssl script

* copy path

* Update list padding

* no throttle and the anthropic cached

* move select to the top

* Update margin inline start

* shrink reasoning vertical space to 2px

* semi bold font for headers

* update animation timing

* haiku

* Add createTodoList tool and integrate into create-todos-step

* chat helper on post chat

* only trigger cicd when change made

* Start created streaming text components

* Refactor analyst agent task to initialize Braintrust logging asynchronously and parallelize database queries for improved performance. Adjusted cleanup timeout for Braintrust traces to reduce delays.

* fixed reasoned for X, so that it rounds down to the minute

* Update users page

* update build pipeline for new web

* document title update

* Named chats for page

* Datasets titles

* Refactor visualization tools and enhance error handling in retryable agent stream. Removed unused metricValueLabel from metrics file tool, updated metric configuration schemas, and improved healing mechanism for tool errors during streaming.

* analyst

* document title updates

* Update useDocumentTitle.tsx

* Refactor tool choice configuration in create-todos-step to use structured object. Remove exponential backoff logic from retryable agent stream for healable errors. Introduce new test for real-world healing scenarios in retryable agent stream.

* Refactor SQL validation logic in modify-metrics-file-tool to skip unnecessary checks when SQL has not changed. Enhance error handling and update validation messages. Clean up code formatting for improved readability.

* update collapse for filecard

* chevron collapse

* Jacob prompt changes (#376)

* prompt changes to improve filtering logic and handle priv/sec errors

* prompt changes to make aggregation better and improved filter best practices

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/agents/think-and-prep-agent/think-and-prep-instructions.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: dal <dallin@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* think and prep

* change header and strong fonts weights

* Update get collection

* combo chart x axis update

* Create a chart schemas as types

* schema types

* simple unit tests for line chart props

* fix the response file ordering iwth active selection.

* copy around reasoning messages taken care of

* fix nullable user message and file processing and such.

* update ticks for chart config

* fix todo parsing.

* app markdown update

* Update splitter to use border instead of width

* change ml

* If no file is found we should auto redirect

* Refactor database connection handling to support SSL modes. Introduced functions to extract SSL parameters and manage connections based on SSL requirements, including a custom verifier for unverified connections.

* black box message update

* chat title updates

* optimizations for trigger.

* some keepalive logic on the anthropic cached

* keep title empty until new one

* no duplicate messages

* null user message on asset pull

* posthog error handling

* 20 sec idle timeout on anthropic

* null req message

* fixed modificiation names missing

* Refactor tool call handling to support new content array format in asset messages and context loaders

* cache most recent file from workflow

* Enhance date and number detection in createDataMetadata function to improve data type handling for metrics files

* group hover effect for message

* logging for chat

* Add messageId handling and file association tracking in dashboard and metrics tools

- Updated runtime context to include messageId in create and modify dashboard and metrics file tools.
- Implemented file association tracking based on messageId in create and modify functions for both dashboards and metrics.
- Ensured type consistency by using AnalystRuntimeContext in runtime context parameters.

* logging for chat

* message type update

* Route to first file instead

* trigger moved to catalog

* Enhance file selection logic to support YAML parsing and improve logging

- Updated `extractMetricIdsFromDashboard` to first attempt JSON parsing, falling back to a regex-based YAML parsing for metric IDs.
- Added detailed debug logging in `selectFilesForResponse` to track file selection process, including metrics and dashboards involved.
- Introduced tests for various scenarios in `file-selection.test.ts` to ensure correct behavior with dashboard context and edge cases.

* trigger dev v4-beta

* Retry + Self Healing (#381)

* Refactor retry logic in analyst and think-and-prep steps

Co-authored-by: dallin <dallin@buster.so>

* some fixes

* console log error

* self healing

* todos retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* remove lots of logs

* Remove chat streaming

* Remove chat streaming

* timeout

* Change to updated at field

* link to home

* Update timeout settings for HTTP and HTTPS agents from 20 seconds to 10 seconds for improved responsiveness.

* Add utils module and integrate message conversion in post_chat_handler

* Implement error handling for extract values (#382)

* Remove chat streaming

* Improve error handling and logging in extract values and chat title steps

Co-authored-by: dallin <dallin@buster.so>

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* loading icon for buster avatar

* finalize tooltip cache

* upgrade mastra

* increase retries

* Add redo functionality for chat messages

- Introduced `redoFromMessageId` parameter in `handleExistingChat` to allow users to specify a message to redo from.
- Implemented validation to ensure the specified message belongs to the current chat.
- Added `softDeleteMessagesFromPoint` function to soft delete a message and all subsequent messages in the same chat, facilitating the redo feature.

* fix electric potential memory leak

* tooltip cache and chart cleanup

* Update bullet to be more indented

* latest version number

* add support endpoint to new server

* Fix jank in combo bar charts

* index check for dashboard

* Collapse only if there are metrics

* Is finished reasoing back

* Update dependencies and enhance chat message handling

- Upgraded `@mastra/core` to version 0.10.8 and added `node-sql-parser` at version 5.3.10 in the lock file.
- Improved integration tests for chat message redo functionality, ensuring correct behavior when deriving `chat_id` from `message_id`.
- Enhanced error handling and validation in the `initializeChat` function to manage cases where `chat_id` is not provided.

* Update pnpm-lock and enhance chat message integration tests

- Added `node-sql-parser` version 5.3.10 to dependencies and updated the lock file.
- Improved integration tests for chat message redo functionality, ensuring accurate deletion and retrieval of messages.
- Enhanced the `initializeChat` function to derive `chat_id` from `message_id` when not provided, improving error handling and validation.

* remove .env import breaking build

* add updated at to the get chat handler

* zmall runtime error fix

* permission tests passing

* return updated at on the get chat handler now

* slq parser fixes

* Implement chat access control logic and add comprehensive tests

- Developed the `canUserAccessChat` function to determine user access to chats based on direct permissions, collection permissions, creator status, and organizational roles.
- Introduced helper functions for checking permissions and retrieving chat information.
- Added integration tests to validate access control logic, covering various scenarios including direct permissions, collection permissions, and user roles.
- Created unit tests to ensure the correctness of the access control function with mocked database interactions.
- Included simple integration tests to verify functionality with existing database data.

* sql parser and int tests working.

* fix test and lint issues

* comment to kick off deployment lo

* access controls on datasets

* electric context bug fix with sql helpers.

* permission and read only

* Add lru-cache dependency and export cache management functions

- Added `lru-cache` as a dependency in the access-controls package.
- Exported new cache management functions from `chats-cached` module, including `canUserAccessChatCached`, `getCacheStats`, `resetCacheStats`, `clearCache`, `invalidateAccess`, `invalidateUserAccess`, and `invalidateChatAccess`.

* packages deploy as well

* wrong workflow lol

* Update AppVerticalCodeSplitter.tsx

* Add error handling for query run and SQL save operations

Co-authored-by: natemkelley <natemkelley@gmail.com>

* Trim whitespace from input values before sending chat prompts

Co-authored-by: natemkelley <natemkelley@gmail.com>

* type in think-and-prep

* use the cached access chat

* update package version

* new asset import message

* Error fallback for login

* Update BusterChart.BarChart.stories.tsx

* Staging changes to fix number card titles, combo chart axis, and using dynamic filters (#386)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* db init command pass through

* combo chart fixes (#387)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* clarifying question and connection logic

* pino pretty error fix

* clarifying is a finishing tool

* change update latest version logic

* Update support endpoint

* fixes for horizontal bar charts and added the combo chart logic to update metrics (#388)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* permission fix on dashboard metric handlers for workspace and data admin

* Add more try catches

* Hide avatar is no more

* Horizontal bar fixes (#389)

* fixes for horizontal bar charts and added the combo chart logic to update metrics

* hopefully fixed horizontal bar charts

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* reasoning shimmer update

* Make the embed flow work with versions

* new account warning update

* Move support modal

* compact number for pie label

* Add final reasoning message tracking and workflow start time to chunk processor and related steps

- Introduced `finalReasoningMessage` to schemas in `analyst-step`, `mark-message-complete-step`, and `create-todos-step`.
- Updated `ChunkProcessor` to calculate and store the final reasoning message based on workflow duration.
- Enhanced various steps to utilize the new `workflowStartTime` for better tracking of execution duration.
- Improved database update logic to include `finalReasoningMessage` when applicable.

* 9 digit cutoff for pie

* trigger update

* test on mastra braintrust

* test deployment

* testing

* pnpm install

* pnpm

* node 22

* pnpm version

* trigger main

* get initial chat file

* hono main deploymenbt

* clear timeouts

* Remove console logs

* migration test to staging

* db url

* try again

* k get rid of tls var

* hmmm lets try this

* mark migrations

* fix migration file?

* drizzle-kit upgrade

* tweaks to the github actions

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Nate Kelley <133379588+nate-kelley-buster@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>

* cert location copy moved (#392)

* biome clean on ai

* add user to flag chat

* attempt to get vercel deployed

* Update tsup.config.ts

* Update pnpm-lock.yaml

* Add @buster/server2 Hono API app with Vercel deployment configuration

* slack oauth integration

* mainly some clean up and biome formatting

* slack oauth

* slack migration + snapshot

* remove unused files

* finalized docker image for porter

* Create porter_app_buster-server_3155.yml file

* Add integration tests for Slack handler and refactor Slack OAuth service

- Introduced integration tests for the Slack handler, covering OAuth initiation, callback handling, and integration status retrieval.
- Refactored Slack OAuth service to improve error handling and ensure proper integration state management.
- Updated token storage implementation to use a database vault instead of Supabase.
- Enhanced existing tests for better coverage and reliability, including cleanup of test data.
- Added new utility functions for managing vault secrets in the database.

* docker image update

* new prompts

* individual tests and a schema fix

* server build

* final working dockerfile

* Update Dockerfile

* new messages to slack messages (#369)

* Update dockerfile

* Update validate-env.js

* update build pipeline

* Update the dockerfile flow

* finalize logging for pino

* stable base

* Update cors middleware logger

* Update cors.ts

* update docker to be more imformative

* Update index.ts

* Update auth.ts

* Update cors.ts

* Update cors.ts

* Update logger.ts

* remove logs

* more cors updates

* build server shared

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies. (#370)

* tons of file parsing errors (#371)

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies.

* tons of file parsing errors

* Dev mode updates

* more stable electric handler

* Dal/agent-self-healing-fixes (#372)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* Dal/agent-self-healing-fixes (#373)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* change porter staging deploy to mastra-braintrust.

* new path for porter deploy

* deploy to staging fix

* Create porter_app_mastra-braintrust-api_3155.yml file (#375)

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>

* Update sizing and opacity

* supe up the instance for mastra

* environment staging

* ssl script

* copy path

* Update list padding

* no throttle and the anthropic cached

* move select to the top

* Update margin inline start

* shrink reasoning vertical space to 2px

* semi bold font for headers

* update animation timing

* haiku

* Add createTodoList tool and integrate into create-todos-step

* chat helper on post chat

* only trigger cicd when change made

* Start created streaming text components

* Refactor analyst agent task to initialize Braintrust logging asynchronously and parallelize database queries for improved performance. Adjusted cleanup timeout for Braintrust traces to reduce delays.

* fixed reasoned for X, so that it rounds down to the minute

* Update users page

* update build pipeline for new web

* document title update

* Named chats for page

* Datasets titles

* Refactor visualization tools and enhance error handling in retryable agent stream. Removed unused metricValueLabel from metrics file tool, updated metric configuration schemas, and improved healing mechanism for tool errors during streaming.

* analyst

* document title updates

* Update useDocumentTitle.tsx

* Refactor tool choice configuration in create-todos-step to use structured object. Remove exponential backoff logic from retryable agent stream for healable errors. Introduce new test for real-world healing scenarios in retryable agent stream.

* Refactor SQL validation logic in modify-metrics-file-tool to skip unnecessary checks when SQL has not changed. Enhance error handling and update validation messages. Clean up code formatting for improved readability.

* update collapse for filecard

* chevron collapse

* Jacob prompt changes (#376)

* prompt changes to improve filtering logic and handle priv/sec errors

* prompt changes to make aggregation better and improved filter best practices

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/agents/think-and-prep-agent/think-and-prep-instructions.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: dal <dallin@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* think and prep

* change header and strong fonts weights

* Update get collection

* combo chart x axis update

* Create a chart schemas as types

* schema types

* simple unit tests for line chart props

* fix the response file ordering iwth active selection.

* copy around reasoning messages taken care of

* fix nullable user message and file processing and such.

* update ticks for chart config

* fix todo parsing.

* app markdown update

* Update splitter to use border instead of width

* change ml

* If no file is found we should auto redirect

* Refactor database connection handling to support SSL modes. Introduced functions to extract SSL parameters and manage connections based on SSL requirements, including a custom verifier for unverified connections.

* black box message update

* chat title updates

* optimizations for trigger.

* some keepalive logic on the anthropic cached

* keep title empty until new one

* no duplicate messages

* null user message on asset pull

* posthog error handling

* 20 sec idle timeout on anthropic

* null req message

* fixed modificiation names missing

* Refactor tool call handling to support new content array format in asset messages and context loaders

* cache most recent file from workflow

* Enhance date and number detection in createDataMetadata function to improve data type handling for metrics files

* group hover effect for message

* logging for chat

* Add messageId handling and file association tracking in dashboard and metrics tools

- Updated runtime context to include messageId in create and modify dashboard and metrics file tools.
- Implemented file association tracking based on messageId in create and modify functions for both dashboards and metrics.
- Ensured type consistency by using AnalystRuntimeContext in runtime context parameters.

* logging for chat

* message type update

* Route to first file instead

* trigger moved to catalog

* Enhance file selection logic to support YAML parsing and improve logging

- Updated `extractMetricIdsFromDashboard` to first attempt JSON parsing, falling back to a regex-based YAML parsing for metric IDs.
- Added detailed debug logging in `selectFilesForResponse` to track file selection process, including metrics and dashboards involved.
- Introduced tests for various scenarios in `file-selection.test.ts` to ensure correct behavior with dashboard context and edge cases.

* trigger dev v4-beta

* Retry + Self Healing (#381)

* Refactor retry logic in analyst and think-and-prep steps

Co-authored-by: dallin <dallin@buster.so>

* some fixes

* console log error

* self healing

* todos retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* remove lots of logs

* Remove chat streaming

* Remove chat streaming

* timeout

* Change to updated at field

* link to home

* Update timeout settings for HTTP and HTTPS agents from 20 seconds to 10 seconds for improved responsiveness.

* Add utils module and integrate message conversion in post_chat_handler

* Implement error handling for extract values (#382)

* Remove chat streaming

* Improve error handling and logging in extract values and chat title steps

Co-authored-by: dallin <dallin@buster.so>

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* loading icon for buster avatar

* finalize tooltip cache

* upgrade mastra

* increase retries

* Add redo functionality for chat messages

- Introduced `redoFromMessageId` parameter in `handleExistingChat` to allow users to specify a message to redo from.
- Implemented validation to ensure the specified message belongs to the current chat.
- Added `softDeleteMessagesFromPoint` function to soft delete a message and all subsequent messages in the same chat, facilitating the redo feature.

* fix electric potential memory leak

* tooltip cache and chart cleanup

* Update bullet to be more indented

* latest version number

* add support endpoint to new server

* Fix jank in combo bar charts

* index check for dashboard

* Collapse only if there are metrics

* Is finished reasoing back

* Update dependencies and enhance chat message handling

- Upgraded `@mastra/core` to version 0.10.8 and added `node-sql-parser` at version 5.3.10 in the lock file.
- Improved integration tests for chat message redo functionality, ensuring correct behavior when deriving `chat_id` from `message_id`.
- Enhanced error handling and validation in the `initializeChat` function to manage cases where `chat_id` is not provided.

* Update pnpm-lock and enhance chat message integration tests

- Added `node-sql-parser` version 5.3.10 to dependencies and updated the lock file.
- Improved integration tests for chat message redo functionality, ensuring accurate deletion and retrieval of messages.
- Enhanced the `initializeChat` function to derive `chat_id` from `message_id` when not provided, improving error handling and validation.

* remove .env import breaking build

* add updated at to the get chat handler

* zmall runtime error fix

* permission tests passing

* return updated at on the get chat handler now

* slq parser fixes

* Implement chat access control logic and add comprehensive tests

- Developed the `canUserAccessChat` function to determine user access to chats based on direct permissions, collection permissions, creator status, and organizational roles.
- Introduced helper functions for checking permissions and retrieving chat information.
- Added integration tests to validate access control logic, covering various scenarios including direct permissions, collection permissions, and user roles.
- Created unit tests to ensure the correctness of the access control function with mocked database interactions.
- Included simple integration tests to verify functionality with existing database data.

* sql parser and int tests working.

* fix test and lint issues

* comment to kick off deployment lo

* access controls on datasets

* electric context bug fix with sql helpers.

* permission and read only

* Add lru-cache dependency and export cache management functions

- Added `lru-cache` as a dependency in the access-controls package.
- Exported new cache management functions from `chats-cached` module, including `canUserAccessChatCached`, `getCacheStats`, `resetCacheStats`, `clearCache`, `invalidateAccess`, `invalidateUserAccess`, and `invalidateChatAccess`.

* packages deploy as well

* wrong workflow lol

* Update AppVerticalCodeSplitter.tsx

* Add error handling for query run and SQL save operations

Co-authored-by: natemkelley <natemkelley@gmail.com>

* Trim whitespace from input values before sending chat prompts

Co-authored-by: natemkelley <natemkelley@gmail.com>

* type in think-and-prep

* use the cached access chat

* update package version

* new asset import message

* Error fallback for login

* Update BusterChart.BarChart.stories.tsx

* Staging changes to fix number card titles, combo chart axis, and using dynamic filters (#386)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* db init command pass through

* combo chart fixes (#387)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* clarifying question and connection logic

* pino pretty error fix

* clarifying is a finishing tool

* change update latest version logic

* Update support endpoint

* fixes for horizontal bar charts and added the combo chart logic to update metrics (#388)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* permission fix on dashboard metric handlers for workspace and data admin

* Add more try catches

* Hide avatar is no more

* Horizontal bar fixes (#389)

* fixes for horizontal bar charts and added the combo chart logic to update metrics

* hopefully fixed horizontal bar charts

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* reasoning shimmer update

* Make the embed flow work with versions

* new account warning update

* Move support modal

* compact number for pie label

* Add final reasoning message tracking and workflow start time to chunk processor and related steps

- Introduced `finalReasoningMessage` to schemas in `analyst-step`, `mark-message-complete-step`, and `create-todos-step`.
- Updated `ChunkProcessor` to calculate and store the final reasoning message based on workflow duration.
- Enhanced various steps to utilize the new `workflowStartTime` for better tracking of execution duration.
- Improved database update logic to include `finalReasoningMessage` when applicable.

* 9 digit cutoff for pie

* trigger update

* test on mastra braintrust

* test deployment

* testing

* pnpm install

* pnpm

* node 22

* pnpm version

* trigger main

* get initial chat file

* hono main deploymenbt

* clear timeouts

* Remove console logs

* migration test to staging

* db url

* try again

* k get rid of tls var

* hmmm lets try this

* mark migrations

* fix migration file?

* drizzle-kit upgrade

* tweaks to the github actions

* clean up workflows db migration

* cert location copy moved

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Nate Kelley <133379588+nate-kelley-buster@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>

* deploy to staging (#393)

* Mastra braintrust (#394)

* Update tsup.config.ts

* Update pnpm-lock.yaml

* Add @buster/server2 Hono API app with Vercel deployment configuration

* slack oauth integration

* mainly some clean up and biome formatting

* slack oauth

* slack migration + snapshot

* remove unused files

* finalized docker image for porter

* Create porter_app_buster-server_3155.yml file

* Add integration tests for Slack handler and refactor Slack OAuth service

- Introduced integration tests for the Slack handler, covering OAuth initiation, callback handling, and integration status retrieval.
- Refactored Slack OAuth service to improve error handling and ensure proper integration state management.
- Updated token storage implementation to use a database vault instead of Supabase.
- Enhanced existing tests for better coverage and reliability, including cleanup of test data.
- Added new utility functions for managing vault secrets in the database.

* docker image update

* new prompts

* individual tests and a schema fix

* server build

* final working dockerfile

* Update Dockerfile

* new messages to slack messages (#369)

* Update dockerfile

* Update validate-env.js

* update build pipeline

* Update the dockerfile flow

* finalize logging for pino

* stable base

* Update cors middleware logger

* Update cors.ts

* update docker to be more imformative

* Update index.ts

* Update auth.ts

* Update cors.ts

* Update cors.ts

* Update logger.ts

* remove logs

* more cors updates

* build server shared

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies. (#370)

* tons of file parsing errors (#371)

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies.

* tons of file parsing errors

* Dev mode updates

* more stable electric handler

* Dal/agent-self-healing-fixes (#372)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* Dal/agent-self-healing-fixes (#373)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* change porter staging deploy to mastra-braintrust.

* new path for porter deploy

* deploy to staging fix

* Create porter_app_mastra-braintrust-api_3155.yml file (#375)

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>

* Update sizing and opacity

* supe up the instance for mastra

* environment staging

* ssl script

* copy path

* Update list padding

* no throttle and the anthropic cached

* move select to the top

* Update margin inline start

* shrink reasoning vertical space to 2px

* semi bold font for headers

* update animation timing

* haiku

* Add createTodoList tool and integrate into create-todos-step

* chat helper on post chat

* only trigger cicd when change made

* Start created streaming text components

* Refactor analyst agent task to initialize Braintrust logging asynchronously and parallelize database queries for improved performance. Adjusted cleanup timeout for Braintrust traces to reduce delays.

* fixed reasoned for X, so that it rounds down to the minute

* Update users page

* update build pipeline for new web

* document title update

* Named chats for page

* Datasets titles

* Refactor visualization tools and enhance error handling in retryable agent stream. Removed unused metricValueLabel from metrics file tool, updated metric configuration schemas, and improved healing mechanism for tool errors during streaming.

* analyst

* document title updates

* Update useDocumentTitle.tsx

* Refactor tool choice configuration in create-todos-step to use structured object. Remove exponential backoff logic from retryable agent stream for healable errors. Introduce new test for real-world healing scenarios in retryable agent stream.

* Refactor SQL validation logic in modify-metrics-file-tool to skip unnecessary checks when SQL has not changed. Enhance error handling and update validation messages. Clean up code formatting for improved readability.

* update collapse for filecard

* chevron collapse

* Jacob prompt changes (#376)

* prompt changes to improve filtering logic and handle priv/sec errors

* prompt changes to make aggregation better and improved filter best practices

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/agents/think-and-prep-agent/think-and-prep-instructions.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: dal <dallin@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* think and prep

* change header and strong fonts weights

* Update get collection

* combo chart x axis update

* Create a chart schemas as types

* schema types

* simple unit tests for line chart props

* fix the response file ordering iwth active selection.

* copy around reasoning messages taken care of

* fix nullable user message and file processing and such.

* update ticks for chart config

* fix todo parsing.

* app markdown update

* Update splitter to use border instead of width

* change ml

* If no file is found we should auto redirect

* Refactor database connection handling to support SSL modes. Introduced functions to extract SSL parameters and manage connections based on SSL requirements, including a custom verifier for unverified connections.

* black box message update

* chat title updates

* optimizations for trigger.

* some keepalive logic on the anthropic cached

* keep title empty until new one

* no duplicate messages

* null user message on asset pull

* posthog error handling

* 20 sec idle timeout on anthropic

* null req message

* fixed modificiation names missing

* Refactor tool call handling to support new content array format in asset messages and context loaders

* cache most recent file from workflow

* Enhance date and number detection in createDataMetadata function to improve data type handling for metrics files

* group hover effect for message

* logging for chat

* Add messageId handling and file association tracking in dashboard and metrics tools

- Updated runtime context to include messageId in create and modify dashboard and metrics file tools.
- Implemented file association tracking based on messageId in create and modify functions for both dashboards and metrics.
- Ensured type consistency by using AnalystRuntimeContext in runtime context parameters.

* logging for chat

* message type update

* Route to first file instead

* trigger moved to catalog

* Enhance file selection logic to support YAML parsing and improve logging

- Updated `extractMetricIdsFromDashboard` to first attempt JSON parsing, falling back to a regex-based YAML parsing for metric IDs.
- Added detailed debug logging in `selectFilesForResponse` to track file selection process, including metrics and dashboards involved.
- Introduced tests for various scenarios in `file-selection.test.ts` to ensure correct behavior with dashboard context and edge cases.

* trigger dev v4-beta

* Retry + Self Healing (#381)

* Refactor retry logic in analyst and think-and-prep steps

Co-authored-by: dallin <dallin@buster.so>

* some fixes

* console log error

* self healing

* todos retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* remove lots of logs

* Remove chat streaming

* Remove chat streaming

* timeout

* Change to updated at field

* link to home

* Update timeout settings for HTTP and HTTPS agents from 20 seconds to 10 seconds for improved responsiveness.

* Add utils module and integrate message conversion in post_chat_handler

* Implement error handling for extract values (#382)

* Remove chat streaming

* Improve error handling and logging in extract values and chat title steps

Co-authored-by: dallin <dallin@buster.so>

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* loading icon for buster avatar

* finalize tooltip cache

* upgrade mastra

* increase retries

* Add redo functionality for chat messages

- Introduced `redoFromMessageId` parameter in `handleExistingChat` to allow users to specify a message to redo from.
- Implemented validation to ensure the specified message belongs to the current chat.
- Added `softDeleteMessagesFromPoint` function to soft delete a message and all subsequent messages in the same chat, facilitating the redo feature.

* fix electric potential memory leak

* tooltip cache and chart cleanup

* Update bullet to be more indented

* latest version number

* add support endpoint to new server

* Fix jank in combo bar charts

* index check for dashboard

* Collapse only if there are metrics

* Is finished reasoing back

* Update dependencies and enhance chat message handling

- Upgraded `@mastra/core` to version 0.10.8 and added `node-sql-parser` at version 5.3.10 in the lock file.
- Improved integration tests for chat message redo functionality, ensuring correct behavior when deriving `chat_id` from `message_id`.
- Enhanced error handling and validation in the `initializeChat` function to manage cases where `chat_id` is not provided.

* Update pnpm-lock and enhance chat message integration tests

- Added `node-sql-parser` version 5.3.10 to dependencies and updated the lock file.
- Improved integration tests for chat message redo functionality, ensuring accurate deletion and retrieval of messages.
- Enhanced the `initializeChat` function to derive `chat_id` from `message_id` when not provided, improving error handling and validation.

* remove .env import breaking build

* add updated at to the get chat handler

* zmall runtime error fix

* permission tests passing

* return updated at on the get chat handler now

* slq parser fixes

* Implement chat access control logic and add comprehensive tests

- Developed the `canUserAccessChat` function to determine user access to chats based on direct permissions, collection permissions, creator status, and organizational roles.
- Introduced helper functions for checking permissions and retrieving chat information.
- Added integration tests to validate access control logic, covering various scenarios including direct permissions, collection permissions, and user roles.
- Created unit tests to ensure the correctness of the access control function with mocked database interactions.
- Included simple integration tests to verify functionality with existing database data.

* sql parser and int tests working.

* fix test and lint issues

* comment to kick off deployment lo

* access controls on datasets

* electric context bug fix with sql helpers.

* permission and read only

* Add lru-cache dependency and export cache management functions

- Added `lru-cache` as a dependency in the access-controls package.
- Exported new cache management functions from `chats-cached` module, including `canUserAccessChatCached`, `getCacheStats`, `resetCacheStats`, `clearCache`, `invalidateAccess`, `invalidateUserAccess`, and `invalidateChatAccess`.

* packages deploy as well

* wrong workflow lol

* Update AppVerticalCodeSplitter.tsx

* Add error handling for query run and SQL save operations

Co-authored-by: natemkelley <natemkelley@gmail.com>

* Trim whitespace from input values before sending chat prompts

Co-authored-by: natemkelley <natemkelley@gmail.com>

* type in think-and-prep

* use the cached access chat

* update package version

* new asset import message

* Error fallback for login

* Update BusterChart.BarChart.stories.tsx

* Staging changes to fix number card titles, combo chart axis, and using dynamic filters (#386)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* db init command pass through

* combo chart fixes (#387)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* clarifying question and connection logic

* pino pretty error fix

* clarifying is a finishing tool

* change update latest version logic

* Update support endpoint

* fixes for horizontal bar charts and added the combo chart logic to update metrics (#388)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* permission fix on dashboard metric handlers for workspace and data admin

* Add more try catches

* Hide avatar is no more

* Horizontal bar fixes (#389)

* fixes for horizontal bar charts and added the combo chart logic to update metrics

* hopefully fixed horizontal bar charts

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* reasoning shimmer update

* Make the embed flow work with versions

* new account warning update

* Move support modal

* compact number for pie label

* Add final reasoning message tracking and workflow start time to chunk processor and related steps

- Introduced `finalReasoningMessage` to schemas in `analyst-step`, `mark-message-complete-step`, and `create-todos-step`.
- Updated `ChunkProcessor` to calculate and store the final reasoning message based on workflow duration.
- Enhanced various steps to utilize the new `workflowStartTime` for better tracking of execution duration.
- Improved database update logic to include `finalReasoningMessage` when applicable.

* 9 digit cutoff for pie

* trigger update

* test on mastra braintrust

* test deployment

* testing

* pnpm install

* pnpm

* node 22

* pnpm version

* trigger main

* get initial chat file

* hono main deploymenbt

* clear timeouts

* Remove console logs

* migration test to staging

* db url

* try again

* k get rid of tls var

* hmmm lets try this

* mark migrations

* fix migration file?

* drizzle-kit upgrade

* tweaks to the github actions

* clean up workflows db migration

* cert location copy moved

* Prism highlighting update

* Fix merge conflicts

---------

Co-authored-by: dal <dallin@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>

* Remove logs

* Remove logs (#395)

* Fix broken web unit tests

* Remove useless test

* Update helpers.test.ts

* Create tsconfig.json

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Nate Kelley <133379588+nate-kelley-buster@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>
2025-07-03 06:32:18 -07:00
Nate Kelley e3990c5f7d
Fix broken web unit tests 2025-07-02 17:00:19 -06:00
dal 58f9d261e6
Mastra braintrust (#391)
* type fixes

* biome clean on ai

* add user to flag chat

* attempt to get vercel deployed

* Update tsup.config.ts

* Update pnpm-lock.yaml

* Add @buster/server2 Hono API app with Vercel deployment configuration

* slack oauth integration

* mainly some clean up and biome formatting

* slack oauth

* slack migration + snapshot

* remove unused files

* finalized docker image for porter

* Create porter_app_buster-server_3155.yml file

* Add integration tests for Slack handler and refactor Slack OAuth service

- Introduced integration tests for the Slack handler, covering OAuth initiation, callback handling, and integration status retrieval.
- Refactored Slack OAuth service to improve error handling and ensure proper integration state management.
- Updated token storage implementation to use a database vault instead of Supabase.
- Enhanced existing tests for better coverage and reliability, including cleanup of test data.
- Added new utility functions for managing vault secrets in the database.

* docker image update

* new prompts

* individual tests and a schema fix

* server build

* final working dockerfile

* Update Dockerfile

* new messages to slack messages (#369)

* Update dockerfile

* Update validate-env.js

* update build pipeline

* Update the dockerfile flow

* finalize logging for pino

* stable base

* Update cors middleware logger

* Update cors.ts

* update docker to be more imformative

* Update index.ts

* Update auth.ts

* Update cors.ts

* Update cors.ts

* Update logger.ts

* remove logs

* more cors updates

* build server shared

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies. (#370)

* tons of file parsing errors (#371)

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies.

* tons of file parsing errors

* Dev mode updates

* more stable electric handler

* Dal/agent-self-healing-fixes (#372)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* Dal/agent-self-healing-fixes (#373)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* change porter staging deploy to mastra-braintrust.

* new path for porter deploy

* deploy to staging fix

* Create porter_app_mastra-braintrust-api_3155.yml file (#375)

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>

* Update sizing and opacity

* supe up the instance for mastra

* environment staging

* ssl script

* copy path

* Update list padding

* no throttle and the anthropic cached

* move select to the top

* Update margin inline start

* shrink reasoning vertical space to 2px

* semi bold font for headers

* update animation timing

* haiku

* Add createTodoList tool and integrate into create-todos-step

* chat helper on post chat

* only trigger cicd when change made

* Start created streaming text components

* Refactor analyst agent task to initialize Braintrust logging asynchronously and parallelize database queries for improved performance. Adjusted cleanup timeout for Braintrust traces to reduce delays.

* fixed reasoned for X, so that it rounds down to the minute

* Update users page

* update build pipeline for new web

* document title update

* Named chats for page

* Datasets titles

* Refactor visualization tools and enhance error handling in retryable agent stream. Removed unused metricValueLabel from metrics file tool, updated metric configuration schemas, and improved healing mechanism for tool errors during streaming.

* analyst

* document title updates

* Update useDocumentTitle.tsx

* Refactor tool choice configuration in create-todos-step to use structured object. Remove exponential backoff logic from retryable agent stream for healable errors. Introduce new test for real-world healing scenarios in retryable agent stream.

* Refactor SQL validation logic in modify-metrics-file-tool to skip unnecessary checks when SQL has not changed. Enhance error handling and update validation messages. Clean up code formatting for improved readability.

* update collapse for filecard

* chevron collapse

* Jacob prompt changes (#376)

* prompt changes to improve filtering logic and handle priv/sec errors

* prompt changes to make aggregation better and improved filter best practices

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/agents/think-and-prep-agent/think-and-prep-instructions.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: dal <dallin@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* think and prep

* change header and strong fonts weights

* Update get collection

* combo chart x axis update

* Create a chart schemas as types

* schema types

* simple unit tests for line chart props

* fix the response file ordering iwth active selection.

* copy around reasoning messages taken care of

* fix nullable user message and file processing and such.

* update ticks for chart config

* fix todo parsing.

* app markdown update

* Update splitter to use border instead of width

* change ml

* If no file is found we should auto redirect

* Refactor database connection handling to support SSL modes. Introduced functions to extract SSL parameters and manage connections based on SSL requirements, including a custom verifier for unverified connections.

* black box message update

* chat title updates

* optimizations for trigger.

* some keepalive logic on the anthropic cached

* keep title empty until new one

* no duplicate messages

* null user message on asset pull

* posthog error handling

* 20 sec idle timeout on anthropic

* null req message

* fixed modificiation names missing

* Refactor tool call handling to support new content array format in asset messages and context loaders

* cache most recent file from workflow

* Enhance date and number detection in createDataMetadata function to improve data type handling for metrics files

* group hover effect for message

* logging for chat

* Add messageId handling and file association tracking in dashboard and metrics tools

- Updated runtime context to include messageId in create and modify dashboard and metrics file tools.
- Implemented file association tracking based on messageId in create and modify functions for both dashboards and metrics.
- Ensured type consistency by using AnalystRuntimeContext in runtime context parameters.

* logging for chat

* message type update

* Route to first file instead

* trigger moved to catalog

* Enhance file selection logic to support YAML parsing and improve logging

- Updated `extractMetricIdsFromDashboard` to first attempt JSON parsing, falling back to a regex-based YAML parsing for metric IDs.
- Added detailed debug logging in `selectFilesForResponse` to track file selection process, including metrics and dashboards involved.
- Introduced tests for various scenarios in `file-selection.test.ts` to ensure correct behavior with dashboard context and edge cases.

* trigger dev v4-beta

* Retry + Self Healing (#381)

* Refactor retry logic in analyst and think-and-prep steps

Co-authored-by: dallin <dallin@buster.so>

* some fixes

* console log error

* self healing

* todos retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* remove lots of logs

* Remove chat streaming

* Remove chat streaming

* timeout

* Change to updated at field

* link to home

* Update timeout settings for HTTP and HTTPS agents from 20 seconds to 10 seconds for improved responsiveness.

* Add utils module and integrate message conversion in post_chat_handler

* Implement error handling for extract values (#382)

* Remove chat streaming

* Improve error handling and logging in extract values and chat title steps

Co-authored-by: dallin <dallin@buster.so>

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* loading icon for buster avatar

* finalize tooltip cache

* upgrade mastra

* increase retries

* Add redo functionality for chat messages

- Introduced `redoFromMessageId` parameter in `handleExistingChat` to allow users to specify a message to redo from.
- Implemented validation to ensure the specified message belongs to the current chat.
- Added `softDeleteMessagesFromPoint` function to soft delete a message and all subsequent messages in the same chat, facilitating the redo feature.

* fix electric potential memory leak

* tooltip cache and chart cleanup

* Update bullet to be more indented

* latest version number

* add support endpoint to new server

* Fix jank in combo bar charts

* index check for dashboard

* Collapse only if there are metrics

* Is finished reasoing back

* Update dependencies and enhance chat message handling

- Upgraded `@mastra/core` to version 0.10.8 and added `node-sql-parser` at version 5.3.10 in the lock file.
- Improved integration tests for chat message redo functionality, ensuring correct behavior when deriving `chat_id` from `message_id`.
- Enhanced error handling and validation in the `initializeChat` function to manage cases where `chat_id` is not provided.

* Update pnpm-lock and enhance chat message integration tests

- Added `node-sql-parser` version 5.3.10 to dependencies and updated the lock file.
- Improved integration tests for chat message redo functionality, ensuring accurate deletion and retrieval of messages.
- Enhanced the `initializeChat` function to derive `chat_id` from `message_id` when not provided, improving error handling and validation.

* remove .env import breaking build

* add updated at to the get chat handler

* zmall runtime error fix

* permission tests passing

* return updated at on the get chat handler now

* slq parser fixes

* Implement chat access control logic and add comprehensive tests

- Developed the `canUserAccessChat` function to determine user access to chats based on direct permissions, collection permissions, creator status, and organizational roles.
- Introduced helper functions for checking permissions and retrieving chat information.
- Added integration tests to validate access control logic, covering various scenarios including direct permissions, collection permissions, and user roles.
- Created unit tests to ensure the correctness of the access control function with mocked database interactions.
- Included simple integration tests to verify functionality with existing database data.

* sql parser and int tests working.

* fix test and lint issues

* comment to kick off deployment lo

* access controls on datasets

* electric context bug fix with sql helpers.

* permission and read only

* Add lru-cache dependency and export cache management functions

- Added `lru-cache` as a dependency in the access-controls package.
- Exported new cache management functions from `chats-cached` module, including `canUserAccessChatCached`, `getCacheStats`, `resetCacheStats`, `clearCache`, `invalidateAccess`, `invalidateUserAccess`, and `invalidateChatAccess`.

* packages deploy as well

* wrong workflow lol

* Update AppVerticalCodeSplitter.tsx

* Add error handling for query run and SQL save operations

Co-authored-by: natemkelley <natemkelley@gmail.com>

* Trim whitespace from input values before sending chat prompts

Co-authored-by: natemkelley <natemkelley@gmail.com>

* type in think-and-prep

* use the cached access chat

* update package version

* new asset import message

* Error fallback for login

* Update BusterChart.BarChart.stories.tsx

* Staging changes to fix number card titles, combo chart axis, and using dynamic filters (#386)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* db init command pass through

* combo chart fixes (#387)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* clarifying question and connection logic

* pino pretty error fix

* clarifying is a finishing tool

* change update latest version logic

* Update support endpoint

* fixes for horizontal bar charts and added the combo chart logic to update metrics (#388)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* permission fix on dashboard metric handlers for workspace and data admin

* Add more try catches

* Hide avatar is no more

* Horizontal bar fixes (#389)

* fixes for horizontal bar charts and added the combo chart logic to update metrics

* hopefully fixed horizontal bar charts

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* reasoning shimmer update

* Make the embed flow work with versions

* new account warning update

* Move support modal

* compact number for pie label

* Add final reasoning message tracking and workflow start time to chunk processor and related steps

- Introduced `finalReasoningMessage` to schemas in `analyst-step`, `mark-message-complete-step`, and `create-todos-step`.
- Updated `ChunkProcessor` to calculate and store the final reasoning message based on workflow duration.
- Enhanced various steps to utilize the new `workflowStartTime` for better tracking of execution duration.
- Improved database update logic to include `finalReasoningMessage` when applicable.

* 9 digit cutoff for pie

* trigger update

* test on mastra braintrust

* test deployment

* testing

* pnpm install

* pnpm

* node 22

* pnpm version

* trigger main

* get initial chat file

* hono main deploymenbt

* clear timeouts

* Remove console logs

* migration test to staging

* db url

* try again

* k get rid of tls var

* hmmm lets try this

* mark migrations

* fix migration file?

* drizzle-kit upgrade

* tweaks to the github actions

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Nate Kelley <133379588+nate-kelley-buster@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>
2025-07-02 14:33:40 -07:00
dal d4bd71b988 Refactor buster-cli model structures and update YAML templates
- Renamed `semantic_models` to `models` in YAML template and related structures for consistency.
- Changed `expr` field type in `PostDatasetsEntityRelationshipsRequest` from `Vec<String>` to `String` to simplify data handling.
- Updated `BusterModel` struct to reflect the new `models` naming and removed unused `ModelDefaults` struct.
- Adjusted tests to align with the updated model structure.

These changes enhance clarity and maintainability in the model representation and data handling within the buster-cli.
2025-01-03 13:26:48 -07:00
dal 1cff6c01e4 Refactor buster-cli model structures and update YAML templates
- Renamed `semantic_models` to `models` in YAML template and related structures for consistency.
- Changed `expr` field type in `PostDatasetsEntityRelationshipsRequest` from `Vec<String>` to `String` to simplify data handling.
- Updated `BusterModel` struct to reflect the new `models` naming and removed unused `ModelDefaults` struct.
- Adjusted tests to align with the updated model structure.

These changes enhance clarity and maintainability in the model representation and data handling within the buster-cli.
2025-01-02 16:48:10 -07:00
Dallin Bentley ed1e28da1c Enhance buster-cli initialization and model file handling
- Added a TODO in the init function to retrieve and store IDs in artifacts.
- Introduced an optional `sql_definition` field in the `PostDatasetsRequest` struct for better model representation.
- Refactored `get_model_files` to process directories recursively, improving model file retrieval.
- Added logging for model file uploads to Buster, enhancing visibility during operations.

These changes improve the initialization process and enhance the handling of model files, setting the stage for future features.
2024-11-26 16:09:48 -07:00
Dallin Bentley 3ddab17ff3 Implement error handling improvements and remove unused templates in buster-cli
- Added conversion from `anyhow::Error` to `BusterError` for better error management.
- Deleted unused `buster_model.rs` and `buster_project.rs` template files to clean up the codebase.
- Enhanced the `deploy` command with improved error handling and credential validation.
- Updated model file handling to support optional model definitions.

This commit streamlines error handling and reduces clutter in the project structure.
2024-11-26 14:36:38 -07:00
Dallin Bentley 1f6e6c68c6 Refactor buster-cli: Simplify Deploy command by removing parameters, update deploy function signature, and enhance model file handling with new data structures for YAML parsing. Cleaned up init function and removed unnecessary comments. 2024-11-26 10:47:43 -07:00
Dallin Bentley d616f98c90 putting the init function back bc I accidentally deleted it 2024-11-26 08:48:08 -07:00
Dallin Bentley 6194f61356 Refactor buster-cli: Remove init and credentials modules, introduce buster_credentials module, and enhance API client. Added ratatui dependency for improved UI. Updated credential handling and API key validation logic. Implemented new data source posting functionality. 2024-11-26 08:46:09 -07:00
Dallin Bentley 785fe6f237 Enhance buster-cli initialization with dbt project integration and user selection. Added MultiSelect for existing dbt projects, improved credential handling, and refactored profile management in YAML. Introduced new dbt module for profiles and connection configurations. 2024-11-25 17:44:10 -07:00
Dallin Bentley 3a40be46cc check api key on init 2024-11-25 15:13:46 -07:00
Dallin Bentley 2642739bdb added in more configs 2024-11-25 12:49:05 -07:00
Dallin Bentley 1c5d13b25d modelfile helper and starting point for fields. 2024-11-25 11:46:48 -07:00
Dallin Bentley a6c5dff778 add in buster-cli and start prepping for analytics/bi app 2024-11-21 15:47:21 -07:00