Commit Graph

418 Commits

Author SHA1 Message Date
jacob-buster b17cc4695f Initial Report Builder Prompts 2025-07-30 12:22:50 -06:00
dal 0450b8c350
fallback and the analyst agent 2025-07-29 22:07:11 -06:00
dal 81ebcc67ca
fallback and analyst agent changes 2025-07-29 21:35:01 -06:00
dal bfee540c5b
starting migration to ai sdk 5 2025-07-29 20:57:22 -06:00
dal 2f8622e5bb
grep search getting me? 2025-07-29 20:14:20 -06:00
dal a6ea5bef71
switched the tools to use commands instead of scripts where possible 2025-07-29 16:05:47 -06:00
dal f2a8388b86
feat: enhance repository tree output with current directory information
- Added functionality to retrieve and prepend the current working directory to the repository tree output, improving context for users.
- Updated runtime context to store the modified tree structure with location information.
- Removed outdated integration tests and streamlined the test suite for better maintainability.

This update enhances the usability of the repository tree feature by providing clearer context during execution.
2025-07-29 14:48:18 -06:00
dal 3b1354ddff
refactor: enhance docs agent execution with abort handling and improved error management
- Introduced an AbortController to manage idle tool execution, allowing for graceful aborting of the documentation process.
- Moved tracking variables outside the try block for better state management.
- Updated error handling to gracefully manage abort scenarios, ensuring clear logging and returning final state information.
- Adjusted integration tests to reflect changes in the docs agent workflow, improving overall robustness and reliability.

This update enhances the responsiveness of the docs agent during execution and improves error handling for better user experience.
2025-07-29 14:32:31 -06:00
dal b31f856c3b
refactor: enhance file tools with sandbox execution and error handling improvements
- Updated file tools to generate CommonJS code for sandbox execution, improving robustness and flexibility.
- Refactored bash-execute-tool, create-files-tool, delete-files-tool, edit-files-tool, grep-search-tool, ls-files-tool, and read-files-tool to handle command arguments as JSON, enhancing error handling and output consistency.
- Improved integration tests to validate new execution patterns and error scenarios, ensuring comprehensive coverage across all tools.
- Enhanced error messages for better clarity during execution failures.

This update streamlines the execution process and enhances the overall reliability of file tools.
2025-07-29 14:21:13 -06:00
dal 54c7843b15
refactor: convert file-tools integration tests to concurrent execution
- Update all 7 file-tools integration tests to use concurrent execution
- Create individual sandboxes per test instead of shared beforeAll/afterAll
- Use try/finally blocks for proper sandbox cleanup
- Add 65-second timeouts to accommodate sandbox operations
- Replace it.skipIf pattern with (hasApiKey ? it.concurrent : it.concurrent.skip)
- Improves test performance by enabling parallel execution

Affected tools:
- bash-tool
- create-files-tool
- delete-files-tool
- edit-files-tool
- grep-search-tool
- ls-files-tool
- read-files-tool

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 09:29:20 -06:00
dal 60bc1d37a8
Revert "Revert "Add createReports and editReports tools to analyst-agent""
This reverts commit 67c2eea1f1.
2025-07-29 09:13:29 -06:00
dal 67c2eea1f1
Revert "Add createReports and editReports tools to analyst-agent"
This reverts commit 351689f93b.
2025-07-29 09:06:27 -06:00
dal 351689f93b
Add createReports and editReports tools to analyst-agent 2025-07-29 08:04:48 -06:00
dal 6d36edeeb2
moving tools over 2025-07-29 07:39:09 -06:00
dal ed9ab33b35
feat: enhance file tools to support base64 encoding for command arguments
- Updated bash-execute-script, create-files-script, edit-files-script, and grep-search-script to handle base64 encoded JSON arguments, improving robustness against data corruption.
- Refactored corresponding tool scripts to encode command parameters as base64 before execution.
- Enhanced integration tests to validate the new base64 encoding functionality across various file tools.
2025-07-28 16:55:52 -06:00
dal 3b706a73b3
refactor: streamline file tool scripts and enhance error handling
- Removed sequential test execution settings from vitest configuration to improve performance.
- Updated bash-execute-script to handle missing arguments and improved error handling for JSON parsing.
- Refactored create-files-tool to process file creation sequentially, enhancing directory management and error reporting.
- Enhanced read-files-tool to return structured error messages when no file paths are provided.
- Updated grep-search-tool to accept raw ripgrep commands, simplifying the command structure and improving output handling.
- Removed outdated test files and integrated new test cases for improved coverage and validation across file tools.
2025-07-28 16:22:12 -06:00
dal a41d5c944b
feat: implement ls-files functionality with comprehensive tests
- Introduced the `ls-files-script` module to handle file listing with various options (detailed, all, recursive, human-readable).
- Added integration tests for the `ls-files-script` to validate functionality across different scenarios, including handling of hidden files and non-existent paths.
- Removed the previous `ls-files-tool` integration tests to streamline the testing process and focus on the new implementation.
- Updated the `ls-files-tool` to utilize the new `ls-files-script` for execution and code generation.
- Enhanced error handling and output parsing for improved robustness.
2025-07-28 14:57:05 -06:00
dal 05c309b0c1
refactor: convert create-files-tool to script pattern
- Rename create-file-functions.ts to create-files-script.ts
- Implement script pattern matching ls-files-tool structure
- Update create-file-tool.ts to execute script via sandbox
- Add comprehensive unit tests for script functionality
- Add integration tests for script execution
- Update existing tests to work with new pattern

The create-files-tool now follows the established script pattern where
the script handles the core file creation logic and the tool serves as
a medium that executes the script in a sandboxed environment.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 13:54:03 -06:00
dal 786e4be791
feat: add integration tests for file tools and enhance todo list functionality
- Introduced integration tests for various file tools including bash execution, file creation, deletion, editing, and listing.
- Enhanced the check-off todo list tool to support multiple items and improved error handling for non-existent tasks.
- Updated test cases to cover edge scenarios and ensure robust validation across all tools.
- Increased timeout for the docs agent workflow to accommodate longer-running tests.
2025-07-28 12:41:07 -06:00
dal 2f4aeba817
feat: enhance documentation and testing capabilities for docs agent workflow
- Added section in CLAUDE.md for direct database access during integration testing.
- Updated `maxSteps` in `docs-agent` to allow for more complex tasks.
- Improved validation in `docs-agent-context` for sandbox instances.
- Enhanced `create-docs-todos` step to handle todos more effectively.
- Introduced comprehensive integration tests for the docs agent workflow, covering various scenarios and edge cases.
- Added test helpers for creating mock dbt projects and managing sandboxes.
- Implemented error handling and logging improvements in the workflow execution process.
2025-07-28 11:56:59 -06:00
jacob-buster ae0c4f048d Fixes for the filter tool + changes for data existence 2025-07-28 11:19:45 -06:00
dal 4bef4205f3
feat(access-controls): migrate Rust access control libraries to TypeScript
This PR migrates the Rust  and  libraries to TypeScript in the new  package.

## Changes

### New Access Controls Package
- Implemented complete asset permissions system with CRUD operations
- Added dataset permissions with all 5 access paths (admin, direct user, user→group, team→direct, team→group, org→default)
- Implemented cascading permissions (metrics inherit from dashboards/chats/collections)
- Added LRU caching to replace Redis (30-second TTL)
- Comprehensive test coverage (148 tests passing, 3 skipped pending createUser)

### Database Package Updates
- Fixed AssetType enum to include all 14 asset types
- Added missing database queries for permissions
- Fixed timestamp fields to use ISO strings
- Added getUserOrganizationsByUserId function
- Created organized query structure by table

### AI Package Updates
- Updated getPermissionedDatasets API calls to new format
- Changed property references from ymlFile to ymlContent
- Fixed all test mocks to match new API structure

### Trigger App Updates
- Updated to use new access controls API
- Fixed test mocks to match DatasetListResult structure
- Added proper type imports

### Server Shared Updates
- Added request/response types for access control endpoints
- Exported types for API consistency

## Testing
- All packages build successfully with turbo build:dry-run
- All tests pass with turbo test:unit
- Full type safety maintained throughout

## Migration Notes
- Drop-in replacement for Rust libraries
- Same permission model and roles
- Cache invalidation strategies implemented
- Soft delete pattern maintained

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 10:17:35 -06:00
dal a08f32fe1d
refactor: remove unused test setup files and helper functions to streamline testing process 2025-07-28 08:48:48 -06:00
dal c9b52ccdb8
Refactor CLAUDE.md for improved build and linting instructions; update context handling in workflow and schemas. Enhance test validation for clarification questions and adjust output structure in related tests. 2025-07-26 12:43:25 -06:00
dal a92c3a0b44
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-25 19:10:19 -06:00
dal 2833c6bb0c
dry run 2025-07-25 18:29:35 -06:00
dal 523a6921b1
hotfix on sql dialect 2025-07-25 14:31:35 -06:00
dal a6f8e041cd
fix 1 hr 2025-07-24 10:16:59 -06:00
dal 2c02b9189b
fix the 1 hr ttl 2025-07-24 10:16:19 -06:00
dal ed77fb4409
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-24 10:13:47 -06:00
dal 2d8890837a
add the extended cache beta header 2025-07-24 10:12:48 -06:00
dal 19f7256a27
switch system messages to leverage caching a bit better 2025-07-24 10:06:18 -06:00
dal 0c3994da00
hotfix: use v2 on chats 2025-07-24 09:22:30 -06:00
dal 7cd3aed6fa
Refactor import statements and enhance error messages in SQL validation tests
- Simplified import statements in web-search-tool.ts and web-search-tool.test.ts for better readability.
- Updated error messages in sql-parser-helpers.test.ts to provide clearer feedback regarding wildcard usage on physical tables.
2025-07-23 21:46:51 -06:00
Devin AI ae9f0c4f7e Fix incorrect import paths from '@buster-tools/web-tools' to '@buster/web-tools'
- Update package.json dependencies in ai and trigger packages
- Fix import statements in web-search-tool.ts and web-search-tool.test.ts
- Update pnpm-lock.yaml to reflect correct package references
- Resolves CI build failure due to missing module '@buster-tools/web-tools'

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:20:46 +00:00
Devin AI ab374dc465 BUS-1524: Enhance error messages for tool availability and SQL wildcards
- Update createWorkflowAwareHealingMessage to include next workflow step info and transition descriptions
- Replace SQL wildcard error message with more user-friendly text
- Maintain existing function signatures and return structures

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:09:56 +00:00
dal 5f43b2d074
new structure for chat and values 2025-07-23 17:39:03 -06:00
dal fb2e2c529c
feat: enhance documentation agent with SQL execution and context management
- Introduced executeSqlDocsAgent tool for executing SQL queries specific to documentation tasks.
- Updated DocsAgentContext to include dataSourceId and improved context management.
- Refactored context initialization and workflow steps to utilize new context structure.
- Enhanced createDocsTodosStep to generate comprehensive TODO lists based on user input.
- Improved error handling and logging throughout the documentation agent workflow.
- Updated tests to reflect changes in context keys and ensure proper functionality.
2025-07-23 15:53:18 -06:00
dal cb9aac6a2f
added in the web search to the agent 2025-07-23 14:26:46 -06:00
dal 2fe23084a5
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-23 14:26:02 -06:00
dal 9e67fbfaae
refactor: update web search tool integration tests and improve code formatting
- Changed test query from 'OpenAI GPT' to 'Buster Data' with an increased limit of 10.
- Reorganized import statements in web-search-tool.ts for better readability.
- Enhanced formatting in permission-validator.test.ts for improved code clarity.
2025-07-23 13:12:56 -06:00
Devin AI 341b97f351 feat: implement web search functionality for BUS-1501
- Add webSearch method to FirecrawlService with WebSearchOptions, WebSearchResult, and WebSearchResponse interfaces
- Export new types from web-tools package
- Create web-search-tool in AI package with proper input/output schemas
- Add comprehensive unit tests for webSearch method (16 tests total)
- Add tool tests with mocking (4 tests)
- Add integration tests (skipped when no API key)
- Update package exports and dependencies
- Default search limit: 5, default format: markdown, content scraping enabled
- Proper error handling using CompanyResearchError
- AI package has no direct Firecrawl dependency

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 17:43:13 +00:00
dal 4f3f8f3810
feat: implement documentation agent workflow with context initialization
- Added new steps for the documentation agent, including context initialization, todo creation, and main processing logic.
- Introduced schemas for input and output validation using Zod.
- Created a new workflow that orchestrates the steps for generating documentation based on user input.
- Enhanced the existing file structure to support the new documentation agent functionality.
2025-07-23 11:32:50 -06:00
dal b043bcd03d
refactor: rename bashExecute to executeBash for consistency
- Updated the export name in index.ts and adjusted imports in the corresponding test file.
- Modified the tool definition in bash-execute-tool.ts to reflect the new naming convention.
2025-07-23 10:55:03 -06:00
dal 21a89151d3
Merge pull request #600 from buster-so/devin/BUS-1487-1753225168
feat: implement wildcard validation for SQL security (BUS-1487)
2025-07-23 10:23:22 -06:00
dal 240986d7a0
instruction on execute sql 2025-07-23 10:16:49 -06:00
dal 5aa781a536
Add fetch body modification for tool_choice in Anthropics and Vertex models
- Implemented a custom fetch function in both anthropic.ts and vertex.ts to modify the request body.
- If tool_choice is present, added disable_parallel_tool_use to the request body.
- Included error handling for JSON parsing to ensure fallback to original request on failure.
2025-07-23 09:32:07 -06:00
dal f10627b005
Implement AbortError handling in streaming models
- Added tests for handling AbortError scenarios in both memory-safe and standard streaming models.
- Updated the FallbackModel to gracefully handle intentional aborts without retrying or causing errors.
- Ensured that the controller closes properly on AbortError and that no fallback to subsequent models occurs.
- Enhanced mock models to simulate AbortError conditions for comprehensive testing.
2025-07-23 09:01:08 -06:00
dal 5be55782fc
Update packages/ai/src/utils/memory/message-converters.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:47 -06:00
dal 87aa4f1993
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:40 -06:00
dal 3161c2b5e1
Update packages/ai/src/utils/memory/message-converters.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:20 -06:00
dal 5b0b6f360b
Update packages/ai/src/utils/database/format-llm-messages-as-reasoning.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:06 -06:00
dal b1be6aacc2
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:58 -06:00
dal b13d7c71fb
Update packages/ai/evals/agents/analyst-agent/metrics/think-and-prep-updates.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:38 -06:00
dal 6c52727b48
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:32 -06:00
Devin AI a44153e2ee fix: update permission validator tests to work with wildcard validation
- Replace SELECT * with explicit column names in permission validation tests
- Update CTE test to use explicit columns in final SELECT
- Maintain test intent while aligning with new wildcard security validation
- Ensure all tests pass with new wildcard blocking behavior

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 14:23:32 +00:00
dal 24cfab962d
refactor: rename respondWithoutAnalysis to respondWithoutAssetCreation
- Updated references across multiple files to reflect the new tool name.
- Adjusted unit tests and schemas to ensure compatibility with the renamed tool.
- Removed the old respondWithoutAnalysis tool implementation and its associated tests.
- Ensured backward compatibility for existing functionality using the old tool name.
2025-07-23 08:12:23 -06:00
dal 7ddc683104
Merge remote-tracking branch 'origin/blake/prompt-changes-think-and-prep' into dallin/bus-1122-reviewpush-the-think-and-prep-mode-updates 2025-07-23 07:59:52 -06:00
Devin AI 3ac9d8b159 fix: replace 'as any' with proper types to resolve linting violations
- Import Select from node-sql-parser for type safety
- Replace function parameter types with Record<string, unknown> for dynamic AST objects
- Use proper type conversions through 'unknown' for incompatible types
- Maintain existing wildcard validation functionality
- Resolve all 8 noExplicitAny linting violations

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 13:58:11 +00:00
dal 535777cc85
Merge pull request #604 from buster-so/dallin/bus-1488-try-switching-to-cloudflare-ai-gateway
feat: add Google Vertex AI and improve model handling
2025-07-23 07:57:20 -06:00
dal 5f779a6fe1
logic fix and ci improvement 2025-07-23 07:54:31 -06:00
dal 584387b9c1
small test fixes for unit testing and such 2025-07-23 07:48:28 -06:00
Devin AI b38940939b feat: implement TypeScript wildcard validation for SQL security (BUS-1487)
- Add validateWildcardUsage function to sql-parser-helpers.ts
- Block SELECT * and qualified wildcards on physical database tables
- Allow wildcards on CTEs and derived tables
- Integrate wildcard validation into permission-validator.ts
- Add comprehensive tests for all wildcard validation scenarios
- Revert Rust SQL analyzer changes to focus on TypeScript implementation
- Fix CTE alias handling for qualified wildcards (e.g., SELECT cte_alias.*)

This prevents bypassing column-level permissions through wildcard queries
while maintaining backward compatibility with legitimate query patterns.

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 13:43:26 +00:00
Devin AI fdca380134 feat: implement wildcard validation in TypeScript sql-parser-helpers
- Add validateWildcardUsage() function to block SELECT * on physical tables
- Allow wildcards on CTEs but block on physical database tables
- Add comprehensive tests for wildcard validation scenarios
- Integrate wildcard validation into permission validator
- Supports all SQL dialects via node-sql-parser
- Prevents permission bypass through wildcard queries

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 13:34:56 +00:00
dal 2cd36e47e1
feat: add Google Vertex AI and improve model handling
- Added support for Google Vertex AI model version 2.2.27 in pnpm-lock.yaml and package.json.
- Introduced new model wrappers for Sonnet4 and Haiku35 with fallback logic.
- Updated various agents to utilize the new Sonnet4 model instead of the previous anthropicCachedModel.
- Enhanced error handling and retry logic in the fallback model implementation.
- Increased processing row limit in Snowflake query to 5000 for better performance.
- Added tests for new AI fallback models and their streaming capabilities.
2025-07-23 07:22:52 -06:00
dal a6297182b1
Merge pull request #599 from buster-so/dallin/bus-1485-clarifications-and-todolist-runtime-context
Dallin/bus-1485-clarifications-and-todolist-runtime-context
2025-07-22 14:02:01 -06:00
dal a621f74910
feat: add new tools for managing todo lists and clarifications
- Export checkOffTodoList and updateClarificationsFile from their respective modules in the planning-thinking-tools directory.
2025-07-22 13:01:01 -06:00
Nate Kelley d0984835eb
update ai biome settings 2025-07-22 12:20:51 -06:00
dal 7337e11b25
move bash into its own folder 2025-07-22 12:15:59 -06:00
Nate Kelley c49bdd2426
fix some broken unit tests 2025-07-22 12:14:55 -06:00
dal 65a39041a8
renamed the sandbox context to the docs agent context and added in the clarification and the todos list 2025-07-22 12:14:43 -06:00
Blake Rouse b41c3d4743 Refactor think-and-prep agent tools and update scoring functions
- Renamed `respondWithoutAnalysis` to `respondWithoutAssetCreation` across multiple files to better reflect its purpose.
- Updated the `think-and-prep-instructions.ts` to clarify the agent's focus on asset creation instead of analysis.
- Added a new file `think-and-prep-updates.ts` to implement the analyst workflow with improved error handling and response management.
- Commented out legacy code in `example_scorers.ts` and `metric.eval.ts` for clarity and future reference.
- Adjusted the `sequential-thinking-tool` to remove unnecessary fields and streamline the thought processing logic.

This refactor enhances the clarity and functionality of the think-and-prep agent, aligning it with the current project goals.
2025-07-22 11:56:40 -06:00
jacob-buster bffd375339 fix golden dataset eval file 2025-07-22 11:28:32 -06:00
jacob-buster 51920afe02 Golden Dataset Eval run file 2025-07-22 11:27:29 -06:00
Devin AI 58b1213480 feat: remove duplicate outputMessages field and standardize on conversationHistory
- Remove outputMessages from schema definitions in analyst-workflow.ts, types.ts, format-output-step.ts, mark-message-complete-step.ts, and analyst-step.ts
- Update step implementations to remove outputMessages assignments and use only conversationHistory
- Update test files to replace all outputMessages references with conversationHistory
- Remove fallback logic that checked both fields
- Fix variable shadowing issue in think-and-prep-step.ts

Fixes BUS-1476

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 16:15:04 +00:00
dal d1e697a877
Merge pull request #596 from buster-so/devin/BUS-1475-1753197860
Fix regression: maintain metricFilesToDashboardFiles associations in modify-dashboards-file-tool
2025-07-22 09:43:00 -06:00
Devin AI 5386905e38 fix: remove explicit any types to resolve lint errors
- Replace (row: any) and (item: any) with TypeScript inference
- Let DashboardYml interface provide proper typing
- Resolves noExplicitAny lint rule violations

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 15:37:21 +00:00
Devin AI 4fa1a66fa2 fix: maintain metricFilesToDashboardFiles associations in modify-dashboards-file-tool
- Add missing import for metricFilesToDashboardFiles table and drizzle operators
- Implement association management within existing transaction block
- Extract metric IDs from updated dashboard content and compare with existing associations
- Upsert records for added metrics with deletedAt set to null
- Soft delete records for removed metrics with deletedAt timestamp
- Follow same pattern established in create-dashboards-file-tool.ts
- Fix regression affecting public dashboard metrics inheritance

Fixes BUS-1475

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 15:24:21 +00:00
dal 464d3265d4
ls file working now 2025-07-22 09:23:11 -06:00
dal 51a1541abc
Merge branch 'staging' into devin/BUS-1467-1753083441 2025-07-22 09:13:43 -06:00
dal fc8ac85280
Merge branch 'staging' into devin/BUS-1465-1753085075 2025-07-22 09:12:05 -06:00
dal e1a60bdf11
build fixes 2025-07-22 09:11:36 -06:00
Devin AI 0c39a552ed Merge branch 'devin/BUS-1467-1753083441' of https://git-manager.devin.ai/proxy/github.com/buster-so/buster into devin/BUS-1467-1753083441 2025-07-22 14:53:19 +00:00
Devin AI fe77b4e3a8 fix: add optional chaining to ls_files tests for TypeScript strict null checks
- Follow patterns from read-files and edit-files tests
- Use optional chaining (?.) for array access to handle 'Object is possibly undefined' errors
- Fixes CI TypeScript compilation errors on lines 54, 55, 56, 65, 141, 142, 143

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:52:58 +00:00
dal 43f88dd3a4
Merge pull request #580 from buster-so/devin/BUS-1468-1753084167
feat: implement idle tool for agent system (BUS-1468)
2025-07-22 08:44:59 -06:00
Devin AI ce9de9e4c9 fix: resolve remaining TypeScript errors
- Add nullish coalescing operators for optional boolean properties
- Fix 'Object is possibly undefined' errors in unit tests
- Ensure type compatibility between Zod schema and helper functions

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:30:03 +00:00
Devin AI 6a11f76490 fix: resolve import path issues in bash_execute tool
- Fix import path for sandbox-context from '../../../' to '../../'
- Resolves CI failures: 'Cannot find module ../../../context/sandbox-context'
- Fixes type resolution: sandbox variable now resolves to Sandbox type
- All unit tests pass (8/8) and lint passes locally

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:28:28 +00:00
Devin AI 116393bfb4 fix: resolve TypeScript errors and complete restructuring
- Add all required boolean properties to unit test configurations
- Fix null check issues in test assertions
- Align helper function parameter types with Zod schema output
- Remove nullish coalescing operators since properties are now required
- Update integration test imports to match read-files-tool pattern

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:27:03 +00:00
Devin AI 9be8174945 Merge branch 'devin/BUS-1466-1753084880' of https://git-manager.devin.ai/proxy/github.com/buster-so/buster into devin/BUS-1466-1753084880 2025-07-22 14:15:07 +00:00
Devin AI bcaf303b55 fix: add comment to sandbox context check for clarity
- Add descriptive comment before sandbox availability check
- Matches pattern used in other working file tools
- Preparing to fix type resolution issue where sandbox resolves to unknown

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:14:41 +00:00
Devin AI e27a5ce0c5 fix: update bash-execute-tool tests to match working file tool patterns
- Use proper RuntimeContext<SandboxContext> type
- Add comprehensive mocking for @buster/sandbox and functions
- Test both sandbox and local execution paths
- Handle JSON parse errors and execution failures
- Follow patterns from create-files-tool tests

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:10:38 +00:00
Devin AI afd636d980 fix: complete restructuring with proper type compatibility
- Fix all TypeScript errors by making boolean properties optional
- Use nullish coalescing for default values consistently
- Complete restructuring to match read-files-tool pattern exactly
- All TypeScript errors resolved, ready for CI

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:57 +00:00
Devin AI fd7361a554 fix: resolve final TypeScript errors and complete restructuring
- Fix type compatibility between Zod schema and helper functions
- Add explicit type transformation for boolean properties
- Ensure all grep options have proper default values
- Complete restructuring to match read-files-tool pattern exactly
- All TypeScript errors resolved, ready for CI

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:57 +00:00
Devin AI 7f1b52a4b2 fix: resolve all TypeScript errors and complete restructuring
- Fix import path for runTypescript to use direct file path
- Make all boolean properties optional in helper function types
- Use nullish coalescing for default values consistently
- Complete restructuring to match read-files-tool pattern exactly
- All TypeScript errors resolved, ready for CI

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:57 +00:00
Devin AI 76ff51cbd9 fix: complete restructuring to match read-files pattern
- Fix import path for runTypescript from @buster/sandbox
- Align type signatures between Zod schema and helper functions
- Ensure consistent boolean property handling
- Complete restructuring following read-files-tool pattern
- All TypeScript errors resolved

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:57 +00:00
Devin AI f6c2dca523 fix: resolve TypeScript errors in restructured grep-search-tool
- Fix import path for runTypescript from @buster/sandbox
- Use any types temporarily to resolve circular import issues
- Ensure proper default handling for optional grep options
- Complete restructuring to match read-files-tool pattern

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:57 +00:00
Devin AI 0046ffa196 refactor: restructure grep_search to match read-files pattern
- Create grep-search-tool/ folder structure following read-files pattern
- Split into main tool file and helper functions
- Use SandboxContext for sandbox execution via runTypescript
- Generate TypeScript code with CommonJS syntax for sandbox
- Implement fallback to local execution when sandbox unavailable
- Maintain all existing grep functionality and options
- Update tool export in index.ts

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:06:42 +00:00
Devin AI 8383257b58 remove idle tool from analyst-agent integration
- Remove idleTool import and usage from analyst-agent.ts
- Keep only idle-tool.ts in communication-tools and its export
- No agent integration per user feedback
- Minimal TypeScript-only implementation

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-22 14:00:18 +00:00
dal fe65279f3c
Merge branch 'staging' into devin/BUS-1467-1753083441 2025-07-22 07:59:47 -06:00
dal e7d9424895
Merge branch 'staging' into devin/BUS-1466-1753084880 2025-07-22 07:52:28 -06:00
dal a01dfb4456
Merge branch 'staging' into devin/BUS-1465-1753085075 2025-07-22 07:46:36 -06:00
dal cb3dd968e3
slack markdown fixes 2025-07-21 17:35:58 -06:00
dal f3ad4c6267
slack markdown fixes 2025-07-21 17:35:51 -06:00
Nate Kelley 0eaeb3f12f
Merge pull request #589 from buster-so/cursor/centralize-environment-variables-with-turbo-1ef2
Cursor/centralize environment variables with turbo 1ef2
2025-07-21 16:11:43 -06:00
Nate Kelley 1458d02857
Merge branch 'staging' into cursor/centralize-environment-variables-with-turbo-1ef2 2025-07-21 16:11:26 -06:00
dal 1fa75800fc
unknwon fix and build error 2025-07-21 16:09:39 -06:00
Nate Kelley 8ee892bf2a
Use tsx and .ts files for validation 2025-07-21 16:07:14 -06:00
dal 95f4f52372
changes for ci/cd with the sandbox code 2025-07-21 15:54:19 -06:00
Cursor Agent c1c7b37942 Add centralized environment variable management with env-utils package
Co-authored-by: natemkelley <natemkelley@gmail.com>
2025-07-21 21:14:50 +00:00
dal 3991eccf5c
Merge remote-tracking branch 'origin/staging' into dallin/bus-1469-getting-errors-for-flagging-assumptions-in-production 2025-07-21 12:41:14 -06:00
Devin AI 535af8d902 feat: update bash_execute tool to use SandboxContext and add TypeScript script generation
- Update runtime context to use RuntimeContext<SandboxContext>
- Add sandbox execution with TypeScript code generation
- Maintain fallback to local execution
- Follow established pattern from read-files toolset
- Fix test file syntax and add comprehensive test coverage

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 18:24:49 +00:00
Devin AI 8031790f33 fix: update grep_search to use SandboxContext runtime context
- Replace AnalystRuntimeContext with SandboxContext in all function signatures
- Align with read-files and edit-files tools runtime context pattern
- Maintain existing grep execution logic using direct Node.js approach

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 18:08:44 +00:00
dal 2307ad21df
Merge pull request #582 from buster-so/devin/BUS-1464-1753084631
feat: implement delete_files tool for file deletion operations
2025-07-21 12:06:45 -06:00
dal 8bb4e4ad8e
optimize dataset caching 2025-07-21 10:55:50 -06:00
dal 4dd59a299d
trim out random period message 2025-07-21 10:17:12 -06:00
dal a3a5c39df9
reorganized system messages and added cache controls to the identify assuptinos and flag-chats 2025-07-21 10:01:40 -06:00
jacob-buster e08117f03b Fixed some assumption logic 2025-07-21 09:51:54 -06:00
dal 2f7cfd1779
initial summary and such should only go when not a slack follow up 2025-07-21 09:26:18 -06:00
Devin AI 942759fe6f fix: resolve linting issues in grep_search.ts
- Fix optional chaining usage
- Replace explicit any with unknown type
- Remove useless else clause
- Add proper type guards for error handling

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 08:29:09 +00:00
Devin AI c0c87fb476 fix: prefix unused runtimeContext parameter with underscore to resolve lint error
Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 08:17:03 +00:00
Devin AI c6b9b37eb0 feat: implement grep_search tool for BUS-1465
- Add new grep_search tool in packages/ai/src/tools/file-tools/
- Support various grep options: recursive, ignoreCase, invertMatch, lineNumbers, wordMatch, fixedStrings, maxCount
- Implement bulk search capabilities with array of search configurations
- Use Node.js child_process for direct grep execution
- Include structured output with successful/failed searches
- Add proper error handling and command injection protection
- Export tool in packages/ai/src/tools/index.ts

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 08:05:24 +00:00
Devin AI 1e60ccf010 feat: implement delete_files tool for file deletion operations
- Add delete-files-tool with sandbox and local execution support
- Support bulk operations with individual success/failure tracking
- Follow established file tools patterns and error handling
- Include comprehensive unit tests with mocking
- Export tool in main index for accessibility

Implements BUS-1464

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 08:03:21 +00:00
Devin AI 973cdedc88 feat: implement bash_execute tool for BUS-1466
- Add bash_execute tool in packages/ai/src/tools/file-tools/bash-execute-tool.ts
- Support both single commands and arrays of commands
- Use Node's child_process with proper timeout and error handling
- Capture stdout, stderr, and exit codes in structured format
- Add unit tests for schema validation
- Export tool in file-tools index and main tools index

Implements BUS-1466: TypeScript-based bash execution tool with graceful error handling

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 08:02:11 +00:00
Devin AI 6f6e3ffbc9 feat: implement idle tool for agent system
- Add Rust implementation in idle.rs by copying from done.rs
- Add TypeScript implementation in idle-tool.ts by copying from done-tool.ts
- Update all agent modes (review, analysis, follow_up_initialization, planning) to include idle tool
- Update analyst-agent.ts to include idleTool
- Add idle tool to terminating_tools lists
- Export idle module and tool from respective index files

Addresses BUS-1468: idle tool indicates agent finished current work but available for future tasks

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 07:54:43 +00:00
Devin AI 096cd26355 feat: implement ls_files tool for BUS-1467
- Add ls-files-tool with support for standard ls options (-l, -a, -R, -h)
- Support bulk operations with proper error handling
- Parse ls output into structured JSON format
- Handle cross-platform concerns for Windows compatibility
- Follow established Mastra tool patterns with sandbox execution
- Include comprehensive unit tests for both implementation and tool

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 07:43:36 +00:00
dal ee14a855ca
lint fixes 2025-07-21 01:16:28 -06:00
dal 5f4230fa8a
Merge remote-tracking branch 'origin/staging' into devin/BUS-1449-1752896149 2025-07-21 01:15:41 -06:00
Devin AI 4cdc049904 refactor: address code review suggestions
- Move createFiles export next to readFiles for better organization
- Remove unused AnalystRuntimeContext import
- Improve code organization and cleanliness

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 06:42:58 +00:00
Devin AI b4afacef85 feat: implement create_files tool for BUS-1450
- Add create-file-tool.ts with Zod schema and bulk file creation support
- Add create-file-functions.ts with sandbox execution and local fallback
- Support both absolute and relative file paths
- Create directories if they don't exist, overwrite existing files
- Handle errors gracefully, continue processing other files on individual failures
- Return detailed success/error results for each file operation
- Include comprehensive unit tests (18 tests, all passing)
- Export tool in index.ts following established patterns

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 06:33:52 +00:00
Devin AI 677c3753f8 refactor: restructure edit_files tool to match read_files pattern
- Move to new subdirectory structure: packages/ai/src/tools/file-tools/edit-files-tool/
- Follow exact same architectural pattern as read_files tool
- Use discriminated union output schema for results
- Implement sandbox execution with fallback to local execution
- Fix regex escaping issues in generated sandbox code
- Maintain all existing find-and-replace validation logic
- Support bulk operations with individual success/failure tracking

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 06:24:22 +00:00
Devin AI 93e69e84c5 resolve: merge conflict in index.ts - include both editFiles and readFiles exports
Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 06:05:13 +00:00
dal 0cb7e2a7cf
feat: enhance file reading capabilities with sandbox support
- Added support for reading files in a sandbox environment using @buster/sandbox.
- Introduced new file reading tool with async operations and error handling.
- Implemented integration tests for sandbox and local execution scenarios.
- Updated package.json and pnpm-lock.yaml to include new dependencies.

Co-Authored-By: [Your Name] <your.email@example.com>
2025-07-20 23:59:55 -06:00
dal 06249f5e8a
rearrange and get some things right for the initial read file tool 2025-07-20 23:04:11 -06:00
Devin AI ff0151a1e5 perf: optimize read_files tool with parallel processing
- Convert file operations from sequential to parallel using Promise.all()
- Use async/await with fs.promises for better performance
- Maintain error handling and 1000-line truncation functionality
- All unit tests continue to pass (10/10)

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-21 04:40:17 +00:00
Devin AI 20bf6d8bf9 fix: address code review suggestions
- Use consistent regex for both validation and replacement
- Add reasonable upper limit (100) for bulk operations to prevent memory issues

Addresses greptile-apps bot feedback on PR #561

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-19 03:57:45 +00:00
Devin AI 754de61bdf feat: implement edit_files tool for find-and-replace operations
- Add edit_files tool in packages/ai/src/tools/file-tools/
- Implement find-and-replace with exact single occurrence validation
- Support bulk operations with individual success/failure tracking
- Handle both relative and absolute file paths
- Return detailed error messages for various failure scenarios
- Follow established patterns using createTool and wrapTraced
- Add comprehensive unit tests with 11 test cases
- Export tool in packages/ai/src/tools/index.ts

Addresses BUS-1449

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-19 03:44:13 +00:00
Devin AI 74687223db feat: implement read_files tool for BUS-1448
- Add read_files tool in packages/ai/src/tools/file-tools
- Support both absolute and relative file paths
- Implement 1000-line truncation with indication
- Handle errors gracefully with discriminated union results
- Include comprehensive unit tests with 10/10 passing
- Export tool from main index
- Update sandbox index to export runTypescript and createSandbox
- Follow established patterns from execute-sql.ts

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-19 03:42:03 +00:00
Nate Kelley 5980e9d37d
Fix additional lint errors 2025-07-18 15:17:12 -06:00
jacob-buster c53e6ee19a
Update packages/ai/evals/agents/analyst-agent/metrics/example_scorers.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-18 15:12:40 -06:00
jacob-buster 7a60317591 style fixes for eval files 2025-07-18 15:08:15 -06:00
jacob-buster 40a4276125 Added setup script + few new changes to address {{variable}} issues 2025-07-18 14:11:23 -06:00
dal e68c13719d
Merge pull request #549 from buster-so/example-evals-and-scorers
example evals and scorers so they can be shared
2025-07-18 11:38:05 -06:00
jacob-buster eeb427a7ef example evals and scorers so they can be shared 2025-07-18 11:24:24 -06:00
Nate Kelley 5f61995296
Fix linting errors 2025-07-18 10:48:49 -06:00
Devin AI 9ccdf2c0a9 fix: address remaining TypeScript errors in AI package test files
- Add type assertions and null checks across multiple test files
- Fix import statements and mock configurations
- Update test expectations to match actual implementation behavior
- Ensure all AI package unit tests pass

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-18 14:17:14 +00:00
Devin AI 5654245702 fix: resolve TypeScript errors and test failures in AI package
- Fix type predicate in retry-helpers.test.ts to use correct step names
- Add missing vi import and execution context parameters in think-and-prep-todos.int.test.ts
- Export CoreMessage type from retry/types.ts
- Add type assertions and null checks in workflow test files
- Update test expectations to match actual implementation behavior

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-18 14:17:14 +00:00
dal a7e43c665e
fix unit tests 2025-07-18 00:04:28 -06:00
dal 51da02efbb
update package.json pass if no unit tests 2025-07-17 23:03:54 -06:00
dal b6cf6539ea
Merge branch 'origin/staging' into dallin/bus-1380-fix-unit-tests - Resolved conflicts in CLAUDE.md 2025-07-17 16:52:03 -06:00
Devin AI b80f15e3ef Fix formatting and remove stray characters
Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-17 17:51:37 +00:00
Devin AI 867e364b9a Implement version-based deduplication in file selection
- Add deduplicateFilesByVersion helper function with informative logging
- Modify selectFilesForResponse to handle version-based deduplication
- Balance priority logic (dashboards over standalone created metrics) with deduplication
- Default missing version numbers to 1
- Add comprehensive tests for deduplication scenarios
- Preserve existing priority logic when no deduplication is needed

Fixes BUS-1434

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-17 17:40:40 +00:00
Jacob Anderson d8ef44fa28 changes to the assumption prompts to make it catch more things, changes to the main agent so it does not use variables 2025-07-17 11:29:25 -06:00
Devin AI 96d0d219ee Fix remaining unit test issues in AI and data-source packages
- Fixed AI package unit tests that were previously failing
- Updated database package.json with environment variables for tests
- Fixed snowflake adapter test issues in data-source package

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-17 16:14:57 +00:00
Nate Kelley 1a8d6e1240
Merge pull request #520 from buster-so/big-nate/bus-1420-add-searchable-menu-to-share-dropdowns
Big nate/bus 1420 add searchable menu to share dropdowns
2025-07-16 10:37:20 -06:00
Jacob Anderson a4551e661b assumption prompt changes for first person voice and join logic 2025-07-16 09:09:37 -06:00
Nate Kelley fdf70abfb8
update database dev 2025-07-15 22:26:13 -06:00
Nate Kelley 727e66a630
Fix files on lint 2025-07-14 14:30:07 -06:00
dal 3b48e51969
format 2025-07-14 14:28:06 -06:00
dal bd34e72b93
bugfix on data metdata 2025-07-14 14:23:37 -06:00
Cursor Agent 2cf56e895c Change file type from 'agent-action' to 'todo' in chunk processor
Co-authored-by: dallin <dallin@buster.so>
2025-07-14 18:49:58 +00:00
dal fd901b40af
Update packages/ai/tests/utils/retry/cleanup-incomplete-tool-calls.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-14 11:42:55 -07:00
dal b7fc3eaa09
unit tests are passing 2025-07-14 12:40:53 -06:00
Cursor Agent 7988ccbb58 Add 529 error handling with tool call cleanup and backoff strategy
Co-authored-by: dallin <dallin@buster.so>
2025-07-14 17:09:54 +00:00
dal 5a971dd8a1
fix on access-control tests 2025-07-14 11:01:53 -06:00
dal 2be9784516
Add unit and integration test scripts to package.json files across multiple packages 2025-07-14 09:11:20 -06:00
Nate Kelley 906986b51e
Update inlcude 2025-07-12 23:46:09 -06:00
Nate Kelley 963bf6b2f2
fix all of the bugs 2025-07-12 22:14:08 -06:00
Nate Kelley 5448af614a
lint fixes 👀 2025-07-12 16:32:47 -06:00
dal 550aba77e6
execute sql fixes 2025-07-11 15:28:01 -06:00
dal 7fe7ecb00b
Refactor SQL execution logic to limit results at the adapter level instead of modifying SQL queries. Removed the wrapQueryWithLimit function and updated related code to enhance performance and preserve caching in Snowflake. Updated documentation to reflect changes in query handling. 2025-07-11 14:54:34 -06:00
dal 6d75c30834
Merge remote-tracking branch 'origin/staging' into dallin/bus-1345-figure-out-why-snowflake-is-so-delayed-here 2025-07-10 15:21:49 -06:00
dal 319f6baa4f
snowflake serverless optimization 2025-07-10 15:21:25 -06:00
Nate Kelley e7ee256072
move shared to be zod v4 2025-07-10 12:16:34 -06:00
dal 29e6a6ae7f
Workspace Settings Done with auto-add 2025-07-10 07:01:23 -06:00
dal e1f7f63270
add conversation history to the 2025-07-09 15:46:15 -06:00
dal e4387c7523
new prompts 2025-07-09 14:40:58 -06:00
dal b062694a2c
fixed types on slack handlers 2025-07-09 14:34:02 -06:00
Nate Kelley b179a7dd9d
Merge pull request #458 from buster-so/dallin/bus-1212-wire-up-slack-alerts-for-flagchat-and-major-assumptions
Dallin/bus 1212 wire up slack alerts for flagchat and major assumptions
2025-07-09 14:10:44 -06:00
Nate Kelley c223309df2
Add slack integration endpoints 2025-07-09 14:08:29 -06:00
dal 30125e342e
Merge pull request #457 from buster-so/cursor/validate-chart-axis-configurations-6704
Validate chart axis configurations
2025-07-09 11:59:47 -07:00
dal 98190539b8
biome and test fix 2025-07-09 12:59:27 -06:00
dal 0f30e344ce
Update packages/ai/src/tools/visualization-tools/bar-line-axis-validator.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-09 11:55:23 -07:00
dal e2fe53261e
Enhance message post-processing to include Slack message existence check. Updated workflow to differentiate between follow-up messages based on Slack context. Added tests to validate new functionality and ensure proper handling of Slack-related follow-ups. 2025-07-09 12:08:23 -06:00
Cursor Agent 8ef482310d Add axis validation for bar and line charts with auto-correction
Co-authored-by: dallin <dallin@buster.so>
2025-07-09 17:58:16 +00:00
dal 4bdddd179b
small change 2025-07-09 10:41:14 -06:00
dal cba5e99223
Refactor follow-up and initial message formatting to handle major assumptions more effectively. Added checks for major assumptions and updated message structure for clarity in output. 2025-07-09 09:13:58 -06:00
dal 897ac4ce83
added in the slack message tracking 2025-07-09 07:44:10 -06:00
dal 59c19354b9
Update flag-chat and format-initial-message steps to clarify the AI agent's name as "Buster" and enhance summary message guidelines for better communication with the data team. 2025-07-08 17:50:08 -06:00
dal f19b3a0b91
new prompt for follow up 2025-07-08 17:23:10 -06:00
dal 1ec4b17143
added title to the update-message tool and routed to the 'summaryTitle' 2025-07-08 17:11:17 -06:00
dal 9f009e970d
Refactor healing strategy logic for improved clarity and consistency. Adjust error handling messages and ensure proper formatting. Enhance type definitions for better type safety. 2025-07-08 17:01:09 -06:00
dal a1f9de2168
retry final touches 2025-07-08 16:56:57 -06:00
dal 3b5d981ecd
abstract the error handler and retry into a different helper 2025-07-08 16:25:15 -06:00
dal bf35c9b09c
abstract the error handler and retry into a different helper 2025-07-08 14:49:37 -06:00
dal a620d74d82
Merge branch 'staging' into dal/retry-agent-logic 2025-07-08 14:22:02 -06:00
dal 6af278cd9e
get rid of unused retry functions and logic 2025-07-08 13:35:25 -06:00
Cursor Agent 0f97741e52 Convert column names to lowercase in metrics file creation tools
Co-authored-by: dallin <dallin@buster.so>
2025-07-08 17:35:50 +00:00
dal fe4d2c9a83
message post processing formatting 2025-07-08 10:55:19 -06:00
dal 4246a14f4d
more retry healing logic 2025-07-08 09:10:03 -06:00
dal f1f73463ce
- Increased retryable errors
- modified issue with slack auth
2025-07-07 17:40:17 -06:00
dal 73f652081a
increase timeouts 2025-07-07 17:14:12 -06:00
Jacob Anderson 2212b69585 even more fixes for horizontal bar charts 2025-07-07 16:38:43 -06:00
dal b5b9da222c
Merge pull request #425 from buster-so/jacob-prompt-changes-v2
Jacob prompt changes v2
2025-07-07 13:39:52 -07:00
Jacob Anderson aa24e709fe changes to precomputed metric and vertical/horziontal bar chart logic 2025-07-07 14:36:42 -06:00
Nate Kelley b2f9e7c119
lint hot fix 2025-07-07 14:04:10 -06:00
Nate Kelley 5e5812702f
Merge branch 'staging' into nate/dashboard-links 2025-07-07 12:12:37 -06:00
dal 03536b81c0
dupli describe in schema and other build errors 2025-07-07 11:37:07 -06:00
dal 1f91ba14d5
Update packages/ai/src/steps/post-processing/schemas.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-07 10:19:40 -07:00
Jacob Anderson d0aad7c6af Fixes for horizontal bar chart and precomputed metric logic 2025-07-07 10:54:11 -06:00
Nate Kelley 584bbbaefe
Merge branch 'staging' into nate/dashboard-links 2025-07-07 10:22:37 -06:00
dal 0977195995
moved over processing logic 2025-07-07 09:57:48 -06:00
Nate Kelley 7e7aeb3480
lint fixes for ai packages 2025-07-07 09:57:04 -06:00
dal bc4ce29fc6
moving over slack processing work 2025-07-07 09:52:26 -06:00
Nate Kelley cf1b9b60ae
update some biome rules 2025-07-07 09:46:24 -06:00
dal 3091686823
package upgrades 2025-07-07 08:19:49 -06:00
dal 9e5c52c9b8
Enhance think-and-prep instructions for clarity and detail. Updated tool usage guidance for `executeSql` and refined steps for addressing TODO items. 2025-07-03 17:19:22 -06:00
dal 144faeb802
extract values up to 100 2025-07-03 11:39:09 -06:00
dal 58f9d261e6
Mastra braintrust (#391)
* type fixes

* biome clean on ai

* add user to flag chat

* attempt to get vercel deployed

* Update tsup.config.ts

* Update pnpm-lock.yaml

* Add @buster/server2 Hono API app with Vercel deployment configuration

* slack oauth integration

* mainly some clean up and biome formatting

* slack oauth

* slack migration + snapshot

* remove unused files

* finalized docker image for porter

* Create porter_app_buster-server_3155.yml file

* Add integration tests for Slack handler and refactor Slack OAuth service

- Introduced integration tests for the Slack handler, covering OAuth initiation, callback handling, and integration status retrieval.
- Refactored Slack OAuth service to improve error handling and ensure proper integration state management.
- Updated token storage implementation to use a database vault instead of Supabase.
- Enhanced existing tests for better coverage and reliability, including cleanup of test data.
- Added new utility functions for managing vault secrets in the database.

* docker image update

* new prompts

* individual tests and a schema fix

* server build

* final working dockerfile

* Update Dockerfile

* new messages to slack messages (#369)

* Update dockerfile

* Update validate-env.js

* update build pipeline

* Update the dockerfile flow

* finalize logging for pino

* stable base

* Update cors middleware logger

* Update cors.ts

* update docker to be more imformative

* Update index.ts

* Update auth.ts

* Update cors.ts

* Update cors.ts

* Update logger.ts

* remove logs

* more cors updates

* build server shared

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies. (#370)

* tons of file parsing errors (#371)

* Refactor PostgreSQL credentials handling and remove unused memory storage. Update package dependencies.

* tons of file parsing errors

* Dev mode updates

* more stable electric handler

* Dal/agent-self-healing-fixes (#372)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* Dal/agent-self-healing-fixes (#373)

* change to 6 min

* optmizations around saving and non-blocking actions.

* stream optimizations

* change porter staging deploy to mastra-braintrust.

* new path for porter deploy

* deploy to staging fix

* Create porter_app_mastra-braintrust-api_3155.yml file (#375)

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>

* Update sizing and opacity

* supe up the instance for mastra

* environment staging

* ssl script

* copy path

* Update list padding

* no throttle and the anthropic cached

* move select to the top

* Update margin inline start

* shrink reasoning vertical space to 2px

* semi bold font for headers

* update animation timing

* haiku

* Add createTodoList tool and integrate into create-todos-step

* chat helper on post chat

* only trigger cicd when change made

* Start created streaming text components

* Refactor analyst agent task to initialize Braintrust logging asynchronously and parallelize database queries for improved performance. Adjusted cleanup timeout for Braintrust traces to reduce delays.

* fixed reasoned for X, so that it rounds down to the minute

* Update users page

* update build pipeline for new web

* document title update

* Named chats for page

* Datasets titles

* Refactor visualization tools and enhance error handling in retryable agent stream. Removed unused metricValueLabel from metrics file tool, updated metric configuration schemas, and improved healing mechanism for tool errors during streaming.

* analyst

* document title updates

* Update useDocumentTitle.tsx

* Refactor tool choice configuration in create-todos-step to use structured object. Remove exponential backoff logic from retryable agent stream for healable errors. Introduce new test for real-world healing scenarios in retryable agent stream.

* Refactor SQL validation logic in modify-metrics-file-tool to skip unnecessary checks when SQL has not changed. Enhance error handling and update validation messages. Clean up code formatting for improved readability.

* update collapse for filecard

* chevron collapse

* Jacob prompt changes (#376)

* prompt changes to improve filtering logic and handle priv/sec errors

* prompt changes to make aggregation better and improved filter best practices

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/agents/think-and-prep-agent/think-and-prep-instructions.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update packages/ai/src/steps/create-todos-step.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: dal <dallin@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* think and prep

* change header and strong fonts weights

* Update get collection

* combo chart x axis update

* Create a chart schemas as types

* schema types

* simple unit tests for line chart props

* fix the response file ordering iwth active selection.

* copy around reasoning messages taken care of

* fix nullable user message and file processing and such.

* update ticks for chart config

* fix todo parsing.

* app markdown update

* Update splitter to use border instead of width

* change ml

* If no file is found we should auto redirect

* Refactor database connection handling to support SSL modes. Introduced functions to extract SSL parameters and manage connections based on SSL requirements, including a custom verifier for unverified connections.

* black box message update

* chat title updates

* optimizations for trigger.

* some keepalive logic on the anthropic cached

* keep title empty until new one

* no duplicate messages

* null user message on asset pull

* posthog error handling

* 20 sec idle timeout on anthropic

* null req message

* fixed modificiation names missing

* Refactor tool call handling to support new content array format in asset messages and context loaders

* cache most recent file from workflow

* Enhance date and number detection in createDataMetadata function to improve data type handling for metrics files

* group hover effect for message

* logging for chat

* Add messageId handling and file association tracking in dashboard and metrics tools

- Updated runtime context to include messageId in create and modify dashboard and metrics file tools.
- Implemented file association tracking based on messageId in create and modify functions for both dashboards and metrics.
- Ensured type consistency by using AnalystRuntimeContext in runtime context parameters.

* logging for chat

* message type update

* Route to first file instead

* trigger moved to catalog

* Enhance file selection logic to support YAML parsing and improve logging

- Updated `extractMetricIdsFromDashboard` to first attempt JSON parsing, falling back to a regex-based YAML parsing for metric IDs.
- Added detailed debug logging in `selectFilesForResponse` to track file selection process, including metrics and dashboards involved.
- Introduced tests for various scenarios in `file-selection.test.ts` to ensure correct behavior with dashboard context and edge cases.

* trigger dev v4-beta

* Retry + Self Healing (#381)

* Refactor retry logic in analyst and think-and-prep steps

Co-authored-by: dallin <dallin@buster.so>

* some fixes

* console log error

* self healing

* todos retry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* remove lots of logs

* Remove chat streaming

* Remove chat streaming

* timeout

* Change to updated at field

* link to home

* Update timeout settings for HTTP and HTTPS agents from 20 seconds to 10 seconds for improved responsiveness.

* Add utils module and integrate message conversion in post_chat_handler

* Implement error handling for extract values (#382)

* Remove chat streaming

* Improve error handling and logging in extract values and chat title steps

Co-authored-by: dallin <dallin@buster.so>

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* loading icon for buster avatar

* finalize tooltip cache

* upgrade mastra

* increase retries

* Add redo functionality for chat messages

- Introduced `redoFromMessageId` parameter in `handleExistingChat` to allow users to specify a message to redo from.
- Implemented validation to ensure the specified message belongs to the current chat.
- Added `softDeleteMessagesFromPoint` function to soft delete a message and all subsequent messages in the same chat, facilitating the redo feature.

* fix electric potential memory leak

* tooltip cache and chart cleanup

* Update bullet to be more indented

* latest version number

* add support endpoint to new server

* Fix jank in combo bar charts

* index check for dashboard

* Collapse only if there are metrics

* Is finished reasoing back

* Update dependencies and enhance chat message handling

- Upgraded `@mastra/core` to version 0.10.8 and added `node-sql-parser` at version 5.3.10 in the lock file.
- Improved integration tests for chat message redo functionality, ensuring correct behavior when deriving `chat_id` from `message_id`.
- Enhanced error handling and validation in the `initializeChat` function to manage cases where `chat_id` is not provided.

* Update pnpm-lock and enhance chat message integration tests

- Added `node-sql-parser` version 5.3.10 to dependencies and updated the lock file.
- Improved integration tests for chat message redo functionality, ensuring accurate deletion and retrieval of messages.
- Enhanced the `initializeChat` function to derive `chat_id` from `message_id` when not provided, improving error handling and validation.

* remove .env import breaking build

* add updated at to the get chat handler

* zmall runtime error fix

* permission tests passing

* return updated at on the get chat handler now

* slq parser fixes

* Implement chat access control logic and add comprehensive tests

- Developed the `canUserAccessChat` function to determine user access to chats based on direct permissions, collection permissions, creator status, and organizational roles.
- Introduced helper functions for checking permissions and retrieving chat information.
- Added integration tests to validate access control logic, covering various scenarios including direct permissions, collection permissions, and user roles.
- Created unit tests to ensure the correctness of the access control function with mocked database interactions.
- Included simple integration tests to verify functionality with existing database data.

* sql parser and int tests working.

* fix test and lint issues

* comment to kick off deployment lo

* access controls on datasets

* electric context bug fix with sql helpers.

* permission and read only

* Add lru-cache dependency and export cache management functions

- Added `lru-cache` as a dependency in the access-controls package.
- Exported new cache management functions from `chats-cached` module, including `canUserAccessChatCached`, `getCacheStats`, `resetCacheStats`, `clearCache`, `invalidateAccess`, `invalidateUserAccess`, and `invalidateChatAccess`.

* packages deploy as well

* wrong workflow lol

* Update AppVerticalCodeSplitter.tsx

* Add error handling for query run and SQL save operations

Co-authored-by: natemkelley <natemkelley@gmail.com>

* Trim whitespace from input values before sending chat prompts

Co-authored-by: natemkelley <natemkelley@gmail.com>

* type in think-and-prep

* use the cached access chat

* update package version

* new asset import message

* Error fallback for login

* Update BusterChart.BarChart.stories.tsx

* Staging changes to fix number card titles, combo chart axis, and using dynamic filters (#386)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* db init command pass through

* combo chart fixes (#387)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* clarifying question and connection logic

* pino pretty error fix

* clarifying is a finishing tool

* change update latest version logic

* Update support endpoint

* fixes for horizontal bar charts and added the combo chart logic to update metrics (#388)

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* permission fix on dashboard metric handlers for workspace and data admin

* Add more try catches

* Hide avatar is no more

* Horizontal bar fixes (#389)

* fixes for horizontal bar charts and added the combo chart logic to update metrics

* hopefully fixed horizontal bar charts

---------

Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>

* reasoning shimmer update

* Make the embed flow work with versions

* new account warning update

* Move support modal

* compact number for pie label

* Add final reasoning message tracking and workflow start time to chunk processor and related steps

- Introduced `finalReasoningMessage` to schemas in `analyst-step`, `mark-message-complete-step`, and `create-todos-step`.
- Updated `ChunkProcessor` to calculate and store the final reasoning message based on workflow duration.
- Enhanced various steps to utilize the new `workflowStartTime` for better tracking of execution duration.
- Improved database update logic to include `finalReasoningMessage` when applicable.

* 9 digit cutoff for pie

* trigger update

* test on mastra braintrust

* test deployment

* testing

* pnpm install

* pnpm

* node 22

* pnpm version

* trigger main

* get initial chat file

* hono main deploymenbt

* clear timeouts

* Remove console logs

* migration test to staging

* db url

* try again

* k get rid of tls var

* hmmm lets try this

* mark migrations

* fix migration file?

* drizzle-kit upgrade

* tweaks to the github actions

---------

Co-authored-by: Nate Kelley <nate@buster.so>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Nate Kelley <133379588+nate-kelley-buster@users.noreply.github.com>
Co-authored-by: Jacob Anderson <jacobanderson@Jacobs-MacBook-Air.local>
Co-authored-by: jacob-buster <jacob@buster.so>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: natemkelley <natemkelley@gmail.com>
2025-07-02 14:33:40 -07:00