Commit Graph

426 Commits

Author SHA1 Message Date
dal 29708e8d7d
create and modify 2025-08-12 09:10:23 -06:00
dal 7872ad2f29
refactor: enhance modify dashboards tool with improved state management and type definitions
- Updated the `modifyDashboardsDelta` and `modifyDashboardsFinish` functions to streamline state handling and improve message entry updates.
- Refactored helper functions for better clarity and type safety in dashboard modifications.
- Introduced new schemas for state management, enhancing type definitions for dashboard files and their statuses.
- Improved test coverage to ensure robust functionality and integration of the modify dashboards tool.

These changes optimize the dashboard modification process and enhance code maintainability.
2025-08-11 22:30:14 -06:00
dal d3870430c2
create dashboard fixes 2025-08-11 19:00:33 -06:00
dal e2d0d332c1
create dashboard functions debugged 2025-08-11 18:37:31 -06:00
dal 70193aad4f
fix lint issues 2025-08-11 16:57:49 -06:00
dal 4e7f6b080e
no reports in todos 2025-08-11 16:22:50 -06:00
dal f1d9293946
refactor: enhance create dashboards tool with improved state management and type definitions
- Removed unused properties from the dashboard state and output schemas for clarity.
- Introduced a new state file schema to better represent dashboard files.
- Updated helper functions to streamline file transformation and reasoning message creation.
- Improved type safety by utilizing the newly defined `StatusSchema`.

These changes optimize the dashboard creation process and enhance code maintainability.
2025-08-11 15:59:29 -06:00
dal 95c29cf507
Refactor analyst and think-and-prep agent instructions to remove report generation capabilities and streamline workflow. Update the analyst agent to exclude report-related tools and adjust think-and-prep instructions for clarity and focus on analysis preparation. Ensure consistent handling of SQL guidance and improve overall structure for better readability. 2025-08-11 15:58:07 -06:00
dal 317b7cb8c3
bump max tokens 2025-08-11 15:41:34 -06:00
dal b5a15c8581
dashboard tool description fix 2025-08-11 15:08:18 -06:00
dal 71d862e51b
sql tools 2025-08-11 14:54:43 -06:00
Nate Kelley a5b175d418
Update create-todos-step.ts 2025-08-11 14:54:12 -06:00
dal 82c1ea7f46
communication tools plus others 2025-08-11 14:53:23 -06:00
dal 2af5f408fa
refactor: enhance create dashboards tool and web search tool integration
- Updated the `createDashboardsStart` function to improve message handling and reasoning entry creation.
- Refactored helper functions for better clarity and type safety in dashboard creation.
- Enhanced web search tool execution to include additional parameters for improved functionality.
- Updated tests to reflect changes in tool execution and ensure robust integration.

These changes streamline the dashboard creation process and enhance the web search tool's capabilities.
2025-08-11 14:50:38 -06:00
dal 698444bb3e
create dashboard fixes with websearch fixes 2025-08-11 14:47:50 -06:00
dal 4ace4d337e
Revert 'Testing some new GPT 5 metaprompts' from commit 1f4f31909
This reverts commit 1f4f319098 which made changes to:
- analyst-agent-instructions.ts
- think-and-prep-instructions.ts
- investigation-instructions.ts
- format-analysis-type-router-prompt.ts
- create-todos-step.ts

Reverting Jacob's experimental prompt changes.
2025-08-11 14:33:07 -06:00
dal 2540f6e195
Revert "Improved Report Prompt"
This reverts commit ed056878e1.
2025-08-11 14:31:35 -06:00
dal 1c0a6b4f0f
move back to anthropic models 2025-08-11 14:27:16 -06:00
dal c6f4ac3902
verbosity low 2025-08-11 14:07:16 -06:00
dal 47f2e38557
priority tier 2025-08-11 13:55:43 -06:00
dal 82d89cf702
clean up builds and errors 2025-08-11 13:41:17 -06:00
dal b96cb39b78
turbo fast web build 2025-08-11 11:33:18 -06:00
dal 1e021f4da1
refactor: introduce new planning tools for todo list and clarifications management
- Added `createCheckOffTodoListTool` and `createUpdateClarificationsFileTool` to enhance planning capabilities.
- Refactored existing tools to utilize the new planning tools, improving code organization and maintainability.
- Updated tests to reflect changes in tool creation and execution, ensuring robust functionality.

These changes streamline the management of todo lists and clarification questions, enhancing overall tool usability.
2025-08-11 10:28:26 -06:00
dal 5d21fd4782
Merge pull request #698 from buster-so/GPT-5-MetaPrompting
Improved Report Prompt
2025-08-11 10:16:04 -06:00
dal 2712697722
refactor: integrate web search tool and enhance metrics processing
- Introduced the `createWebSearchTool` function to streamline web search capabilities.
- Updated various metrics tools to improve type handling and input processing.
- Refactored metrics delta and finish functions to utilize `ToolCallOptions` for better integration.
- Removed deprecated report creation and editing tools to clean up the codebase.

These changes enhance functionality and maintainability across the tools.
2025-08-11 10:15:34 -06:00
jacob-buster ed056878e1 Improved Report Prompt 2025-08-11 10:13:22 -06:00
dal e93a82a3f5
refactor: enhance dashboard and metrics tools with improved descriptions and type handling
- Added the `GPT5` import to the analyst agent for enhanced functionality.
- Updated the `createModifyDashboardsStart` function to utilize `ToolCallOptions` for better input handling.
- Refactored dashboard and metrics tools to use helper functions for descriptions, improving maintainability.
- Ensured consistent type handling and removed redundant code in metrics delta processing.

These changes streamline the tools and improve the overall code quality.
2025-08-11 08:39:05 -06:00
dal 74ae581f73
refactor: update ESLint rules and clean up socket query files
- Disabled specific TypeScript ESLint rules to allow for more flexibility in type definitions.
- Removed unused socket query files and related types to streamline the codebase.
- Enhanced various components and layouts for better performance and readability.

These changes improve code maintainability and reduce clutter in the project.
2025-08-11 07:57:14 -06:00
Nate Kelley 95d081766e
update imports 2025-08-08 20:39:15 -06:00
Nate Kelley 9a88c67b9b
Merge pull request #689 from buster-so/GPT-5-MetaPrompting
Testing some new GPT 5 metaprompts
2025-08-08 17:01:08 -06:00
jacob-buster 1f4f319098 Testing some new GPT 5 metaprompts 2025-08-08 16:10:46 -06:00
dal 51b7d4e691
Remove serviceTier option from OpenAI configuration in analyst and think-and-prep agents 2025-08-08 15:25:52 -06:00
dal b00b5c6858
Add serviceTier option to OpenAI configuration in analyst and think-and-prep agents 2025-08-08 14:32:21 -06:00
dal e21ac9d1e1
low verbosity on anlysis router 2025-08-08 14:31:14 -06:00
dal 63450d31d7
low verbosity 2025-08-08 14:30:04 -06:00
dal 663bafdb41
dumb down and make smarter 2025-08-08 14:21:51 -06:00
dal 14092b08be
move everything over to gpt5 2025-08-08 13:29:59 -06:00
dal 5e97fbffd2
enhance create-todos-system-prompt: add tool calling guidelines and clarify user request interpretation
- Introduced a section on tool calling rules to ensure proper usage of available tools.
- Added clarifications to the TODO list items regarding user requests, emphasizing the need to determine if the user is asking for a single metric, a report, or a dashboard.

These updates improve the clarity and functionality of the analyst agent's prompt, ensuring more accurate task breakdowns.
2025-08-08 13:18:10 -06:00
dal 9156e7a06e
merge staging 2025-08-08 13:17:58 -06:00
dal e75f90ceab
refactor: update dashboard types and schemas for improved validation
- Removed unused import for `createMetricsRawLlmMessageEntry` in visualization tools.
- Introduced Zod schemas for `Dashboard` and `DashboardConfig` to enhance type validation and structure.
- Cleaned up the interface definitions for better clarity and maintainability.

These changes improve the robustness of the dashboard configuration and streamline the codebase.
2025-08-08 13:00:20 -06:00
dal ff8530a08f
a little reorg 2025-08-08 11:51:06 -06:00
dal dada5cb0eb
refactor: remove unused dependencies and clean up visualization tools
- Deleted references to the `@mastra/core` package from various configuration files.
- Removed multiple unused test files and helper functions related to visualization tools, including axis validation and version history management.
- Cleaned up import statements for better organization and readability.

These changes streamline the codebase by eliminating unnecessary dependencies and improving maintainability.
2025-08-08 11:48:53 -06:00
dal 601a13c402
analyst agent steps is clean 2025-08-08 11:33:04 -06:00
dal 7f1bfc3bc5
fix: improve error handling and refactor test mocks for SQL execution
- Added a check to ensure the DataSource is available before executing SQL statements, throwing an error if not.
- Refactored test mocks to use `vi.mocked` for better clarity and consistency in handling DataSource connection errors.
- Removed unnecessary whitespace for cleaner code.

These changes enhance the robustness of SQL execution and improve the reliability of related tests.
2025-08-08 09:36:26 -06:00
dal fb65ac42ea
test: update integration tests to include additional properties for agent options
- Added userId, chatId, organizationId, dataSourceId, and dataSourceSyntax to the integration tests for the Think and Prep Agent and Mark Message Complete Step.
- Cleaned up import statements for better organization and readability.
- Ensured that the tests reflect the latest schema changes for improved tracking and functionality.

These updates enhance the clarity and robustness of the integration tests, aligning them with recent changes in agent options.
2025-08-08 09:32:53 -06:00
dal 3d12455f69
refactor: enhance agent options and streamline step functions
- Added new properties (chatId, organizationId, dataSourceId, dataSourceSyntax, userId) to the ThinkAndPrepAgentOptionsSchema for improved tracking and functionality.
- Renamed createTodos function to runCreateTodosStep for consistency across step functions.
- Updated integration tests to reflect the new function names and ensure proper functionality.
- Removed outdated test files related to the think-and-prep step, reducing clutter and focusing on relevant tests.

These changes improve the modularity and clarity of the agent's configuration while enhancing the overall maintainability of the codebase.
2025-08-08 09:29:59 -06:00
dal 402b27d178
integration tests for the steps are great 2025-08-08 09:08:16 -06:00
dal e4b70eb1da
more clean up and exports 2025-08-08 09:04:16 -06:00
dal 0c1a0525e5
clean up 2025-08-08 08:59:57 -06:00
dal 135d95b46e
chore: remove outdated test files for analyst step functionality
- Deleted multiple test files related to the analyst step, including tests for database save, deduplication, failed tool handling, file ID mapping, and file selection.
- This cleanup reduces clutter in the codebase and focuses on maintaining relevant tests.

These changes aim to streamline the testing process and improve overall maintainability.
2025-08-08 08:46:23 -06:00
dal f3dd5c1ccb
refactor: export AnalystAgentOptionsSchema and AnalystStreamOptionsSchema
- Changed the declaration of AnalystAgentOptionsSchema and AnalystStreamOptionsSchema to export them for use in other modules.
- Removed the test files related to analyst-agent steps, streamlining the codebase and improving maintainability.

This update enhances the modularity of the agent's configuration schemas while cleaning up unused test files.
2025-08-08 08:25:41 -06:00
dal 1d92be2a71
more fixes 2025-08-08 08:06:55 -06:00
dal 283d74ca09
tons of cleanup 2025-08-08 08:03:22 -06:00
dal 1b70c6e2b3
fix: clean up imports and enhance dashboard file handling
- Removed duplicate import of ModelMessage in create-dashboards-execute.ts for clarity.
- Updated createDashboardsReasoningMessage function to conditionally include version_number only if defined, improving the handling of dashboard file properties.

All changes aim to streamline the code and enhance maintainability.
2025-08-08 07:14:13 -06:00
dal c08fdbc30f
ok now i actually removed them 2025-08-08 07:10:50 -06:00
dal a3a078c8cd
removed unnecessary streaming tools for the file tools 2025-08-08 07:10:42 -06:00
dal 4e19eddbb8
communication and cleanup 2025-08-08 06:55:54 -06:00
dal c432df1326
fix: update list-files-tool streaming test expectations to match actual implementation
Updated test expectations in list-files-tool-streaming.test.ts to match the actual structure of MessageEntry objects being passed to updateMessageEntries. The implementation passes complete objects with all properties rather than partial objects, and args are serialized as JSON strings.

Changes:
- Updated test expectations to check for complete MessageEntry objects instead of partial matches
- Fixed args field to expect JSON strings instead of objects
- Added missing properties like type, result, and started_at
- Fixed state.paths expectations for partial JSON parsing scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-07 23:39:30 -06:00
dal 8c2c94b31c
feat: enhance file selection and tool creation in communication tools
- Introduced new file selection logic to extract and prioritize files based on their status and relationships.
- Added helper functions for failure indicator checks and file extraction from reasoning history.
- Updated the communication tools to utilize the new file selection methods, improving response message creation.
- Refactored file tools to follow a consistent factory pattern for better maintainability and clarity.

All tests updated and passing for new functionality.
2025-08-07 23:22:29 -06:00
dal b2e5d702cc
feat: refactor create-files-tool to follow factory pattern
- Update create-file-tool.ts to use factory function from create-files-tool.ts
- Fix docs-agent to properly use factory functions for file tools
- Remove experimental_context usage in favor of proper context passing
- Fix lint issues with type definitions in transform helper
- All tests passing for create-files-tool functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-07 23:17:55 -06:00
dal d052f56f86
execute sql and sequential tests 2025-08-07 22:21:20 -06:00
dal ca2d3e940c
Refactor SQL execution tools to improve code readability and maintainability. Consolidate function parameters into single lines and enhance error handling in execution results. Update imports for consistency across files. 2025-08-07 22:10:11 -06:00
dal ee99a835c3
Refactor Analyst and Think-and-Prep agents to enforce required messageId field in options schema. Update tool creation logic to streamline context handling and enhance error reporting. Introduce new execute-sql tool with improved state management and SQL execution capabilities, including delta processing and message updates. Remove deprecated execute-sql functionality and related tests. 2025-08-07 22:09:58 -06:00
dal affa91ce4f
done with communication tools 2025-08-07 21:38:43 -06:00
dal f3436f7a04
Remove deprecated dashboard and metrics tools, including associated tests. Refactor the create and modify dashboard/metrics execution logic to streamline processing and enhance error handling. Update schemas and validation logic to ensure consistency across tools. 2025-08-07 20:36:48 -06:00
dal ead86c5d39
Refactor agent tools to enhance error handling and state management. Update healing message structure in Analyst and Think-and-Prep agents to provide more detailed tool availability information. Introduce state management for streaming in Modify and Create dashboards/metrics tools, improving database interaction and processing feedback. Remove deprecated tools and streamline tool creation processes. 2025-08-07 17:31:13 -06:00
dal 8987ca17a1
parallelToolCalls 2025-08-07 17:26:47 -06:00
dal 8c8d1f37cc
big migration of tools 2025-08-07 17:10:24 -06:00
dal 468cf44cca
done tool 2025-08-07 16:36:01 -06:00
dal 91290530ba
tons of changes 2025-08-07 16:09:45 -06:00
dal be13ce221d
think and prep minimal effort 2025-08-07 14:58:59 -06:00
Nate Kelley 09c96ab4f9
update agents 2025-08-07 14:07:20 -06:00
Nate Kelley 40a44cf1f9
Merge branch 'staging' into dallin-bus-1589-create-and-modify-report-tools 2025-08-07 14:05:44 -06:00
dal 290b698807
temp 1 2025-08-07 12:27:59 -06:00
dal 5bc3feb6f7
no temp 2025-08-07 12:23:15 -06:00
dal 9cafd4e0ff
remove max tokens 2025-08-07 12:22:16 -06:00
dal 92c4c1e257
Update AI package to use GPT-5 model and add OpenAI SDK dependency 2025-08-07 11:20:55 -06:00
dal 07eb5aa0c9
ok going to make tools follow factory pattern and prepping to move chunk processor to be tool specific 2025-08-07 09:58:28 -06:00
dal a065612deb
Refactor generateChatTitle function to improve title generation and database updates. Introduce helper functions for LLM title generation and database record updates, enhancing code clarity and maintainability. 2025-08-06 22:58:25 -06:00
dal 5370b8cbf6
generate cfhat title 2025-08-06 22:56:27 -06:00
dal 1d545a009c
healing messages 2025-08-06 17:57:04 -06:00
dal aaae50a32f
remove useless tests 2025-08-06 17:21:50 -06:00
dal 218bdf8eb3
Refactor Docs Agent to include additional context parameters and streamline SQL execution. Introduce user, chat, data source, organization, and optional message IDs in the DocsAgentOptions schema. Update createDocsAgent function to utilize the new context structure. Remove obsolete parseStreamingArgs function from execute-sql-docs-agent as AI SDK v5 handles streaming parsing internally. 2025-08-06 17:10:25 -06:00
dal 04ae594d3a
no need to have extra tool for tool call repair 2025-08-06 16:57:06 -06:00
dal 2b5efe790e
repiar tool try catch errors. 2025-08-06 16:52:21 -06:00
dal d211a2cd25
tons of ai sdk v5 changes 2025-08-06 16:52:05 -06:00
dal 74400c6b9d
Enhance analyst agent integration by adding user and organization context. Update the analyst agent to include additional parameters for user and organization IDs, and refactor tool integration to utilize new dashboard and metrics functions. Remove unused context injection helper to streamline the codebase. 2025-08-06 14:31:37 -06:00
dal f694c534c7
Refactor analyst agent tools integration and SQL execution context. Update the analyst agent to utilize new tool functions for metrics and dashboards, and modify SQL execution to accept context directly from the analyst agent options, enhancing clarity and maintainability. 2025-08-06 14:31:30 -06:00
dal 6b72213992
Update dependencies to stable versions and refactor analyst agent context. Replace beta versions with specific stable versions in pnpm-lock and package.json. Adjust context handling in various files to improve clarity and maintainability. Remove unused DocsAgentContext and related schemas. 2025-08-06 11:47:12 -06:00
dal 11c071ffe6
Refactor retry logic in analyst and think-and-prep steps to remove unnecessary error handling and improve healing strategy. Update tests to reflect changes in retry behavior and remove obsolete tests related to network error handling. 2025-08-05 22:56:02 -06:00
dal f17d81ba30
migration 2025-08-05 22:11:48 -06:00
dal a2d90f01a7
ai fallback on ai sdk v5 2025-08-05 18:24:32 -06:00
dal 69209e086c
Refactor edit reports functionality to support sequential edit operations with improved error handling and user feedback. Update schemas to include detailed operation results, including duration and error messages, enhancing overall report content management. 2025-08-05 14:45:55 -06:00
dal 2b926c3e91
Refactor report creation tool to support multiple report files with enhanced validation and error handling. Update schemas and processing logic to accommodate markdown content, ensuring robust file management and improved user feedback during report generation. 2025-08-05 14:10:09 -06:00
dal a26242c46d
Enhance report handling by integrating markdown to PlateJS conversion in the report retrieval process, improving content formatting and error handling. Update report update schema to accept string content for markdown streaming. 2025-08-05 12:02:40 -06:00
dal bb07af8cbd
route to different think and prep based on the context from the router 2025-08-05 11:15:59 -06:00
dal 5219cc0004
Enhance report generation capabilities by adding new report creation and editing tools, updating agent instructions to include report generation, and refining the analyst agent's workflow for improved narrative and analysis integration. 2025-08-05 11:09:55 -06:00
dal 03ccc51b5f
Refactor analysis type router to use a dedicated prompt formatting function and clean up code structure 2025-08-05 11:02:45 -06:00
dal d63aa94011
Implement analysis router enhancements and optimize report tools 2025-08-05 10:47:25 -06:00
dal db69b8bce5
Added in the analysis router using sonnet4 2025-08-05 10:47:05 -06:00
jacob-buster 0f78400dc4 Adding Router and Two Prompts 2025-08-05 10:28:19 -06:00
dal fcbe1838a1
migrating over to sdk v5 2025-08-05 09:40:05 -06:00
dal 5883fc8762
migrating over the agent 2025-08-05 09:39:56 -06:00
dal 9214075cad
analyst agent is clean 2025-08-05 00:54:23 -06:00
jacob-buster ab441d17cb Improved Report Builder Prompt 2025-08-04 15:33:33 -06:00
jacob-buster 65a1780b2c Further Reporting Improvements 2025-08-01 13:31:03 -06:00
dal f17df330d6
catch all for jacob testing 2025-07-31 15:34:27 -06:00
dal 14f34a7b41
Refactor report tools to include runtime context in parameters for create and edit functions 2025-07-31 15:30:32 -06:00
jacob-buster b17cc4695f Initial Report Builder Prompts 2025-07-30 12:22:50 -06:00
dal 0450b8c350
fallback and the analyst agent 2025-07-29 22:07:11 -06:00
dal 81ebcc67ca
fallback and analyst agent changes 2025-07-29 21:35:01 -06:00
dal bfee540c5b
starting migration to ai sdk 5 2025-07-29 20:57:22 -06:00
dal 2f8622e5bb
grep search getting me? 2025-07-29 20:14:20 -06:00
dal a6ea5bef71
switched the tools to use commands instead of scripts where possible 2025-07-29 16:05:47 -06:00
dal f2a8388b86
feat: enhance repository tree output with current directory information
- Added functionality to retrieve and prepend the current working directory to the repository tree output, improving context for users.
- Updated runtime context to store the modified tree structure with location information.
- Removed outdated integration tests and streamlined the test suite for better maintainability.

This update enhances the usability of the repository tree feature by providing clearer context during execution.
2025-07-29 14:48:18 -06:00
dal 3b1354ddff
refactor: enhance docs agent execution with abort handling and improved error management
- Introduced an AbortController to manage idle tool execution, allowing for graceful aborting of the documentation process.
- Moved tracking variables outside the try block for better state management.
- Updated error handling to gracefully manage abort scenarios, ensuring clear logging and returning final state information.
- Adjusted integration tests to reflect changes in the docs agent workflow, improving overall robustness and reliability.

This update enhances the responsiveness of the docs agent during execution and improves error handling for better user experience.
2025-07-29 14:32:31 -06:00
dal b31f856c3b
refactor: enhance file tools with sandbox execution and error handling improvements
- Updated file tools to generate CommonJS code for sandbox execution, improving robustness and flexibility.
- Refactored bash-execute-tool, create-files-tool, delete-files-tool, edit-files-tool, grep-search-tool, ls-files-tool, and read-files-tool to handle command arguments as JSON, enhancing error handling and output consistency.
- Improved integration tests to validate new execution patterns and error scenarios, ensuring comprehensive coverage across all tools.
- Enhanced error messages for better clarity during execution failures.

This update streamlines the execution process and enhances the overall reliability of file tools.
2025-07-29 14:21:13 -06:00
dal 54c7843b15
refactor: convert file-tools integration tests to concurrent execution
- Update all 7 file-tools integration tests to use concurrent execution
- Create individual sandboxes per test instead of shared beforeAll/afterAll
- Use try/finally blocks for proper sandbox cleanup
- Add 65-second timeouts to accommodate sandbox operations
- Replace it.skipIf pattern with (hasApiKey ? it.concurrent : it.concurrent.skip)
- Improves test performance by enabling parallel execution

Affected tools:
- bash-tool
- create-files-tool
- delete-files-tool
- edit-files-tool
- grep-search-tool
- ls-files-tool
- read-files-tool

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 09:29:20 -06:00
dal 60bc1d37a8
Revert "Revert "Add createReports and editReports tools to analyst-agent""
This reverts commit 67c2eea1f1.
2025-07-29 09:13:29 -06:00
dal 67c2eea1f1
Revert "Add createReports and editReports tools to analyst-agent"
This reverts commit 351689f93b.
2025-07-29 09:06:27 -06:00
dal 351689f93b
Add createReports and editReports tools to analyst-agent 2025-07-29 08:04:48 -06:00
dal 6d36edeeb2
moving tools over 2025-07-29 07:39:09 -06:00
dal ed9ab33b35
feat: enhance file tools to support base64 encoding for command arguments
- Updated bash-execute-script, create-files-script, edit-files-script, and grep-search-script to handle base64 encoded JSON arguments, improving robustness against data corruption.
- Refactored corresponding tool scripts to encode command parameters as base64 before execution.
- Enhanced integration tests to validate the new base64 encoding functionality across various file tools.
2025-07-28 16:55:52 -06:00
dal 3b706a73b3
refactor: streamline file tool scripts and enhance error handling
- Removed sequential test execution settings from vitest configuration to improve performance.
- Updated bash-execute-script to handle missing arguments and improved error handling for JSON parsing.
- Refactored create-files-tool to process file creation sequentially, enhancing directory management and error reporting.
- Enhanced read-files-tool to return structured error messages when no file paths are provided.
- Updated grep-search-tool to accept raw ripgrep commands, simplifying the command structure and improving output handling.
- Removed outdated test files and integrated new test cases for improved coverage and validation across file tools.
2025-07-28 16:22:12 -06:00
dal a41d5c944b
feat: implement ls-files functionality with comprehensive tests
- Introduced the `ls-files-script` module to handle file listing with various options (detailed, all, recursive, human-readable).
- Added integration tests for the `ls-files-script` to validate functionality across different scenarios, including handling of hidden files and non-existent paths.
- Removed the previous `ls-files-tool` integration tests to streamline the testing process and focus on the new implementation.
- Updated the `ls-files-tool` to utilize the new `ls-files-script` for execution and code generation.
- Enhanced error handling and output parsing for improved robustness.
2025-07-28 14:57:05 -06:00
dal 05c309b0c1
refactor: convert create-files-tool to script pattern
- Rename create-file-functions.ts to create-files-script.ts
- Implement script pattern matching ls-files-tool structure
- Update create-file-tool.ts to execute script via sandbox
- Add comprehensive unit tests for script functionality
- Add integration tests for script execution
- Update existing tests to work with new pattern

The create-files-tool now follows the established script pattern where
the script handles the core file creation logic and the tool serves as
a medium that executes the script in a sandboxed environment.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 13:54:03 -06:00
dal 786e4be791
feat: add integration tests for file tools and enhance todo list functionality
- Introduced integration tests for various file tools including bash execution, file creation, deletion, editing, and listing.
- Enhanced the check-off todo list tool to support multiple items and improved error handling for non-existent tasks.
- Updated test cases to cover edge scenarios and ensure robust validation across all tools.
- Increased timeout for the docs agent workflow to accommodate longer-running tests.
2025-07-28 12:41:07 -06:00
dal 2f4aeba817
feat: enhance documentation and testing capabilities for docs agent workflow
- Added section in CLAUDE.md for direct database access during integration testing.
- Updated `maxSteps` in `docs-agent` to allow for more complex tasks.
- Improved validation in `docs-agent-context` for sandbox instances.
- Enhanced `create-docs-todos` step to handle todos more effectively.
- Introduced comprehensive integration tests for the docs agent workflow, covering various scenarios and edge cases.
- Added test helpers for creating mock dbt projects and managing sandboxes.
- Implemented error handling and logging improvements in the workflow execution process.
2025-07-28 11:56:59 -06:00
jacob-buster ae0c4f048d Fixes for the filter tool + changes for data existence 2025-07-28 11:19:45 -06:00
dal 4bef4205f3
feat(access-controls): migrate Rust access control libraries to TypeScript
This PR migrates the Rust  and  libraries to TypeScript in the new  package.

## Changes

### New Access Controls Package
- Implemented complete asset permissions system with CRUD operations
- Added dataset permissions with all 5 access paths (admin, direct user, user→group, team→direct, team→group, org→default)
- Implemented cascading permissions (metrics inherit from dashboards/chats/collections)
- Added LRU caching to replace Redis (30-second TTL)
- Comprehensive test coverage (148 tests passing, 3 skipped pending createUser)

### Database Package Updates
- Fixed AssetType enum to include all 14 asset types
- Added missing database queries for permissions
- Fixed timestamp fields to use ISO strings
- Added getUserOrganizationsByUserId function
- Created organized query structure by table

### AI Package Updates
- Updated getPermissionedDatasets API calls to new format
- Changed property references from ymlFile to ymlContent
- Fixed all test mocks to match new API structure

### Trigger App Updates
- Updated to use new access controls API
- Fixed test mocks to match DatasetListResult structure
- Added proper type imports

### Server Shared Updates
- Added request/response types for access control endpoints
- Exported types for API consistency

## Testing
- All packages build successfully with turbo build:dry-run
- All tests pass with turbo test:unit
- Full type safety maintained throughout

## Migration Notes
- Drop-in replacement for Rust libraries
- Same permission model and roles
- Cache invalidation strategies implemented
- Soft delete pattern maintained

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 10:17:35 -06:00
dal a08f32fe1d
refactor: remove unused test setup files and helper functions to streamline testing process 2025-07-28 08:48:48 -06:00
dal c9b52ccdb8
Refactor CLAUDE.md for improved build and linting instructions; update context handling in workflow and schemas. Enhance test validation for clarification questions and adjust output structure in related tests. 2025-07-26 12:43:25 -06:00
dal a92c3a0b44
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-25 19:10:19 -06:00
dal 2833c6bb0c
dry run 2025-07-25 18:29:35 -06:00
dal 523a6921b1
hotfix on sql dialect 2025-07-25 14:31:35 -06:00
dal a6f8e041cd
fix 1 hr 2025-07-24 10:16:59 -06:00
dal 2c02b9189b
fix the 1 hr ttl 2025-07-24 10:16:19 -06:00
dal ed77fb4409
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-24 10:13:47 -06:00
dal 2d8890837a
add the extended cache beta header 2025-07-24 10:12:48 -06:00
dal 19f7256a27
switch system messages to leverage caching a bit better 2025-07-24 10:06:18 -06:00
dal 0c3994da00
hotfix: use v2 on chats 2025-07-24 09:22:30 -06:00
dal 7cd3aed6fa
Refactor import statements and enhance error messages in SQL validation tests
- Simplified import statements in web-search-tool.ts and web-search-tool.test.ts for better readability.
- Updated error messages in sql-parser-helpers.test.ts to provide clearer feedback regarding wildcard usage on physical tables.
2025-07-23 21:46:51 -06:00
Devin AI ae9f0c4f7e Fix incorrect import paths from '@buster-tools/web-tools' to '@buster/web-tools'
- Update package.json dependencies in ai and trigger packages
- Fix import statements in web-search-tool.ts and web-search-tool.test.ts
- Update pnpm-lock.yaml to reflect correct package references
- Resolves CI build failure due to missing module '@buster-tools/web-tools'

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:20:46 +00:00
Devin AI ab374dc465 BUS-1524: Enhance error messages for tool availability and SQL wildcards
- Update createWorkflowAwareHealingMessage to include next workflow step info and transition descriptions
- Replace SQL wildcard error message with more user-friendly text
- Maintain existing function signatures and return structures

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:09:56 +00:00
dal 5f43b2d074
new structure for chat and values 2025-07-23 17:39:03 -06:00
dal fb2e2c529c
feat: enhance documentation agent with SQL execution and context management
- Introduced executeSqlDocsAgent tool for executing SQL queries specific to documentation tasks.
- Updated DocsAgentContext to include dataSourceId and improved context management.
- Refactored context initialization and workflow steps to utilize new context structure.
- Enhanced createDocsTodosStep to generate comprehensive TODO lists based on user input.
- Improved error handling and logging throughout the documentation agent workflow.
- Updated tests to reflect changes in context keys and ensure proper functionality.
2025-07-23 15:53:18 -06:00
dal cb9aac6a2f
added in the web search to the agent 2025-07-23 14:26:46 -06:00
dal 2fe23084a5
Merge remote-tracking branch 'origin/staging' into dallin/bus-1498-piece-together-docs-agent-with-instructions-tools-workflow 2025-07-23 14:26:02 -06:00
dal 9e67fbfaae
refactor: update web search tool integration tests and improve code formatting
- Changed test query from 'OpenAI GPT' to 'Buster Data' with an increased limit of 10.
- Reorganized import statements in web-search-tool.ts for better readability.
- Enhanced formatting in permission-validator.test.ts for improved code clarity.
2025-07-23 13:12:56 -06:00
Devin AI 341b97f351 feat: implement web search functionality for BUS-1501
- Add webSearch method to FirecrawlService with WebSearchOptions, WebSearchResult, and WebSearchResponse interfaces
- Export new types from web-tools package
- Create web-search-tool in AI package with proper input/output schemas
- Add comprehensive unit tests for webSearch method (16 tests total)
- Add tool tests with mocking (4 tests)
- Add integration tests (skipped when no API key)
- Update package exports and dependencies
- Default search limit: 5, default format: markdown, content scraping enabled
- Proper error handling using CompanyResearchError
- AI package has no direct Firecrawl dependency

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 17:43:13 +00:00