Commit Graph

756 Commits

Author SHA1 Message Date
dal 594f08202b
Refactor metric file handling and improve error handling
- Updated type definitions for better clarity and type safety in metric file download and export tasks.
- Enhanced error handling in the metric files API to utilize HTTPException for more consistent error responses.
- Added environment variable validation in export tasks to ensure necessary credentials are present before execution.
- Improved type assertions for content in metric export queries to ensure proper handling of JSONB data.
2025-08-13 14:01:46 -06:00
dal a3c4238da1
lint fix 2025-08-13 13:48:28 -06:00
dal 3eb80ae630
Add metric file export functionality with Cloudflare R2 support
- Introduced new API endpoint for downloading metric files as CSV.
- Implemented export logic to handle large datasets (up to 1 million rows) and generate presigned URLs for secure downloads.
- Added cleanup task to remove exported files from R2 storage after 60 seconds.
- Updated environment configuration to include Cloudflare R2 credentials.
- Enhanced error handling for various export scenarios.
- Refactored related database queries and schemas for better integration.
- Updated documentation for new features and usage instructions.
2025-08-13 13:36:42 -06:00
dal d02d809e59
migration, optimization on entry querires, etc. 2025-08-13 11:50:56 -06:00
dal abc070449b
update the reasoning entries for all of hte tools 2025-08-13 11:04:37 -06:00
dal dab0bce7d2
Refactor tool imports and enhance naming consistency in Think and Prep Agent
- Updated import statements for communication and database tools to improve clarity and organization.
- Introduced constants for tool names in the `execute-sql` and `sequential-thinking-tool` modules to ensure consistent usage across the codebase.
- Enhanced the `createThinkAndPrepAgent` function to utilize these constants, improving maintainability and readability.

These changes streamline the integration of tools within the Think and Prep Agent, enhancing overall code organization.
2025-08-13 10:52:28 -06:00
dal 3e1181c121
more type safety 2025-08-13 10:49:35 -06:00
dal 8299e47e4e
more migration, streaming, type safety, etc 2025-08-13 10:48:07 -06:00
dal eb14e26972
dashboard helpers and tools 2025-08-12 23:26:46 -06:00
dal d9e6c3dee2
done tool no overwrite 2025-08-12 23:23:35 -06:00
dal 6c989b7c25
Refactor tool names and enhance context handling in AnalystAgent
- Updated tool name constants for visualization and communication tools to improve clarity and consistency.
- Modified the AnalystAgent to utilize these constants, ensuring better context handling during tool calls.
- Enhanced the STOP_CONDITIONS to reference the DONE_TOOL_NAME constant, improving maintainability.

These changes streamline the integration of tools within the AnalystAgent, enhancing overall code organization and readability.
2025-08-12 23:05:45 -06:00
dal 0f0a5ed7d1
fix prompts of dashboard and metric tools 2025-08-12 22:42:21 -06:00
dal 5d2631b848
Enhance agent streaming and logging in Analyst and Think and Prep steps
- Added consumption of text streams in both `runAnalystAgentStep` and `runThinkAndPrepAgentStep` to ensure continuous processing of agent responses.
- Introduced detailed logging at various stages of the agent's execution to improve traceability and debugging.
- These changes enhance the robustness of the agent workflow and provide better insights during execution.
2025-08-12 21:47:09 -06:00
dal 855e7b1a55
Integrate dataset handling into Analyst and Think and Prep agents
- Added support for fetching and managing user-specific datasets in the `analystAgentTask`, enhancing the context available for processing.
- Updated the `AnalystAgent` and `ThinkAndPrepAgent` schemas to include datasets, ensuring they are passed correctly in system messages.
- Implemented error handling for dataset retrieval to prevent workflow interruptions.
- Adjusted integration tests to accommodate the new datasets structure, ensuring comprehensive coverage.

These changes improve the agents' ability to utilize relevant datasets, enhancing their functionality and user experience.
2025-08-12 19:31:25 -06:00
dal cc6c407023
Enhance TODO creation process with streaming support
- Introduced a new streaming mechanism for generating TODO lists, allowing for real-time updates and improved user experience.
- Added context handling for datasets in both `AnalystAgent` and `ThinkAndPrepAgent`, ensuring relevant data is included in system messages.
- Implemented new helper functions for managing TODO state and reasoning messages, improving code organization and clarity.
- Updated integration tests to validate the new streaming functionality and ensure consistent message roles.

These changes significantly enhance the functionality and maintainability of the TODO creation process within the agent workflow.
2025-08-12 18:32:56 -06:00
dal 1e594ac4ec
Refactor chat message handling by removing unused fields
- Eliminated `responseMessages`, `reasoning`, and `rawLlmMessages` from chat creation and handling functions to streamline data processing.
- Updated the `generateJsonbArraySql` function to accept pre-stringified JSON for better performance and clarity.

These changes enhance the efficiency of chat message management and reduce unnecessary complexity in the codebase.
2025-08-12 16:38:20 -06:00
dal e52c0c7d72
move sequential thinking away from old inherited context 2025-08-12 16:15:37 -06:00
dal 0b8f8665d6
Refactor agent steps and message roles
- Removed unused `steps` array from `createThinkAndPrepAgent` function to simplify the code.
- Updated message role from 'assistant' to 'user' in `create-todos-step` for better context alignment.
- Adjusted `valuesMessage` role in `extract-values-search-step` to 'user' when extracted values are present, enhancing message clarity.

These changes improve code clarity and ensure consistent message roles across agent steps.
2025-08-12 16:13:41 -06:00
dal a7ced93c6e
move to newer braintrust version 2025-08-12 16:06:00 -06:00
dal 1d0d312ff3
lots of tests passing 2025-08-12 15:54:46 -06:00
dal da4f3605cc
Refactor prompt loading and streamline agent steps
- Replaced file system reads with direct imports of prompt text files in various agent and step files to improve performance and reduce complexity.
- Updated the `runDocsAgentStep` and `runDocsAgentWorkflow` functions to return `void` instead of structured outputs, simplifying their interfaces.
- Enhanced test cases to validate prompt loading without relying on file system checks, ensuring faster and more reliable tests.
- Cleaned up unused imports and optimized code structure across multiple files.

These changes aim to enhance code clarity, maintainability, and performance in the agent workflow and related components.
2025-08-12 14:57:23 -06:00
dal 8c27a3dfff
Merge remote-tracking branch 'origin/staging' into dal/upgrade-to-ai-sdk-v5 2025-08-12 12:53:48 -06:00
dal c6a28e39d5
migration changes 2025-08-12 12:44:15 -06:00
dal 79103376f1
more migratsion 2025-08-12 12:30:38 -06:00
dal 7e6e406727
more changes 2025-08-12 12:10:21 -06:00
dal 8f58ff84de
lots of migration and report tooling 2025-08-12 12:08:30 -06:00
dal 4871f49f59
Refactor identify assumptions step and related tests
- Updated type imports from `CoreMessage` to `ModelMessage` for consistency across the identify assumptions step.
- Changed argument names from `args` to `input` and `result` to `output` in the integration tests to align with the new structure.
- Removed legacy step export for `identifyAssumptionsStepLegacy` to clean up the codebase.
- Adjusted the reporting tools to import `StatusSchema` from the correct path.
- Deleted outdated test files and helper functions related to the docs agent workflow to streamline the project structure.

These changes improve code clarity and maintainability while ensuring the integration tests reflect the latest design patterns.
2025-08-12 11:58:15 -06:00
dal 299efd1c13
format follow up messages 2025-08-12 11:50:29 -06:00
dal 4a241ce419
repository tree cleanup 2025-08-12 11:46:15 -06:00
dal 9052bc6251
build issues cleaned up 2025-08-12 11:43:54 -06:00
dal cf3dc3387b
modify metrics finish 2025-08-12 11:33:39 -06:00
dal eb68b4edff
flag chat step clean 2025-08-12 11:33:01 -06:00
dal b699b9f2c9
versioned asset details 2025-08-12 11:32:37 -06:00
dal 72150c2ee1
clean bar line tests 2025-08-12 11:32:28 -06:00
dal 31c3cbc1b7
refactor: enhance reporting tools with new create and modify functionalities
- Introduced `createReports` and `modifyReports` tools, enabling users to create and modify reports with improved state management.
- Added new schemas for report input and output, enhancing type safety and clarity.
- Implemented helper functions for transforming report state and generating reasoning entries.
- Improved test coverage for the new reporting functionalities to ensure robust integration.

These changes streamline the reporting process and enhance overall code maintainability.
2025-08-12 11:19:07 -06:00
dal 819b2dd0d6
tons of migration from mastra to ai sdk 2025-08-12 10:36:55 -06:00
dal a5422360ec
migrating over docs agetn 2025-08-12 10:13:16 -06:00
dal 9d9690bb35
analysis type router 2025-08-12 09:36:40 -06:00
dal 77bc071f4b
more migrations 2025-08-12 09:32:07 -06:00
dal 29708e8d7d
create and modify 2025-08-12 09:10:23 -06:00
dal 7872ad2f29
refactor: enhance modify dashboards tool with improved state management and type definitions
- Updated the `modifyDashboardsDelta` and `modifyDashboardsFinish` functions to streamline state handling and improve message entry updates.
- Refactored helper functions for better clarity and type safety in dashboard modifications.
- Introduced new schemas for state management, enhancing type definitions for dashboard files and their statuses.
- Improved test coverage to ensure robust functionality and integration of the modify dashboards tool.

These changes optimize the dashboard modification process and enhance code maintainability.
2025-08-11 22:30:14 -06:00
dal d3870430c2
create dashboard fixes 2025-08-11 19:00:33 -06:00
dal e2d0d332c1
create dashboard functions debugged 2025-08-11 18:37:31 -06:00
dal 70193aad4f
fix lint issues 2025-08-11 16:57:49 -06:00
dal 4e7f6b080e
no reports in todos 2025-08-11 16:22:50 -06:00
dal f1d9293946
refactor: enhance create dashboards tool with improved state management and type definitions
- Removed unused properties from the dashboard state and output schemas for clarity.
- Introduced a new state file schema to better represent dashboard files.
- Updated helper functions to streamline file transformation and reasoning message creation.
- Improved type safety by utilizing the newly defined `StatusSchema`.

These changes optimize the dashboard creation process and enhance code maintainability.
2025-08-11 15:59:29 -06:00
dal 95c29cf507
Refactor analyst and think-and-prep agent instructions to remove report generation capabilities and streamline workflow. Update the analyst agent to exclude report-related tools and adjust think-and-prep instructions for clarity and focus on analysis preparation. Ensure consistent handling of SQL guidance and improve overall structure for better readability. 2025-08-11 15:58:07 -06:00
dal 317b7cb8c3
bump max tokens 2025-08-11 15:41:34 -06:00
Nate Kelley 48fa337303
can parse dollar signs 2025-08-11 15:10:55 -06:00
dal b5a15c8581
dashboard tool description fix 2025-08-11 15:08:18 -06:00
Nate Kelley 438bec4ce4
Fix caption parser 2025-08-11 15:01:08 -06:00
dal 71d862e51b
sql tools 2025-08-11 14:54:43 -06:00
Nate Kelley a5b175d418
Update create-todos-step.ts 2025-08-11 14:54:12 -06:00
Nate Kelley ec7d27167f
Merge branch 'staging' of https://github.com/buster-so/buster into staging 2025-08-11 14:53:43 -06:00
Nate Kelley 9dd37de7cd
Add tests for metric caption parsing 2025-08-11 14:53:41 -06:00
dal 82c1ea7f46
communication tools plus others 2025-08-11 14:53:23 -06:00
dal 2af5f408fa
refactor: enhance create dashboards tool and web search tool integration
- Updated the `createDashboardsStart` function to improve message handling and reasoning entry creation.
- Refactored helper functions for better clarity and type safety in dashboard creation.
- Enhanced web search tool execution to include additional parameters for improved functionality.
- Updated tests to reflect changes in tool execution and ensure robust integration.

These changes streamline the dashboard creation process and enhance the web search tool's capabilities.
2025-08-11 14:50:38 -06:00
dal 698444bb3e
create dashboard fixes with websearch fixes 2025-08-11 14:47:50 -06:00
dal 4ace4d337e
Revert 'Testing some new GPT 5 metaprompts' from commit 1f4f31909
This reverts commit 1f4f319098 which made changes to:
- analyst-agent-instructions.ts
- think-and-prep-instructions.ts
- investigation-instructions.ts
- format-analysis-type-router-prompt.ts
- create-todos-step.ts

Reverting Jacob's experimental prompt changes.
2025-08-11 14:33:07 -06:00
dal 2540f6e195
Revert "Improved Report Prompt"
This reverts commit ed056878e1.
2025-08-11 14:31:35 -06:00
dal 1c0a6b4f0f
move back to anthropic models 2025-08-11 14:27:16 -06:00
dal c6f4ac3902
verbosity low 2025-08-11 14:07:16 -06:00
dal 47f2e38557
priority tier 2025-08-11 13:55:43 -06:00
dal 82d89cf702
clean up builds and errors 2025-08-11 13:41:17 -06:00
Nate Kelley 23411f25c6
report editor updates 2025-08-11 13:40:01 -06:00
Nate Kelley 78374e7ddf
Update server editor 2025-08-11 13:13:33 -06:00
dal b96cb39b78
turbo fast web build 2025-08-11 11:33:18 -06:00
dal 1e021f4da1
refactor: introduce new planning tools for todo list and clarifications management
- Added `createCheckOffTodoListTool` and `createUpdateClarificationsFileTool` to enhance planning capabilities.
- Refactored existing tools to utilize the new planning tools, improving code organization and maintainability.
- Updated tests to reflect changes in tool creation and execution, ensuring robust functionality.

These changes streamline the management of todo lists and clarification questions, enhancing overall tool usability.
2025-08-11 10:28:26 -06:00
dal 5d21fd4782
Merge pull request #698 from buster-so/GPT-5-MetaPrompting
Improved Report Prompt
2025-08-11 10:16:04 -06:00
dal 2712697722
refactor: integrate web search tool and enhance metrics processing
- Introduced the `createWebSearchTool` function to streamline web search capabilities.
- Updated various metrics tools to improve type handling and input processing.
- Refactored metrics delta and finish functions to utilize `ToolCallOptions` for better integration.
- Removed deprecated report creation and editing tools to clean up the codebase.

These changes enhance functionality and maintainability across the tools.
2025-08-11 10:15:34 -06:00
jacob-buster ed056878e1 Improved Report Prompt 2025-08-11 10:13:22 -06:00
dal e93a82a3f5
refactor: enhance dashboard and metrics tools with improved descriptions and type handling
- Added the `GPT5` import to the analyst agent for enhanced functionality.
- Updated the `createModifyDashboardsStart` function to utilize `ToolCallOptions` for better input handling.
- Refactored dashboard and metrics tools to use helper functions for descriptions, improving maintainability.
- Ensured consistent type handling and removed redundant code in metrics delta processing.

These changes streamline the tools and improve the overall code quality.
2025-08-11 08:39:05 -06:00
dal 74ae581f73
refactor: update ESLint rules and clean up socket query files
- Disabled specific TypeScript ESLint rules to allow for more flexibility in type definitions.
- Removed unused socket query files and related types to streamline the codebase.
- Enhanced various components and layouts for better performance and readability.

These changes improve code maintainability and reduce clutter in the project.
2025-08-11 07:57:14 -06:00
Nate Kelley 95d081766e
update imports 2025-08-08 20:39:15 -06:00
Nate Kelley 9a88c67b9b
Merge pull request #689 from buster-so/GPT-5-MetaPrompting
Testing some new GPT 5 metaprompts
2025-08-08 17:01:08 -06:00
jacob-buster 1f4f319098 Testing some new GPT 5 metaprompts 2025-08-08 16:10:46 -06:00
dal 51b7d4e691
Remove serviceTier option from OpenAI configuration in analyst and think-and-prep agents 2025-08-08 15:25:52 -06:00
dal b00b5c6858
Add serviceTier option to OpenAI configuration in analyst and think-and-prep agents 2025-08-08 14:32:21 -06:00
dal e21ac9d1e1
low verbosity on anlysis router 2025-08-08 14:31:14 -06:00
dal 63450d31d7
low verbosity 2025-08-08 14:30:04 -06:00
dal 663bafdb41
dumb down and make smarter 2025-08-08 14:21:51 -06:00
dal 14092b08be
move everything over to gpt5 2025-08-08 13:29:59 -06:00
dal 5e97fbffd2
enhance create-todos-system-prompt: add tool calling guidelines and clarify user request interpretation
- Introduced a section on tool calling rules to ensure proper usage of available tools.
- Added clarifications to the TODO list items regarding user requests, emphasizing the need to determine if the user is asking for a single metric, a report, or a dashboard.

These updates improve the clarity and functionality of the analyst agent's prompt, ensuring more accurate task breakdowns.
2025-08-08 13:18:10 -06:00
dal 9156e7a06e
merge staging 2025-08-08 13:17:58 -06:00
dal e75f90ceab
refactor: update dashboard types and schemas for improved validation
- Removed unused import for `createMetricsRawLlmMessageEntry` in visualization tools.
- Introduced Zod schemas for `Dashboard` and `DashboardConfig` to enhance type validation and structure.
- Cleaned up the interface definitions for better clarity and maintainability.

These changes improve the robustness of the dashboard configuration and streamline the codebase.
2025-08-08 13:00:20 -06:00
dal ff8530a08f
a little reorg 2025-08-08 11:51:06 -06:00
dal dada5cb0eb
refactor: remove unused dependencies and clean up visualization tools
- Deleted references to the `@mastra/core` package from various configuration files.
- Removed multiple unused test files and helper functions related to visualization tools, including axis validation and version history management.
- Cleaned up import statements for better organization and readability.

These changes streamline the codebase by eliminating unnecessary dependencies and improving maintainability.
2025-08-08 11:48:53 -06:00
dal 601a13c402
analyst agent steps is clean 2025-08-08 11:33:04 -06:00
Nate Kelley 5456561734
add a better markdown converter 2025-08-08 09:57:05 -06:00
dal 7f1bfc3bc5
fix: improve error handling and refactor test mocks for SQL execution
- Added a check to ensure the DataSource is available before executing SQL statements, throwing an error if not.
- Refactored test mocks to use `vi.mocked` for better clarity and consistency in handling DataSource connection errors.
- Removed unnecessary whitespace for cleaner code.

These changes enhance the robustness of SQL execution and improve the reliability of related tests.
2025-08-08 09:36:26 -06:00
dal fb65ac42ea
test: update integration tests to include additional properties for agent options
- Added userId, chatId, organizationId, dataSourceId, and dataSourceSyntax to the integration tests for the Think and Prep Agent and Mark Message Complete Step.
- Cleaned up import statements for better organization and readability.
- Ensured that the tests reflect the latest schema changes for improved tracking and functionality.

These updates enhance the clarity and robustness of the integration tests, aligning them with recent changes in agent options.
2025-08-08 09:32:53 -06:00
dal 3d12455f69
refactor: enhance agent options and streamline step functions
- Added new properties (chatId, organizationId, dataSourceId, dataSourceSyntax, userId) to the ThinkAndPrepAgentOptionsSchema for improved tracking and functionality.
- Renamed createTodos function to runCreateTodosStep for consistency across step functions.
- Updated integration tests to reflect the new function names and ensure proper functionality.
- Removed outdated test files related to the think-and-prep step, reducing clutter and focusing on relevant tests.

These changes improve the modularity and clarity of the agent's configuration while enhancing the overall maintainability of the codebase.
2025-08-08 09:29:59 -06:00
dal 402b27d178
integration tests for the steps are great 2025-08-08 09:08:16 -06:00
dal e4b70eb1da
more clean up and exports 2025-08-08 09:04:16 -06:00
dal 0c1a0525e5
clean up 2025-08-08 08:59:57 -06:00
dal 135d95b46e
chore: remove outdated test files for analyst step functionality
- Deleted multiple test files related to the analyst step, including tests for database save, deduplication, failed tool handling, file ID mapping, and file selection.
- This cleanup reduces clutter in the codebase and focuses on maintaining relevant tests.

These changes aim to streamline the testing process and improve overall maintainability.
2025-08-08 08:46:23 -06:00
dal f3dd5c1ccb
refactor: export AnalystAgentOptionsSchema and AnalystStreamOptionsSchema
- Changed the declaration of AnalystAgentOptionsSchema and AnalystStreamOptionsSchema to export them for use in other modules.
- Removed the test files related to analyst-agent steps, streamlining the codebase and improving maintainability.

This update enhances the modularity of the agent's configuration schemas while cleaning up unused test files.
2025-08-08 08:25:41 -06:00
dal 1d92be2a71
more fixes 2025-08-08 08:06:55 -06:00
dal 283d74ca09
tons of cleanup 2025-08-08 08:03:22 -06:00
dal 1b70c6e2b3
fix: clean up imports and enhance dashboard file handling
- Removed duplicate import of ModelMessage in create-dashboards-execute.ts for clarity.
- Updated createDashboardsReasoningMessage function to conditionally include version_number only if defined, improving the handling of dashboard file properties.

All changes aim to streamline the code and enhance maintainability.
2025-08-08 07:14:13 -06:00