Commit Graph

445 Commits

Author SHA1 Message Date
dal 47f413e111
fix: update access-controls tests for new cache structure
Updated dataset permission tests to expect the full DatasetAccessResult
object instead of just a boolean value in cache operations. This aligns
with the PR feedback to cache the complete access result including
accessPath and userRole.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 12:29:15 -06:00
dal 273fbc36c4
fix: address non-critical PR review comments
- Optimized N+1 query in get-permissioned-datasets.ts using inArray for batch lookup
- Removed unused bulkRemoveAssetPermissions import
- Fixed import organization in find-user-by-email.ts
- Updated CLAUDE.md to reflect tests are written and fixed API example
- Clarified TODO comment in lookup.ts to prevent potential infinite recursion

These are style and performance improvements that don't affect functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 12:29:02 -06:00
dal 2e04af1785
fix: address additional critical PR review comments
- Fixed permission hierarchy: full_access (5) now correctly ranks higher than viewer (3)
- Removed 'querier' role from admin permission check as it was too permissive
- Added organizationId parameter to dashboard access checks to prevent unauthorized access
- Fixed public dashboard expiry date check to verify dates are in the future
- Added cycle detection for collection permissions to prevent infinite recursion
- Fixed bulk remove permissions to correctly filter by identityIds array
- Updated date comparison to use ISO string format for PostgreSQL timestamp columns

These fixes address critical security and logic issues identified in the PR review.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 12:28:58 -06:00
dal d078830a18
fix: address PR review comments for access control migration
- Fixed AssetTypeSchema to include all active asset types (excluding deprecated 'metric' and 'dashboard')
- Added missing 'restricted_querier' role to UserOrganizationRole type
- Fixed dataset access caching to store full result object including accessPath and userRole
- Updated count query to use SQL COUNT for better performance
- Fixed IdentityType consistency across dataset permissions
- Removed unused 'ne' import from list-asset-permissions.ts
- Updated comments to correctly reference 6 access paths instead of 5

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-29 12:28:52 -06:00
dal 4bef4205f3
feat(access-controls): migrate Rust access control libraries to TypeScript
This PR migrates the Rust  and  libraries to TypeScript in the new  package.

## Changes

### New Access Controls Package
- Implemented complete asset permissions system with CRUD operations
- Added dataset permissions with all 5 access paths (admin, direct user, user→group, team→direct, team→group, org→default)
- Implemented cascading permissions (metrics inherit from dashboards/chats/collections)
- Added LRU caching to replace Redis (30-second TTL)
- Comprehensive test coverage (148 tests passing, 3 skipped pending createUser)

### Database Package Updates
- Fixed AssetType enum to include all 14 asset types
- Added missing database queries for permissions
- Fixed timestamp fields to use ISO strings
- Added getUserOrganizationsByUserId function
- Created organized query structure by table

### AI Package Updates
- Updated getPermissionedDatasets API calls to new format
- Changed property references from ymlFile to ymlContent
- Fixed all test mocks to match new API structure

### Trigger App Updates
- Updated to use new access controls API
- Fixed test mocks to match DatasetListResult structure
- Added proper type imports

### Server Shared Updates
- Added request/response types for access control endpoints
- Exported types for API consistency

## Testing
- All packages build successfully with turbo build:dry-run
- All tests pass with turbo test:unit
- Full type safety maintained throughout

## Migration Notes
- Drop-in replacement for Rust libraries
- Same permission model and roles
- Cache invalidation strategies implemented
- Soft delete pattern maintained

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 10:17:35 -06:00
dal 57dd2895d2
dry run for these two bad boys 2025-07-25 19:00:07 -06:00
dal 2833c6bb0c
dry run 2025-07-25 18:29:35 -06:00
dal 523a6921b1
hotfix on sql dialect 2025-07-25 14:31:35 -06:00
dal 1f52bd9b69
Enhance asset chat handling by introducing `handleAssetChatWithPrompt` for processing user prompts alongside asset imports. Update `createChatHandler` to manage asset-only and asset+prompt scenarios, ensuring correct message order and preventing unnecessary analyst task triggers. Add tests for new functionality and refactor related services for improved clarity and maintainability. 2025-07-25 12:47:01 -06:00
Nate Kelley 5e6eb76f5b
move get asset types around 2025-07-24 17:41:36 -06:00
Nate Kelley b2cf1725fc
Merge branch 'staging' into devin/BUS-1494-1753276227 2025-07-24 17:21:04 -06:00
dal a6f8e041cd
fix 1 hr 2025-07-24 10:16:59 -06:00
dal 2d8890837a
add the extended cache beta header 2025-07-24 10:12:48 -06:00
dal 19f7256a27
switch system messages to leverage caching a bit better 2025-07-24 10:06:18 -06:00
dal 0c3994da00
hotfix: use v2 on chats 2025-07-24 09:22:30 -06:00
Nate Kelley af090d467e
fix tests 2025-07-23 22:53:54 -06:00
Nate Kelley e6d4062847
Merge branch 'staging' into big-nate/bus-1483-quick-win-for-filter-dashboard-drill-downexplore-metric 2025-07-23 22:45:54 -06:00
Nate Kelley 5eb43d0373
Add expanded threedot menu for menus 2025-07-23 22:33:16 -06:00
dal 7cd3aed6fa
Refactor import statements and enhance error messages in SQL validation tests
- Simplified import statements in web-search-tool.ts and web-search-tool.test.ts for better readability.
- Updated error messages in sql-parser-helpers.test.ts to provide clearer feedback regarding wildcard usage on physical tables.
2025-07-23 21:46:51 -06:00
Devin AI ae9f0c4f7e Fix incorrect import paths from '@buster-tools/web-tools' to '@buster/web-tools'
- Update package.json dependencies in ai and trigger packages
- Fix import statements in web-search-tool.ts and web-search-tool.test.ts
- Update pnpm-lock.yaml to reflect correct package references
- Resolves CI build failure due to missing module '@buster-tools/web-tools'

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:20:46 +00:00
Devin AI ab374dc465 BUS-1524: Enhance error messages for tool availability and SQL wildcards
- Update createWorkflowAwareHealingMessage to include next workflow step info and transition descriptions
- Replace SQL wildcard error message with more user-friendly text
- Maintain existing function signatures and return structures

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-24 03:09:56 +00:00
dal 2140f08163
package renamo 2025-07-23 21:06:42 -06:00
dal 5f43b2d074
new structure for chat and values 2025-07-23 17:39:03 -06:00
Nate Kelley df3533ffc7
pass in better props 2025-07-23 17:18:45 -06:00
dal 482564528a
lint fixes 2025-07-23 16:56:32 -06:00
Nate Kelley 503c9fafa4
response types update 2025-07-23 16:56:17 -06:00
dal 42d1da5389
context fixes on messages 2025-07-23 16:56:06 -06:00
Nate Kelley a11ce3bfd2
Update responses and requests 2025-07-23 16:35:14 -06:00
Nate Kelley 1ea1d6c97c
move around some types 2025-07-23 16:29:00 -06:00
Nate Kelley 0601a3cd09
fix weird vite 7 error 2025-07-23 15:57:44 -06:00
dal 9e67fbfaae
refactor: update web search tool integration tests and improve code formatting
- Changed test query from 'OpenAI GPT' to 'Buster Data' with an increased limit of 10.
- Reorganized import statements in web-search-tool.ts for better readability.
- Enhanced formatting in permission-validator.test.ts for improved code clarity.
2025-07-23 13:12:56 -06:00
Devin AI 341b97f351 feat: implement web search functionality for BUS-1501
- Add webSearch method to FirecrawlService with WebSearchOptions, WebSearchResult, and WebSearchResponse interfaces
- Export new types from web-tools package
- Create web-search-tool in AI package with proper input/output schemas
- Add comprehensive unit tests for webSearch method (16 tests total)
- Add tool tests with mocking (4 tests)
- Add integration tests (skipped when no API key)
- Update package exports and dependencies
- Default search limit: 5, default format: markdown, content scraping enabled
- Proper error handling using CompanyResearchError
- AI package has no direct Firecrawl dependency

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 17:43:13 +00:00
dal 21a89151d3
Merge pull request #600 from buster-so/devin/BUS-1487-1753225168
feat: implement wildcard validation for SQL security (BUS-1487)
2025-07-23 10:23:22 -06:00
dal 240986d7a0
instruction on execute sql 2025-07-23 10:16:49 -06:00
dal 77ffeab37a
Fix Snowflake adapter tests for lowercase column names
The Snowflake adapter implementation transforms column names to lowercase
for consistency, but the tests were expecting uppercase column names.
This commit updates the tests to match the implementation:

- Update test expectations to use lowercase column names (id, name)
- Fix hasMoreRows assertions to match implementation logic (only true when rowCount > limit)
- Ensure all Snowflake-related tests pass with the current adapter behavior

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-23 10:09:51 -06:00
dal d371a65524
Update Snowflake adapter tests and implementation for improved row handling
- Increased allowed variance in cached query time checks to accommodate network fluctuations.
- Corrected property name in test assertions to match expected lowercase format.
- Enhanced SnowflakeAdapter to transform column names to lowercase and adjusted logic for determining if more rows are available from the stream.
2025-07-23 10:01:34 -06:00
dal 03ba461dc4
Merge remote-tracking branch 'origin/staging' into devin/BUS-1495-1753278753 2025-07-23 09:43:18 -06:00
dal 5aa781a536
Add fetch body modification for tool_choice in Anthropics and Vertex models
- Implemented a custom fetch function in both anthropic.ts and vertex.ts to modify the request body.
- If tool_choice is present, added disable_parallel_tool_use to the request body.
- Included error handling for JSON parsing to ensure fallback to original request on failure.
2025-07-23 09:32:07 -06:00
dal f10627b005
Implement AbortError handling in streaming models
- Added tests for handling AbortError scenarios in both memory-safe and standard streaming models.
- Updated the FallbackModel to gracefully handle intentional aborts without retrying or causing errors.
- Ensured that the controller closes properly on AbortError and that no fallback to subsequent models occurs.
- Enhanced mock models to simulate AbortError conditions for comprehensive testing.
2025-07-23 09:01:08 -06:00
dal 5be55782fc
Update packages/ai/src/utils/memory/message-converters.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:47 -06:00
dal 87aa4f1993
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:40 -06:00
dal 3161c2b5e1
Update packages/ai/src/utils/memory/message-converters.test.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:20 -06:00
dal 5b0b6f360b
Update packages/ai/src/utils/database/format-llm-messages-as-reasoning.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:29:06 -06:00
dal b1be6aacc2
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:58 -06:00
dal b13d7c71fb
Update packages/ai/evals/agents/analyst-agent/metrics/think-and-prep-updates.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:38 -06:00
dal 6c52727b48
Update packages/ai/src/tools/communication-tools/respond-without-asset-creation.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-07-23 08:28:32 -06:00
Devin AI a44153e2ee fix: update permission validator tests to work with wildcard validation
- Replace SELECT * with explicit column names in permission validation tests
- Update CTE test to use explicit columns in final SELECT
- Maintain test intent while aligning with new wildcard security validation
- Ensure all tests pass with new wildcard blocking behavior

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 14:23:32 +00:00
Devin AI efd56f90a7 Fix max-rows-limiting test mocks for streaming implementation
- Update Snowflake adapter test to use streamResult: true
- Mock streamRows method with proper stream event handling
- Remove TypeScript error from destroyed property
- Verify streamRows called with correct start/end parameters

Co-Authored-By: Dallin Bentley <dallinbentley98@gmail.com>
2025-07-23 14:12:25 +00:00
dal 24cfab962d
refactor: rename respondWithoutAnalysis to respondWithoutAssetCreation
- Updated references across multiple files to reflect the new tool name.
- Adjusted unit tests and schemas to ensure compatibility with the renamed tool.
- Removed the old respondWithoutAnalysis tool implementation and its associated tests.
- Ensured backward compatibility for existing functionality using the old tool name.
2025-07-23 08:12:23 -06:00
dal 7ddc683104
Merge remote-tracking branch 'origin/blake/prompt-changes-think-and-prep' into dallin/bus-1122-reviewpush-the-think-and-prep-mode-updates 2025-07-23 07:59:52 -06:00