Ling0925
/
buster
mirror of https://github.com/buster-so/buster.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

reformatted prds

This commit is contained in:
dal 2025-04-07 15:13:25 -06:00
parent ec916767c4
commit 84b797cbf7
No known key found for this signature in database
GPG Key ID: 16F4B0E1E9F61122
5 changed files with 675 additions and 983 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

426
api/prds/active/api_http_status_fix.md
View File

@ -9,296 +9,210 @@ ticket: BUS-1067
# HTTP Status Code Fix
## Parent Project
This is a sub-PRD of the [Bug Fixes and Testing Improvements](project_bug_fixes_and_testing.md) project. Please refer to the parent PRD for the overall project context, goals, and implementation plan.
## Problem Statement
The API is currently returning incorrect HTTP status codes in several scenarios, particularly in error cases. This inconsistency makes it difficult for clients to properly handle errors and leads to confusion in error handling. The main issues are:
<!--
Clearly articulate the problem you're solving. Include:
- Current state and behavior
- Expected behavior
- How this fits into the larger project
- Specific pain points this component addresses
-->
Current behavior:
- Some error responses return 200 OK with error in body
- Inconsistent use of 4xx status codes
- Missing proper status codes for specific error cases
- Lack of standardization across handlers
- Permission denied errors return 404 instead of 403
- Version not found errors have inconsistent handling
- Error status codes differ between metrics and dashboards
- Error messages in status codes don't match handler messages
- No standardized error response format
Expected behavior:
- Proper HTTP status codes for all responses
- Consistent error status codes
- Clear mapping between error types and status codes
- Permission denied returns 403 Forbidden
- Version not found returns 404 Not Found
- Consistent error handling across all asset types
- Clear mapping between handler errors and status codes
- Standardized error response format
## Goals
1. Standardize HTTP status codes across all handlers
2. Implement proper error status codes
3. Create error-to-status code mapping
4. Add tests for status code verification
5. Document status code usage
1. Standardize HTTP status codes for asset handlers
2. Implement proper error status codes for permission and version errors
3. Create consistent error-to-status code mapping
4. Add comprehensive tests for status code verification
## Non-Goals
1. Changing error message format
1. Changing handler error messages
2. Adding new error types
3. Modifying success response format
4. Changing API contracts
## Technical Design
### Overview
The fix involves creating a standardized error-to-status code mapping and updating all handlers to use this mapping consistently.
### Error Status Code Mapping
```rust
// libs/handlers/src/error.rs
#[derive(Debug)]
pub enum HandlerError {
NotFound(String),
Unauthorized(String),
Forbidden(String),
BadRequest(String),
Conflict(String),
Internal(String),
}
impl HandlerError {
pub fn status_code(&self) -> StatusCode {
match self {
HandlerError::NotFound(_) => StatusCode::NOT_FOUND,
HandlerError::Unauthorized(_) => StatusCode::UNAUTHORIZED,
HandlerError::Forbidden(_) => StatusCode::FORBIDDEN,
HandlerError::BadRequest(_) => StatusCode::BAD_REQUEST,
HandlerError::Conflict(_) => StatusCode::CONFLICT,
HandlerError::Internal(_) => StatusCode::INTERNAL_SERVER_ERROR,
}
}
}
impl From<HandlerError> for Response {
fn from(error: HandlerError) -> Self {
let status = error.status_code();
let body = json!({
"error": {
"message": error.to_string(),
"code": status.as_u16()
}
});
Response::builder()
.status(status)
.header("Content-Type", "application/json")
.body(body.to_string())
.unwrap_or_else(|_| {
Response::builder()
.status(StatusCode::INTERNAL_SERVER_ERROR)
.body("Internal server error".to_string())
.unwrap()
})
}
}
```
### Handler Updates
Example handler update:
```rust
// libs/handlers/src/assets/get_asset.rs
pub async fn get_asset_handler(
asset_id: &Uuid,
user: &AuthenticatedUser,
) -> Result<Response, HandlerError> {
let asset = match Asset::find_by_id(asset_id).await {
Ok(asset) => asset,
Err(_) => return Err(HandlerError::NotFound(
format!("Asset {} not found", asset_id)
)),
};
if !user.can_view(&asset) {
return Err(HandlerError::Forbidden(
"User does not have permission to view this asset".to_string()
));
}
Ok(Response::builder()
.status(StatusCode::OK)
.body(json!(asset).to_string())
.unwrap())
}
```
### Test Cases
```rust
// libs/handlers/tests/error_status_test.rs
#[tokio::test]
async fn test_not_found_status() -> Result<()> {
// Create test setup with admin user
let setup = TestSetup::new(Some(UserOrganizationRole::Admin)).await?;
let fake_id = Uuid::new_v4();
let response = get_asset_handler(
&fake_id,
&setup.user
).await;
assert!(response.is_err());
let err = response.unwrap_err();
assert_eq!(err.status_code(), StatusCode::NOT_FOUND);
Ok(())
}
#[tokio::test]
async fn test_forbidden_status() -> Result<()> {
// Create test setup with viewer role
let setup = TestSetup::new(Some(UserOrganizationRole::Viewer)).await?;
// Create asset without permissions
let asset_id = AssetTestHelpers::create_test_asset(
&setup.db,
"Test Asset",
setup.organization.id
).await?;
let response = get_asset_handler(
&asset_id,
&setup.user
).await;
assert!(response.is_err());
let err = response.unwrap_err();
assert_eq!(err.status_code(), StatusCode::FORBIDDEN);
// Verify error is logged
let mut conn = setup.db.diesel_conn().await?;
let error_log = error_logs::table
.filter(error_logs::asset_id.eq(asset_id))
.filter(error_logs::user_id.eq(setup.user.id))
.first::<ErrorLog>(&mut conn)
.await?;
assert_eq!(error_log.status_code, StatusCode::FORBIDDEN.as_u16() as i32);
Ok(())
}
#[tokio::test]
async fn test_error_response_format() -> Result<()> {
// Create test setup with viewer role
let setup = TestSetup::new(Some(UserOrganizationRole::Viewer)).await?;
// Create test asset
let asset_id = AssetTestHelpers::create_test_asset(
&setup.db,
"Test Asset",
setup.organization.id
).await?;
let response = get_asset_handler(
&asset_id,
&setup.user
).await;
assert!(response.is_err());
let err = response.unwrap_err();
// Convert error to response
let error_response: Response = err.into();
// Verify response format
let body = hyper::body::to_bytes(error_response.into_body()).await?;
let error_json: serde_json::Value = serde_json::from_slice(&body)?;
assert!(error_json.get("error").is_some());
assert!(error_json["error"].get("message").is_some());
assert!(error_json["error"].get("code").is_some());
Ok(())
}
```
### Dependencies
1. Test infrastructure from [Test Infrastructure Setup](api_test_infrastructure.md)
2. Existing handler implementations
3. HTTP status code definitions
4. Error type definitions
4. Changing handler logic
## Implementation Plan
### Phase 1: Error Type Updates
### Phase 1: Error Mapping ⏳ (In Progress)
1. Create/update error types
2. Implement status code mapping
3. Add error response formatting
4. Update documentation
#### Technical Design
### Phase 2: Handler Updates
```rust
// Error mapping structure
pub struct ErrorMapping {
pub pattern: &'static str,
pub status: StatusCode,
pub message: &'static str,
}
1. Update handlers to use new error types
2. Add proper status code returns
3. Implement error handling
4. Add tests
// Error mappings
const ERROR_MAPPINGS: &[ErrorMapping] = &[
ErrorMapping {
pattern: "don't have permission",
status: StatusCode::FORBIDDEN,
message: "Permission denied",
},
ErrorMapping {
pattern: "not found",
status: StatusCode::NOT_FOUND,
message: "Resource not found",
},
// ... more mappings
];
```
### Phase 3: Testing
#### Implementation Steps
1. [ ] Create error mapping structure
- Define error patterns
- Map to status codes
- Standardize messages
- Testing requirements:
- Pattern matching
- Message formatting
- Edge cases
1. Add status code tests
2. Test error scenarios
3. Verify response formats
4. Test edge cases
2. [ ] Update metric route handler
- Add error mapping
- Update response format
- Testing requirements:
- All error types
- Status code verification
- Message validation
## Testing Strategy
3. [ ] Update dashboard route handler
- Add error mapping
- Update response format
- Testing requirements:
- All error types
- Status code verification
- Message validation
### Unit Tests
#### Tests
- Test error type mapping
- Test status code assignment
- Test error response format
- Test handler error cases
##### Unit Tests
```rust
#[cfg(test)]
mod tests {
use super::*;
### Integration Tests
#[test]
fn test_permission_denied_mapping() {
let error = anyhow!("User doesn't have permission to access this resource");
let result = map_error(error);
assert_eq!(result.status(), StatusCode::FORBIDDEN);
assert_eq!(result.message(), "Permission denied");
}
- Test complete request flow
- Verify status codes
- Test error scenarios
- Test response format
#[test]
fn test_not_found_mapping() {
let error = anyhow!("Metric not found");
let result = map_error(error);
assert_eq!(result.status(), StatusCode::NOT_FOUND);
assert_eq!(result.message(), "Resource not found");
}
## Success Criteria
#[test]
fn test_version_not_found_mapping() {
let error = anyhow!("Version 123 not found");
let result = map_error(error);
assert_eq!(result.status(), StatusCode::NOT_FOUND);
assert_eq!(result.message(), "Version not found");
}
1. All handlers return correct status codes
2. Error responses are properly formatted
3. Tests pass for all scenarios
4. Documentation is updated
#[test]
fn test_unknown_error_mapping() {
let error = anyhow!("Some unexpected error");
let result = map_error(error);
assert_eq!(result.status(), StatusCode::INTERNAL_SERVER_ERROR);
}
}
```
## Rollout Plan
##### Integration Tests
- Test Scenario: Permission Denied
- Setup:
- Create test metric
- Create user without permissions
- Steps:
1. Attempt to access metric
2. Verify response
- Assertions:
- Status code is 403
- Message is "Permission denied"
- Edge Cases:
- Inherited permissions
- Public resources
- Invalid user
1. Implement error type changes
2. Update handlers incrementally
3. Deploy to staging
4. Monitor for issues
5. Deploy to production
- Test Scenario: Resource Not Found
- Setup:
- Create test user
- Generate invalid UUID
- Steps:
1. Attempt to access non-existent resource
2. Verify response
- Assertions:
- Status code is 404
- Message is "Resource not found"
- Edge Cases:
- Deleted resources
- Case sensitivity
- Special characters
## Appendix
#### Success Criteria
- [ ] All error mappings implemented
- [ ] Unit tests passing
- [ ] Integration tests passing
- [ ] Consistent behavior across asset types
### Related Files
### Phase 2: Handler Updates 🔜 (Not Started)
- `libs/handlers/src/error.rs`
- `libs/handlers/src/assets/*.rs`
- `libs/handlers/tests/error_status_test.rs`
- `libs/handlers/tests/assets/*.rs`
[Similar structure to Phase 1]
### HTTP Status Code Reference
## Dependencies on Other Components
Common status codes used:
- 200: OK
- 201: Created
- 400: Bad Request
- 401: Unauthorized
- 403: Forbidden
- 404: Not Found
- 409: Conflict
- 500: Internal Server Error
1. Test Infrastructure
- Interface: TestDb for database access
- Testing: Permission setup utilities
2. Asset Handlers
- Interface: Error types and messages
- Testing: Error generation scenarios
## Security Considerations
- Consideration 1: Error Information Exposure
- Risk: Detailed errors could expose system info
- Mitigation: Standardized error messages
- Testing: Message content validation
- Consideration 2: Permission Checks
- Risk: Incorrect status codes bypass frontend checks
- Mitigation: Comprehensive error mapping
- Testing: All permission scenarios
## References
- [HTTP Status Code Standards](link_to_standards)
- [Error Handling Guidelines](link_to_guidelines)
- [Testing Best Practices](link_to_practices)

335
api/prds/active/project_bug_fixes_and_testing.md
View File

@ -9,14 +9,33 @@ status: Draft
## Problem Statement
Several bugs have been identified in our asset handling and permission management systems that need to be addressed:
<!--
Clearly articulate the problem you're solving. Include:
- Current state and behavior
- Expected behavior
- Pain points
- Impact on users/system
- Why it needs to be solved now
-->
1. Metric status updates are not being properly propagated to the metric_file object (BUS-1069)
2. Asset access control is not returning appropriate HTTP status codes (BUS-1067)
3. Public sharing parameters are not being properly updated (BUS-1064)
Current behavior:
1. Metric status updates not propagating to metric_file object (BUS-1069)
2. Asset access control returning incorrect HTTP status codes (BUS-1067)
3. Public sharing parameters not updating properly (BUS-1064)
4. Permission field inconsistencies across asset types (BUS-1063)
5. Lack of standardized testing utilities and patterns
These issues affect core functionality around asset access, sharing, and status management. Additionally, our testing infrastructure needs improvement to prevent similar issues in the future.
Expected behavior:
1. Metric status updates correctly reflected in all objects
2. Consistent and correct HTTP status codes for all asset operations
3. Public sharing parameters properly validated and updated
4. Consistent permission field format across all asset types
5. Robust testing infrastructure with standardized patterns
Impact:
- User Impact: Inconsistent error handling and permissions cause confusion
- System Impact: Bugs affecting core asset functionality
- Testing Impact: Duplicate test code and missing edge cases
## Goals
@ -34,192 +53,184 @@ These issues affect core functionality around asset access, sharing, and status
3. Modifying the underlying database schema
4. Changing the existing API contracts
## Technical Design
## Implementation Plan
### Overview
### Phase 1: Test Infrastructure ⏳ (In Progress)
The implementation will focus on fixing bugs while establishing better testing infrastructure. The changes will be made in a way that maintains backward compatibility and doesn't require database migrations.
#### Technical Design
```mermaid
graph TD
A[Test Infrastructure] --> B[Metric Status Fix]
A --> C[Access Control Fix]
A --> D[Sharing Parameters Fix]
A --> E[Permission Field Fix]
B --> F[Integration Tests]
C --> F
D --> F
E --> F
A[Test Infrastructure] --> B[Database Utils]
A --> C[Permission Utils]
A --> D[Asset Utils]
B --> E[Integration Tests]
C --> E
D --> E
```
### Component Breakdown
#### Component 1: Test Infrastructure
- Purpose: Establish common test utilities for database and permission testing
- Sub-PRD: [Test Infrastructure Setup](api_test_infrastructure.md)
- Interfaces:
- Input: Test configuration
- Output: Test utilities and helpers
#### Component 2: Metric Status Fix
- Purpose: Fix metric status update propagation
- Sub-PRD: [Metric Status Update Fix](api_metric_status_fix.md)
- Interfaces:
- Input: Metric update request
- Output: Updated metric with correct status
#### Component 3: Access Control Fix
- Purpose: Standardize HTTP status codes
- Sub-PRD: [Asset Access Control Fix](api_access_control_fix.md)
- Interfaces:
- Input: Asset access request
- Output: Appropriate HTTP status code
#### Component 4: Sharing Parameters Fix
- Purpose: Fix public sharing parameter updates
- Sub-PRD: [Sharing Parameters Fix](api_sharing_parameters_fix.md)
- Interfaces:
- Input: Sharing update request
- Output: Updated sharing settings
#### Component 5: Permission Field Fix
- Purpose: Standardize permission field handling
- Sub-PRD: [Permission Field Fix](api_permission_field_fix.md)
- Interfaces:
- Input: Asset access request
- Output: Consistent permission field
### Dependencies
1. Database access for testing
2. Existing permission models and enums
3. HTTP status code definitions
4. Asset type definitions
## Implementation Plan
The implementation will be broken down into the following sub-PRDs, with their dependencies and concurrent development opportunities clearly defined:
1. [Test Infrastructure Setup](api_test_infrastructure.md) - **Must be completed first**
- Establishes testing utilities needed by all other components
#### Sub-PRDs
1. [Test Infrastructure Setup](api_test_infrastructure.md)
- Purpose: Establish common test utilities
- Dependencies: None
- Required for: All other PRDs
- Status: 🆕 Not Started
- Testing Requirements:
- Unit Tests:
- Database connection management
- Test data creation/cleanup
- Permission setup helpers
- Integration Tests:
- Full test workflow
- Database state verification
- Permission validation
After Test Infrastructure is complete, the following tickets can be worked on concurrently:
#### Implementation Steps
1. [ ] Database test utilities
- Connection pool management
- Test data isolation
- Automatic cleanup
2. [HTTP Status Code Fix](api_http_status_fix.md) - **Can be developed concurrently with 3, 4, and 5**
- BUS-1067: Asset access control HTTP status codes
2. [ ] Permission test utilities
- User/role setup
- Permission verification
- Access control testing
3. [ ] Asset test utilities
- Test asset creation
- Version management
- Status verification
#### Tests
##### Unit Tests
```rust
#[cfg(test)]
mod tests {
#[test]
fn test_database_connection() {
// Test connection pool
}
#[test]
fn test_permission_setup() {
// Test permission helpers
}
#[test]
fn test_asset_creation() {
// Test asset utilities
}
}
```
##### Integration Tests
- Test Scenario: Full Asset Lifecycle
- Setup:
- Test database
- Test users with roles
- Test organization
- Steps:
1. Create asset
2. Update permissions
3. Verify access
- Assertions:
- Database state
- Permission checks
- Edge Cases:
- Connection failures
- Concurrent access
- Invalid permissions
#### Success Criteria
- [ ] All test utilities implemented
- [ ] Documentation complete
- [ ] Integration tests passing
- [ ] Example usage provided
### Phase 2: Bug Fixes (Concurrent) 🔜 (Not Started)
#### Sub-PRDs
1. [HTTP Status Code Fix](api_http_status_fix.md)
- Dependencies: Test Infrastructure
- Can be worked on concurrently with: Metric Status Fix, Sharing Parameters Fix, Permission Field Fix
- Reason for concurrency: Modifies error handling layer which is independent of other changes
- Status: 🆕 Not Started
- Can be developed concurrently with: 2, 3, 4
- Testing Requirements:
- Unit Tests:
- Error mapping
- Status code validation
- Integration Tests:
- Error scenarios
- Permission checks
3. [Metric Status Update Fix](api_metric_status_fix.md) - **Can be developed concurrently with 2, 4, and 5**
- BUS-1069: Metric status propagation
2. [Metric Status Update Fix](api_metric_status_fix.md)
- Dependencies: Test Infrastructure
- Can be worked on concurrently with: HTTP Status Fix, Sharing Parameters Fix, Permission Field Fix
- Reason for concurrency: Modifies metric-specific logic that doesn't affect other components
- Status: 🆕 Not Started
- Can be developed concurrently with: 1, 3, 4
- Testing Requirements:
- Unit Tests:
- Status propagation
- Update validation
- Integration Tests:
- Full update flow
- State verification
4. [Sharing Parameters Fix](api_sharing_parameters_fix.md) - **Can be developed concurrently with 2, 3, and 5**
- BUS-1064: Public sharing parameters
3. [Sharing Parameters Fix](api_sharing_parameters_fix.md)
- Dependencies: Test Infrastructure
- Can be worked on concurrently with: HTTP Status Fix, Metric Status Fix, Permission Field Fix
- Reason for concurrency: Focuses on sharing validation logic that is separate from other changes
- Status: 🆕 Not Started
- Can be developed concurrently with: 1, 2, 4
- Testing Requirements:
- Unit Tests:
- Parameter validation
- Update logic
- Integration Tests:
- Sharing scenarios
- Permission checks
5. [Permission Field Fix](api_permission_field_fix.md) - **Can be developed concurrently with 2, 3, and 4**
- BUS-1063: Permission field consistency
4. [Permission Field Fix](api_permission_field_fix.md)
- Dependencies: Test Infrastructure
- Can be worked on concurrently with: HTTP Status Fix, Metric Status Fix, Sharing Parameters Fix
- Reason for concurrency: Changes permission response structure without affecting core permission logic
- Status: 🆕 Not Started
- Can be developed concurrently with: 1, 2, 3
- Testing Requirements:
- Unit Tests:
- Field format
- Inheritance logic
- Integration Tests:
- Cross-asset consistency
- Permission validation
### Concurrent Development Strategy
To enable efficient concurrent development without conflicts:
#### Concurrent Development Strategy
1. **Independent Code Paths**
- HTTP Status Fix: Modifies error handling layer
- Metric Status Fix: Updates metric-specific update logic
- Sharing Parameters Fix: Changes sharing validation
- Permission Field Fix: Standardizes permission response format
- Each fix modifies separate layers
- Clear interface boundaries
- Isolated test data
2. **Isolated Test Data**
- Each test will use unique identifiers with test_id prefix
- Test database connections managed by TestDb
- Automatic cleanup after each test
3. **Clear Integration Points**
- HTTP Status Fix: Other fixes will adopt new error types as they're completed
- Permission Field Fix: Other components will use new permission field format
- No circular dependencies between fixes
4. **Merge Strategy**
- Test Infrastructure must be merged first
- Other fixes can be merged in any order
- Each fix includes its own tests and documentation
### Development Phases
Phase 1: Foundation (Sequential)
- Complete Test Infrastructure
- Review and merge
Phase 2: Bug Fixes (Concurrent)
- Assign teams to each fix
- Develop and test independently
- Regular sync meetings to discuss integration
Phase 3: Integration (Sequential)
- Merge completed fixes
- Run full test suite
- Verify all features working together
## Testing Strategy
### Unit Tests
- Each fix must include comprehensive unit tests
- Test both success and failure cases
- Mock external dependencies where appropriate
- Use test utilities for common setup
### Integration Tests
- Test complete workflows
- Verify HTTP status codes
- Test permission combinations
- Verify database state after operations
2. **Integration Testing**
- Component interaction tests
- System-wide integration
- Performance validation
## Security Considerations
- Maintain existing permission checks
- No exposure of sensitive data in error messages
- Proper validation of public access parameters
- Audit logging of permission changes
- Consideration 1: Permission Validation
- Risk: Incorrect access control
- Mitigation: Comprehensive permission tests
- Testing: Edge case validation
## Monitoring and Logging
- Consideration 2: Error Messages
- Risk: Information exposure
- Mitigation: Standardized error responses
- Testing: Message content validation
- Log permission changes
- Track HTTP status code distribution
- Monitor test execution times
- Alert on test failures
## Dependencies
## Rollout Plan
1. Database Layer
- Interface: Connection pool
- Testing: Transaction isolation
1. Deploy test infrastructure changes
2. Roll out fixes one at a time
3. Monitor for any issues
4. Update documentation
2. Permission System
- Interface: Access control
- Testing: Role validation
## Appendix
## References
### Related PRDs
- [Test Infrastructure Setup](api_test_infrastructure.md)
- [Metric Status Update Fix](api_metric_status_fix.md)
- [Asset Access Control Fix](api_access_control_fix.md)
- [Sharing Parameters Fix](api_sharing_parameters_fix.md)
- [Permission Field Fix](api_permission_field_fix.md)
- [Test Infrastructure Design](link_to_design_doc)
- [Permission System Documentation](link_to_docs)
- [HTTP Status Code Standards](link_to_standards)

301
api/prds/project_template.md
View File

@ -3,6 +3,7 @@ title: Project Name
author: Your Name
date: YYYY-MM-DD
status: Draft
ticket: TICKET-ID
---
# Project Name
@ -11,38 +12,52 @@ status: Draft
<!--
Clearly articulate the high-level problem this project solves. Include:
- Current state
- Current state and behavior
- Expected behavior
- Pain points
- Impact on users/system
- Why it needs to be solved now
- Any relevant metrics or data
-->
Current behavior:
- [Current behavior 1]
- [Current behavior 2]
- [Current behavior 3]
Expected behavior:
- [Expected behavior 1]
- [Expected behavior 2]
- [Expected behavior 3]
## Goals
<!-- List the high-level goals of this project -->
<!-- List specific, measurable goals this project will achieve -->
1.
2.
3.
## Non-Goals
<!-- List what is explicitly out of scope for this project -->
<!-- List what is explicitly out of scope -->
1.
2.
3.
## Technical Design
### Overview
## Implementation Plan
<!--
Provide a high-level overview of the technical approach:
- Main components
- How they interact
- Key technologies used
Break down the implementation into sub-PRDs and phases. Each phase should:
- Have clear dependencies and concurrent development opportunities
- Include its own technical design
- Have comprehensive tests
- Have clear success criteria
-->
### Phase 1: Foundation ⏳ (In Progress)
#### Technical Design
```mermaid
graph TD
A[Component A] --> B[Component B]
@ -50,181 +65,125 @@ graph TD
// Add your architecture diagram
```
### Component Breakdown
<!--
Break down the project into logical components. For each component:
- Describe its purpose
- Reference the sub-PRD that will implement it
- Define interfaces with other components
-->
#### Component 1: [Name]
- Purpose:
- Sub-PRD: [Link to sub-PRD]
- Interfaces:
- Input:
- Output:
#### Component 2: [Name]
- Purpose:
- Sub-PRD: [Link to sub-PRD]
- Interfaces:
- Input:
- Output:
### Dependencies
<!-- List all dependencies for this project -->
1.
2.
3.
## Implementation Plan
<!--
Break down the implementation into sub-PRDs and clearly define:
1. The order in which sub-PRDs should be implemented
2. Which sub-PRDs can be developed concurrently
3. Dependencies between sub-PRDs
4. How to avoid conflicts between concurrent development efforts
Track the status of each sub-PRD:
- Complete
- In Progress
- Upcoming
-->
### Sub-PRD Implementation Order and Dependencies
The implementation will be broken down into the following sub-PRDs, with their dependencies and development order clearly defined:
1. [Foundation Component](link_to_foundation_prd.md) - **Must be completed first**
- This PRD establishes the core functionality needed by all other components
#### Sub-PRDs
1. [Foundation Component](link_to_foundation_prd.md)
- Purpose: [Describe purpose]
- Dependencies: None
- Required for: All other PRDs
- Status: 🆕 Not Started
- Testing Requirements:
- Unit Tests:
- Test case 1
- Test case 2
- Integration Tests:
- Scenario 1
- Scenario 2
2. [Component A](link_to_component_a_prd.md) - **Can be developed concurrently with Components B and C**
#### Implementation Steps
1. [ ] Step 1
- Technical details
- Edge cases to handle
- Testing requirements
2. [ ] Step 2
- Technical details
- Edge cases to handle
- Testing requirements
#### Tests
##### Unit Tests
```rust
#[cfg(test)]
mod tests {
#[test]
fn test_normal_case() {
// Test implementation
}
#[test]
fn test_edge_case_1() {
// Test implementation
}
}
```
##### Integration Tests
- Test Scenario: [Description]
- Setup:
- Required data
- System state
- Steps:
1. Action 1
2. Action 2
- Assertions:
- Expected state
- Error cases
- Edge Cases:
- Case 1
- Case 2
#### Success Criteria
- [ ] Foundation component implemented
- [ ] All tests passing
- [ ] Documentation complete
### Phase 2: Parallel Component Development 🔜 (Not Started)
#### Sub-PRDs
1. [Component A](link_to_component_a_prd.md)
- Dependencies: Foundation Component
- Required for: Component D
- Potential conflict areas with Component B: [Describe and provide mitigation]
- Can be developed concurrently with: Components B and C
- Testing Requirements:
- Unit Tests:
- Test case 1
- Test case 2
- Integration Tests:
- Scenario 1
- Scenario 2
3. [Component B](link_to_component_b_prd.md) - **Can be developed concurrently with Components A and C**
2. [Component B](link_to_component_b_prd.md)
- Dependencies: Foundation Component
- Required for: Component E
- Potential conflict areas with Component A: [Describe and provide mitigation]
- Can be developed concurrently with: Components A and C
- Testing Requirements:
- Unit Tests:
- Test case 1
- Test case 2
- Integration Tests:
- Scenario 1
- Scenario 2
4. [Component C](link_to_component_c_prd.md) - **Can be developed concurrently with Components A and B**
- Dependencies: Foundation Component
- Required for: None
- No conflicts with other concurrent components
[Similar structure for other components]
5. [Component D](link_to_component_d_prd.md) - **Must wait for Component A**
- Dependencies: Foundation Component, Component A
- Required for: None
#### Concurrent Development Strategy
6. [Component E](link_to_component_e_prd.md) - **Must wait for Component B**
- Dependencies: Foundation Component, Component B
- Required for: None
1. **Clear Component Boundaries**
- Interface definitions
- Data isolation
- Test data separation
### Concurrent Development Strategy
To enable efficient concurrent development without conflicts:
1. **Clear Component Interfaces**: Each sub-PRD must define clear interfaces for how other components interact with it
2. **Separate Database Concerns**: Ensure database schema changes are coordinated to prevent conflicts
3. **Modular Code Structure**: Organize code to minimize overlap between components
4. **Regular Integration**: Plan for regular integration points to catch conflicts early
5. **Feature Flags**: Use feature flags to allow merging incomplete features without affecting production
### Phase 1: Foundation
<!-- Describe the first phase of implementation -->
**Components:**
- Foundation Component
**Success Criteria:**
- Core interfaces defined and implemented
- Database schema changes completed
- Unit tests passing at 90%+ coverage
- Integration tests defined
### Phase 2: Parallel Component Development
<!-- Describe the second phase of implementation -->
**Components:**
- Component A
- Component B
- Component C
**Success Criteria:**
- All components implemented according to their respective PRDs
- Components successfully integrated with Foundation Component
- No regressions in Foundation Component functionality
- Unit and integration tests passing
### Phase 3: Dependent Components
<!-- Describe the third phase of implementation -->
**Components:**
- Component D
- Component E
**Success Criteria:**
- All components implemented according to their respective PRDs
- Full system integration completed
- End-to-end tests passing
- Performance benchmarks met
## Testing Strategy
<!--
Provide a high-level testing strategy for the entire project.
Each sub-PRD will have more detailed testing plans.
-->
### Unit Tests
-
-
-
### Integration Tests
-
-
-
2. **Integration Testing**
- Component interaction tests
- System-wide integration tests
- Performance tests
## Security Considerations
<!-- List security considerations for the entire project -->
-
-
-
<!-- List security implications and how they're addressed -->
- Consideration 1
- Risk:
- Mitigation:
- Testing:
## Monitoring and Logging
## Dependencies
<!-- Describe monitoring and logging requirements -->
-
-
-
<!-- List external dependencies and affected components -->
1. Component 1
- Interface changes:
- Testing requirements:
## Rollout Plan
## References
<!-- Describe the plan for rolling out this project -->
1.
2.
3.
4.
## Appendix
### Related PRDs
<!-- List all sub-PRDs that are part of this project -->
- [Component 1 Name](link_to_sub_prd1.md)
- [Component 2 Name](link_to_sub_prd2.md)
- [Component 3 Name](link_to_sub_prd3.md)
<!-- Include links to relevant documentation -->
- [Link to design docs]
- [Link to related PRDs]

205
api/prds/sub_prd_template.md
View File

@ -4,6 +4,7 @@ author: Your Name
date: YYYY-MM-DD
status: Draft
parent_prd: project_name.md
ticket: TICKET-ID
---
# Component Name
@ -16,162 +17,136 @@ This is a sub-PRD of the [Project Name](project_name.md) project. Please refer t
<!--
Clearly articulate the specific problem this component solves within the larger project. Include:
- Current state and behavior
- Expected behavior
- How this fits into the larger project
- Specific pain points this component addresses
- Why this component is necessary
-->
Current behavior:
- [Current behavior 1]
- [Current behavior 2]
- [Current behavior 3]
Expected behavior:
- [Expected behavior 1]
- [Expected behavior 2]
- [Expected behavior 3]
## Goals
<!-- List the specific goals of this component -->
<!-- List specific, measurable goals this component will achieve -->
1.
2.
3.
## Non-Goals
<!-- List what is explicitly out of scope for this component -->
<!-- List what is explicitly out of scope -->
1.
2.
3.
## Technical Design
### Component Overview
## Implementation Plan
<!--
Provide a detailed overview of this component:
- How it fits into the larger system
- Interfaces with other components
- Key technologies used
Break down the implementation into phases. Each phase should:
- Be independently deployable
- Include its own technical design
- Have clear success criteria
- Include comprehensive tests
-->
```mermaid
graph TD
A[This Component] --> B[Other Component]
C[Another Component] --> A
// Add your component diagram
```
### Phase 1: [Name] ⏳ (In Progress)
### Interfaces
<!--
Define all interfaces this component exposes to or consumes from other components:
- API endpoints
- Function signatures
- Data structures
- Events/messages
-->
#### Exposed Interfaces
#### Technical Design
<!-- Include technical details specific to this phase -->
```rust
// Include actual code structures/types for interfaces this component exposes
struct ExposedInterface {
// Include fields and their purposes
// Include actual code structures/types
struct ComponentName {
field1: Type1, // Purpose of field1
field2: Type2, // Purpose of field2
}
```
#### Consumed Interfaces
#### Implementation Steps
1. [ ] Step 1
- Technical details
- Edge cases to handle
- Testing requirements
2. [ ] Step 2
- Technical details
- Edge cases to handle
- Testing requirements
#### Tests
##### Unit Tests
```rust
// Include actual code structures/types for interfaces this component consumes
struct ConsumedInterface {
// Include fields and their purposes
field1: Type1, // Purpose of field1
field2: Type2, // Purpose of field2
#[cfg(test)]
mod tests {
#[test]
fn test_normal_case() {
// Test implementation
}
#[test]
fn test_edge_case_1() {
// Test implementation
}
#[test]
fn test_error_case_1() {
// Test implementation
}
}
```
### Implementation Details
##### Integration Tests
- Test Scenario: [Description]
- Setup:
- Required data
- System state
- Steps:
1. Action 1
2. Action 2
- Assertions:
- Expected state
- Error cases
- Edge Cases:
- Case 1
- Case 2
<!--
Provide detailed implementation information:
- Core algorithms
- Data flow
- Error handling
- Edge cases
-->
#### Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] All tests passing
#### Core Logic
### Phase 2: [Name] 🔜 (Not Started)
```rust
// Include actual code for core logic
fn core_function() -> Result<Type, Error> {
// Implementation details
}
```
### File Changes
<!--
List all files that will be:
- Created
- Modified
- Deleted
Include the purpose of each change
-->
#### New Files
- `src/new_module/new_file.rs`
- Purpose: [Describe purpose]
- Key components: [List key components]
- Dependencies: [List dependencies]
#### Modified Files
- `src/existing_module/existing_file.rs`
- Changes: [Describe changes]
- Purpose: [Describe purpose of changes]
## Testing Strategy
<!-- Provide a detailed testing strategy for this component -->
### Unit Tests
- Test case 1: [Description]
- Input: [Input description]
- Expected output: [Expected output description]
- Edge cases: [List edge cases]
### Integration Tests
- Test scenario 1: [Description]
- Components involved: [List components]
- Test steps: [List steps]
- Expected outcome: [Describe expected outcome]
## Security Considerations
<!-- List security considerations specific to this component -->
-
-
-
[Similar structure to Phase 1]
## Dependencies on Other Components
<!--
List dependencies on other components in the project:
- Which components must be completed before this one
- Which components can be developed concurrently
- Potential conflict areas with concurrent development
-->
<!-- List dependencies on other components in the project -->
1. Component 1
- Interface changes:
- Testing requirements:
- Integration points:
### Required Components
- [Component Name](component_prd.md): [Describe dependency]
## Security Considerations
### Concurrent Development
- [Component Name](component_prd.md): [Describe how concurrent development will work]
- Potential conflicts: [Describe potential conflicts]
- Mitigation strategy: [Describe mitigation strategy]
<!-- List security implications and how they're addressed -->
- Consideration 1
- Risk:
- Mitigation:
- Testing:
## Implementation Timeline
## References
<!-- Provide a timeline for implementing this component -->
- Task 1: [Estimated time]
- Task 2: [Estimated time]
- Task 3: [Estimated time]
Total estimated time: [Total time]
<!-- Include links to relevant documentation -->
- [Link to design docs]
- [Link to related PRDs]

391
api/prds/template.md
View File

@ -1,319 +1,152 @@
# PRD Template
<!--
This template is designed to help you create comprehensive Product Requirements Documents (PRDs).
Key points to remember:
- Be specific and detailed in each section
- Include technical details where relevant
- Consider all stakeholders
- Think about edge cases and potential issues
- Document assumptions and dependencies
-->
---
title: Component Name
author: Your Name
date: YYYY-MM-DD
status: Draft
ticket: TICKET-ID
---
# Component Name
## Problem Statement
## Problem Statement ✅
<!--
Clearly articulate the problem you're solving. Include:
- Current state
- Current state and behavior
- Expected behavior
- Pain points
- Impact on users/system
- Why it needs to be solved now
- Any relevant metrics or data
-->
The current Buster CLI's `deploy` command and the API's deployment endpoint have diverged in their handling of dataset creation and validation. The CLI supports advanced features like:
1. Global configuration via buster.yml
2. Model-level overrides
3. Searchable dimensions (stored values)
4. Cross-project entity relationships
Current behavior:
- [Current behavior 1]
- [Current behavior 2]
- [Current behavior 3]
This misalignment can cause deployment inconsistencies and confusion for users.
Expected behavior:
- [Expected behavior 1]
- [Expected behavior 2]
- [Expected behavior 3]
Key issues:
- Configuration handling differs between CLI and API
- Stored values flag naming inconsistency (searchable vs stored_values)
- Entity relationships handled differently
- Success/failure reporting differences
- Validation scope misalignment
## Goals
### Current Limitations
<!-- List specific limitations or issues in the current system -->
- [Limitation 1]
- [Limitation 2]
- ...
<!-- List specific, measurable goals this change will achieve -->
1.
2.
3.
### Impact
<!-- Describe the impact of not solving this problem -->
- User Impact: [Describe]
- System Impact: [Describe]
- Business Impact: [Describe]
## Non-Goals
## Requirements
### Functional Requirements ✅
<!--
List all functional requirements. Each should be:
- Specific
- Measurable
- Achievable
- Relevant
- Time-bound
-->
#### Core Functionality
- Requirement 1
- Details:
- Acceptance Criteria:
- Dependencies:
#### User Interface
- Requirement 1
- Details:
- Acceptance Criteria:
- Dependencies:
#### Data Management
- Requirement 1
- Details:
- Acceptance Criteria:
- Dependencies:
### Non-Functional Requirements ✅
<!--
Include requirements for:
- Performance
- Security
- Scalability
- Maintainability
- Reliability
-->
- Performance Requirements
- [Specific metrics]
- Security Requirements
- [Security considerations]
- Scalability Requirements
- [Scalability expectations]
## Technical Design ✅
### System Architecture
<!--
Describe the high-level architecture:
- Components
- Interactions
- Data flow
- System boundaries
-->
```mermaid
graph TD
A[Component A] --> B[Component B]
B --> C[Component C]
// Add your architecture diagram
```
### Core Components ✅
<!--
For each component, include:
- Purpose
- Responsibilities
- Interfaces
- Dependencies
-->
#### Component 1: [Name]
```rust
// Include actual code structures/types
struct ComponentName {
// Include fields and their purposes
field1: Type1, // Purpose of field1
field2: Type2, // Purpose of field2
}
// Include relevant methods/functions
impl ComponentName {
// Document method purpose and behavior
fn method1() -> Result<Type, Error> {
// Implementation details
}
}
```
### Database Changes (If applicable)
<!--
Document all database changes:
- New tables
- Modified tables
- Migrations
- Indexes
-->
```sql
-- Include actual SQL for schema changes
CREATE TABLE new_table (
id UUID PRIMARY KEY,
-- Document each column's purpose
column1 TYPE1, -- Purpose of column1
column2 TYPE2 -- Purpose of column2
);
-- Include indexes
CREATE INDEX idx_name ON table_name(column_name);
```
### API Changes (If applicable)
<!--
Document all API changes:
- New endpoints
- Modified endpoints
- Request/Response structures
- Authentication/Authorization changes
-->
```rust
// Document new/modified API structures
struct NewRequest {
field1: Type1, // Purpose and validation requirements
field2: Type2, // Purpose and validation requirements
}
struct NewResponse {
field1: Type1, // Purpose and possible values
field2: Type2, // Purpose and possible values
}
```
### File Changes (If applicable)
<!--
List all files that will be:
- Created
- Modified
- Deleted
Include the purpose of each change
-->
#### New Files
- `src/new_module/new_file.rs`
- Purpose: [Describe purpose]
- Key components: [List key components]
- Dependencies: [List dependencies]
#### Modified Files
- `src/existing_module/existing_file.rs`
- Changes: [Describe changes]
- Impact: [Describe impact]
- Dependencies: [List dependencies]
#### Deleted Files
- `src/old_module/old_file.rs`
- Reason for deletion: [Explain why]
- Impact: [Describe impact]
- Migration plan: [Describe how to handle deletion]
<!-- List what is explicitly out of scope -->
1.
2.
3.
## Implementation Plan
<!--
Break down the implementation into phases:
- Each phase should be independently deployable
- Include clear success criteria
- List dependencies between phases
- Include testing strategy for each phase
Break down the implementation into phases. Each phase should:
- Be independently deployable
- Include its own technical design
- Have clear success criteria
- Include comprehensive tests
-->
### Phase 1: Configuration Alignment ⏳ (In Progress)
### Phase 1: [Name] ⏳ (In Progress)
1. Update request mapping
- [x] Add yml_file field to DeployDatasetsRequest
- [x] Update stored_values mapping from searchable ✅
- [x] Add schema and data_source_name resolution from buster.yml ✅
#### Technical Design
<!-- Include technical details specific to this phase -->
2. Implement config resolution
- [x] Honor model-level overrides ✅
- [x] Support buster.yml fallbacks ✅
- [x] Add validation for required fields ✅
```rust
// Include actual code structures/types
struct ComponentName {
field1: Type1, // Purpose of field1
field2: Type2, // Purpose of field2
}
```
3. Add yml content preservation
- [x] Store complete YML in dataset record ✅
- [x] Add YML validation ✅
- [x] Ensure proper escaping/formatting ✅
#### Database Changes
<!-- If applicable -->
```sql
-- Include actual SQL
CREATE TABLE new_table (
id UUID PRIMARY KEY,
field1 TYPE1 -- Purpose
);
```
### Phase 2: Testing & Documentation 🔜 (Not Started)
#### Implementation Steps
1. [ ] Step 1
- Technical details
- Edge cases to handle
- Testing requirements
1. Add comprehensive tests
- [ ] Unit tests for config resolution
- [ ] Integration tests for deployment flow
- [ ] Error scenario tests
- [ ] Validation tests
2. [ ] Step 2
- Technical details
- Edge cases to handle
- Testing requirements
2. Update documentation
- [ ] API endpoint documentation
- [ ] Request/response format docs
- [ ] Configuration precedence docs
- [ ] Error handling docs
#### Tests
3. Add migration guide
- [ ] Document breaking changes
- [ ] Provide upgrade steps
- [ ] Add examples
(You can use as many phases as you deem relevant.)
### Phase 3:....
### Phase 4:....
### Phase 5:....
## Testing Strategy ✅
### Unit Tests
<!--
Detail unit testing approach:
- Test scenarios
- Edge cases
- Mocking strategy
-->
#### Component 1 Tests
##### Unit Tests
```rust
#[cfg(test)]
mod tests {
// Document test purpose and scenarios
#[test]
fn test_scenario_1() {
fn test_normal_case() {
// Test implementation
}
#[test]
fn test_edge_case_1() {
// Test implementation
}
}
```
### Integration Tests
<!--
Detail integration testing approach:
- Test scenarios
- System interactions
- Data flow validation
-->
##### Integration Tests
- Test Scenario: [Description]
- Setup:
- Required data
- System state
- Steps:
1. Action 1
2. Action 2
- Assertions:
- Expected state
- Error cases
- Edge Cases:
- Case 1
- Case 2
#### Scenario 1: [Description]
- Setup: [Describe test setup]
- Steps: [List test steps]
- Expected Results: [Describe expected outcome]
- Validation Criteria: [List validation points]
#### Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] All tests passing
### Security Considerations
<!-- Detail security implications -->
- Security Requirement 1
- Description: [Describe requirement]
- Implementation: [How it's implemented]
- Validation: [How it's validated]
### Phase 2: [Name] 🔜 (Not Started)
### Performance Considerations
<!-- Detail performance implications -->
- Performance Requirement 1
- Description: [Describe requirement]
- Implementation: [How it's implemented]
- Validation: [How it's validated]
[Similar structure to Phase 1]
### References
<!-- Include relevant documentation -->
- [Link to relevant docs]
- [Link to related PRDs]
- [Link to design docs]
## Security Considerations
<!-- List security implications and how they're addressed -->
- Consideration 1
- Risk:
- Mitigation:
- Testing:
## Dependencies
<!-- List external dependencies and affected components -->
1. Component 1
- Interface changes:
- Testing requirements:
## References
<!-- Include links to relevant documentation -->
- [Link to design docs]
- [Link to related PRDs]