diff --git a/api/CLAUDE.md b/api/CLAUDE.md index 9d5de5ecf..fa4ee2197 100644 --- a/api/CLAUDE.md +++ b/api/CLAUDE.md @@ -1,7 +1,7 @@ # Buster API Repository Navigation Guide > **Last Updated**: April 7, 2025 -> **Version**: 1.0.1 +> **Version**: 1.0.2 ## Architecture Overview @@ -66,15 +66,17 @@ While these files contain best practices for writing tests, REST patterns, etc., - All libraries depend on common workspace dependencies ## Repository Structure -- `src/` - Main server code - - `routes/` - API endpoints (REST, WebSocket) - - `utils/` - Shared utilities - - `types/` - Common type definitions -- `libs/` - Shared libraries - - Each lib has its own Cargo.toml and docs -- `migrations/` - Database migrations -- `tests/` - Integration tests -- `documentation/` - Detailed docs +- `src/` - Main server code (Axum application wiring) + - `routes/` - API endpoints (REST, WebSocket) - Defines routes, applies middleware + - `utils/` - Shared server-specific utilities + - `types/` - Common type definitions for the server layer +- `libs/` - Shared libraries (Core logic) + - `database/` - Database interactions, schema, models, migrations, **test utilities** + - `handlers/` - Request handling logic for specific routes/features + - `agents/`, `sharing/`, `query_engine/`, etc. - Other core logic libraries + - Each lib has its own `Cargo.toml`, `src/`, and `tests/` (for lib-specific integration tests) +- `server/tests/` - Focused integration tests for the `server` crate (routing, middleware), **mocks handlers**. +- `documentation/` - Detailed docs (`*.mdc` files) - `prds/` - Product requirements ## Build Commands @@ -88,29 +90,45 @@ While these files contain best practices for writing tests, REST patterns, etc., ### Run Specific Tests ```bash -# Run tests for a specific library +# Run tests for a specific library (e.g., database integration tests) cargo test -p database -# Run a specific test function +# Run tests for the handlers library +cargo test -p handlers + +# Run focused server integration tests (routing, middleware) +cargo test -p server + +# Run a specific test function (e.g., in handlers) cargo test -p handlers -- test_get_dashboard_handler -# Run tests with filter -cargo test metrics +# Run tests with filter (e.g., all tests containing "metric") +cargo test metric # Run with output visible and single-threaded cargo test -- --test-threads=1 --nocapture ``` -### Test Database Environment -```bash -# Test database setup pattern -let test_db = TestDb::new().await?; +### Test Database Environment (Using `database::test_utils`) +```rust +use database::test_utils::{TestDb, insert_test_metric_file, cleanup_test_data}; -# Clean up test data -test_db.cleanup().await?; - -# Full user+org setup -let setup = TestSetup::new(Some(UserOrganizationRole::Admin)).await?; +#[tokio::test] async fn my_db_test() -> anyhow::Result<()> { + // Assumes pools are initialized beforehand (e.g., via #[ctor]) + let test_db = TestDb::new().await?; + + // Create test data structs + let metric = test_db.create_test_metric_file(&test_db.user_id).await?; + let metric_id = metric.id; + + // Insert data using helpers + insert_test_metric_file(&metric).await?; + + // ... perform test logic ... + + // Clean up specific test data + cleanup_test_data(&[metric_id]).await?; + Ok(())} ``` ## Core Guidelines @@ -121,8 +139,10 @@ let setup = TestSetup::new(Some(UserOrganizationRole::Admin)).await?; - Never log secrets or sensitive data - All dependencies inherit from workspace using `{ workspace = true }` - Use database connection pool from `get_pg_pool().get().await?` -- Write tests with `tokio::test` for async tests -- Use test infrastructure utilities in `libs/database/tests/common/` for database tests +- Write tests with `#[tokio::test]` for async tests. +- Use database test utilities from `libs/database/src/test_utils.rs` for managing test data. +- Use mocking (`mockall`, `mockito`) for unit tests. +- Use `axum_test_helper::TestClient` for server integration tests (`server/tests/`). ## Common Database Pattern ```rust @@ -136,27 +156,18 @@ diesel::update(table) .await? ``` -## Common Concurrency Pattern -```rust -let futures: Vec<_> = items - .into_iter() - .map(|item| process_item(item)) - .collect(); -let results = try_join_all(futures).await?; -``` - ## Troubleshooting Guide ### Common Issues 1. **Test Database Connection Issues** - - **Symptom**: Tests fail with connection pool errors - - **Solution**: Check that test database is running and DATABASE_URL is correct in .env.test + - **Symptom**: Tests fail with connection pool errors or timeouts. + - **Solution**: Ensure test database service is running. Verify `DATABASE_URL` in `.env.test` is correct. Check pool initialization logic (e.g., `#[ctor]` setup). - **Example Error**: `Failed to get diesel connection: connection pool timeout` 2. **Test Cleanup Issues** - - **Symptom**: Tests fail with duplicate records or constraint violations - - **Solution**: Make sure `test_db.cleanup().await?` is called at the end of tests + - **Symptom**: Tests fail with duplicate records, unique constraint violations, or unexpected data from previous runs. + - **Solution**: Ensure `cleanup_test_data(&[asset_ids])` is called at the end of *every* integration test that modifies the database, cleaning up precisely the data it created. - **Example Error**: `duplicate key value violates unique constraint` 3. **Missing Permissions in Handlers** @@ -171,4 +182,4 @@ let results = try_join_all(futures).await?; ### Library-Specific Troubleshooting -Check individual CLAUDE.md files in each library directory for specific troubleshooting guidance. \ No newline at end of file +Check individual CLAUDE.md files or READMEs in each library directory for specific troubleshooting guidance. \ No newline at end of file diff --git a/api/documentation/testing.mdc b/api/documentation/testing.mdc index 88ad27f0d..7803835d8 100644 --- a/api/documentation/testing.mdc +++ b/api/documentation/testing.mdc @@ -13,1880 +13,294 @@ For consistent mocking across the codebase, use these libraries: | Library | Purpose | Use Case | |---------|---------|----------| -| **mockito** | HTTP service mocking | Mocking external API calls | -| **mockall** | Trait/struct mocking | Mocking database and service interfaces | -| **mock-it** | Simple mocking | Quick mocks for simple interfaces | -| **wiremock** | Advanced HTTP mocking | Complex API scenarios with extended matching | +| **mockito** | HTTP service mocking | Mocking external API calls in unit tests | +| **mockall** | Trait/struct mocking | Mocking database or service interfaces in unit tests | +| **axum-test-helper** / similar | In-memory HTTP requests | Testing server routing/middleware without real HTTP | Always prefer dependency injection patterns to enable easy mocking of dependencies. ### Test Utilities and Setup -#### Core Test Types +#### Core Database Test Utilities (`libs/database/src/test_utils.rs`) -The test infrastructure provides several core test utilities for consistent test setup: +The `database` library provides core utilities for setting up database state consistently across tests that need it (both within `libs/` and in `server/tests/`). ```rust -// Example of TestDb and TestSetup implementation +// Standard TestDb helper from libs/database/src/test_utils.rs pub struct TestDb { pub test_id: String, pub organization_id: Uuid, pub user_id: Uuid, - initialized: bool, } impl TestDb { - pub async fn new() -> Result { - // Load environment variables - dotenv().ok(); - - // Initialize test pools using existing pools - let test_id = format!("test-{}", Uuid::new_v4()); - let organization_id = Uuid::new_v4(); - let user_id = Uuid::new_v4(); - - Ok(Self { - test_id, - organization_id, - user_id, - initialized: true, - }) - } - - // Get connections from pre-initialized pools - pub async fn diesel_conn(&self) -> Result { - get_pg_pool() - .get() - .await - .map_err(|e| anyhow!("Failed to get diesel connection: {}", e)) - } - - pub async fn sqlx_conn(&self) -> Result { - get_sqlx_pool() - .acquire() - .await - .map_err(|e| anyhow!("Failed to get sqlx connection: {}", e)) - } - - pub async fn redis_conn(&self) -> Result { - get_redis_pool() - .get() - .await - .map_err(|e| antml!("Failed to get redis connection: {}", e)) - } - - // Create test organization - pub async fn create_organization(&self) -> Result { - let org = Organization { - id: Uuid::new_v4(), - name: "Test Organization".to_string(), - domain: Some("test.org".to_string()), - created_at: Utc::now(), - updated_at: Utc::now(), - deleted_at: None, - }; - - let mut conn = self.diesel_conn().await?; - diesel::insert_into(organizations::table) - .values(&org) - .execute(&mut conn) - .await?; - - Ok(org) - } + /// Creates a new test database environment context. + /// Note: This does NOT initialize pools, assumes they are already initialized. + pub async fn new() -> Result { /* ... */ } - // Create test user - pub async fn create_user(&self) -> Result { - let user = User { - id: Uuid::new_v4(), - email: format!("test-{}@example.com", Uuid::new_v4()), - name: Some("Test User".to_string()), - config: json!({}), - created_at: Utc::now(), - updated_at: Utc::now(), - attributes: json!({}), - avatar_url: None, - }; - - let mut conn = self.diesel_conn().await?; - diesel::insert_into(users::table) - .values(&user) - .execute(&mut conn) - .await?; - - Ok(user) - } + /// Get a database connection from the pre-initialized pool. + pub async fn get_conn(&self) -> Result> { /* ... */ } - // Create user-organization relationship - pub async fn create_user_to_org( + /// Creates a basic User struct (does not insert). + pub async fn create_test_user(&self) -> Result { /* ... */ } + + /// Creates a basic MetricFile struct (does not insert). + pub async fn create_test_metric_file(&self, owner_id: &Uuid) -> Result { /* ... */ } + + /// Creates a basic DashboardFile struct (does not insert). + pub async fn create_test_dashboard_file(&self, owner_id: &Uuid) -> Result { /* ... */ } + + /// Creates a basic AssetPermission struct (does not insert). + pub async fn create_asset_permission( &self, - user_id: Uuid, - org_id: Uuid, - role: UserOrganizationRole, - ) -> Result { - let user_org = UserToOrganization { - user_id, - organization_id: org_id, - role, - sharing_setting: SharingSetting::Private, - edit_sql: true, - upload_csv: true, - export_assets: true, - email_slack_enabled: true, - created_at: Utc::now(), - updated_at: Utc::now(), - deleted_at: None, - created_by: user_id, - updated_by: user_id, - deleted_by: None, - status: UserOrganizationStatus::Active, - }; - - let mut conn = self.diesel_conn().await?; - diesel::insert_into(users_to_organizations::table) - .values(&user_org) - .execute(&mut conn) - .await?; - - Ok(user_org) - } - - // Create authenticated user for testing - pub async fn create_authenticated_user( - &self, - role: Option, - ) -> Result<(AuthenticatedUser, Organization)> { - let org = self.create_organization().await?; - let user = self.create_user().await?; - - let role = role.unwrap_or(UserOrganizationRole::Admin); - self.create_user_to_org(user.id, org.id, role).await?; - - let auth_user = AuthenticatedUser { - id: user.id, - email: user.email, - name: user.name, - organization_id: org.id, - role, - sharing_setting: SharingSetting::Private, - edit_sql: true, - upload_csv: true, - export_assets: true, - email_slack_enabled: true, - }; - - Ok((auth_user, org)) - } + asset_id: &Uuid, + asset_type: AssetType, + identity_id: &Uuid, + role: AssetPermissionRole, + ) -> Result { /* ... */ } } -// Helper struct for test setup -pub struct TestSetup { - pub user: AuthenticatedUser, - pub organization: Organization, - pub db: TestDb, -} +// Helper functions for inserting test data (using pre-initialized pools) -impl TestSetup { - pub async fn new(role: Option) -> Result { - let test_db = TestDb::new().await?; - let (user, org) = test_db.create_authenticated_user(role).await?; - - Ok(Self { - user, - organization: org, - db: test_db, - }) - } -} +/// Insert a test asset permission. +pub async fn insert_test_permission(permission: &AssetPermission) -> Result<()> { /* ... */ } + +/// Insert a test metric file. +pub async fn insert_test_metric_file(metric_file: &MetricFile) -> Result<()> { /* ... */ } + +/// Insert a test dashboard file. +pub async fn insert_test_dashboard_file(dashboard_file: &DashboardFile) -> Result<()> { /* ... */ } + +/// Clean up test data (assets and permissions) by asset IDs. +pub async fn cleanup_test_data(asset_ids: &[Uuid]) -> Result<()> { /* ... */ } +``` + +**Note:** These helpers assume database pools (`pg_pool`, `sqlx_pool`, `redis_pool`) are initialized *before* tests run, typically via a mechanism like `#[ctor]` or a test runner setup. ### Library Code Testing Structure -- **Unit Tests**: Include unit tests inside library source files using `#[cfg(test)]` modules -- **Integration Tests**: Place integration tests in the lib's `tests/` directory -- **Test Utils**: Create a private `test_utils.rs` module in each lib for test utilities specific to that lib -- **Scope Isolation**: Test utilities should only be shared within their scope (e.g., libs/database, libs/agents) -- **Conditional Testing**: Use conditional compilation or runtime checks to make tests non-blocking (e.g., skip tests if a required API key is missing) +- **Unit Tests**: Include unit tests inside library source files using `#[cfg(test)]` modules. **Unit tests must mock all external dependencies (DB, HTTP) and never connect to real services.** +- **Integration Tests**: Place integration tests in the lib's `tests/` directory (e.g., `libs/database/tests/`, `libs/handlers/tests/`). These tests *can* interact with external services (like the test database) using the test utilities. +- **Test Utils**: Library-specific helpers can reside in `src/test_utils.rs` (for internal use) or `tests/common/mod.rs` (for integration tests within that library). -### API (src) Testing Structure -- Maintain API-specific tests in the `/api/tests/` directory -- Organize integration tests to mirror the API's route structure -- Keep API-specific test utilities in the shared test common directory +### Server (`server/`) Testing Structure +- **Location**: API server tests reside in `server/tests/`. +- **Goal**: These tests focus *only* on the server's responsibilities: routing, middleware application (auth, CORS, logging), request parsing, and response formatting at the framework level. **They should NOT test handler business logic.** +- **Strategy**: Use Axum's in-memory testing capabilities. +- **Test Utilities (`server/tests/common/mod.rs`)**: Create server-specific helpers here: + - `build_test_app()`: Constructs the Axum `Router`, wiring up routes and middleware. + - **Handler Mocking**: Replace real handlers (from `libs/handlers`) with mock functions that have the *same signature* but return simple `StatusCode` responses (e.g., `StatusCode::OK`, `StatusCode::CREATED`). + - **Mock Authentication**: Implement a test-only authentication extractor (`MockAuth`) that bypasses real token validation. + - **Mock State**: If necessary, provide a minimal `MockAppState`. +- **Testing Tool**: Use `axum_test_helper::TestClient` (or similar) to send requests directly to the in-memory `Router`. +- **Database Interaction**: If a middleware being tested *requires* specific database state (e.g., checking organization membership), use the standard `database::test_utils` helpers (`TestDb`, `insert_...`, `cleanup_test_data`) to set up that state. ### Testing Configuration -- Use cargo features to make running specific test groups easier -- Consider using cargo-nextest for parallel test execution -- Add test tags to group tests by functionality or component +- Use cargo features to make running specific test groups easier (e.g., `cargo test --features integration-tests`). +- Consider using `cargo-nextest` for parallel test execution. ### Running Tests in Different Scopes -- **Run all tests**: `cargo test` +- **Run all tests**: `cargo test` or `cargo nextest run` - **Run library-specific tests**: `cargo test -p ` +- **Run server tests**: `cargo test -p server` - **Run a specific test**: `cargo test ` - **Run tests with specific pattern**: `cargo test -- ` -- **Run tests conditionally**: Use features to enable/disable test categories - -```toml -# In Cargo.toml -[features] -integration-tests = [] -unit-tests = [] -``` - -```rust -// In code -#[cfg(test)] -#[cfg(feature = "integration-tests")] -mod integration_tests { - // Test code... -} -``` - -Then, run: `cargo test --features integration-tests` ### Examples of Proper Test Organization -#### Example: Library Unit Tests (`#[cfg(test)]` Module) +#### Example: Library Unit Test (Mocking DB) ```rust -// File: libs/braintrust/src/helpers.rs +// File: libs/handlers/src/some_handler.rs +use crate::db_trait::DbConnection; // Using a trait for DI +use mockall::mock; -// Implementation code... - -/// Get system message from a stored prompt -pub async fn get_prompt_system_message(client: &BraintrustClient, prompt_id: &str) -> Result { - // Function implementation... +async fn handle_request(conn: &impl DbConnection, item_id: Uuid) -> Result { + conn.find_item(item_id).await } #[cfg(test)] mod tests { use super::*; - use std::env; - use dotenv::dotenv; - - #[tokio::test] - async fn test_get_prompt_system_message() -> Result<()> { - // Load environment variables - dotenv().ok(); - - // Skip test if no API key is available (non-blocking) - if env::var("BRAINTRUST_API_KEY").is_err() { - println!("Skipping test_get_prompt_system_message: No API key available"); - return Ok(()); + use mockall::predicate::*; + + // Define the trait + #[async_trait::async_trait] + trait DbConnection { + async fn find_item(&self, id: Uuid) -> Result; + } + + // Create the mock + mock! { + MyDbConnection {} // Name of the mock struct + #[async_trait::async_trait] + impl DbConnection for MyDbConnection { // Implement the trait + async fn find_item(&self, id: Uuid) -> Result; } - - // Test implementation... - + } + + #[tokio::test] + async fn test_handle_request_success() -> Result<()> { + let item_id = Uuid::new_v4(); + let expected_item = Item { id: item_id, name: "Test".to_string() }; + + let mut mock_conn = MockMyDbConnection::new(); + mock_conn.expect_find_item() + .with(eq(item_id)) + .times(1) + .returning(move |_| Ok(expected_item.clone())); + + let result = handle_request(&mock_conn, item_id).await?; + + assert_eq!(result, expected_item); Ok(()) } } ``` -#### Example: Library Integration Tests (Separate Directory) +#### Example: Library Integration Test (Using `TestDb`) ```rust -// File: libs/handlers/tests/metrics/delete_metric_test.rs - -use handlers::metrics::delete_metric_handler; -use database::models::MetricFile; -use crate::common::fixtures::create_test_metric; +// File: libs/handlers/tests/metrics/delete_metric_handler_test.rs +use database::test_utils::{TestDb, insert_test_metric_file, cleanup_test_data}; +use database::pool::get_pg_pool; // To verify deletion +use database::schema::metric_files; +use diesel::prelude::*; +use diesel_async::RunQueryDsl; +use handlers::metrics::delete_metric_handler; // The real handler use anyhow::Result; +use uuid::Uuid; -/// Integration test for the delete_metric_handler #[tokio::test] async fn test_delete_metric_integration() -> Result<()> { - // Setup test environment - let test_db = setup_test_db().await?; - - // Create and insert test data - let test_metric = create_test_metric(&test_db).await?; - - // Test the handler functionality - let result = delete_metric_handler(&test_metric.id, &test_db.user_id).await?; - - // Verify results + // Setup test data using database utilities + let test_db = TestDb::new().await?; + let owner_id = test_db.user_id; + let metric_file = test_db.create_test_metric_file(&owner_id).await?; + let metric_id = metric_file.id; + insert_test_metric_file(&metric_file).await?; + + // Call the actual handler function + let result = delete_metric_handler(metric_id, &owner_id).await?; + + // Verify the handler's effect (e.g., soft delete) assert!(result.deleted_at.is_some()); - - // Clean up test data - test_db.cleanup().await?; - + + // Optional: Verify directly in DB + let mut conn = get_pg_pool().get().await?; + let db_metric = metric_files::table + .filter(metric_files::id.eq(metric_id)) + .select(metric_files::deleted_at) + .first::>>(&mut conn) + .await?; + assert!(db_metric.is_some()); + + // Clean up test data using the helper + cleanup_test_data(&[metric_id]).await?; + Ok(()) } ``` -#### Example: Library-Specific Test Utilities +#### Example: Server Integration Test (Mocking Handler) ```rust -// File: libs/database/src/test_utils.rs +// File: server/tests/routing_test.rs +mod common; // Assuming common/mod.rs contains build_test_app and mocks -use crate::models::{User, Organization}; -use crate::pool::PgPool; +use axum::http::StatusCode; +use axum_test_helper::TestClient; +use common::build_test_app; use uuid::Uuid; -use anyhow::Result; -/// Test utilities specific to the database library -pub struct DatabaseTestUtils { - pub pool: PgPool, - pub test_id: String, +#[tokio::test] +async fn test_get_item_route_exists() { + let app = build_test_app(); // Builds router with MOCK handlers + let client = TestClient::new(app); + let item_id = Uuid::new_v4(); + + // Send request to the endpoint + let response = client.get(&format!("/items/{}", item_id)).send().await; + + // Assert that the route exists and the MOCK handler returned OK + // We are NOT testing the real handler's logic here. + assert_eq!(response.status(), StatusCode::OK); } -impl DatabaseTestUtils { - pub async fn new() -> Result { - // Initialize test database connection - let pool = create_test_pool().await?; - let test_id = Uuid::new_v4().to_string(); - - Ok(Self { pool, test_id }) - } - - pub async fn create_test_user(&self) -> Result { - // Create a test user in the database - // ... - } - - pub async fn cleanup(&self) -> Result<()> { - // Clean up test data - // ... - } +#[tokio::test] +async fn test_create_item_auth_middleware() { + let app = build_test_app(); // Assumes app uses a mock auth middleware + let client = TestClient::new(app); + + // Send request - mock auth should allow it + let response = client + .post("/items") + .json(&serde_json::json!({ "name": "Test" })) + .send() + .await; + + // Assert the MOCK handler's success code (e.g., CREATED) + assert_eq!(response.status(), StatusCode::CREATED); + + // How to test *failure* depends on the mock auth implementation. + // It might involve sending a specific header to trigger failure in the mock. } ``` ## General Testing Guidelines -- All tests must be async and use tokio test framework -- Tests should be well-documented with clear test case descriptions and expected outputs -- Each test should focus on testing a single piece of functionality -- Tests should be independent and not rely on the state of other tests -- Use meaningful test names that describe what is being tested -- **Design for testability**: Code should be written with testing in mind - - Favor dependency injection for easier mocking - - Separate business logic from external dependencies - - Use interfaces/traits to abstract dependencies - - Write small, focused functions that do one thing well - - Avoid global state and side effects where possible +- All tests must be async and use `#[tokio::test]`. +- Tests should be well-documented. +- Each test should focus on a single piece of functionality. +- Tests should be independent. +- Use meaningful test names. +- **Design for testability**: Favor dependency injection, separate logic from IO, use traits. ## Unit Tests -- Unit tests should be inline with the code they are testing using `#[cfg(test)]` modules -- Each public function should have corresponding unit tests -- **Important**: Unit tests should NEVER connect to external services or databases -- Mock all external dependencies: - - Use mockito for HTTP services - - Use trait-based mocks for database operations - - Create mock implementations of dependencies -- **IMPORTANT**: Always use the async version of mockito: - ```rust - // Correct async approach - let server = mockito::Server::new_async().await; - ``` - Instead of: - ```rust - // Incorrect: Not compatible with tokio runtime - let server = mockito::Server::new(); - ``` -- When using mockito in async tests, ensure all mock setup is done asynchronously -- Test both success and error cases -- Test edge cases and boundary conditions -- Structure unit tests to maximize code coverage -- Use dependency injection to make code easily testable with mocks +- **Location**: Inline with code using `#[cfg(test)]`. +- **Scope**: Test individual functions/modules in isolation. +- **External Dependencies**: MUST be mocked (Database, HTTP APIs, file system, time). Use `mockall` for traits/structs, `mockito` (async) for HTTP. +- **Database**: Never connect to a real DB. Use trait-based mocks if DB interaction is needed. +- **HTTP**: Use `mockito::Server::new_async().await`. ## Integration Tests -Integration tests are specifically designed to verify the interaction with external services and should be separate from the main codebase. +### Library Integration Tests (`libs/*/tests/`) +- **Location**: In each library's `tests/` directory. +- **Scope**: Test the public API of the library, including its interaction with external services relevant to that library (e.g., database tests interact with the DB). +- **Database**: Use `database::test_utils` (`TestDb`, `insert_...`, `cleanup_test_data`) to manage test data in a real test database. +- **External APIs**: Can use `mockito` if testing library functions that call external services. +- **Isolation**: Tests should ideally not call functions from *other* libraries directly, focus on the current library's integration. -### Integration Test Structure +### Server Integration Tests (`server/tests/`) +- **Location**: `server/tests/`. +- **Scope**: Test the assembled Axum application: routing, middleware, request/response cycle. +- **Handlers**: Mocked (as described above) - do *not* test handler logic here. +- **Database**: Only interact with the database via `database::test_utils` if *required* for testing middleware behavior (e.g., setting up a user for auth middleware). +- **Testing Tool**: `axum_test_helper::TestClient` or similar in-memory tester. -- **Location**: Integration tests should always be in a separate `tests/` directory, never mixed with the main code -- **External Interaction**: Unlike unit tests, integration tests are explicitly designed to interact with external services (databases, APIs, etc.) -- **Configuration**: - - Configuration should come from environment variables via `.env` files - - Use `dotenv` to load environment variables during test setup - - Example: Database connection parameters should come from `.env` - - Prefer `.env.test` for test-specific configurations +### Environment and Configuration +- Use `.env.test` for test-specific environment variables (Database URLs, API keys). +- Load config via `dotenv`. The `database` test setup might rely on environment variables being loaded beforehand. +- Ensure `DATABASE_URL` points to a dedicated test database. -```rust -// Example of proper integration test setup -use dotenv::dotenv; -use diesel_async::AsyncPgConnection; -use diesel_async::pooled_connection::AsyncDieselConnectionManager; -use deadpool_diesel::postgres::Pool; - -// Setup function that loads from environment -async fn setup_test_environment() -> Result { - // Load environment variables, preferring .env.test if available - if std::path::Path::new(".env.test").exists() { - dotenv::from_filename(".env.test").ok(); - } else { - dotenv().ok(); - } - - // Create database pool from environment variables - let database_url = std::env::var("DATABASE_URL") - .expect("DATABASE_URL must be set for integration tests"); - - let config = AsyncDieselConnectionManager::::new(database_url); - let pool = Pool::builder(config).max_size(5).build()?; - - // Return test context with real connections to external services - Ok(TestContext { pool, ... }) -} -``` - -### Library-Specific Integration Tests - -- Each library should have its own integration tests in its `tests/` directory -- These tests should focus ONLY on the library's public API -- Integration tests should NOT cross library boundaries -- Example structure: - -``` -libs/ - ├── database/ - │ ├── src/ - │ └── tests/ # Database-specific integration tests - │ ├── common/ # Database test utilities - │ └── models/ # Tests for database models - ├── handlers/ - │ ├── src/ - │ └── tests/ # Handler-specific integration tests - │ ├── common/ # Handler test utilities - │ └── users/ # Tests for user handlers -``` - -### API Integration Tests - -- API tests should be in the main `/api/tests/` directory -- Structure should mirror the API routes -- Focus on end-to-end testing of the entire API -- Example structure: - -``` -api/ - ├── src/ - └── tests/ - ├── common/ # Shared API test utilities - └── integration/ # Organized by API resource - ├── users/ # User endpoint tests - ├── threads/ # Thread endpoint tests - └── messages/ # Message endpoint tests -``` - -### Modular Test Utilities - -- **Scope Isolation**: Test utilities should be scoped to their specific domain: - - Database utilities in `libs/database/tests/common/` - - Handler utilities in `libs/handlers/tests/common/` - - API utilities in `/api/tests/common/` - -- **Modular Structure**: Organize test utilities by function: - -``` -tests/common/ - ├── mod.rs # Re-exports common utilities - ├── db.rs # Database test helpers - ├── http.rs # HTTP client testing - ├── auth.rs # Authentication test helpers - └── fixtures/ # Test data fixtures - ├── mod.rs # Re-exports all fixtures - ├── users.rs # User test data - └── messages.rs # Message test data -``` - -- **No Global Utilities**: Avoid creating global test utilities shared across all components -- **Library-Specific Fixtures**: Test fixtures should be specific to their domain - -### Test Data Isolation - -- Each test should create its own isolated data -- Use unique identifiers (UUIDs) to mark test data -- Clean up after tests complete -- Use test_id pattern to track and clean up test data: - -```rust -#[tokio::test] -async fn test_user_creation() -> Result<()> { - let test_id = Uuid::new_v4().to_string(); - let pool = get_db_pool().await?; - - // Create test data with test_id marker - let test_user = UserBuilder::new() - .with_email("test@example.com") - .with_test_id(&test_id) // Mark this as test data - .build(); - - // Insert test data - test_user.insert(&pool).await?; - - // Run test assertions... - - // Clean up all data with this test_id - cleanup_test_data(&pool, &test_id).await?; - - Ok(()) -} -``` - -### Integration Test Setup Best Practices - -#### Environment Configuration -- Create a centralized test environment setup system: +### Database Setup for Integration Tests +- **Initialization**: Assumes connection pools are initialized globally before tests start (e.g., using `#[ctor]` in `libs/database/src/lib.rs` or `tests/mod.rs`). +- **Test Data Management**: ALWAYS use `database::test_utils`: ```rust - // tests/common/env.rs - use std::sync::Once; - use dotenv::dotenv; - - static ENV_SETUP: Once = Once::new(); - - /// Initialize test environment once per test process - pub fn init_test_env() { - ENV_SETUP.call_once(|| { - // First check for .env.test - if std::path::Path::new(".env.test").exists() { - dotenv::from_filename(".env.test").ok(); - } else { - // Fall back to regular .env - dotenv().ok(); - } - - // Set additional test-specific env vars - if std::env::var("TEST_ENV").is_err() { - std::env::set_var("TEST_ENV", "test"); - } - - // Initialize logger for tests - if std::env::var("TEST_LOG").is_ok() { - tracing_subscriber::fmt() - .with_env_filter("debug") - .with_test_writer() - .init(); - } - }); + #[tokio::test] + async fn test_something_with_db() -> Result<()> { + // Assumes pools are initialized + let test_db = TestDb::new().await?; + let user = test_db.create_test_user().await?; + let dashboard = test_db.create_test_dashboard_file(&user.id).await?; + insert_test_dashboard_file(&dashboard).await?; + let dashboard_id = dashboard.id; + + // ... perform test actions ... + + // Clean up the specific data created for this test + cleanup_test_data(&[dashboard_id]).await?; + Ok(()) } ``` +- **Schema/Models**: Use the standard definitions from `database::schema` and `database::models`. +- **Isolation**: `TestDb` provides context, but actual data isolation relies on careful use of `cleanup_test_data` at the end of each test. -#### Database Setup -- All integration tests must import and utilize the application's schema from [schema.rs](mdc:src/database/schema.rs) -- Database models from [models.rs](mdc:src/database/models.rs) should be used for test data setup and verification -- Use the `testkit` library for test IDs and directly access pools from the database library: - ```rust - // tests/common/db.rs - use anyhow::Result; - use diesel_async::AsyncPgConnection; - use database::pool::get_pg_pool; - use testkit::test_id; - use uuid::Uuid; - - pub struct TestDb { - pub test_id: String, // Unique identifier for this test run - } - - impl TestDb { - /// Creates a new isolated test database environment - pub fn new() -> Self { - // Generate unique test identifier - let test_id = test_id(); - - Self { test_id } - } - - /// Get a database connection from the pre-initialized pool - pub async fn get_conn(&self) -> Result> { - Ok(get_pg_pool().get().await?) - } - - /// Clean up test data created by this test instance - pub async fn cleanup(&self) -> Result<()> { - let mut conn = self.get_conn().await?; - - // Delete data with test_id marker - // Reset sequences if needed - - Ok(()) - } - - /// Create test data with proper isolation using the test_id - pub async fn create_test_data(&self) -> Result<()> { - let mut conn = self.get_conn().await?; - - // Create test data tagged with test_id for isolation - Ok(()) - } - } - - impl Drop for TestDb { - fn drop(&mut self) { - // Optionally perform synchronous cleanup on drop - // This ensures cleanup even if tests panic - } - } - ``` - -#### Required Environment Variables -The testkit automatically creates a `.env.test` file if one doesn't exist, with necessary default test configurations. You can customize it with: - -```env -TEST_DATABASE_URL=postgres://user:pass@localhost/test_db -TEST_POOLER_URL=postgres://user:pass@localhost/test_db -TEST_REDIS_URL=redis://localhost:6379 -TEST_DATABASE_POOL_SIZE=10 -TEST_SQLX_POOL_SIZE=10 -TEST_API_KEY=test-key -TEST_ENV=test -# Optional: enable test logging -# TEST_LOG=debug -``` - -**IMPORTANT**: All integration tests should use the pools from the database library (already initialized by the testkit at build time). Tests should never be written with hardcoded service configurations or manual pool creation. - -#### Service Mocks -Create reusable mock services for common external dependencies: -```rust -// tests/common/mocks/http_client.rs -pub struct MockHttpClient { - server: mockito::Server, -} - -impl MockHttpClient { - pub async fn new() -> Self { - Self { - server: mockito::Server::new_async().await, - } - } - - pub fn url(&self) -> String { - self.server.url() - } - - pub fn mock_success_response(&self, path: &str, body: &str) -> mockito::Mock { - self.server - .mock("GET", path) - .with_status(200) - .with_header("content-type", "application/json") - .with_body(body) - .create() - } - - pub fn mock_error_response(&self, path: &str, status: usize) -> mockito::Mock { - self.server - .mock("GET", path) - .with_status(status) - .create() - } -} -``` - -## Test Structure -```rust -#[cfg(test)] -mod tests { - use super::*; - use mockito; - use tokio; - - // Optional: Setup function for common test initialization - async fn setup() -> TestContext { - // Setup code here - } - - #[tokio::test] - async fn test_name() { - // Test case description in comments - // Expected output in comments - - // Arrange - // Setup test data and dependencies - - // Act - // Execute the function being tested - - // Assert - // Verify the results - } -} -``` - -## Mocking Guidelines for Async Testing - -### Mockito for HTTP Service Mocking - -Mockito is the primary tool for mocking HTTP services in tests. When using mockito in an async context: - -```rust -// CORRECT: Create async mockito server -#[tokio::test] -async fn test_http_client() -> Result<()> { - // Always use the async version for tokio compatibility - let server = mockito::Server::new_async().await; - - // Setup the mock with expected request and response - let mock = server - .mock("GET", "/api/data") - .with_status(200) - .with_header("content-type", "application/json") - .with_body(r#"{"key": "value"}"#) - .create(); - - // Use the server's URL with your client - let client = YourHttpClient::new(&server.url()); - let response = client.get_data().await?; - - // Verify the mock was called as expected - mock.assert(); - - Ok(()) -} -``` - -#### Mockito Best Practices: - -- **Always use `Server::new_async()`** instead of `Server::new()` for tokio compatibility -- **Match complex requests** with matchers: - ```rust - // Match JSON request bodies - server.mock("POST", "/api/users") - .match_body(mockito::Matcher::Json(json!({"name": "Test User"}))) - .with_status(201) - .create(); - - // Match headers - server.mock("GET", "/api/protected") - .match_header("authorization", "Bearer token") - .with_status(200) - .create(); - ``` -- **Test different response scenarios** (success, errors, timeouts): - ```rust - // Success response - let success_mock = server.mock("GET", "/api/resource/1") - .with_status(200) - .with_body(r#"{"id": "1", "name": "Resource"}"#) - .create(); - - // Error response - let error_mock = server.mock("GET", "/api/resource/999") - .with_status(404) - .with_body(r#"{"error": "Not found"}"#) - .create(); - - // Timeout simulation - let timeout_mock = server.mock("GET", "/api/slow") - .with_delay(std::time::Duration::from_secs(10)) - .with_status(200) - .create(); - ``` -- **Create reusable mock setup functions** for common patterns - -### Additional Mocking Libraries for Async Rust - -#### 1. Mock_it for trait mocking - -For mocking traits and interfaces: - -```rust -use mock_it::Mock; - -#[async_trait] -pub trait Database { - async fn get_user(&self, id: &str) -> Result; -} - -// Create a mock implementation -let mut db_mock = Mock::new(); - -// Set up expectations with async behavior -db_mock.expect_call(matching!("get_user", arg if arg == "123")) - .returns_async(Ok(User { - id: "123".to_string(), - name: "Test User".to_string(), - })); - -// Test using the mock -let result = db_mock.get_user("123").await?; -assert_eq!(result.name, "Test User"); -``` - -#### 2. Async-std's Async-Mock - -For async-std runtime testing: - -```rust -use async_std::test; -use async_mock::AsyncMock; - -#[derive(AsyncMock)] -#[async_mock(UserRepository)] -pub trait UserRepository { - async fn find_user(&self, id: &str) -> Result; -} - -#[test] -async fn test_user_service() { - // Create a mock repository - let mock_repo = MockUserRepository::new(); - - // Setup expectations - mock_repo.expect_find_user() - .with(eq("123")) - .returns(Ok(User { id: "123".to_string(), name: "User" })); - - // Test service with mock - let service = UserService::new(mock_repo); - let user = service.get_user_data("123").await?; - - assert_eq!(user.name, "User"); -} -``` - -#### 3. WireMock for Complex HTTP Mocking - -For extensive API mocking scenarios: - -```rust -use wiremock::{MockServer, Mock, ResponseTemplate}; -use wiremock::matchers::{method, path}; - -#[tokio::test] -async fn test_api_client() -> Result<()> { - // Start mock server - let mock_server = MockServer::start().await; - - // Setup mock - Mock::given(method("GET")) - .and(path("/api/users")) - .respond_with(ResponseTemplate::new(200) - .set_body_json(json!([ - {"id": "1", "name": "User 1"}, - {"id": "2", "name": "User 2"} - ])) - ) - .mount(&mock_server) - .await; - - // Create client using mock URL - let client = ApiClient::new(&mock_server.uri()); - - // Test the client - let users = client.list_users().await?; - assert_eq!(users.len(), 2); - - Ok(()) -} -``` - -#### 4. Tower's `mock` Module for Service Mocking - -For mocking Tower services: - -```rust -use tower_test::{mock, assert_request_eq}; -use tower::Service; - -#[tokio::test] -async fn test_tower_service() { - // Create mock service - let (mut mock, handle) = mock::pair::, Response<()>>(); - - // Spawn a task that processes requests - tokio::spawn(async move { - // Send successful response - if let Ok(request) = handle.recv().await { - handle.send_response( - Response::builder() - .status(200) - .body(()) - .unwrap() - ); - } - }); - - // Create your service that uses the mock - let service = YourService::new(mock); - - // Make request through your service - let response = service - .call(Request::get("/test").body(()).unwrap()) - .await - .unwrap(); - - assert_eq!(response.status(), 200); -} -``` - -### Mocking Database Access for Unit Tests - -For unit tests, completely mock the database access rather than using real connections: - -```rust -use mock_it::Mock; -use async_trait::async_trait; -use uuid::Uuid; - -// Define a repository trait that can be mocked -#[async_trait] -pub trait UserRepository { - async fn find_user_by_id(&self, id: &str) -> Result>; - async fn create_user(&self, email: &str, name: &str) -> Result; - async fn delete_user(&self, id: &str) -> Result<()>; -} - -// In unit tests, create a mock implementation -#[tokio::test] -async fn test_user_service_with_mocked_db() -> Result<()> { - // Create mock repository - let mut repo_mock = Mock::new(); - - // Setup expectations for database operations - let test_user = User { - id: Uuid::new_v4().to_string(), - email: "test@example.com".to_string(), - name: "Test User".to_string(), - created_at: chrono::Utc::now(), - }; - - // Mock the find_user_by_id method - repo_mock.expect_call(matching!("find_user_by_id", arg if arg == "123")) - .returns_async(Ok(Some(test_user.clone()))); - - // Create the service with the mocked repository - let service = UserService::new(repo_mock); - - // Test the service logic - let user = service.get_user_profile("123").await?; - - // Assertions to verify service logic - assert_eq!(user.email, "test@example.com"); - - Ok(()) -} -``` - -### Mocking Diesel Models for Unit Tests - -For mocking Diesel model operations: - -```rust -use mockall::predicate::*; -use mockall::mock; - -// Create a mock for database operations -mock! { - pub DieselDb { - async fn find_user(&self, id: &str) -> Result>; - async fn create_user(&self, data: NewUser) -> Result; - } -} - -// User service that depends on database -struct UserService { - db: Box, -} - -impl UserService { - fn new(db: impl DieselDbTrait + 'static) -> Self { - Self { db: Box::new(db) } - } - - async fn get_user(&self, id: &str) -> Result> { - self.db.find_user(id).await - } -} - -// Unit test with mocked database -#[tokio::test] -async fn test_get_user() -> Result<()> { - // Create mock DB - let mut mock_db = MockDieselDb::new(); - - // Setup expectations - let test_user = User { - id: "user123".to_string(), - name: "Test User".to_string(), - email: "test@example.com".to_string(), - created_at: chrono::Utc::now(), - }; - - mock_db.expect_find_user() - .with(eq("user123")) - .times(1) - .returning(move |_| Ok(Some(test_user.clone()))); - - // Create service with mock - let service = UserService::new(mock_db); - - // Test service method - let user = service.get_user("user123").await?; - - // Verify results - assert!(user.is_some()); - let user = user.unwrap(); - assert_eq!(user.email, "test@example.com"); - - Ok(()) -} -``` - -### Combining Different Mocking Approaches - -For complex systems, combine different mocking libraries: - -```rust -#[tokio::test] -async fn test_complex_system() -> Result<()> { - // Set up HTTP mock - let server = mockito::Server::new_async().await; - - // Set up mock responses - let api_mock = server - .mock("GET", "/api/data") - .with_status(200) - .with_body(r#"{"data": "value"}"#) - .create(); - - // Create mock database implementation - let mut db_mock = Mock::new(); - db_mock.expect_call(matching!("save_data", _)) - .returns_async(Ok(())); - - // Initialize system with mocks - let client = HttpClient::new(&server.url()); - let system = YourSystem::new(client, db_mock); - - // Test system behavior - let result = system.process_and_save().await?; - assert!(result.is_success()); - - // Verify HTTP mock was called - api_mock.assert(); - - Ok(()) -} -``` - -## Error Testing -- Test error conditions and error handling -- Verify error messages and error types -- Test timeout scenarios -- Test connection failures -- Test invalid input handling - -## Database Testing -- Use a separate test database for integration tests -- Clean up test data after tests complete -- Test database connection error handling - -## Test Output -- Tests should provide clear error messages -- Use descriptive assert messages -- Print relevant debug information in test failures -- Log test execution progress for long-running tests - -## CI/CD Considerations -- All tests must pass in CI environment -- Tests should be reproducible -- Tests should not have external dependencies that could fail CI -- Test execution time should be reasonable - -## Example Unit Test with Mockito -```rust -#[cfg(test)] -mod tests { - use super::*; - use mockito; - use tokio; - use anyhow::Result; - - #[tokio::test] - async fn test_api_call_success() -> Result<()> { - // Test case: Successful API call returns expected response - // Expected: Response contains user data with status 200 - - // Create async mockito server - let server = mockito::Server::new_async().await; - - // Setup the mock with expected request and response - let mock = server - .mock("GET", "/api/user") - .match_header("authorization", "Bearer test-token") - .with_status(200) - .with_body(r#"{"id": "123", "name": "Test User"}"#) - .create(); - - // Create API client with mock server URL - let client = ApiClient::new(server.url()); - - // Execute the function being tested - let response = client.get_user("test-token").await?; - - // Verify results - assert_eq!(response.id, "123"); - assert_eq!(response.name, "Test User"); - - // Verify that the mock was called as expected - mock.assert(); - - Ok(()) - } - - #[tokio::test] - async fn test_api_call_error_handling() -> Result<()> { - // Test case: API returns error status - // Expected: Function returns appropriate error - - let server = mockito::Server::new_async().await; - - // Setup mock with error response - let mock = server - .mock("GET", "/api/user") - .with_status(401) - .with_body(r#"{"error": "Unauthorized"}"#) - .create(); - - let client = ApiClient::new(server.url()); - - // Execute and verify it returns an error - let result = client.get_user("invalid-token").await; - assert!(result.is_err()); - - // Verify the error contains the expected message - let err = result.unwrap_err(); - assert!(err.to_string().contains("Unauthorized")); - - mock.assert(); - - Ok(()) - } -} -``` - -## Example Application Integration Test - -```rust -use crate::tests::common::db::TestDb; -use crate::database::{models, schema, pool}; -use crate::handlers::users::get_user_handler; -use anyhow::Result; - -#[tokio::test] -async fn test_user_creation_flow() -> Result<()> { - // Setup test database with isolation - // TestDb handles test ID generation and connection access - let test_db = TestDb::new(); - - // Create test user using fixture or builder pattern - let test_user = models::User::builder() - .email("test@example.com") - .name("Test User") - .test_id(&test_db.test_id) // Set test ID for isolation - .build(); - - // Insert test data with proper error handling - let mut conn = test_db.get_conn().await?; - diesel::insert_into(schema::users::table) - .values(&test_user) - .execute(&mut conn) - .await?; - - // Create HTTP client for API testing - let client = reqwest::Client::new(); - - // Test the API endpoint - let response = client - .get(&format!("{}/api/users/{}", TEST_API_URL, test_user.id)) - .header("Authorization", "Bearer test-token") - .send() - .await?; - - // Verify response status - assert_eq!(response.status(), 200); - - // Parse and verify response body - let user_response = response.json::().await?; - assert_eq!(user_response.email, "test@example.com"); - - // Clean up test data - test_db.cleanup().await?; - - Ok(()) -} -``` - -## Example Library Integration Test - -This example shows how to structure integration tests for libraries in the `libs/` directory: - -```rust -// File: libs/my_library/tests/integration_test.rs - -use my_library::{Client, Config}; -use anyhow::Result; - -// Import test utilities if needed -// Testing utilities can be in the library's tests/common/ directory - -#[tokio::test] -async fn test_client_performs_operation() -> Result<()> { - // Setup mock server for external API - let server = mockito::Server::new_async().await; - - // Configure mock responses - let mock = server - .mock("POST", "/api/resource") - .with_status(201) - .with_body(r#"{"id": "new-resource-id"}"#) - .create(); - - // Configure the client to use the mock server - let config = Config::builder() - .base_url(server.url()) - .timeout(std::time::Duration::from_secs(5)) - .build(); - - let client = Client::new(config); - - // Call the library function being tested - let result = client.create_resource("test-resource").await?; - - // Verify the library correctly processed the response - assert_eq!(result.id, "new-resource-id"); - - // Verify the mock was called as expected - mock.assert(); - - Ok(()) -} -``` - -## Common Test Utilities - -A comprehensive set of testing utilities has been implemented in the `tests/common/` directory. These utilities provide a standardized approach to testing throughout the codebase and should be used for all new tests. The utilities address common testing patterns and provide a consistent interface for test code. - -### Common Test Module Structure -``` -tests/ -├── common/ -│ ├── mod.rs # Main module that re-exports all utilities -│ ├── env.rs # Environment setup utilities -│ ├── db.rs # Database testing utilities with test isolation -│ ├── http/ # HTTP testing utilities -│ │ ├── mock_server.rs # MockServer wrapper for mockito -│ │ ├── client.rs # TestHttpClient for API requests -│ │ └── mod.rs # HTTP module exports -│ ├── fixtures/ # Test data fixtures -│ │ ├── mod.rs # Exports all fixtures -│ │ ├── builder.rs # FixtureBuilder trait and helpers -│ │ ├── users.rs # User-related test data -│ │ └── threads.rs # Thread-related test data -│ ├── assertions/ # Test assertion utilities -│ │ ├── mod.rs # Exports all assertions -│ │ ├── response.rs # HTTP response assertions -│ │ └── model.rs # Model validation assertions -│ ├── matchers/ # Matcher utilities for mockito -│ │ ├── mod.rs # Exports all matchers -│ │ ├── json.rs # JSON matchers for request bodies -│ │ └── headers.rs # Header matchers for requests -│ └── helpers.rs # General test helper functions -└── integration/ # Integration test files -``` - -### Environment Setup Utilities -The `env.rs` module provides utilities for setting up the test environment: - -```rust -// tests/common/env.rs -use std::sync::Once; -use dotenv::dotenv; - -static ENV_SETUP: Once = Once::new(); - -/// Initialize test environment once per test process -pub fn init_test_env() { - ENV_SETUP.call_once(|| { - // First check for .env.test - if std::path::Path::new(".env.test").exists() { - dotenv::from_filename(".env.test").ok(); - } else { - // Fall back to regular .env - dotenv().ok(); - } - - // Set additional test-specific env vars - if std::env::var("TEST_ENV").is_err() { - std::env::set_var("TEST_ENV", "test"); - } - - // Initialize logger for tests - if std::env::var("TEST_LOG").is_ok() { - tracing_subscriber::fmt() - .with_env_filter("debug") - .with_test_writer() - .init(); - } - }); -} - -/// Get a config value from environment with fallback -pub fn get_test_config(key: &str, default: &str) -> String { - std::env::var(key).unwrap_or_else(|_| default.to_string()) -} -``` - -### Database Testing Utilities -The `db.rs` module provides utilities for database testing with proper test isolation: - -```rust -// tests/common/db.rs -use anyhow::Result; -use diesel_async::{AsyncPgConnection, AsyncConnection}; -use diesel_async::pooled_connection::AsyncDieselConnectionManager; -use deadpool_diesel::postgres::Pool; -use uuid::Uuid; - -pub struct TestDb { - pub pool: Pool, - pub test_id: String, // Unique identifier for this test run -} - -impl TestDb { - /// Creates a new isolated test database environment - pub async fn new() -> Result { - // Initialize environment - crate::common::env::init_test_env(); - - // Generate unique test identifier - let test_id = Uuid::new_v4().to_string(); - - // Get database config from env - let database_url = std::env::var("TEST_DATABASE_URL") - .expect("TEST_DATABASE_URL must be set for tests"); - - // Create connection manager and pool - let config = AsyncDieselConnectionManager::::new(database_url); - let pool = Pool::builder(config) - .max_size(5) - .build()?; - - let db = Self { pool, test_id }; - - // Optional: Setup initial test data - // db.setup_schema().await?; - - Ok(db) - } - - /// Get a connection from the pool - pub async fn get_conn(&self) -> Result { - Ok(self.pool.get().await?) - } - - /// Clean up test data created by this test instance - pub async fn cleanup(&self) -> Result<()> { - // Example: Clean up tables used in tests - // let conn = &mut self.get_conn().await?; - // diesel::delete(schema::users::table) - // .filter(schema::users::test_id.eq(&self.test_id)) - // .execute(conn) - // .await?; - - Ok(()) - } - - /// Get the unique test ID for this test instance - pub fn test_id(&self) -> &str { - &self.test_id - } -} - -impl Drop for TestDb { - fn drop(&mut self) { - // Optional synchronous cleanup fallback - } -} -``` - -### HTTP Testing Utilities -The `http/` directory contains utilities for HTTP testing, including a MockServer wrapper and a TestHttpClient: - -```rust -// tests/common/http/mock_server.rs -use anyhow::Result; -use mockito::{self, Mock, Server}; -use serde_json::Value; - -/// MockServer is a wrapper around mockito::Server -pub struct MockServer { - server: Server, -} - -impl MockServer { - /// Create a new MockServer - pub async fn new() -> Result { - Ok(Self { - server: Server::new_async().await, - }) - } - - /// Get the base URL of the mock server - pub fn url(&self) -> String { - self.server.url() - } - - /// Mock a GET request with JSON response - pub fn mock_get_json(&self, path: &str, response: &T) -> Result { - let body = serde_json::to_string(response)?; - - Ok(self.server - .mock("GET", path) - .with_status(200) - .with_header("content-type", "application/json") - .with_body(body) - .create()) - } - - /// Mock a POST request with JSON request and response - pub fn mock_post_json(&self, - path: &str, - request_matcher: Option, - response: &T) -> Result { - let body = serde_json::to_string(response)?; - let mut mock = self.server - .mock("POST", path) - .with_status(200) - .with_header("content-type", "application/json") - .with_body(body); - - // Add request body matcher if provided - if let Some(req_body) = request_matcher { - mock = mock.match_body(crate::common::matchers::json::json_contains(req_body)); - } - - Ok(mock.create()) - } - - /// Mock an error response - pub fn mock_error(&self, path: &str, method: &str, status: usize, message: &str) -> Result { - let error_body = serde_json::json!({ - "error": message, - }); - - Ok(self.server - .mock(method, path) - .with_status(status) - .with_header("content-type", "application/json") - .with_body(serde_json::to_string(&error_body)?) - .create()) - } -} - -// tests/common/http/client.rs -use anyhow::Result; -use reqwest::{Client, Method, RequestBuilder, Response, StatusCode}; -use serde::de::DeserializeOwned; -use serde_json::Value; -use std::time::Duration; - -/// TestHttpClient provides a fluent API for making HTTP requests in tests -pub struct TestHttpClient { - client: Client, - base_url: String, -} - -impl TestHttpClient { - /// Create a new TestHttpClient with the given base URL - pub fn new(base_url: &str) -> Self { - let client = Client::builder() - .timeout(Duration::from_secs(5)) - .build() - .expect("Failed to create HTTP client"); - - Self { - client, - base_url: base_url.to_string(), - } - } - - /// Create a request builder with the given method and path - pub fn request(&self, method: Method, path: &str) -> RequestBuilder { - let url = format!("{}{}", self.base_url, path); - self.client.request(method, url) - } - - /// Make a GET request and return the response - pub async fn get(&self, path: &str) -> Result { - Ok(self.request(Method::GET, path).send().await?) - } - - /// Make a POST request with the given body and return the response - pub async fn post(&self, path: &str, body: &T) -> Result { - Ok(self.request(Method::POST, path) - .json(body) - .send() - .await?) - } - - /// Make a GET request and parse the response as JSON - pub async fn get_json(&self, path: &str) -> Result { - let response = self.get(path).await?; - - if response.status().is_success() { - Ok(response.json::().await?) - } else { - let status = response.status(); - let error_text = response.text().await?; - anyhow::bail!("Request failed with status {}: {}", status, error_text) - } - } - - /// Make a POST request with the given body and parse the response as JSON - pub async fn post_json( - &self, - path: &str, - body: &T - ) -> Result { - let response = self.post(path, body).await?; - - if response.status().is_success() { - Ok(response.json::().await?) - } else { - let status = response.status(); - let error_text = response.text().await?; - anyhow::bail!("Request failed with status {}: {}", status, error_text) - } - } -} -``` - -### Fixture Builder Pattern -The `fixtures/builder.rs` file provides a builder pattern for creating test fixtures: - -```rust -// tests/common/fixtures/builder.rs -use uuid::Uuid; - -/// TestFixture is a trait for test fixtures -pub trait TestFixture { - /// Get the test ID for this fixture - fn test_id(&self) -> Option<&str>; - - /// Set the test ID for this fixture - fn with_test_id(self, test_id: &str) -> Self; -} - -/// FixtureBuilder is a trait for building test fixtures -pub trait FixtureBuilder { - /// Build the fixture - fn build(&self) -> T; - - /// Build the fixture with the given test ID - fn build_with_test_id(&self, test_id: &str) -> T; -} - -/// Example builder for a User fixture -#[derive(Default)] -pub struct UserBuilder { - pub email: Option, - pub name: Option, - pub test_id: Option, -} - -impl UserBuilder { - pub fn new() -> Self { - Self::default() - } - - pub fn email(mut self, email: &str) -> Self { - self.email = Some(email.to_string()); - self - } - - pub fn name(mut self, name: &str) -> Self { - self.name = Some(name.to_string()); - self - } - - pub fn test_id(mut self, test_id: &str) -> Self { - self.test_id = Some(test_id.to_string()); - self - } -} - -impl FixtureBuilder for UserBuilder { - fn build(&self) -> User { - User { - id: Uuid::new_v4(), - email: self.email.clone().unwrap_or_else(|| format!("user-{}@example.com", Uuid::new_v4())), - name: self.name.clone(), - test_id: self.test_id.clone(), - created_at: chrono::Utc::now(), - updated_at: chrono::Utc::now(), - } - } - - fn build_with_test_id(&self, test_id: &str) -> User { - let mut builder = self.clone(); - builder.test_id = Some(test_id.to_string()); - builder.build() - } -} -``` - -### Assertion Utilities -The `assertions/` directory contains utilities for making assertions in tests: - -```rust -// tests/common/assertions/response.rs -use anyhow::Result; -use reqwest::Response; -use reqwest::StatusCode; -use serde::de::DeserializeOwned; -use serde_json::Value; - -/// Extension trait for reqwest::Response providing assertion methods -pub trait ResponseAssertions { - /// Assert that the response has the given status code - fn assert_status(self, status: StatusCode) -> Self; - - /// Assert that the response has a success status code (2xx) - fn assert_success(self) -> Self; - - /// Assert that the response has an error status code (4xx or 5xx) - fn assert_error(self) -> Self; - - /// Assert that the response contains the given header - fn assert_header(self, name: &str, value: &str) -> Self; - - /// Assert that the response body contains the given JSON value - fn assert_json_contains(self, expected: Value) -> Result; - - /// Deserialize the response body as JSON and apply the given assertion function - fn assert_json(self, assert_fn: F) -> Result - where - T: DeserializeOwned, - F: FnOnce(&T) -> bool; -} - -impl ResponseAssertions for Response { - fn assert_status(self, status: StatusCode) -> Self { - assert_eq!(self.status(), status, "Expected status code {}, got {}", status, self.status()); - self - } - - fn assert_success(self) -> Self { - assert!(self.status().is_success(), "Expected success status, got {}", self.status()); - self - } - - fn assert_error(self) -> Self { - assert!(self.status().is_client_error() || self.status().is_server_error(), - "Expected error status, got {}", self.status()); - self - } - - fn assert_header(self, name: &str, value: &str) -> Self { - let header_value = self.headers().get(name) - .expect(&format!("Header {} not found", name)) - .to_str() - .expect(&format!("Header {} is not valid UTF-8", name)); - - assert_eq!(header_value, value, "Expected header {} to be {}, got {}", name, value, header_value); - self - } - - async fn assert_json_contains(self, expected: Value) -> Result { - let json = self.json::().await?; - - // Check if expected is a subset of json - assert!(json_contains(&json, &expected), - "Expected JSON to contain {:?}, got {:?}", expected, json); - - Ok(self) - } - - async fn assert_json(self, assert_fn: F) -> Result - where - T: DeserializeOwned, - F: FnOnce(&T) -> bool, - { - let json = self.json::().await?; - - assert!(assert_fn(&json), "JSON assertion failed"); - - Ok(self) - } -} - -// Helper function to check if one JSON value contains another -fn json_contains(json: &Value, expected: &Value) -> bool { - match (json, expected) { - (Value::Object(json_obj), Value::Object(expected_obj)) => { - expected_obj.iter().all(|(k, v)| { - json_obj.get(k).map_or(false, |json_v| json_contains(json_v, v)) - }) - } - (Value::Array(json_arr), Value::Array(expected_arr)) => { - expected_arr.iter().all(|expected_v| { - json_arr.iter().any(|json_v| json_contains(json_v, expected_v)) - }) - } - _ => json == expected, - } -} -``` - -### JSON and Header Matchers -The `matchers/` directory contains utilities for matching JSON and headers in mockito: - -```rust -// tests/common/matchers/json.rs -use mockito::Matcher; -use serde_json::Value; -use std::fmt; - -/// Create a matcher that checks if a JSON request body contains the given JSON value -pub fn json_contains(expected: Value) -> JsonContainsMatcher { - JsonContainsMatcher { expected } -} - -/// Matcher for checking if a JSON request body contains a JSON value -pub struct JsonContainsMatcher { - expected: Value, -} - -impl Matcher for JsonContainsMatcher { - fn matches(&self, body: &[u8]) -> bool { - let actual: Value = match serde_json::from_slice(body) { - Ok(v) => v, - Err(_) => return false, - }; - - contains_json(&actual, &self.expected) - } -} - -impl fmt::Display for JsonContainsMatcher { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - write!(f, "JSON containing {}", self.expected) - } -} - -/// Helper function to check if one JSON value contains another -fn contains_json(json: &Value, expected: &Value) -> bool { - match (json, expected) { - (Value::Object(json_obj), Value::Object(expected_obj)) => { - expected_obj.iter().all(|(k, v)| { - json_obj.get(k).map_or(false, |json_v| contains_json(json_v, v)) - }) - } - (Value::Array(json_arr), Value::Array(expected_arr)) => { - expected_arr.iter().all(|expected_v| { - json_arr.iter().any(|json_v| contains_json(json_v, expected_v)) - }) - } - _ => json == expected, - } -} - -// tests/common/matchers/headers.rs -use mockito::Matcher; -use std::collections::HashMap; -use std::fmt; - -/// Create a matcher that checks if request headers match the expected headers -pub fn header_matcher(expected_headers: HashMap) -> HeaderMatcher { - HeaderMatcher { expected_headers } -} - -/// Matcher for checking request headers -pub struct HeaderMatcher { - expected_headers: HashMap, -} - -impl Matcher for HeaderMatcher { - fn matches(&self, request_headers: &[u8]) -> bool { - let headers_str = match std::str::from_utf8(request_headers) { - Ok(s) => s, - Err(_) => return false, - }; - - // Parse headers - let headers: HashMap = headers_str - .split("\r\n") - .filter(|line| !line.is_empty()) - .filter_map(|line| { - let mut parts = line.splitn(2, ": "); - let name = parts.next()?; - let value = parts.next()?; - Some((name.to_lowercase(), value.to_string())) - }) - .collect(); - - // Check if expected headers are present - self.expected_headers.iter().all(|(name, value)| { - headers.get(&name.to_lowercase()).map_or(false, |v| v == value) - }) - } -} - -impl fmt::Display for HeaderMatcher { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - write!(f, "Headers containing {:?}", self.expected_headers) - } -} -``` - -### Using These Test Utilities - -Here's an example of using these test utilities in an integration test: - -```rust -use crate::tests::common::{ - db::TestDb, - env::init_test_env, - http::{MockServer, TestHttpClient}, - fixtures::UserBuilder, - assertions::ResponseAssertions, -}; -use anyhow::Result; -use serde_json::json; - -#[tokio::test] -async fn test_user_api() -> Result<()> { - // Initialize environment - init_test_env(); - - // Setup test database - let test_db = TestDb::new().await?; - - // Create test user using builder - let user = UserBuilder::new() - .email("test@example.com") - .name("Test User") - .build_with_test_id(test_db.test_id()); - - // Insert test user into database - // ... - - // Setup mock server for external API - let mock_server = MockServer::new().await?; - - // Configure mock responses - let mock = mock_server.mock_get_json("/api/external", &json!({ - "status": "success" - }))?; - - // Create test HTTP client - let client = TestHttpClient::new("http://localhost:8000"); - - // Test the API endpoint - let response = client.get(&format!("/api/users/{}", user.id)).await?; - - // Assert on the response - response - .assert_status(StatusCode::OK) - .assert_header("content-type", "application/json") - .assert_json_contains(json!({ - "id": user.id.to_string(), - "email": "test@example.com" - }))?; - - // Verify the mock was called - mock.assert(); - - // Clean up - test_db.cleanup().await?; - - Ok(()) -} -``` \ No newline at end of file +**IMPORTANT**: Integration tests should use the shared connection pools initialized once for the test run. Tests should *never* create their own pools manually.