- Updated the SQL migration to enforce a unique constraint on the combination of `database_name` and `data_source_id` in the datasets table, ensuring data integrity.
- Refactored the `deploy_datasets_handler` to separate datasets with and without IDs, allowing for concurrent upsert operations based on their presence.
- Enhanced the upsert logic to handle datasets more efficiently, improving performance during dataset deployment.
These changes improve the robustness and efficiency of the dataset deployment process within the API.
- Introduced a new `/sql/run` endpoint for executing SQL queries against datasets and data sources.
- Created a dedicated `sql` module and a `run_sql` handler to manage SQL execution logic.
- Implemented access checks to ensure users have the necessary permissions to execute SQL queries.
- Enhanced data retrieval and metadata processing for SQL results, improving overall API functionality.
These changes expand the API's capabilities by allowing users to run custom SQL queries, facilitating more flexible data interactions.
- Added a new `post_dataset` module and corresponding route to handle dataset creation.
- Updated the router to include the new POST endpoint for datasets, improving API functionality.
- Maintained existing routes while ensuring modular organization of dataset-related logic.
These changes improve the API's capabilities for dataset management by providing a dedicated endpoint for dataset creation, enhancing overall usability.
- Replaced the existing `post_datasets` endpoint with a new `deploy_datasets` endpoint to better reflect its purpose.
- Deleted the `post_datasets` module and its associated logic, streamlining the codebase.
- Updated the request and response structures to use `DeployDatasetsRequest` and related types, enhancing clarity and maintainability.
- Adjusted the BusterClient to utilize the new endpoint for deploying datasets, ensuring consistency across the API.
These changes improve the API's functionality by providing a clearer and more focused approach to dataset deployment, facilitating better data management.
- Introduced a new route to the datasets API for fetching data samples associated with a specific dataset.
- Added the `get_dataset_data_sample` module to handle the logic for retrieving dataset data samples.
- Updated the router configuration to include the new endpoint, enhancing the API's functionality for dataset management.
These changes improve the API's capabilities by allowing users to access sample data for datasets, facilitating better data exploration and analysis.
- Simplified the GetDatasetResponse struct by removing unnecessary fields and renaming existing ones for clarity.
- Updated the dataset retrieval logic to focus on essential dataset attributes, enhancing performance and readability.
- Improved user role checks for dataset access, ensuring clearer error messages for permission issues.
- Removed unused imports and streamlined the code for better maintainability.
These changes enhance the API's efficiency in retrieving dataset information and improve the clarity of user permissions related to dataset access.
- Introduced a new optional `model` field in the Dataset struct to store model references.
- Updated the dataset routes to include a new endpoint for retrieving datasets by ID.
- Modified dataset creation logic to accommodate the new `model` field.
- Refactored dataset queries to utilize `datasets::all_columns` for improved readability and maintainability.
These changes enhance the dataset management capabilities by allowing the association of models with datasets, improving data organization and retrieval.
- Introduced a new optional `yml_file` field in the Dataset model to store YAML file references.
- Updated the database schema to include the `yml_file` column in the datasets table.
- Modified various API request and response structures to accommodate the new `yml_file` field.
- Enhanced dataset handling functions to support the inclusion of `yml_file` in dataset operations.
These changes improve the dataset management capabilities by allowing the association of YAML files with datasets, facilitating better data organization and retrieval.
- Added queries to retrieve datasets and permission groups associated with users for improved access tracking in the dataset overview response.
- Implemented logic to include direct dataset access and permission group lineage in the user overview, enhancing clarity on user permissions.
- Improved error handling for database interactions related to dataset and permission group queries.
- Added debug print statements for datasets and permission groups queries to facilitate troubleshooting.
These changes improve the API's ability to manage and report user permissions effectively, providing a clearer overview of user access to datasets and permission groups.
- Updated the UserPermissionLineage and UserOverviewItem structs to provide a clearer representation of user permissions and lineage in the dataset overview response.
- Simplified the get_dataset_overview function by removing redundant queries and consolidating user access checks.
- Improved error handling for database interactions related to user permissions.
- Streamlined the overall structure of the dataset overview response to focus on user details and their access capabilities.
These changes enhance the API's clarity and efficiency in managing and reporting user permissions.
- Introduced UserPermissionLineage struct to provide detailed user access information in the dataset overview response.
- Updated get_dataset_overview function to include comprehensive checks for user permissions, dataset group access, and direct access.
- Improved error handling for database queries related to user permissions and access checks.
- Added TODO comments in list_dataset_assets and put_dataset_assets routes to address future dataset group integration.
These changes enhance the API's capability to manage and report on user permissions effectively.
- Introduced a new DatasetPermission model in the database schema to manage dataset access permissions, including fields for organization_id, dataset_id, and permission_type.
- Updated the API routes to nest asset-related routes under dataset_id, enhancing the organization of dataset-related functionalities.
These changes improve the structure for managing dataset permissions and streamline the API for asset handling.
- Created dataset_groups and dataset_permissions tables in the database schema, including organization_id as a foreign key with ON DELETE CASCADE.
- Added corresponding indexes for organization_id in both tables to optimize query performance.
- Updated the Rust models and schema to reflect the new tables and their relationships.
- Integrated dataset_groups into the API routes for improved data organization and management.
These changes enhance the database structure and facilitate better handling of dataset-related permissions and groupings.
- Changed the role of a user in the teams_to_users table from 'admin' to 'manager' for better role clarity.
- Refactored the users_to_organizations table to include a new 'status' field and updated multiple user roles to align with the new role structure (workspace_admin, querier, data_admin).
- Added a new permission_groups module to the API routes for improved permission management.
- Updated the security module to include a new checks module for enhanced security handling.
- Integrated dotenv in the Next.js configuration to manage environment variables more effectively.
These changes improve the clarity and functionality of user roles and permissions within the application.
- Introduced a new UserOrganizationStatus enum to manage user organization statuses (Active, Inactive, Pending, Guest) in the database schema.
- Updated the UserToOrganization model to include a status field.
- Refactored role checks across various routes to replace the previous UserOrganizationRole values (Owner, Admin) with new roles (WorkspaceAdmin, DataAdmin) for better role management.
- Enhanced data source handling in multiple routes to align with the updated role structure.
These changes improve the clarity and functionality of user organization management within the application.
- Updated the UserOrganizationRole enum to include new roles: WorkspaceAdmin, DataAdmin, Querier, RestrictedQuerier, and Viewer, replacing the previous roles of Owner, Member, and Admin.
- Modified the TeamToUserRole enum to change the Owner role to Manager.
- Added new database tables for dataset_groups, dataset_permissions, datasets_to_dataset_groups, and permission_groups_to_users to support enhanced data management.
- Introduced UserOrganizationStatusEnum to the schema for better organization status tracking.
These changes improve role management and expand the database schema for better data organization and permissions handling.
- Deleted the `package-lock.json` file from the root directory.
- Modified the `dev` target in the API Makefile to start Redis using Docker Compose from the parent directory, improving service orchestration.
- Updated the `next.config.mjs` to load environment variables from the parent directory during development.
- Added `dotenv` as a dependency in both `package.json` and `package-lock.json` to manage environment variables effectively.
These changes streamline the development setup and enhance the management of environment variables.
- Simplified the API service build configuration in `docker-compose.yml` by consolidating the build context and Dockerfile path.
- Added `diesel_migrations` dependency to `Cargo.toml` for database migration management.
- Implemented database migration logic in `main.rs`, including error handling and logging for migration success or failure.
- Introduced a new mail service in `supabase/docker-compose.yml` for handling SMTP, POP3, and web interface.
- Removed version specification from `supabase/dev/docker-compose.dev.yml` for cleaner configuration.
These changes improve the overall structure and functionality of the application, facilitating better database management and service orchestration.
Nate Kelley
dal
RakeshNara10
dal