- Updated the `list_assets` function to include organization ID filtering in dataset permissions queries.
- Removed redundant organization ID filters from the dataset permissions queries to streamline the logic.
- Ensured that only relevant dataset assets are returned based on the user's organization, improving data security and relevance.
These changes enhance the API's ability to serve organization-specific data, aligning with recent improvements in dataset asset APIs.
- Introduced a new `is_simple` flag in the `deploy_datasets` function to differentiate between full and simple dataset deployments.
- Updated the `deploy_datasets_handler` to accept the `is_simple` parameter, allowing for conditional processing of inserted datasets.
- Modified the `DeployDatasetsRequest` struct to include an optional `id` and `type_` field, enhancing the request's flexibility.
- Adjusted the handling of the `yml_file` field to be optional in the `DeployDatasetsRequest` struct.
- Updated the `process_batch` function to handle "USER-DEFINED" data types in addition to existing types.
These changes improve the dataset deployment process by allowing for more granular control and flexibility in handling different dataset types.
- Added functionality to retrieve the user's organization ID in both `get_dataset_overview` and `list_assets` endpoints.
- Updated database queries to filter users and permissions based on the organization ID, ensuring that only relevant data is returned for the user's organization.
- Improved error handling for organization ID retrieval, logging errors appropriately.
These changes improve data security and relevance by ensuring that users only access assets associated with their organization.
- Added the user's email and name to the UserOverviewItem struct for improved clarity in user details.
- Updated the database query to select the user's name alongside their ID and email, ensuring comprehensive user information is retrieved.
- Refactored the mapping logic to accommodate the new name field, enhancing the dataset overview response.
These changes improve the dataset overview API by providing more detailed user information, facilitating better understanding of user access and roles.
- Modified the role adjustment logic in the teams_to_users table to treat 'admin' roles as 'manager'.
- Set the default role to 'member' for all other cases, improving clarity in role assignments.
These changes enhance the migration process for dataset groups and permissions management.
- Simplified certificate handling logic by allowing the COPY command to proceed without failure if cert.pem is missing.
- Updated the Dockerfile to ensure that the update-ca-certificates command is run unconditionally after copying the certificate.
- Maintained the existing build process for the bi_api application.
These changes improve the Docker image build process by making certificate handling more robust and less dependent on the environment.
- Updated the SQL migration to enforce a unique constraint on the combination of `database_name` and `data_source_id` in the datasets table, ensuring data integrity.
- Refactored the `deploy_datasets_handler` to separate datasets with and without IDs, allowing for concurrent upsert operations based on their presence.
- Enhanced the upsert logic to handle datasets more efficiently, improving performance during dataset deployment.
These changes improve the robustness and efficiency of the dataset deployment process within the API.
- Introduced a new `/sql/run` endpoint for executing SQL queries against datasets and data sources.
- Created a dedicated `sql` module and a `run_sql` handler to manage SQL execution logic.
- Implemented access checks to ensure users have the necessary permissions to execute SQL queries.
- Enhanced data retrieval and metadata processing for SQL results, improving overall API functionality.
These changes expand the API's capabilities by allowing users to run custom SQL queries, facilitating more flexible data interactions.
- Added a new `post_dataset` module and corresponding route to handle dataset creation.
- Updated the router to include the new POST endpoint for datasets, improving API functionality.
- Maintained existing routes while ensuring modular organization of dataset-related logic.
These changes improve the API's capabilities for dataset management by providing a dedicated endpoint for dataset creation, enhancing overall usability.
Nate Kelley
dal