buster/packages/server-shared
dal e60a8420c4
feat: implement schema sync and validation cron job
- Add daily cron job that runs at midnight MST to validate schemas
- Introspect customer data sources and compare with dataset YML configs
- Send Slack notifications when discrepancies are found
- Group datasets by database/schema for efficient introspection
- Add comprehensive logging for debugging schema comparisons
- Fix credential retrieval to use vault pattern from AI package
- Move schema-sync types from server-shared to trigger app

This implementation includes:
- Database queries for fetching organizations and datasets
- YML parser with Zod validation
- Schema comparator with type normalization
- Slack notification formatter using Block Kit
- CloudWatch metrics for monitoring
- Comprehensive unit tests

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 09:05:12 -06:00
..
.cursor Added readme and other stuff for cursor 2025-07-17 12:56:42 -06:00
scripts Add prebuild script to server-shared 2025-07-12 16:07:55 -06:00
src move get asset types around 2025-07-24 17:41:36 -06:00
.gitignore Mastra braintrust (#391) 2025-07-02 14:33:40 -07:00
README.md Added readme and other stuff for cursor 2025-07-17 12:56:42 -06:00
biome.json fix broken linting 2025-07-17 13:18:32 -06:00
index.ts feat: implement schema sync and validation cron job 2025-07-31 09:05:12 -06:00
package.json feat: implement schema sync and validation cron job 2025-07-31 09:05:12 -06:00
tsconfig.json fix all of the bugs 2025-07-12 22:14:08 -06:00
turbo.json update database dev 2025-07-15 22:26:13 -06:00
vitest.config.ts Add some unit tests 2025-07-04 01:38:58 -06:00

README.md

@buster/server-shared

Shared TypeScript types and schemas for the Buster ecosystem

Overview

The @buster/server-shared package provides a centralized location for all shared TypeScript types, Zod schemas, and type utilities used across the Buster monorepo. This package ensures type safety and consistency between frontend and backend services.

Architecture Principles

1. Zod-First Approach

All types are defined using Zod schemas first, then TypeScript types are inferred from them:

// ✅ Good: Define schema first
export const UserSchema = z.object({
  id: z.string(),
  email: z.string().email(),
  name: z.string(),
});

export type User = z.infer<typeof UserSchema>;

// ❌ Bad: Defining types without schemas
export interface User {
  id: string;
  email: string;
  name: string;
}

2. Database Type Imports

When referencing database types from @buster/database, ALWAYS import them as types to avoid compilation errors:

// ✅ Good: Import as type
import type { organizations, userOrganizationRoleEnum } from '@buster/database';

// ❌ Bad: Import as value
import { organizations } from '@buster/database';

3. Enum Pattern for Database Parity

When creating enums that mirror database enums, use frozen objects to maintain type safety:

import type { userOrganizationRoleEnum } from '@buster/database';

type OrganizationRoleBase = (typeof userOrganizationRoleEnum.enumValues)[number];

// Create a frozen object that mirrors the database enum
export const OrganizationRoleEnum: Record<OrganizationRoleBase, OrganizationRoleBase> = 
  Object.freeze({
    viewer: 'viewer',
    workspace_admin: 'workspace_admin',
    data_admin: 'data_admin',
    querier: 'querier',
    restricted_querier: 'restricted_querier',
  });

// Create Zod schema from the enum
export const OrganizationRoleSchema = z.enum(
  Object.values(OrganizationRoleEnum) as [OrganizationRoleBase, ...OrganizationRoleBase[]]
);

export type OrganizationRole = z.infer<typeof OrganizationRoleSchema>;

4. Type Parity Checks

When types are direct copies of database models, use the Expect and Equal utilities to ensure parity:

import type { organizations } from '@buster/database';
import type { Equal, Expect } from '../type-utilities';

export type Organization = z.infer<typeof OrganizationSchema>;

// This will cause a TypeScript error if the types don't match
type _OrganizationEqualityCheck = Expect<Equal<Organization, typeof organizations.$inferSelect>>;

Directory Structure

src/
├── index.ts                 # Main barrel export (currently incomplete)
├── chats/                   # Chat-related types
│   ├── index.ts            # Barrel exports for chats
│   ├── chat.types.ts       # Core chat types
│   ├── chat-message.types.ts
│   └── ...
├── organization/            # Organization types
│   ├── index.ts            # Barrel exports
│   ├── organization.types.ts
│   ├── requests.ts         # Request schemas/types
│   ├── responses.ts        # Response schemas/types
│   └── ...
├── metrics/                 # Metrics types
│   ├── index.ts
│   ├── requests.types.ts   # Note: Some use .types.ts suffix
│   ├── responses.types.ts
│   └── ...
├── type-utilities/          # Generic utility types
│   ├── index.ts
│   ├── pagination.ts       # Pagination utilities
│   ├── isEqual.ts          # Type equality checks
│   └── ...
└── ... (other modules)

Module Structure Guidelines

Each module should follow this structure:

1. Barrel Export (index.ts)

Export all public types from the module:

export * from './organization.types';
export * from './roles.types';
export * from './requests';
export * from './responses';

2. Request Types (requests.ts)

Define all request schemas for the module:

import { z } from 'zod';

export const CreateUserRequestSchema = z.object({
  email: z.string().email(),
  name: z.string(),
});

export type CreateUserRequest = z.infer<typeof CreateUserRequestSchema>;

3. Response Types (responses.ts)

Define all response schemas for the module:

import { z } from 'zod';
import { UserSchema } from './user.types';

export const GetUserResponseSchema = UserSchema;
export type GetUserResponse = z.infer<typeof GetUserResponseSchema>;

export const ListUsersResponseSchema = z.object({
  users: z.array(UserSchema),
  total: z.number(),
});
export type ListUsersResponse = z.infer<typeof ListUsersResponseSchema>;

Usage Examples

Importing Types

// Import from specific modules
import { Organization, UpdateOrganizationRequest } from '@buster/server-shared/organization';
import { Chat, ChatMessage } from '@buster/server-shared/chats';
import { PaginationParams } from '@buster/server-shared/type-utilities';

Using Schemas for Validation

import { UpdateOrganizationRequestSchema } from '@buster/server-shared/organization';

// Validate incoming data
const validatedData = UpdateOrganizationRequestSchema.parse(requestBody);

// Or safe parse with error handling
const result = UpdateOrganizationRequestSchema.safeParse(requestBody);
if (!result.success) {
  console.error(result.error);
}

Adding New Modules

  1. Create a new directory under src/
  2. Add the following files:
    • index.ts - Barrel exports
    • requests.ts - Request schemas/types
    • responses.ts - Response schemas/types
    • Additional type files as needed
  3. Update package.json exports:
    "./your-module": {
      "types": "./dist/your-module/index.d.ts",
      "default": "./dist/your-module/index.js"
    }
    

Common Patterns

Hex Color Validation

const HexColorSchema = z
  .string()
  .regex(
    /^#([0-9A-Fa-f]{3}|[0-9A-Fa-f]{6})$/,
    'Must be a valid 3 or 6 digit hex color code'
  );

Pagination

Use the generic pagination types from type-utilities:

import { PaginationParams } from '../type-utilities';

export const ListUsersRequestSchema = z.object({
  ...PaginationParams.shape,
  filter: z.string().optional(),
});

Important Notes

  1. Never import database constants directly - This will cause build failures in frontend packages
  2. Always define schemas before types - Types should be inferred from schemas
  3. Maintain type parity - Use Expect<Equal<>> pattern for database model copies
  4. Export through package.json - Each module needs its own export entry
  5. Follow naming conventions - Use either requests.ts/responses.ts or requests.types.ts/responses.types.ts consistently within a module

Contributing

When adding new types or modifying existing ones:

  1. Follow the Zod-first approach
  2. Ensure proper barrel exports
  3. Add request/response types where applicable
  4. Update package.json exports if adding new modules
  5. Run type checking before committing