Usage with Zod (and rather complex discriminated unions)

[severi]

Im trying to find a way to integrate Electrodb to our application. Currently we use zod for handling typing/data validation. Our data models can be rather complex and we for example quite extensively use discriminated unions.

If we want to take Electrodb into use we would need somewhat duplicate our model definitions (zod + electrodb entity attributes). Does anyone else have experience with this, and are there any ways to reduce the amount of copy paste needed.

Would there for example be some simple way to only define some basic set of attributes for electrodb entities (for example those used for partition keys, sortkeys, etc, but also allow "any other elements" to be stored to the object (currently they are removed automatically).

[pjvds]

Wondering which route you took @severi, care to share you experience?

[severi]

I know the answer comes with a loong delay, but better late than never I guess.

Unfortunately the only solution I found was to to have two separate schemas, one for electrodb and one for zod

I decided to keep the types in the Electrodb schema rather simple (usually just someAttribute: { type: 'any'} ) and do the proper validation with zod before storing / after retrieving the data

I also created a helper function that verifies that both the zod schema and the electrodb definition have the same attributes listed

import { z } from 'zod';
type DbModel = {
  zod: z.ZodObject<any>;
  electrodb: {
    model: { entity: string; version: string; service: string };
    attributes: Record<string, unknown>;
    indexes: unknown;
  };
};

export const verifyModel = <T extends DbModel>(model: T) => {
  const zodKeys = (model.zod.keyof().options as string[]).sort();
  const attributes = Object.keys(model.electrodb.attributes).sort();
  if (zodKeys.join(',') !== attributes.join(',')) {
    throw new Error(`Zod keys and electrodb attributes do not match: ${zodKeys.join(',')} vs ${attributes.join(',')}`);
  }
  return model;
};

This does not work with more complex zod schemas but has been enough for us for now.

All in all I think this definitely is an issue that I'd love to see being addressed by the framework directly in someway

[JeremyJonas]

In case it help anyone, I have been doing a type assertion style conversion which has been working quiet well. It doesn't convert zod to electrodb attributes/schema, which I don't think would be possible beyond super simple models, but it does give type completion, compilation errors, and intellisense to move pretty quick.

Has decent coverage of Zod and ElectroDB types, but missing MapAttributes as I don't use them and just use any.

ZodSchemaToElectroAttributes Typescript Utility Type

Take a Zod object type as input, and casts it to ElectroDB Attributes type definition.

import type { RequiredKeysOf, IsLiteral } from 'type-fest';
import type * as electro from 'electrodb';
import type { z } from 'zod';

// Utility to map TypeScript types to ElectroDB attributes
type TypeToElectroAttribute<T> =
  IsLiteral<T> extends true ? Omit<electro.EnumAttribute, 'type'> & { type: ReadonlyArray<T> } :
  T extends Set<string> ? electro.StringSetAttribute :
  T extends Set<number> ? electro.NumberSetAttribute :
  T extends string ? electro.StringAttribute :
  T extends number ? electro.NumberAttribute :
  T extends boolean ? electro.BooleanAttribute :
  T extends ReadonlyArray<string> ? electro.StringListAttribute : // Use List instead of Set for readonly arrays
  T extends ReadonlyArray<number> ? electro.NumberListAttribute : // Use List instead of Set for readonly arrays
  T extends Array<string> ? electro.StringListAttribute : // Mutable List for arrays of strings
  T extends Array<number> ? electro.NumberListAttribute : // Mutable List for arrays of numbers
  T extends Record<string, any> ? electro.AnyAttribute : // Use AnyAttribute for dynamic object types
  T extends Array<Record<string, any>> ? electro.AnyListAttribute : // Mutable List for arrays of dynamic objects
  never;

type ZodReadOnlyKeysOf<T extends z.AnyZodObject> = keyof { [P in keyof T['shape'] as T['shape'][P] extends z.ZodReadonly<any> ? P : never]: true }

// Utility to handle `readonly`, `optional`, etc.
type InferElectroAttributes<T extends object, TSchema extends z.AnyZodObject, TRequiredKeys = RequiredKeysOf<T>, TReadOnlyKeys = ZodReadOnlyKeysOf<TSchema>> = {
  // Handle Readonly fields
  [K in keyof T]: TypeToElectroAttribute<Exclude<T[K], undefined | null>> // support nullish
    & (K extends TReadOnlyKeys ? electro.ReadOnlyAttribute : { readOnly?: false })
    & (K extends TRequiredKeys ? electro.RequiredAttribute : { required?: false })
}

// Main utility to map a Zod schema to its ElectroDB attributes using inferred TypeScript type
export type ZodSchemaToElectroAttributes<T extends z.ZodObject<any>> = InferElectroAttributes<z.infer<T>, T>;

Usage

import { z as zod } from 'zod'

const TestEnum = zod.enum(['active', 'inactive', 'pending'])

// example zod schema
const TestSchema = zod.object({
  string: zod.string(), // StringAttribute
  numberOptional: zod.number().optional(), // NumberAttribute & Required: false
  booleanNullable: zod.boolean().nullable(), // BooleanAttribute & Required: false
  enum: TestEnum, // EnumAttribute (Enum)
  enumNullish: TestEnum.nullish(), // EnumAttribute (Enum)
  object: zod.object({ theme: zod.string() }), // AnyAttribute (Nested object)
  listString: zod.array(zod.string()), // StringListAttribute (Array of strings)
  listNumber: zod.array(zod.number()), // NumberListAttribute (Array of numbers)
  literalString: zod.literal('LITERAL'), // EnumAttribute (Literal)
  stringReadonly: zod.string().readonly(),
});

// example electrodb attributes type inferred by zod schema
const TestAttributes = {
  string: { type: 'string', required: true },
  numberOptional: { type: 'number' },
  booleanNullable: { type: 'boolean', required: true },
  enum: { type: TestEnum.options, required: true },
  enumNullish: { type: TestEnum.options, required: false },
  object: { type: 'any', required: true },
  listString: { type: 'list', items: { type: 'string'}, required: true },
  listNumber: { type: 'list', items: { type: 'number'}, required: true },
  literalString: { type: ['LITERAL'], required: true },
  stringReadonly: { type: 'string', required: true, readOnly: true },
} satisfies ZodSchemaToElectroAttributes<typeof TestSchema>; // list line casts and asserts

And then use that as the attributes property of electrodb schema

export const AccountTask = createSchema({
  model: {
    service: 'Example',
    entity: 'Test',
    version: 'v0',
  },
  attributes: TestAttributes,
  // ...
} as const);

The key is using satisfies ZodSchemaToElectroAttributes<typeof TestSchema> for assertion and casting.

Intellisense if works great

Hope that help someone else! 🥂

[zirkelc]

That is awesome, thanks for sharing it!

Any reason you used createSchema() instead of new Entity()?

[JeremyJonas]

new Entity() would work the same as createSchema, I have a complex setup with other type inference based on the schema as well, so I prefer to create the schema separately and then use it to create Entity. But for this case there is no difference 😄

[JeremyJonas]

Just added CustomAttribute support to the type inference for Record types - slowly chipping away as run into cases.

...
  T extends Record<string, any> ? (electro.CustomAttributeDefinition<T> & { type: electro.CustomAttributeTypeName<T> }) | electro.AnyAttribute : 

Will support either "any" or explicit custom type

...
  object: { type: 'any', required: true },
  objectCustom: { type: CustomAttributeType<{ x: number; y: number}>('any'), required: true },

Here is full updated code

import type { RequiredKeysOf, IsLiteral } from 'type-fest';
import type * as electro from 'electrodb';
import type { z } from 'zod';

// Utility to map TypeScript types to ElectroDB attributes
type TypeToElectroAttribute<T> =
  IsLiteral<T> extends true ? Omit<electro.EnumAttribute, 'type'> & { type: ReadonlyArray<T> } :
  T extends Set<string> ? electro.StringSetAttribute :
  T extends Set<number> ? electro.NumberSetAttribute :
  T extends string ? electro.StringAttribute :
  T extends number ? electro.NumberAttribute :
  T extends boolean ? electro.BooleanAttribute :
  T extends ReadonlyArray<string> ? electro.StringListAttribute : // Use List instead of Set for readonly arrays
  T extends ReadonlyArray<number> ? electro.NumberListAttribute : // Use List instead of Set for readonly arrays
  T extends Array<string> ? electro.StringListAttribute : // Mutable List for arrays of strings
  T extends Array<number> ? electro.NumberListAttribute : // Mutable List for arrays of numbers
  T extends Record<string, any> ? (electro.CustomAttributeDefinition<T> & { type: electro.CustomAttributeTypeName<T> }) | electro.AnyAttribute : // Use AnyAttribute for dynamic object types
  T extends Array<Record<string, any>> ? electro.AnyListAttribute : // Mutable List for arrays of dynamic objects
  never;

type ZodReadOnlyKeysOf<T extends z.AnyZodObject> = keyof { [P in keyof T['shape'] as T['shape'][P] extends z.ZodReadonly<any> ? P : never]: true }

// Utility to handle `readonly`, `optional`, etc.
type InferElectroAttributes<T extends object, TSchema extends z.AnyZodObject, TRequiredKeys = RequiredKeysOf<T>, TReadOnlyKeys = ZodReadOnlyKeysOf<TSchema>> = {
  // Handle Readonly fields
  [K in keyof T]: TypeToElectroAttribute<Exclude<T[K], undefined | null>> // support nullish
    & (K extends TReadOnlyKeys ? electro.ReadOnlyAttribute : { readOnly?: false })
    & (K extends TRequiredKeys ? electro.RequiredAttribute : { required?: false })
}

// Main utility to map a Zod schema to its ElectroDB attributes using inferred TypeScript type
export type ZodSchemaToElectroAttributes<T extends z.ZodObject<any>> = InferElectroAttributes<z.infer<T>, T>;

////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////
// NOTE: Preserve following comment for testing - uncomment to test types
////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////
import { z as zod } from 'zod'
import { CustomAttributeType } from 'electrodb';
const TestEnum = zod.enum(['active', 'inactive', 'pending'])
const TestSchema = zod.object({
  string: zod.string(), // StringAttribute
  numberOptional: zod.number().optional(), // NumberAttribute & Required: false
  booleanNullable: zod.boolean().nullable(), // BooleanAttribute & Required: false
  enum: TestEnum, // EnumAttribute (Enum)
  enumNullish: TestEnum.nullish(), // EnumAttribute (Enum)
  object: zod.object({ theme: zod.string() }), // AnyAttribute (Nested object)
  objectCustom: zod.object({ x: zod.number(), y: zod.number() }), // AnyAttribute (Nested object)
  listString: zod.array(zod.string()), // StringListAttribute (Array of strings)
  listNumber: zod.array(zod.number()), // NumberListAttribute (Array of numbers)
  literalString: zod.literal('LITERAL'), // EnumAttribute (Literal)
  stringReadonly: zod.string().readonly(),
});

const TestAttributes = {
  string: { type: 'string', required: true },
  numberOptional: { type: 'number' },
  booleanNullable: { type: 'boolean', required: true },
  enum: { type: TestEnum.options, required: true },
  enumNullish: { type: TestEnum.options, required: false },
  object: { type: 'any', required: true },
  objectCustom: { type: CustomAttributeType<{ x: number; y: number}>('any'), required: true },
  listString: { type: 'list', items: { type: 'string'}, required: true },
  listNumber: { type: 'list', items: { type: 'number'}, required: true },
  literalString: { type: ['LITERAL'], required: true },
  stringReadonly: { type: 'string', required: true, readOnly: true },
} satisfies ZodSchemaToElectroAttributes<typeof TestSchema>;

[zirkelc]

Do you think turning this into zod-to-electrodb that converts a Zod schema into ElectroDB attributes would work? Of course ignoring additional Zod functions like email(), datetime() or validation like transform(), preprocess(), refine(), etc. for now.