Proper introspection APIs for handy utils?

[kentcdodds]

I'd like to be able to parse searchParams with Zod. Here's a utility I built for that:

import { z } from 'zod'

export function parseSearchParams<SchemaType extends z.ZodRawShape>(
	sp: URLSearchParams,
	schema: z.ZodObject<SchemaType>,
) {
	const rawValues: Record<string, Array<string> | string> = {}
	const schemaProps = schema._def.shape()
	for (const key of sp.keys()) {
		const values = sp.getAll(key)
		const propSchema = schemaProps[key]
		if (propSchema && values.length === 1 && !isExpectingArray(propSchema)) {
			rawValues[key] = values[0]
		} else {
			rawValues[key] = values
		}
	}

	return schema.parse(rawValues)
}

function isExpectingArray(schema?: z.ZodTypeAny): boolean {
	if (!schema?._def) return false

	const { typeName } = schema._def
	if (typeName === 'ZodArray') return true
	if (['ZodUnion', 'ZodIntersection'].includes(typeName)) {
		return schema._def.options.some(isExpectingArray)
	}
	if (['ZodOptional', 'ZodNullable', 'ZodDefault'].includes(typeName)) {
		return isExpectingArray(schema._def.innerType)
	}
	if (typeName === 'ZodLazy') {
		return isExpectingArray(schema._def.getter())
	}
	return false
}

const searchParams = new URLSearchParams()
searchParams.append('starportIds', '123')
searchParams.append('starportIds', '456')
searchParams.append('cityIds', '789')
searchParams.append('cityIds', '101112')
searchParams.append('brandIds', '131415')
searchParams.append('brandIds', '161718')
searchParams.append('modelIds', '192021')
searchParams.append('modelIds', '222324')
searchParams.append('hostIds', '252627')
searchParams.append('hostIds', '282930')
searchParams.append('capacityMin', '31')
searchParams.append('capacityMax', '32')
searchParams.append('dailyChargeMin', '33')
searchParams.append('dailyChargeMax', '34')
searchParams.append('hostRatingMin', '5')
searchParams.append('shipRatingMin', '4')
searchParams.append('availabilityStartDate', '2021-01-01')
searchParams.append('availabilityEndDate', '2021-01-02')

const result = parseSearchParams(
	searchParams,
	z.object({
		starportIds: z.array(z.string()),
		cityIds: z.array(z.string()),
		brandIds: z.array(z.string()),
		modelIds: z.array(z.string()),
		hostIds: z.array(z.string()),
		capacityMin: z.coerce.number().positive(),
		capacityMax: z.coerce.number().positive(),
		dailyChargeMin: z.coerce.number().positive(),
		dailyChargeMax: z.coerce.number().positive(),
		hostRatingMin: z.coerce.number().min(0).max(5),
		shipRatingMin: z.coerce.number().min(0).max(5),
		availabilityStartDate: z.string().refine(
			v => /\d{4}-\d{2}-\d{2}/.test(v),
			val => ({ message: `Invalid date: ${val}` }),
		),
		availabilityEndDate: z.string().refine(
			v => /\d{4}-\d{2}-\d{2}/.test(v),
			val => ({ message: `Invalid date: ${val}` }),
		),
	}),
)

console.log(result)

/*
Logs:

{
  starportIds: [ '123', '456' ],
  cityIds: [ '789', '101112' ],
  brandIds: [ '131415', '161718' ],
  modelIds: [ '192021', '222324' ],
  hostIds: [ '252627', '282930' ],
  capacityMin: 31,
  capacityMax: 32,
  dailyChargeMin: 33,
  dailyChargeMax: 34,
  hostRatingMin: 5,
  shipRatingMin: 4,
  availabilityStartDate: '2021-01-01',
  availabilityEndDate: '2021-01-02'
}
*/

The tricky bit is the fact that searchParams can have multiple of the same key (so you can't just do searchParams.entries()). So what my utility does is to first get all the properties on searchParams with getAll, then determines whether you were hoping for an array of values and if you are it will double check there's only one value and then add only one value to the resulting object. If you're hoping for an array then it leaves it as an array. If you were not hoping for an array but there's more than one value it leaves it as an array and the later parse call will result in an error (as desired).

My problem is that to determine whether you want an array I need to inspect the schema you've provided and it looks like I'm using a pseudo-private API for that (and TypeScript isn't happy with that). Specifically the _def property.

Is there a better way to do what I'm doing?

EDIT: I changed from schema: z.ZodSchema<SchemaType> to schema: z.ZodObject<SchemaType> and that made TypeScript much happier. But I still want to make sure that there's no better way to do this since I'm using a pseudo-private property 😅

EDIT 2: I handled other zod types that wrap the array type so you can use default, optional, union etc.

[igalklebanov]

Also discussed in #1681.

[JacobWeisenburger]

@kentcdodds,

Here's how I would do it. I think it's a bit cleaner and no messing around with pseudo-private properties 😀. Also it allows you to use safeParse or parseAsync, etc... without having to make new functions.

Let me know if you have any questions.

function valuesFromSearchParams ( searchParams: URLSearchParams ) {
    return Array.from( searchParams.keys() ).reduce( ( values, key ) => ( {
        ...values, [ key ]: searchParams.getAll( key )
    } ), {} as Record<string, Array<string> | string> )
}

const searchParamSchema = z.string().array().min( 1 )

const schema = z.object( {
    many: searchParamSchema,
    string: searchParamSchema.max( 1 ).pipe(
        z.coerce.string()
    ),
    posNumber: searchParamSchema.max( 1 ).pipe(
        z.coerce.number().positive()
    ),
    range: searchParamSchema.max( 1 ).pipe(
        z.coerce.number().min( 0 ).max( 5 )
    ),
    plainDate: searchParamSchema.max( 1 ).pipe(
        z.coerce.string().refine(
            v => /\d{4}-\d{2}-\d{2}/.test( v ),
            val => ( { message: `Invalid plain date: ${ val }` } ),
        ),
    ),
} )

const searchParams = new URLSearchParams( [
    [ 'many', 'hello' ],
    [ 'many', 'world' ],
    [ 'string', 'foo' ],
    [ 'posNumber', '42' ],
] )

searchParams.set( 'range', '4' )
searchParams.set( 'plainDate', '2021-01-01' )

const values = valuesFromSearchParams( searchParams )

console.log( schema.parse( values ) )
// {
//     many: [ 'hello', 'world' ],
//     string: 'foo',
//     posNumber: 42,
//     range: 4,
//     plainDate: '2021-01-01'
// }

[kentcdodds]

Thanks! I'll give you suggestion a look!

[JacobWeisenburger]

@kentcdodds
I wrote a different way here: https://gist.github.com/JacobWeisenburger/9256eae415f6b0a04b718d633266a4e0
I think it's better than my first answer. Hopefully one of my answers works for you.

[kentcdodds]

Thanks, unfortunately this won't work. There's no way you can know just from the params whether it's ok to have only one raw value or if that one value should be within an array.

[kentcdodds]

I'm using /~https://github.com/kiliman/remix-params-helper right now which handles this for me, but it's also using _def. So my original request for introspection APIs is still necessary.

[JacobWeisenburger]

... There's no way you can know just from the params whether it's ok to have only one raw value or if that one value should be within an array.

https://gist.github.com/JacobWeisenburger/9256eae415f6b0a04b718d633266a4e0
I have updated my gist to include oneStringInArray. Does this address what you are talking about?

[ruslanguns]

I was about to reinvent the wheel doing the same thing, thanks for your contribution! It works as a champ! 🎉

[wenchonglee]

This thread is stale but in case anyone else stumbles on this like I did, I also wanted to inspect the schema but for a more specific use case.

This is what I came up with, it doesn't cover everything because my use case doesn't require it. (thankfully, because there're a lot more classes to check)

I also had to use _def for pipe though.

const ExampleSchema = z.object({
  a: z.string(),
  b: z.string().optional(),
  c: z.string().refine((val) => typeof val === "string"),
  d: z.string().transform((val) => val),
  e: z.string().array(),
  f: z
    .string()
    .transform((val) => val.length)
    .pipe(z.number().min(5)),
  g: z
    .string()
    .min(1)
    .transform((val) => val.length)
    .pipe(z.number().min(5))
    .array()
    .refine((val) => val)
    .nullish(),
});

console.log("Naively comparing");
console.log(ExampleSchema.shape.a instanceof ZodString); // true
console.log(ExampleSchema.shape.b instanceof ZodString); // false 
console.log(ExampleSchema.shape.c instanceof ZodString); // false
console.log(ExampleSchema.shape.d instanceof ZodString); // false
console.log(ExampleSchema.shape.e instanceof ZodString); // false
console.log(ExampleSchema.shape.f instanceof ZodString); // false
console.log(ExampleSchema.shape.g instanceof ZodString); // false

console.log("Do some recursive type checks");
const findBaseType = (schema: ZodSchema<any>) => {
  if (schema instanceof ZodEffects) {
    return findBaseType(schema.sourceType());
  }

  if (schema instanceof ZodOptional || schema instanceof ZodNullable) {
    return findBaseType(schema.unwrap());
  }

  if (schema instanceof ZodPipeline) {
    return findBaseType(schema._def.in);
  }

  if (schema instanceof ZodArray) {
    return findBaseType(schema.element);
  }

  return schema;
};
console.log(findBaseType(ExampleSchema.shape.a) instanceof ZodString); // true 
console.log(findBaseType(ExampleSchema.shape.b) instanceof ZodString); // true
console.log(findBaseType(ExampleSchema.shape.c) instanceof ZodString); // true
console.log(findBaseType(ExampleSchema.shape.d) instanceof ZodString); // true
console.log(findBaseType(ExampleSchema.shape.e) instanceof ZodString); // true
console.log(findBaseType(ExampleSchema.shape.f) instanceof ZodString); // true
console.log(findBaseType(ExampleSchema.shape.g) instanceof ZodString); // true

Codesandbox if it helps anyone

[paulcalcraft]

Thanks for this, was just about to start writing something like this this morning!

My notes from yesterday eve were literally:

schema._def.shape()[key]._def gives us an object that can have the right .typeName but could also have .typeName of “optional” if that was the right-most thing. And then the _innerType is string or whatever. So we could do this by convention if we can have the type as the rightmost thing always? Or the leftmost thing always? So it chases down _innerType until it gets to the beginning.

Glad to see there's a more flexible way that doesn't rely on ordering / control over generating the schemas.

[humblemodulo]

@colinhacks I just want to confirm that _def.shape (and the like) is actually safe to use and there are no massive "internals" changing any time soon.

I'm currently building a tool that uses Zod objects as their schema "source of truth" and translates them into related objects/types for use with other modules/services. Just want to make sure that won't break any time soon.

[codinsonn]

Same, actually, pinning the supported zod version just in case

[paulcalcraft]

Me as well!