CLI Typegen expand references

[nikmatswace]

Is your feature request related to a problem? Please describe.
When generating types you only get an "internalGroqTypeReferenceTo" a related schema.

export type ProgressBar = {
  _id: string;
  _type: "progressBar";
  _createdAt: string;
  _updatedAt: string;
  _rev: string;
  name: string;
  steps: Array<{
    _ref: string;
    _type: "reference";
    _weak?: boolean;
    _key: string;
    [internalGroqTypeReferenceTo]?: "progressBarStep";
  }>;
  locale?: string;
};

This means that you can't access the actual properties on an indiviual "step", which becomes troublesome when you have several references to other schemas or some kind of multi-level-reference structure (e.g. I have a Page with a Progress bar that has Progress bar steps).

Describe the solution you'd like
It would be great if we during the type generation could define if we wanted all/some "internalGroqTypeReferences" expanded. Perhaps either via the typegen config or some kind of flag on the schema itself.

The result after such an expansion could be something like:

export type ProgressBar = {
  _id: string;
  _type: "progressBar";
  _createdAt: string;
  _updatedAt: string;
  _rev: string;
  name: string;
  steps: Array<{
    _ref: string;
    _type: "reference";
    _weak?: boolean;
    _key: string;
    [internalGroqTypeReferenceTo]?: "progressBarStep";
    // Progress bar step properties
    _id: string;
    _type: "progressBarStep";
    _createdAt: string;
    _updatedAt: string;
    _rev: string;
    text: string;
    percentage: number
    locale?: string;
  }>;
  locale?: string;
};

Describe alternatives you've considered
I've considered creating a "dummy query" that explicitly expands the type but isn't used anywhere, however I don't think that's a very clean solution. I've also tried to alter the main query that contains multi-level-references to try and get the query-typegen to expand the "deeper levels of nesting" but with no success.

Additional context
Typegen is a super helpful feature, and the current state is a very promising! I do however think that there is room for some improvement, an this would bring it a big step forward!

[sgulseth]

Not sure I understand the ask. steps is a reference object, and it doesn't container the progress bar step properties?
If you want to get the actual progress bar step properties you can use -> to dereference it.

[nikmatswace]

Hello @sgulseth, thank you for your answer and question! Let me elaborate a bit to hopefully clear up my ask a bit.

An example of my use case is the following, I am running the query below:

export const PAGE_QUERY = groq`*[_type == "page" && 
slug == $slug][0] { 
  ...,
  sections[]->
}`

A section could be a e.g. ProgressBar, a Carousel or a Hero. It's basically the blocks that will build the page itself. I am never fetching a singular entity of above, I am always fetching them as part of a page. And therefore I don't have a query that dereferences above entities directly. Below is then what I would like to do to render the sections:

data.sections.map((section) => {
    switch (section._type) {
      case 'progressBar':
        return (
          <ProgressBarSection
            key={section._id}
            data={section as ProgressBar}
          />
        )
      case 'carousel':
        return (
          <CarouselSection
            key={section._id}
            data={section as Carousel}
          />
        )
      case 'hero':
        return (
          <HeroSection
            key={section._id}
            data={section as Hero}
          />
        )
      default:
        break
    }
  })

But ProgressBar as a type does not contain the ProgressBarSteps-properties (which presents a problem in the ProgressBarSection-component that needs those), only a reference to it (as shown above). I could create a query that I never use just to get the "complete" ProgressBar-type with the steps dereferenced, but it's a workaround I would like to avoid.

So basically, I would like to access the complete dereferenced types of all generated types, as above is just an example of where we need access to them in cases where we actually aren't fetching the object itself, but a parent of it.

I hope that this made it clearer! If not, or if there is another possible way to achieve above, please do tell me so 🙂

[sgulseth]

What's the result of the PAGE_QUERY query? And what's the generated types for this query? 🤔 This looks to be different from the one you pasted in the issue.

Typegen should return all the fields that are available on the actual result

[nikmatswace]

The first post was basically just a try to give "general example" to describe the issue in "broader" terms 🙂 . But since you asked, here is my very specific one:

The result of PAGE_QUERY is basically something like (a bit simplified):
data: {title: "Page Title", slug: "page-slug", sections: [ProgressBar, Carousel] }
if the page contains a ProgressBar and a Carousel-section.

This is the generated type of the PAGE_QUERY:

export type PAGE_QUERYResult = {
  _id: string
  _type: 'page'
  _createdAt: string
  _updatedAt: string
  _rev: string
  title: string
  slug: string
  sections: Array<
    | {
        _id: string
        _type: 'progressBar'
        _createdAt: string
        _updatedAt: string
        _rev: string
        title: string
        steps: Array<{
          _ref: string
          _type: 'reference'
          _weak?: boolean
          _key: string
          [internalGroqTypeReferenceTo]?: 'progressBarStep'
        }>
        locale?: string
        brand: 'brand1' | 'brand2' | 'brand3'
      }
    | {
        _id: string
        _type: 'carousel'
        _createdAt: string
        _updatedAt: string
        _rev: string
        title: string
        description?: string
        listTitle: string
        listItems: Array<string>
        findYourProduct: string
        summaryMicrocopy?: Array<{
          _ref: string
          _type: 'reference'
          _weak?: boolean
          _key: string
          [internalGroqTypeReferenceTo]?: 'microcopy'
        }>
        locale?: string
        brand: 'brand1' | 'brand2' | 'brand3'
      }
    | {
        _id: string
        _type: 'yourDetails'
        _createdAt: string
        _updatedAt: string
        _rev: string
        title: string
        description?: string
        actionText: string
        formFields: Array<{
          _ref: string
          _type: 'reference'
          _weak?: boolean
          _key: string
          [internalGroqTypeReferenceTo]?: 'formField'
        }>
        summaryMicrocopy?: Array<{
          _ref: string
          _type: 'reference'
          _weak?: boolean
          _key: string
          [internalGroqTypeReferenceTo]?: 'microcopy'
        }>
        locale?: string
        brand: 'brand1' | 'brand2' | 'brand3'
      }
  > | null
  locale?: string
  brand: 'brand1' | 'brand2' | 'brand3'
} | null

(You may notice that the "Hero section" isn't there, it's because I just wanted to give it as a general example since a Hero is a very common part of a page)

[nikmatswace]

@sgulseth - Is the use case clearer now?

If not, please do tell so and I will try to do my best to expand upon/explain in more detail.

[sgulseth]

(Sorry for the late reply here)

Yes, I think so. But, if I understand it correctly this is also expected behavior. The types should reflect the raw document stored in Sanity. If you execute a query like *[_type == "page"][0] in for example Sanity Vision you should see that the result matches the types returned. If you want to expand(dereference) them you can do:

*[_type == "page"][0] {
  ...,
  sections[] {
    ...,
    _type == "progressBar" => {
      steps[]->
    },
    _type == "carousel" => {
      summaryMicrocopy[]->
    }
  }
}

I agree that it could get a bit "clunky" to write these queries if you have lots of different reference attributes, but improving that should either be a native GROQ feature(We have an issue tracking it here), or a query builder feature.

[evankirkiles]

The main problem at hand here is the inability to create generic components for objects without direct 1:1 queries. It would be incredibly beneficial and (I assume) low-haul to autogen a lookup table of document _types to their generated schema TypeScript types so we can resolve references by hand.

For example, consider the following generated schema:

// This is an object, not a document
export type InternalLink = {
  _type: 'internalLink';
  linkTarget?:
    | {
        _ref: string;
        _type: 'reference';
        _weak?: boolean;
        [internalGroqTypeReferenceTo]?: 'typeA';
      }
    | {
        _ref: string;
        _type: 'reference';
        _weak?: boolean;
        [internalGroqTypeReferenceTo]?: 'typeB';
      };
};

// Link text comes from `nameA` field
export type TypeA = {
  _id: string;
  _type: 'typeA';
  slug?: Slug;
  nameA?: string;
};

// Link text comes from `nameB` field
export type TypeB = {
  _id: string;
  _type: 'typeB';
  slug?: Slug;
  nameB?: string;
};

Some sort of generic link component would look like:

export default function InternalLink({ link }: { link: InternalLink }) {
  // Opaquely typed linkTarget, e.g. { _type: "reference", [sym]?: "typeA"  } | { _type: "reference", [sym]?: "typeB" }
  const linkTarget = link.linkTarget;
  let text;
  // The below switch statement is invalid because the reference hasn't been expanded
  switch (linkTarget._type) {
     case 'typeA':
        text = linkTarget.nameA;
        break;
     case 'typeB':
        text = linkTarget.nameB;
        break;
  }
  text = link.text;
  return <a href={`/${linkTarget.slug.current}`}>{text}</a>
}

To fix the opaque _type: "reference" union, we can implement some sort of resolveReference function, which needs some way to resolve document TypeScript types from the reference's [internalGroqTypeReferenceTo] field values. This is currently not possible without manually creating a lookup table for all document types like so with DoctypeTable:

// Please auto-generate this!
type DoctypeTable = {
  typeA: TypeA;
  typeB: TypeB;
}

// Exported as strongly-typed generic resolveReference<"typeA" | "typeB">
export function resolveReference<T extends keyof DoctypeTable>(ref: {
  _type: 'reference';
  [internalGroqTypeReferenceTo]?: T;
}): DoctypeTable[T] {
  if (obj._type === 'reference')
    throw new Error('Asset reference has not been expanded!');
  return obj as unknown as DoctypeTable[T];
}

Meaning our generic link component could now convert references into their referenced types as well as throw when asset references haven't been expanded:

- // Opaquely typed linkTarget, e.g. { _type: "reference", [sym]?: "typeA"  } | { _type: "reference", [sym]?: "typeB" }
- const linkTarget = link.linkTarget;
+ // Correctly typed linkTarget, e.g. TypeA | TypeB
+ const linkTarget = resolveReference(link.linkTarget);

If sanity typegen could automatically create and export that DoctypeTable (with a more aptly fitting name), that would save an annoying piece of manual labor in updating the table every time a new document type is added. This is a similar approach to that taken of other TypeScript schema generation tools like Prisma's codegen, for example.

[nikmatswace]

(It's okay!)

Thank you for your answer sgulseth. I do realize that this is probably the "expected behavior" as of right now, but this issue is more of a feature request rather than a bug report.

While your example query does expand/de-reference the type result of the specific "page query" correctly, it still doesn't fulfill the desired behavior that we're asking for.

I am never explictly fetching only ProgressBar, and therefore I can't access a de-referenced version of the type ProgressBar, making it impossible to create a generic component (as evankirkiles stated) for it.

I don't think that GROQ has to be changed to give the possibility of implementing a type-generation that (has the possibility to) expand/de-reference relations. evankirkiles gives a good example of a possible solution above.

[linusstaf]

We would very much like some way of resolving all the references during type generation. We have a pretty complex schema setup with a lot of references. For context, the generated types file is well over 9000 lines long.

Currently we are using the old archived sanity-codegen plugin which does provide this (in a way). For example a document that has a category referenced to it is generated as this in the plugin: category: SanityReference<Category>. It still does require some TS type magic to resolve the referred type though.

Being able to generate some form of lookup table, or using generics like the old plugin does is basically required for us to move over to TypeGen. You don't need to have that many types for it to be unsustainable to manage manually, and we have a lot of them.

[nikmatswace]

@sgulseth , do you have any kind of update here? It looks like we are quite a few that would very much like this functionality added to the sanity typegen.

[sgulseth]

Hi @nikmatswace - we are in the process of scoping the next work for typegen. However, we should have some outlines shortly.

Would returning a type like:

asset: {
      _ref: string
      _type: 'reference'
      _weak?: boolean
      [internalGroqTypeReferenceTo]?: SanityImageAsset
    }

Be better than having a map? Trying to think of pros/cons, and we don't want to be stuck in a corner 🤔

[nikmatswace]

I think it would be step in the right direction! With the above implementation and some custom code to resolve the internalGroqTypeReference I think it would be possible to achieve the desired behavior. But I would have to spend some time to play around with it to verify it.

My primary wish would be to be able to generate the types and add some kind of flag to automatically dereference any references in the generated types though. Meaning something like:

asset: {
      _ref: string
      _type: 'reference'
      _weak?: boolean
      ...whateverPropsSanityImageAssetHas
    }

[sgulseth]

The goal with typegen is to able to introspect groq queries and return a conforming type for what the query would return. Automatically dereferencing the types would break that goal.
The idea behind internalGroqTypeReferenceTo was that it's something that could be consumed by third party libraries, ie a query builder to help generating types for each project.