Types for find, update etc. for a generic repository?

[norahmaria]

I'm having a little bit of an issue with creating a generic repository - primarily with the types to allow the correct inputs when there are refs involved.

I have my schema, that uses a couple references.

@modelOptions({ schemaOptions: { collection: "ticket" } })
export class TicketSchema {
  _id!: Mongoose.ObjectId;

  @prop({ ref: () => UserSchema })
  public creator!: Ref<UserSchema>;

  @prop({ ref: () => UserSchema })
  public responsible?: Ref<UserSchema>;

  @prop({ ref: () => GuildSchema })
  public guild!: Ref<GuildSchema>;

  @prop()
  opened!: Date;

  @prop()
  message!: string;
}

My interface used in every other repository (Typegoose is only used in a shared package that exports the db repositories). This is also used client side which is why there is no use of the ObjectId type.

export interface Ticket {
  _id: string;

  creator: User | string;
  guild: Guild | string;
  responsible?: User | string;

  opened: Date;
  message: string;
}

And I'm trying to figure out the types for generic functions like find and update etc. Below is what I have right now, I also earlier had just a version were it accepted Partial<U>, U being the interface I use for the data in other repositories for the project, so in this example Ticket - but then mongoose itself complained about findOne.

export type Schema<T> = Typegoose.AnyParamConstructor<T>;

export class Repository<U, S> {
  model: ReturnModelType<Schema<S>, Typegoose.BeAnObject>;

  constructor(readonly entity: Schema<S>) {
    this.model = getModelForClass(entity);
  }

  async create(input: Omit<WithoutId<U>, "date">) {
    return this.model.create(input);
  }
  
  findOne(filter: Partial<DocumentType<U>>) {
    return this.model.findOne(filter);
  }
 }

The TicketRepository then extends this with the types <Ticket, TicketSchema> from above.

So I'm using DocumentType, but when I then try to use find for example - none of the inputs I'd like are accepted. For example when I fetch a user, and try to pass it in as the creator in the filter for findOne, it complains:

Type '{ _id: string; __v?: any; $locals: Record<string, unknown>; $op: "remove" | "save" | "validate" | null; $where: Record<string, unknown>; baseModelName?: string | undefined; collection: Collection<...>; ... 7 more ...; typegooseName(): string; }' is not assignable to type 'string | User | undefined'.
  Type '{ _id: string; __v?: any; $locals: Record<string, unknown>; $op: "remove" | "save" | "validate" | null; $where: Record<string, unknown>; baseModelName?: string | undefined; collection: Collection<...>; ... 7 more ...; typegooseName(): string; }' is not assignable to type 'User'.
      Types of property 'memberships' are incompatible.
      Type 'Ref<GuildMemberSchema>[]' is not assignable to type '(string | GuildMember)[]'.
        Type 'Ref<GuildMemberSchema>' is not assignable to type 'string | GuildMember'.
          Type 'ObjectId' is not assignable to type 'string | GuildMember'.
            Type 'ObjectId' is missing the following properties from type 'GuildMember': discordId, guild, warnings, notes, and 2 more.ts(2322)
            
Ticket.d.ts(5, 5): The expected type comes from property 'creator' which is declared here on type 'Partial<DocumentType<Ticket>>'

But if I try to pass in just the ID, it's also not happy.

Most times when I have a reference, I'd like to just pass the id (as a string or an ObjectId, as mongoose are ok with both - and same for reference arrays, I'd usually just pass in (string | ObjectId)[]).

Or, in the example case - pass in the reference directly if it's already fetched from the db in the same code.

However, I'm not able to find a utility type for this? It seems DocumentType<U> is not the type to use as the input/filter types. I'd expect a type that allows any reference (+ nested refs) to have the type Ref<REF_TYPE> | REF_TYPE | string | ObjectId, as those are all valid inputs in mongoose for a reference.

How do I get the type of just a valid input or valid filter? I'm sure I'm just missing something very obvious, any help appreciated! ☺️

[hasezoey]

i dont know if this is actually related to the error, but Ref (and by extension mongoose's PopulatedDoc) work by only allowing undefined | REF_TYPE | REF_ID_TYPE where REF_TYPE would be the class (like GuildMemberSchema) and REF_ID_TYPE the id type (by default ObjectId), so only one of them can be used at the same time (meaning you cannot use string | ObjectId, only one at a time)

also i am not quite sure what your generics on Repository are (or what Schema<> is), so you likely have a conflict there

[hasezoey]

Oh, andSchema is just a shorthand, I updated the post with it and what types I'm extending the repository base with ☺️

i still dont quite know what U and S are supposed to mean, could you explain? from my quick guessing S is meant to be the class, but what is U? and if it is the class, then you likely also want to use the same in DocumentType<S>

[norahmaria]

Sorry this is long

Alright, I've put all the input types on the repository functions to Partial<DocumentType<S>>, as you're right about it needing the class, rather than the interface. So that solves the issue of not being able to put a fetched entity from the db as the input for a ref field, my bad. Thanks!

However, it still won't let me pass ObjectId instead of a full reference. And, the <U> is the interface for the data actually used in every other repository (front end etc), for example the GuildMember interface:

import type { Guild } from "./Guild.js";
import type { Note } from "./Note.js";
import type { Ticket } from "./Ticket.js";
import type { User } from "./User.js";
import type { Warning } from "./Warning.js";

interface Action {
  date: Date;
  reason?: string;
  type: "Kick" | "Mute" | "Deafen" | "Timeout" | "Ban";
  caller: User | string;
}

export interface GuildMember {
  _id: string;

  discordId: string;

  guild: Guild | string;

  warnings: (Warning | string)[];
  notes: (Note | string)[];
  tickets: (Ticket | string)[];

  actions: Action[];
}

Then the GuildMember typegoose definition

import { type Ref, prop, modelOptions } from "@typegoose/typegoose";

import { GuildSchema } from "./GuildSchema.js";
import { WarningSchema } from "./WarningSchema.js";
import { NoteSchema } from "./NoteSchema.js";
import { TicketSchema } from "./TicketSchema.js";
import { UserSchema } from "./UserSchema.js";

import type * as Mongoose from "mongoose";

export class ActionSchema {
  @prop()
  date!: Date;

  @prop()
  reason?: string;

  @prop()
  type!: "kick" | "mute" | "deafen" | "timeout" | "ban";

  @prop({ ref: () => UserSchema })
  caller!: Ref<UserSchema>;
}

@modelOptions({ schemaOptions: { collection: "guild-member" } })
export class GuildMemberSchema {
  _id!: Mongoose.ObjectId;

  @prop()
  public discordId!: string;

  @prop({ ref: () => GuildSchema })
  guild!: Ref<GuildSchema>;

  @prop({ ref: () => WarningSchema, default: [] })
  warnings!: Ref<WarningSchema>[];

  @prop({ ref: () => NoteSchema, default: [] })
  notes!: Ref<NoteSchema>[];

  @prop({ ref: () => TicketSchema, default: [] })
  tickets!: Ref<TicketSchema>[];

  @prop({ type: () => [ActionSchema], default: [] })
  actions!: ActionSchema[];
}

Then the repo, plus an example of an error when trying to put in the objectId for the reference.

import { ObjectId, type WithoutId } from "mongodb";
import { type GuildMember } from "@maev/types";

import { Repository } from "~/Repository.js";
import { GuildMemberSchema } from "~/schemas/index.js";

export class GuildMemberDb extends Repository<GuildMember, GuildMemberSchema> {
  constructor() {
    super(GuildMemberSchema);
  }

  async findByDiscordIdInGuild(guildId: ObjectId, discordId: string) {
   // findOne expects type Partial<DocumentType<GuildMemberSchema>>
    return this.findOne({
      guild: guildId,
      discordId,
    });
  }
}

In findByDiscordIdInGuild, when passing ObjectId as guild, it complains:

Type 'ObjectId' is not assignable to type 'Ref<GuildSchema> | undefined'.
  Property '_id' is missing in type 
  
GuildMemberSchema.ts(35, 3): The expected type comes from property 'guild' which is declared here on type 'Partial<DocumentType<GuildMemberSchema>>'

And then I thought maybe I just need to pass in { _id: guildId } to the guild, so I did but then, even more confusing it errors out:

Type 'ObjectId' is missing the following properties from type 'ObjectId': auto, defaultOptions, OptionsConstructor, cast, and 18 more.ts(2740)

Does typegoose and mongoose not use the same type for ObjectId? Thanks again for the help! ☺️

[norahmaria]

I now tried the type Mongoose.ObjectId as I notice it seems it differs from the directly imported ObjectId.

import type * as Mongoose from "mongoose";

In which, I again get the error:

Type '{ _id: Mongoose.Schema.Types.ObjectId; }' is not assignable to type 'Ref<GuildSchema> | undefined'.

Or when not putting it in an object:

Type 'ObjectId' is not assignable to type 'Ref<GuildSchema> | undefined'.
  Type 'ObjectId' is not assignable to type 'DocumentType<GuildSchema, BeAnObject>'.
    Type 'ObjectId' is missing the following properties from type 'Document<unknown, BeAnObject, GuildSchema>': $assertPopulated, $clone, $getAllSubdocs, $ignore, and 42 more

[norahmaria]

Using Mongoose.ObjectId would not be ideal tho, as then I can't do something like the example below, as there doesn't seem to be a way of creating a new Mongoose.ObjectId().

async findByDiscordIdInGuild(guildId: string | ObjectId, discordId: string) {
  return this.findOne({
    guild: new ObjectId(guildId),
    discordId,
  });
}

[hasezoey]

Type 'ObjectId' is not assignable to type 'Ref | undefined'.
Property '_id' is missing in type

this error is because mongoose modifies the objectid, which is (in older versions) not reflected in the the lower types and only in the mongoose's re-exported type (see #13515, fixed since 7.4.0), this issue essentially means you can use Mongoose.ObjectId to be passed into a thing expecting a mongodb.ObjectId, but not vise-versa

Type '{ _id: Mongoose.Schema.Types.ObjectId; }' is not assignable to type 'Ref | undefined'.

mongoose has 2 types for each (where there is not a instance type already), a SchemaType and a InstanceType, the schema type is what is used by mongoose internally to describe a type in a schema (example SchemaNumber), which could be used directly in the schema definition (example exampleProp: SchemaNumber), and a instance type is what actually holds the data (example Number), and mongoose automatically translates those instance types to schema types where necessary (example exampleProp: Number)

as there doesn't seem to be a way of creating a new Mongoose.ObjectId().

are you sure about that? mongoose at runtime just modifies (monkey-patches) the bson ObjectId, which makes bson.ObjectId, mongodb.ObjectId and mongoose.Types.ObjectId equivalent classes (same reference).

what is your error, and did you already try using mongoose.Types.ObjectId instead of mongoose.ObjectId? (if that makes it work, then it is likely a mongoose bug)