data (batch) loading / caching (N+1)

[itamar82]

Anybody have good libraries/patterns for dealing with fetching data from remote sources like DBs to reduce number of queries?

example:

type Cart {
  id: ID!
  userId: Int
  items: [CartItem!]!
}

type CartItem {
  id: ID!
  sku: String
  price: Int
  quantity: Int
}

type Query {
  getUserCarts(userId: Int): [Cart!]!
}

In a suboptimal approach to implementing getUserCarts this would query the database for all carts, then an additional query for each cart to query cart items.

Preferred approach would be to collect all the Cart IDs and then batch load all the cart items in one query...

Anyone implement something like this for Ariadne?

[rafalp]

This isn't documented (yet!) but its well supported by Ariadne. Only caveat is that this requires your server to be running using ASGI instead of WSGI, so you are having async query executor.

Here's python port of dataloader library that is confirmed to work with Ariadne.

Example abstraction on top of it could look like this:

from aiodataloader import DataLoader

async def get_users_by_id_from_db(ids: Sequence[int]):
    data = {u.id: u for u in await some_orm_db_query(id__in=ids)}
    return [data.get(i) for i in ids]  # this is because dataloader expects sequence of objects or None following order of ids in ids 


CONTEXT_KEY = "__users_loader"

def load_user_by_id(info, id: str | int) -> Optional[User]:
    if CONTEXT_KEY not in info.context:
        info.context[CONTEXT_KEY] = DataLoader(get_users_by_id_from_db)

    return info.context[CONTEXT_KEY].load(id)


# GraphQL resolver
def resolve_user(_, info, *, id: str):
    return load_user_by_id(info, int(id))

[sMorm]

Is there an elegant solution like this for WSGI apps like Flask?

[maxmorlocke]

I'm by no means recommending you do this, but for WSGI apps, the best solution I've thought of is to do something where you have a top level resolver that handles all the queries/prefetching, and then have field resolvers that just access the prefetched data.

[sMorm]

I see. That can probably be done through root value. Just curious though, do we know why the support for promise was removed in graphql-core v3?

[rafalp]

It wasn't. v3 started out as fresh blank-sheet Python port of graphql.js and from start aimed to support only then new Python 3 features like asyncio.

[pylipp]

@sMorm how was your experience using the promise library (or another sync one) with ariadne and Flask?

[dorianamouroux]

In the GraphQL Ruby implementation, they have a concept of "lookahead": https://graphql-ruby.org/queries/lookahead.html#using-a-lookahead It's a helper function to know if a child is requested in the type, at the resolver level.

I think it would be a good option for some cases, for example in your case, assuming you are using Django you could do something like:

# resolver
def resolve_get_user_carts(_, info):
    queryset = Card.objects.get(pk=user_id)
    if has_child_field(info, "items", user_id):
        queryset = queryset.select_related('items') # or prefetch_related
    return queryset

I believe has_child_field doesn't necessarily need to be part of Ariadne, as info is a GraphQLResolveInfo which is part of GraphQL-core, and I think contains everything we need to implement that function, but I didn't get to dive into it yet.

[maxmorlocke]

Django's prefetch & select_related tools work really nicely here, allowing us to minimize queries quite efficiently:

from django.db.models import Prefetch

def resolve_get_user_carts(object, info):
    return info.context["request"].user.cart_set.prefetch_related(
        Prefetch(
            'cartitem',
            queryset=CartItem.objects.select_related(
                'manufacturer',
            ),
        ),
    )

[dorianamouroux]

However, I don't think we want to call prefetch_relatec or selected_related when the cartitem / manufacturer isn't requested by the client, hence the has_child_field function which could be a fair solution to optimize

[maxmorlocke]

I very much agree. I'm looking forward to adding that to our prefetch/select related queries :) Your point is entirely valid, I just wanted to make sure folks were aware of the expanded capabilities of prefetch in the django orm.

[howbazaar]

💩 came looking for the answer to this, but only found out that it isn't really implemented yet

[rafalp]

What do you mean by "it's not really implemented yet"? Batching is already supported for both async and sync servers: https://ariadnegraphql.org/docs/dataloaders#sync-dataloader

[howbazaar]

I did not know about this, but it doesn't solve my issue. I needed to look ahead into the nested fields to determine behaviour of a pager.

[howbazaar]

My particular problem is that we have a resolver that is returning a Connection type object, using the Cursor spec, so it has edges and nodes, so the query is something like this:

{ policies { edges { nodes { name }}}

and I'm in the resolver for the policies query, and I'm wanting to know if complexThing is selected as a field of the node.

[howbazaar]

Crazy as this may seem, this works:

from graphql import FieldNode, GraphQLResolveInfo, SelectionNode, SelectionSetNode


def _find_field(name: str, fields: Iterable[FieldNode]) -> Optional[FieldNode]:
    for node in fields:
        if node.name.value == name:
            return node
    # Couldn't find the node.
    return None


def _get_field_nodes(fields: Iterable[SelectionNode]) -> list[FieldNode]:
    return [node for node in fields if isinstance(node, FieldNode)]


def look_ahead_fields(info: GraphQLResolveInfo, *names: str) -> list[str]:
    # There is always one `field_nodes` and it is the AST of the current field.
    return [
        node.name.value
        for node in _look_ahead_fields(info.field_nodes[0].selection_set, names)
    ]


def look_ahead_field(info: GraphQLResolveInfo, *names: str) -> Optional[FieldNode]:
    look_ahead = cast(tuple[str], names[:-1])
    if look_ahead:
        fields = _look_ahead_fields(info.field_nodes[0].selection_set, look_ahead)
    else:
        selection_set = info.field_nodes[0].selection_set
        if selection_set is None:
            return None
        fields = _get_field_nodes(selection_set.selections)
    return _find_field(names[-1], fields)


def _look_ahead_fields(
    fields: Optional[SelectionSetNode], names: tuple[str]
) -> list[FieldNode]:
    if not names or not fields:
        return []
    field_name = names[0]
    field = _find_field(field_name, _get_field_nodes(fields.selections))
    if not field or field.selection_set is None:
        return []
    # If there are more names, walk down, otherwise return the names of the fields.
    if len(names) > 1:
        return _look_ahead_fields(field.selection_set, cast(tuple[str], names[1:]))
    return _get_field_nodes(field.selection_set.selections)

If I'm in the resolver for policies I can now say:
look_ahead_fields(info, "edges") and get ['node']
look_ahead_fields(info, "edges", "node") and get ['name', 'resourceMatchCount']

and

look_ahead_field(info, "edges", "node", "resourceMatchCount") and get the FieldNode object that would allow me to look at any args (which I need to do).

[vollossy]

Not 100% sure that this discussion is suitable, but will try to post my question here.
What is correct way for solving one to many relations with dataloader in context of rdbms?
Here is example. I have 2 tables:

create table authors(name text primary key);
create table books(title text primary key, author text not null references authors(name));

And my graphql schema will be like:

type Query{
    authors: [Author!]!
}

type Author{
    name: String!
    books: [Book!]!
}

type Book{
    title: String!
}

Ariadne doc says:

DataLoader.load() takes id of an object to load, and returns awaitable for this object or None

So, according to this, to retrieve all books for author with dataloader, I must first retrieve book titles:

async def get_books(titles: list[str]) -> list[Book]:
    return order_books_according_to_titles(await fetch_books_by_titles(titles), titles)

Or it is ok to use dataloader with author name for books fetch? Something like this:

async def get_books(author_names: list[str]) -> list[Book]:
    return order_books_according_to_author_names(await fetch_books_by_author_names(author_names), author_names)

[pylipp]

@vollossy your second approach is totally fine. DataLoader.load can take anything needed for a lookup.

I.e. in our app we have the SizeRange.sizes resolver with this loader definition.

This SO post explains it, too.

[vollossy]

Great, thanks for clarifying. Maybe, this case should be added to project docs, what do you think?

[pylipp]

Not sure, it's something to be mentioned in the docs of the batch-loading packages rather than in ariadne IMHO