Support for Querying using UDF Type

[adarsh2397]

Goal and Motivation

To enable the following functionality:

SELECT id, ObjectRecognition(data) FROM MyMovies;

Which will automatically select the corresponding UDF that can perform Object Recognition from the list of created UDFs. The motivation behind this is to enable users to be able to just specify the task instead of selecting a specific UDF to run. Additionally, by controlling the selection of the UDF ourselves, we would be able to focus on carefully selecting the best UDF considering the available resources, input and task requirements.

Assumption: All UDFs of the same type should have the same set of inputs and outputs.

Implementation

We are already storing the TYPE attribute while creating the UDF and this attribute is currently not being used anywhere. We can reuse this attribute to store the type of task the UDF can be used for (eg. FaceRecognition, ObjectDetection, EmotionClassification etc).

Sample Query:

CREATE UDF IF NOT EXISTS FastRCNNObjectDetector
      INPUT  (Frame_Array NDARRAY UINT8(3, ANYDIM, ANYDIM))
      OUTPUT (labels NDARRAY STR(ANYDIM), bboxes NDARRAY FLOAT32(ANYDIM, 4),
                scores NDARRAY FLOAT32(ANYDIM))
      TYPE  ObjectDetection
      IMPL  '{}/udfs/fastrcnn_object_detector.py';

When a query such as the following:

SELECT id, ObjectRecognition(data) FROM MyMovies;

is triggered, we will first try to match the name of the UDF (ObjectRecognition) to (in order):

1. An existing UDF that matches the name
2. Any UDF whose type matches the name passed

Steps:

1. If there is a match for (1), we would load the UDF with the given name and proceed as currently exists
2. Else, if there is a match for (2):
   2.1. In statement_binder.py, we would set expr.function to None and expr.function_type to the passed name.
   2.2. During Logical Plan to Physical Plan conversion, for each FunctionExpression encountered, we will check if expr.function is None and if true, we will load any UDF that matches the type set in expr.function_type

For Reference: Check for changes in statement_binder.py, rules.py and function_expression.py in PR#441

Pending Tasks

• [] Add new test cases that support this scenario
• [] Decide on how to select the UDF by type. Keep it random for now?

[gaurav274]

@jarulraj @xzdandy @pchunduri6 @kaushikravichandran @LordDarkula For visibility

[kaushikravichandran]

If there is an match for (1), we would try and load any UDF that belongs to this type (ObjectRecognition)
If there is no match for (1), we would then try to search for a UDF with that name (ObjectRecognition)

Isn't this a bit confusing? Should we make the differentiation b/w ObjectRecognition being a UDF or a TYPE more explicit?

[gaurav274]

I think the idea is to allow users to assign arbitrary TYPEs to the UDFs.
IMO, the order should be (2) -> (1). We should give preference to the UDF name. What do you think?

[adarsh2397]

That's another decision we have to take I guess. Should the TYPEs be arbitrary that any user can decide as they setup the UDFs or should we provide the TYPEs that can be set. This list can be a growing list and always have an 'UNSPECIFIED' type for any non-supported ones.

Since the number of TYPEs can be really big, it might make sense to make it arbitrary. But if we do this, we would lose control over the TYPE and any future optimizations we could apply based on the TYPE of model or task.

@kaushikravichandran So that's between choosing method (1) and (2). We can make the distinction more explicit by adding the TYPEs at the parsing level (if we don't use arbitrary TYPEs). I am not that much familiar with parsers but since this list can grow to be quite big, would this be a problem here?

@gaurav274 In terms of (1) vs (2), I guess what I was thinking is block the creation of UDF with a name that matches a TYPE. That way (1) -> (2) or (2) -> (1) would yield the same result. I am good with both methods.

[gaurav274]

The ultimate idea is to let users specify the relationship between models. Automating it won't be easy because different models will have different labels. So even though the two models are of object detection type, they might differ in output labels. Model A's labels can be a subset of model B's labels.

[gaurav274]

I guess we are moving toward the idea of relational hints. To simplify things, we can make some assumptions as of now.

[adarsh2397]

Got it. Then I guess we go ahead with method 2 with the modification that instead of storing as list of Enums, we check the catalog itself directly.
Also, shall we keep a check in CREATE UDF to ensure that no UDF Name clashes with a TYPE and vice versa. This can help reduce some amount of ambiguity during usage.

[gaurav274]

We wouldn't need to keep track of enums. Instead, the TYPE information already persists in the catalog when the UDF is created. We can fetch the UDFs based on a specific TYPE from the catalog.

[LordDarkula]

Could you put some more detail into the actual design. What is the motivation for this? Why is it beneficial for the user to specify a UDF type rather than a specific UDF? Perhaps a concrete example could clear this up.

[LordDarkula]

Maybe rename it to AutomaticUDF or something like that. Basically the value provided to the user is to allow them to use EVA without having to choose a UDF and that should be obvious.

[adarsh2397]

@LordDarkula I will update this with more information as we go along. Currently I have summarized the future scope in just one line to keep it open.

The benefit is mainly when someone wants to run some task, say ObjectDetection, but isn't bothered about the model that does this for them. Now that we have the control to choose the model from within the subset of ObjectDetection models, we can select the best model to execute based on factors like accuracy and error requirements, size of input, available resources (eg. CPU vs GPU) etc. But this part is for the future scope.

For example: Let us say two users run this query on different setups: User 1 with a GPU, User 2 without a GPU.

SELECT ObjectDetection(data) FROM MyMovies;

Maybe for User 1, we could pick a larger model with good accuracy that would execute in <5s using the power of GPUs. But for User 2, we would pick a smaller model achieving almost similar accuracy in <5s using CPUs only.

I like the idea of naming them as Auto<something>, like AutoObjectDetection or AutoFaceRecognition. So maybe we utilize this approach only when the query is of the form:

SELECT Auto<model_type>(data) FROM MyMovies;

The only hiccup I see in this is what if a model already exists start with the term Auto.

[gaurav274]

Isn't ObjectDetection obvious? I mean, AutoObjectDetection sounds weird to me.
Further, I strongly advocate supporting arbitrary TYPE names (Else it will be difficult for us later)

[adarsh2397]

Yes. If we are going ahead with arbitrary TYPE names then having something like AutoObjectDetection isn't necessary at all. We'll have the catalog anyways to lookup for what the token refers to.

[xzdandy]

I think the choosing the physical UDF for a logical UDF should be a rule in the optimizer instead of during query parsing or query binding.
On the other hand, we do need to be careful about the alias assigned in the binder.

[adarsh2397]

Agreed on this. We might need to store the name of the logical type in the FunctionExpression as well as some flag to indicate that we would be resolving this UDF in the optimizer.

[adarsh2397]

Have a rough/draft PR that works with LATERAL JOIN query using the Logical Type.
#441

As mentioned above, we might need to decide on:

1. Storing some flag and type in FunctionExpression during Statement Binding to be used at a later stage
2. Add a simple / dummy method to choose the best UDF possible. Currently can be set to the smallest UDF, alter based on optimizations later on.
3. Add a disclaimer that all UDFs with similar Logical Types have same Output and Input types

Will add this shortly in the Discussion also

[adarsh2397]

Updated Discussion with the new changes in the PR