Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Coverage - Implement SINTER and SINTERSTORE #180

Closed
TalZaccai opened this issue Mar 28, 2024 · 13 comments · Fixed by #334
Closed

API Coverage - Implement SINTER and SINTERSTORE #180

TalZaccai opened this issue Mar 28, 2024 · 13 comments · Fixed by #334
Assignees
Labels
API good first issue Good for newcomers help wanted Extra attention is needed

Comments

@TalZaccai
Copy link
Contributor

Here's your opportunity to implement a couple of RESP commands in Garnet!

SINTER

Syntax:
SINTER key [key ...]

Command description:
Returns the members of the set resulting from the intersection of all the given sets.

For example:

key1 = {a,b,c,d}
key2 = {c}
key3 = {a,c,e}
SINTER key1 key2 key3 = {c}

Keys that do not exist are considered to be empty sets. With one of the keys being an empty set, the resulting set is also empty (since set intersection with an empty set always results in an empty set).

Response:

Array reply: a list with members of the resulting set.

SINTERSTORE

Syntax:
SINTERSTORE destination key [key ...]

Command description:
This command is equal to SINTER, but instead of returning the resulting set, it is stored in destination.

If destination already exists, it is overwritten.

Response:

Integer reply: the number of elements in the resulting set.


How to approach this task:

  1. Add the new command info to the setCommandsInfoMap in the RespCommandsInfo class (Garnet.server/Resp/RespCommandsInfo.cs)
  2. Add the new command string to the returned HashSet in the RespInfo.GetCommands method (Garnet.server/Resp/RespInfo.cs)
  3. Add the appropriate fast parsing logic for the new command in the RespCommand.FastParseArrayCommand method (use other commands as reference) (Garnet.server/Resp/RespCommand.cs)
  4. Implement the wrapper method to the storage layer in StorageSession (Garnet.server/Storage/Session/ObjectStore/SetOps.cs)
  5. Define a new method in IGarnetAPI and implement it in GarnetApiObjectCommands, calling the method you have implemented in step #4 (Garnet.server/API/IGarnetAPI.cs, Garnet.server/API/GarnetApiObjectCommands.cs)
  6. Add a new method to the RespServerSession class which will call the storage API and return the appropriate response (Garnet.server/Resp/Objects/SetCommands.cs)
  7. Add a new method to SetObjImpl which will perform the required operation on the hash object (Garnet.server/Objects/Set/SetObjectImpl.cs)
  8. Add a new enum value to SetOperation and add the appropriate case to the switch in SetObject.Operate, in which you will call the method defined in step #8 (Garnet.server/Objects/Set/SetObject.cs)
  9. Add an appropriate case to the switch in TransactionManager.SetObjectKeys (Garnet.server/Transaction/TxnKeyManager.cs)
  10. Add tests for the new command in the RespSetTests class (Garnet.test/RespSetTests.cs)

Tip: First add a simple test that calls the new command and use it to debug as you develop, it will make your life much easier!

If you have any questions, the Garnet team is here to help!

@TalZaccai TalZaccai added good first issue Good for newcomers help wanted Extra attention is needed API labels Mar 28, 2024
@jlao
Copy link

jlao commented Mar 30, 2024

I'm interested in taking a stab at this!

@TalZaccai
Copy link
Contributor Author

I'm interested in taking a stab at this!

Go ahead! LMK if you have any questions!

@MojtabaTajik
Copy link

MojtabaTajik commented Apr 8, 2024

@TalZaccai , can you please update the assignees? This will help others easily find the unpicked items.
If this one has not been picked up yet, I'm interested.

@badrishc
Copy link
Contributor

If more than one person wants to attempt an open work item, please go ahead. If a PR comes in quicker and meets the requirements, then we would be happy to merge it.

@jlao
Copy link

jlao commented Apr 11, 2024

Someone else can take this. I don't think I have time to tackle this in a timely manner right now.

@TalZaccai
Copy link
Contributor Author

@MojtabaTajik Let me know if you'd like to try and tackle this one!

@MojtabaTajik
Copy link

@TalZaccai yes, pls assign it to me.

@TalZaccai
Copy link
Contributor Author

@TalZaccai yes, pls assign it to me.

Done! :)

@badrishc
Copy link
Contributor

@MojtabaTajik - any update on this?

@MojtabaTajik
Copy link

@TalZaccai @badrishc

I just started working on it this week. Since it's my first contribution to Garnet, it's taking me more time to understand everything.

I've completed all the steps, but in step 7, there's a partial class called "SetObject" which contains a HashSet<byte[]> field named "set".
I've written the "SetIntersect" method there, allowing me to retrieve set names requested by the user (setsToIntersect variable). I also have the current set in the "set" field. However, I'm not sure how to retrieve other set members here and perform the intersection operation.

Am I proceeding correctly here?

        private void SetIntersect(byte* input, int length, ref SpanByteAndMemory output)
        {
            var _input = (ObjectInputHeader*)input;
            int numSets = _input->count;

            bool isMemory = false;
            MemoryHandle ptrHandle = default;
            
            // Parse the input to get the sets you want to intersect
            List<HashSet<byte[]>> setsToIntersect = new List<HashSet<byte[]>>();
            byte* inputCurrPtr = input + sizeof(ObjectInputHeader);
            for (int i = 0; i < numSets; i++)
            {
                if (!RespReadUtils.ReadByteArrayWithLengthHeader(out var setBytes, ref inputCurrPtr, input + length))
                    return; // Handle parsing error

                var tempSect = new HashSet<byte[]>() { setBytes };
                setsToIntersect.Add(tempSect);
            }

            // Perform the intersection
            var currentSet = set;
            
        }

@TalZaccai
Copy link
Contributor Author

@TalZaccai @badrishc

I just started working on it this week. Since it's my first contribution to Garnet, it's taking me more time to understand everything.

I've completed all the steps, but in step 7, there's a partial class called "SetObject" which contains a HashSet<byte[]> field named "set". I've written the "SetIntersect" method there, allowing me to retrieve set names requested by the user (setsToIntersect variable). I also have the current set in the "set" field. However, I'm not sure how to retrieve other set members here and perform the intersection operation.

Am I proceeding correctly here?

        private void SetIntersect(byte* input, int length, ref SpanByteAndMemory output)
        {
            var _input = (ObjectInputHeader*)input;
            int numSets = _input->count;

            bool isMemory = false;
            MemoryHandle ptrHandle = default;
            
            // Parse the input to get the sets you want to intersect
            List<HashSet<byte[]>> setsToIntersect = new List<HashSet<byte[]>>();
            byte* inputCurrPtr = input + sizeof(ObjectInputHeader);
            for (int i = 0; i < numSets; i++)
            {
                if (!RespReadUtils.ReadByteArrayWithLengthHeader(out var setBytes, ref inputCurrPtr, input + length))
                    return; // Handle parsing error

                var tempSect = new HashSet<byte[]>() { setBytes };
                setsToIntersect.Add(tempSect);
            }

            // Perform the intersection
            var currentSet = set;
            
        }

Hey @MojtabaTajik!

I understand your confusion, I think originally when I wrote these instructions they were meant for adding a single-object operation, so given your input I might update those for future work items :)

To your question, your parsing logic should go into SetCommands.cs and the main command logic would go into SetOps.cs, where you can obtain the appropriate locks on the relevant keys and you could call the StorageSession's GET and SET operations to read & write to & from the object store.

I would recommend that you review some similar commands that were implemented recently, such as LMOVE, SDIFF, SDIFFSTORE & SUNION for reference.

Thanks again for your question,
Don't hesitate to reach out again.

Tal.

lapkarol pushed a commit to lapkarol/garnet that referenced this issue Apr 26, 2024
@MojtabaTajik
Copy link

@TalZaccai
I was working on this on my weekends. I just saw a PR from Lapkarol that will implement them so you can use them.

@TalZaccai TalZaccai linked a pull request Apr 30, 2024 that will close this issue
@TalZaccai
Copy link
Contributor Author

@TalZaccai I was working on this on my weekends. I just saw a PR from Lapkarol that will implement them so you can use them.

Sounds good, I'll have a look at that new PR. Didn't know that someone else was working on it...

TalZaccai pushed a commit that referenced this issue May 20, 2024
* API Coverage - Implement SINTER and SINTERSTORE (#180)

* Fix forrmating

* Changes after review

* fix formating

---------

Co-authored-by: Karol Łapiński <karol.lapinski@softwarehut.com>
Co-authored-by: Karol <lapkarol.gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants