Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional use QueueSubscribe for NATS events source #3131

Merged
merged 13 commits into from
May 21, 2024

Conversation

sysr9
Copy link
Contributor

@sysr9 sysr9 commented May 13, 2024

resolves #2936

Checklist:

adds queue parameter to NATS EventSource to implement queue-based architecture

@sysr9 sysr9 requested a review from whynowy as a code owner May 13, 2024 19:05
@sysr9 sysr9 changed the title Use QueueSubscribe for NATS events source Optional use QueueSubscribe for NATS events source May 13, 2024
@praveenperera
Copy link

Wouldn't it make more sense to make this the default, and switch the active-active failover?

@sysr9
Copy link
Contributor Author

sysr9 commented May 14, 2024

Wouldn't it make more sense to make this the default, and switch the active-active failover?

in some cases this may cause unpredictable effects

  • queued subscription may use more resources on NATS server then regular one
  • messages can be duplicated in case of mixed setup - queued and regulars
  • queued subscription is an overhead when you have single producer and single consumer

@praveenperera
Copy link

@sysr9 makes sense, if we are using the QueueGroup how can we make it use active-active HA mode? Or should that be a different PR?

@sysr9
Copy link
Contributor Author

sysr9 commented May 15, 2024

@praveenperera

  • use multiple replicas in same group for active-active failover
  • use multiple event sources in multiple groups for message duplication into this groups

in my setup i have global NATS bus and multiple clusters with argo events deployed, now i need global events for all clusters

// Queue is the name of the queue group to subscribe as if specified. Uses QueueSubscribe
// logic to subscribe as queue group. If the queue is empty, uses default Subscribe logic.
// +optional
Queue string `json:"queue" protobuf:"bytes,9,opt,name=queue"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use *string, all the rest looks good to me, thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switched to *string

sysr9 and others added 12 commits May 21, 2024 13:27
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
…1.0 (argoproj#3138)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
…rgoproj#3139)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
…proj#3140)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
…goproj#3141)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
…goproj#3113)

Signed-off-by: florian-feicht <florian.feicht@adesso-afida.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Signed-off-by: sysr9 <sysr999@gmail.com>
Copy link
Member

@whynowy whynowy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@whynowy whynowy merged commit 7ee9f8b into argoproj:master May 21, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use QueueSubscribe for NATS events source?
4 participants