Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EVE-34] Process managers #176

Open
alexeyzimarev opened this issue Jan 12, 2023 · 6 comments · May be fixed by #177
Open

[EVE-34] Process managers #176

alexeyzimarev opened this issue Jan 12, 2023 · 6 comments · May be fixed by #177
Labels
enhancement New feature or request linear Sync the issue to Linear

Comments

@alexeyzimarev
Copy link
Contributor

alexeyzimarev commented Jan 12, 2023

I get lots of questions about process managers. Personally, I am not a big fan because:

  • Processes are indeed everywhere
  • Most processes are natural and not orchestrated
  • Orchestration adds complexity

But, for the sake of it, I can try implementing it by adding some syntactic sugar to the existing building blocs:

  • Aggregate -> Process
  • AggregateState -> ProcessState
  • ApplicationService -> ProcessManager

The manager needs to be called by a subscription. Possible by multiple subscriptions, but that can be done later. It also needs a producer to send commands. Here comes the question if I need to wire an existing app service that processes commands coming via HTTP also to consume command messages. It's possible even today, but maybe I can make it easier.

With a proper registration extensions, it should provide an easy to use API to wire all the things together.

EVE-34

@alexeyzimarev alexeyzimarev added the enhancement New feature or request label Jan 12, 2023
@alexeyzimarev alexeyzimarev linked a pull request Jan 12, 2023 that will close this issue
@alexeyzimarev alexeyzimarev added the linear Sync the issue to Linear label Mar 3, 2023
@alexeyzimarev alexeyzimarev changed the title Process managers [EVE-34] Process managers Mar 3, 2023
@diegosasw
Copy link

diegosasw commented Jun 26, 2023

I have a few questions about this.

Aren't process managers defined as "the opposite to an aggregate" as in

  • an aggregate receives commands and emits domain events
  • a process manager receives domain events and emits commands

or are they something else?

But, for the sake of it, I can try implementing it by adding some syntactic sugar to the existing building blocs

If Eventuous supports them as syntactic sugar, would its state be also event sourced? Because process managers don't usually have the technical limitation to be short lived, right?

Here comes the question if I need to wire an existing app service that processes commands coming via HTTP also to consume command messages.

To me, if there is a process manager is because there is no need for http commands, as the transaction is initiated automatically without the need of a UI. In other words, I see process managers as a replacement to UI and http communication.


For example, in a Restaurant domain where there are orders, waiters and each waiter wants to have a list of things that could do next each time it goes to the kitchen, this could be modelled with the following 3 aggregates.

Waiter producing events like

  • AvailableSet
  • UnavailableSet

and Order producing events like

  • OrderPrepared
  • OrderPicked (with waiter Id)
  • OrderDelivered

and WaiterTodoQueue, as a queue of tasks associated to a specific waiter, producing events like

  • OrderSuggestionAdded
  • OrderSuggestionRemoved

Customers create orders and someone prepares them. Waiters pick prepared orders.
A waiter goes to a read model (fed by WaiterTodoQueue) and picks one of the suggestions. When picked, this suggestion disappears from other waiters' todo queues. When a waiter delivers it, the order is delivered.

If I want a "process manager" called Dispatcher in charge of keeping waiters busy by suggesting to them orders to be picked, I would want this dispatcher to have state + behaviour (probably business logic).

So the Dispatcher would keep track of available waiters and orders prepared in order to dispatch/send commands to *WaiterTodoQueue, this Dispatcher is indeed something that receives events and emits commands (towards WaiterTodoQueue aggregate).

The problem is that the Dispatcher state, if event sourced, could grow rapidly and become slower unless it uses snapshots or similar.

Most processes are natural and not orchestrated
Does that mean that the Dispatcher in the example above could be modelled differently without using orchestration?

TL;DR: What's the definition of process manager? Is its state event sourced? Can things always be modelled without introducing orchestration (something that receives events and emits commands)?

@alexeyzimarev
Copy link
Contributor Author

What you describe as "aggregate" is basically a decider, which can be done using aggregates. Eventuous command services are quote close to that pattern.

Process managers, in general, receive messages and send messages. Whether the PM reacts to commands is discussable, but a process is usually started by a command anyway.

What I was looking in this issue initially is event-sourced state. As we have the state store now, the state object can be used in functional services, as well as in PMs. So, that's covered.

My assumption is that atm the only missing thing is multi-subscription support. Probably it's not even needed for the first iteration. But, definitely, it is needed later as I'd expect PMs to react on integration events, which might be emitted to different topics/streams. The question is more about the API rather than what's behind it. Doing AddSubscription(...).AddProcessManager<T>() and using the same instance of T across subscriptions is the easiest, and it will definitely work. I was thinking into a PM-first registration API like AddProcess<T>().SubscribeTo<T1, T2>().SubscribeTo<T3, T4>(), but it's much more work.

@gius
Copy link

gius commented Jun 7, 2024

There is one thing I don't get - let's assume that I need to conduct two aggregate types - A and B. Then I create a new instance of ProcessManager/Aggregate that handles the process, let's call it PM. How should I property identify instance of PM based on events from A and B that only contain AId, or BId respectively? Do I need to create a projection that allows finding PMId based on AId or BId?

In other words, how should I identify a particular PMId for handling an event coming from a particular A (thus containing just AId).

@bartelink
Copy link

bartelink commented Jun 7, 2024

As with anything else, you need to figure out some way to correlate the two sides
in some cases there's a parent-child relationship, and you don't need multiple instances - for those cases maybe the child id will sufffice

In the more general case, if you consider the process a workflow, you might mint a fresh identity for the instance

One benefit of this is that each instance of the process has an independent set of events and/or state and you don't get long streams. Of course that can potentially also be a problem if you have a need to coordinate the multiple running instances.

The you can make anything enlisted in it aware of same; either

  • by feeding the id of the flow into each side out of band/naturally
  • or when kicking off the process, you seed it with the ids of the participants, and make the first step to make those aware of their enlistment

(Will be interested to see/hear if Eventuous has any specific affordances wrt this, but for me process managers are just programming)

@gius
Copy link

gius commented Jun 7, 2024

You are right, especially with the parent-child relationship, you don't need another entity. My concern is exactly about the second case you mentioned - creating a new process manager instance with ids of its participants. What is the best way to notify the manager back when an event from any of its participants arrive.

The only idea I have is to create a projection that provides links from participantIds back to managerId and event subscription for all participant events that 'gateways' them to the respective owning manager. This should work (provided that there is not partitioning for events and thus participant or manager cannot get behind?) but just feels like a lot of boilerplate code for quite a typical case.

Feeding managerId to all participants feels a little bit like leaking abstraction, but might be appropriate in many cases, though.

Thank you for these ideas and brainstorming ;-)

@bartelink
Copy link

bartelink commented Jun 7, 2024

I would tend to have a reactor that sees all the events.
It should be handling the events in bulk across streams and forwarding them en masse
The question is how, based on solely the event, it determines the PM to route to either

  • the event carries enough context to let you route it
  • or you need to go back to the source to determine the PM to feed it to
    Once you know where to go, submitting an event to the PM is just a transaction against the workflow aggregate

(IIRC in there is a that sample in https://github.com/jet/dotnet-templates that does that, but the general principle applies regardless)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request linear Sync the issue to Linear
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants