You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We propose adding a new API methods in the client for allowing users to create and manage atomic counters associated to a stream. As an example of when this could be useful, in some processing scenarios, there could be processing tasks that use as input the output of other tasks (i.e., pipelines). If we think in this scenario in the context of Pravega, there would be writer and reader tasks writing and consuming data from a stream. However, given that we do not necessarily refer to stream processing tasks, readers may need to know when writers have completed their work so they can stop reading further events from a stream and complete their processing. In this scenario, having a "counter" that writers and readers could update and check, would be interesting to provide a simple way of synchronizing groups of tasks. Note that this is just an example, but there could be other cases in which atomic counters could be useful as a client-facing primitive.
Describe the solution you'd like
Segment Attributes can offer multiple types of updates at the Segment Store level. Once of them is the Accumulate update type, which adds the value passed in the WireCommand to the one existing in the server, if any. This primitive is exactly what we need to implement a client-facing atomic counter with minimal changes at the server side.
Describe alternatives you've considered
Certainly, the proposed feature could be implemented using the StateSychronizer. However, thanks to the attribute update types, we could implement counters in a much simpler way, both for the client and the server perspective. For simplicity, just exposing attribute update types at the WireCommand level solves the problem, and opens up for adding more attribute update types that could be automatically exposed as well.
Additional context
The need for this feature appeared when doing experiments with Pravega and serverless frameworks.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
We propose adding a new API methods in the client for allowing users to create and manage atomic counters associated to a stream. As an example of when this could be useful, in some processing scenarios, there could be processing tasks that use as input the output of other tasks (i.e., pipelines). If we think in this scenario in the context of Pravega, there would be writer and reader tasks writing and consuming data from a stream. However, given that we do not necessarily refer to stream processing tasks, readers may need to know when writers have completed their work so they can stop reading further events from a stream and complete their processing. In this scenario, having a "counter" that writers and readers could update and check, would be interesting to provide a simple way of synchronizing groups of tasks. Note that this is just an example, but there could be other cases in which atomic counters could be useful as a client-facing primitive.
Describe the solution you'd like
Segment Attributes can offer multiple types of updates at the Segment Store level. Once of them is the
Accumulate
update type, which adds the value passed in theWireCommand
to the one existing in the server, if any. This primitive is exactly what we need to implement a client-facing atomic counter with minimal changes at the server side.Describe alternatives you've considered
Certainly, the proposed feature could be implemented using the
StateSychronizer
. However, thanks to the attribute update types, we could implement counters in a much simpler way, both for the client and the server perspective. For simplicity, just exposing attribute update types at theWireCommand
level solves the problem, and opens up for adding more attribute update types that could be automatically exposed as well.Additional context
The need for this feature appeared when doing experiments with Pravega and serverless frameworks.
The text was updated successfully, but these errors were encountered: