Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Union streams #2275

Open
lpf4254302 opened this issue Sep 22, 2023 · 3 comments
Open

Union streams #2275

lpf4254302 opened this issue Sep 22, 2023 · 3 comments
Assignees
Labels
kind/feature New feature or request

Comments

@lpf4254302
Copy link
Contributor

What would you like to be added/modified:
Multi Topic Source
{"sql":"create stream lpf_stream () WITH ( datasource = "topic/t1,topic/t2,topic/t3", FORMAT = "json", TYPE="mqtt")"}
or
{
"id": "multi1",
"name": "multi1",
"graph": {
"nodes": {
"device1": {
"type": "source",
"nodeType": "httppush",
"props": {
"datasource": "/iot/alarm1"
}
},
"device2": {
"type": "source",
"nodeType": "httppush",
"props": {
"datasource": "/iot/alarm2"
}
},
"humidityFilter": {
"type": "operator",
"nodeType": "filter",
"props": {
"expr": "humidity > 30"
}
},
"logout1": {
"type": "sink",
"nodeType": "log"
}
},
"topo": {
"sources": [
"device1", "device2"
],
"edges": {
"device1": ["humidityFilter"],
"device2": ["humidityFilter"],
"humidityFilter": ["logout1"]
}
}
}
}
Why is this needed:
Wildcard mode allows you to subscribe to multiple topics, but sometimes multiple topics cannot be represented by wildcards

@ngjaying
Copy link
Collaborator

ngjaying commented Sep 22, 2023

Thank you for bringing this up. A general "merge" node may be needed to merge not only different topics of the same source but also any type of data sources into one. It is not a problem in graph API. Any idea if SQL has such similar syntax?

@ngjaying ngjaying added the kind/feature New feature or request label Sep 22, 2023
@lpf4254302
Copy link
Contributor Author

Thank you for bringing this up. A general "merge" node may be needed to merge not only different topics of the same source but also any type of data sources into one. It is not a problem in graph API. Any idea if SQL has such similar syntax?

sql:
(select * from root.sg1.t1) union all (select * from root.sg1.t2)
or
select * from root.sg1.t1,root.sg1.t2

@ngjaying ngjaying changed the title Multi Topic Source Union streams Dec 27, 2023
@ngjaying ngjaying added this to 1.14 Q2 in 2024 Roadmap Jan 4, 2024
@wfnuser
Copy link
Contributor

wfnuser commented Apr 8, 2024

I'd like to have a try on this issue. Can you assign it to me. 😊 @ngjaying

@ngjaying ngjaying moved this from 1.14 Q2 to Backlog in 2024 Roadmap May 23, 2024
@ngjaying ngjaying moved this from Backlog to 2.1 Q4 in 2024 Roadmap May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
Development

No branches or pull requests

3 participants