Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Request] Add livedata MQTT topic #1845

Open
phol opened this issue Mar 19, 2024 · 0 comments
Open

[Request] Add livedata MQTT topic #1845

phol opened this issue Mar 19, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@phol
Copy link

phol commented Mar 19, 2024

Is your feature request related to a problem? Please describe.

I'm logging the data produced by OpenDTU into a database using a small tool I wrote and open sourced yesterday, opendtu-logger. To do this, I'm using the websocket, as it contains a data structure more suitable for me than the MQTT topics. Whereas currently, the MQTT implementation is key-value based, I prefer to store historical data into normalised SQL tables as this can be much more efficient.

I initially built a proof-of-concept OpenDTU Logger using MQTT, storing data as "timestamp", "topic", "value". This works but is very inefficient.
With my new tool, I can store data as "timestamp", "value1", "value2", "valueN...". This is much more efficient in terms of storage and also makes querying data much more convenient.

OpenDTU Logger currently subscribes to the OpenDTU websocket as a client. However, this approach has a few drawbacks:

  • It requires the tool to run within the same network as the OpenDTU.
  • If the IP address of the OpenDTU changes, the opendtu-logger program needs to be reconfigured as well.
  • OpenDTU Logger is restarted whenever OpenDTU is restarted as the websocket will suddenly drop its connection.

Describe the solution you'd like

I'd like to request the addition of a new MQTT topic, dtu/livedata, which I could use instead of the websocket.

This would fix all 3 listed drawbacks.

To this topic, the OpenDTU would only have to push the exact same contents as it currently pushes to the websocket.
I don't mind if in a future release, the structure would change, as my tool is specifically developed to work with OpenDTU and can be updated again if required.

Describe alternatives you've considered

An alternative I considered was to request the addition of an "offline" notification in the websocket, but a new MQTT topic seems like a better solution to me as it would be more robust and also cover cases such as sudden power loss.

An added benefit of the MQTT topic is that future OpenDTU GUIs could be run standalone, on a different network, and could show data based on a MQTT subscription as Vue supports MQTT over websockets. With a livedata MQTT topic, the ESP32 is no longer the limiting factor in terms of the maximum number of clients supported, as much of the load is shifted to the MQTT server.

Another alternative I considered was to request including the possibility to connect OpenDTU to my logger using a WireGuard connection, but this would basically be a giant workaround. Moreover, a pull request to include WireGuard was denied, so I think the approach above would be better.

Additional context

OpenDTU Logger is available in a git repository: OpenDTU logger
To explain a bit of background and my motivation to write this tool, I wrote a blog post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant