Stats4Lox mit Grafana

[killeriq]

Hallo, wie weit sind wir? Danke

[Christian Fenzl]

Wir machen gerade ganz was anderes!
Prof.Mobilux baut gerade das Musicserver-Gateway ins Squeezelite Plugin ein.
Und ich baue gerade ein neues Dach ;-)

[nickman]

Schade, ich hätte dafür auch Verwendung....
Rein aus Interesse: Wie wird denn das neue Dach automatisiert

[Christian Fenzl]

Da gibt's schon was zu automatisieren :-)
Wir bauen einen Teil des Dachbodens zu einer Galerie aus, mit mehreren Lichtkreisen, Bewegungsmelder, Fenster- und Türkontakten und Dachflächenfenster mit elektrischem Rollo, und natürlich eine neue Zone für's Multiroom-Audio.
Diese Galerie ist wie prädestiniert für den Dingz-Taster, den ich dort probieren möchte. Die API ist ja ein Hammer, ich bin gespannt, was ich da rauskitzeln kann :-)

[killeriq]

so bis ist vertig nur diese losung ist moglich...

https://www.loxwiki.eu/pages/viewpage.action?pageId=67371963

[t_heinrich]

killeriq thanks for the link that sounds fantastic. Is this your tutorial?
Please can you give a little bit more details for:

Download and install the above 3 packages. Follow the instructions for each. The standard installation options for all are recommended.

Thanks in advance.

Best Thomas

[killeriq]

man if you are not able to search and install those tools, dont even start. there are multi ways and depending what you have at home.
InfluxDB - a time series database.
Grafana - an open source analytics and monitoring solution which will read statistical data from InfluxDB.
ioBroker - an IoT platform designed for orchestrating data between home automation systems which can be used to pull values from Loxone and insert into InfluxDB.

[t_heinrich]

killeriq perhaps it's because English isn't my mother language but I don't see the connection between your two sentences:
"i guess all is well written there" and "if you are not able to search and install those tools, dont even start"

[killeriq]

well read it again, seems like others dont have YOUR problem

[svethi]

t_heinrich das geht auch mit node-red anstatt ioBroker. Wie es einem wohl besser gefällt. Ich kenne den ioBroker jetzt nicht, aber im Node-Red wird eine Websocket Verbindung zum MiniServer aufgebaut und alle Änderungen werden direkt an node-red übergeben. So braucht man nicht zu pollen und bekommt die Werte direkt bei Änderung

[svethi]

Okay, klar, wenn Du den ioBroker eh schon nutzt, macht das ja Sinn

[killeriq]

hab ich gestern uber die anleitung alles fertig gemacht im 15min...aber uber IOBroker ist zu vergleich zum NodeRed, zu viel extra arbeit:
jedes wert muss man extra einchecken und noch "alias ID" schreiben
So dann sieht NodeRed Losung besser aus - alles wurde zum InfluxDB/Grafana uber Stream ALL geschickt und dann kann ich auswahlen direkt im Dashboard.

[t_heinrich]

killeriq danke für die Info. Wenn es nur "einmaliger" Aufwand ist, würde ich den in Kauf nehmen, wenn ich dafür nicht noch was zusätzliches nutzen/installieren muss. Wenn es natürlich technisch deutlich besser wäre, es über Node-Red zu lösen, "müsste" ich mir das auch mal anschauen. ;-)

[Leo Kirch]

t_heinrich schau mal, hier gibt es ebenfalls eine hilfreiche Beschreibung in der Muttersprache :
https://www.verdrahtet.info/2020/02/14/installation-von-influxdb-grafana-inkl-verbindung-zu-iobroker/

[t_heinrich]

Vielen Dank, schau ich mir mal an.

[Gast]

Hi (I can read German, but do not feel confident enough to write...)
I tried several solutions mentioned in this thread (incl. ioBroker and Node Red). At the end of the day I have decided to dump all these solutions based on Websockets. There is an easier way to get your Loxone data into InfluxDB: UDP logger.

All you need is a python script, listening for these UDP messages and inserting the data into InfluxDB. No need for Node Red or ioBroker, because you can configure your measurements (incl. aliases, tags, periodic logging) directly in Loxone Config. Here is my solution, incl. tutorial:

https://github.com/budulinek/easy-loxone-influx

Sure, this is "only" UDP, but as long as you send the data within your own LAN and you use periodic logging for critical measurements, it is pretty reliable and robust.
Happy to hear (English or German) your comments or suggestions (for exampe how to improve the syntax of the UDP message).

[TomekWaw]

Wow, this is a great idea! I'm happy I've found out this post accidentally! Using ioBroker now but I feel this might be a better solution!

Hi Budulinek, I'm going to use your little server for the logging - should you ever need to connect via websockets again. I just completed a demo-python-prog that works: https://github.com/alxgross/Loxone_Websockets_Demo

Hi alx_g, thanks for the link.
I will not go back to websockets, I am very satisfied with UDP. Actually, I no longer use my python script (https://github.com/budulinek/easy-loxone-influx), because I realised that:
- InfluxDB has a built-in UDP listener, all you need is to enable it in /etc/influxdb/influxdb.conf
- The syntax of InfluxDB line protocol (https://docs.influxdata.com/influxdb...ocol_tutorial/) is very simple. You can actually make direct UDP (or HTTP/HTTPS) connection between Loxone and InfluxDB. No need for python scripts or other intermediaries. Just use Loxone "Status" block and "Virtual UDP output" to dynamically build your UDP message.

I did not have time to update github, but you can find a simple tutorial here (in Czech, please use Google Translate):
https://www.vodnici.net/community/di.../6/#post-24903

Of course, you need to establish few Virtual outputs and Status blocks, but it is OK if you do not have hundreds of measurements.

[svethi]

You are not right. In your case the phyton-Script have to pull the informations immediately. That causes a lot of cputime for nothing and you don’t get all informations which are available over Websocket. Each change of a value is automatically pushed over Websocket without a need to pull. It’s the simplest way.

[Gast]

The Python script is not pulling data from Loxone. It's a simple UDP listener. Loxone UDP logger pushes each change of a value automatically without a need to pull. So in this sense, Loxone Websocket API and Loxone UDP logger work the same (push each change of value). The difference is in reliability:

• Loxone UDP logger is doing "push and forget"
• Loxone WS is doing "push and wait for ACK, if you do not receive ACK, try to push again..."

IMHO, within your own LAN, UDP logger ("push and forget") is reliable enough, especially for recording analog values which change frequently. But if you really worry about UDP packets getting lost in the labyrinth of your LAN, you can implement periodic logging (see my tutorial). But it is Loxone (not the python script) who is responsible for periodic logging. With this solution, Loxone UDP logger is doing both "push each value change" and "push after time period".

You are right in your second point (amount of information available via UDP and via WS). UDP logger sends only measurement name + value. On the other side, websocket sends more information (incl. room and category). IMHO, measurement name + value is enough in most use cases. But if you really need more information (such as room and category), you can manually edit the UDP message sent by the Loxone UDP logger (see my tutorial). The python script will store this additional information as a tag. Then you can use this tag in order to filter out measurements in Grafana.

Cheers

[svethi]

Ah okay, but there you have another problem. Some people have round about 100 measurements. So you have to configure 100 virtual outputs for this and each of them has to create its own connection. In the worst case 100 connections at the time. Over Websockets its only 1 connection for all. The next is, you are wrong. Over Websocket will only pushed the ID and the value. All other informations are coming from the information file downloaded from the miniserver only one time loaded to configure the node red flow.
In the second point you are wrong. Some value of complex blocks are only available in Websockets. For this try to send the the smartactor informations (color transitions; transition times) over UDP virtual outputs. This is only one example. Sure for a small amount of data this is good alternative, but for a large amount of data or specific data, I see problems.

[Gast]

Ah okay, but there you have another problem. Some people have round about 100 measurements. So you have to configure 100 virtual outputs for this

Please read my tutorial first:

https://github.com/budulinek/easy-loxone-influx

There is not a single word about "UDP virtual outputs"! All the time I am talking about UDP logger. If you have 100 measurements, select them all in Lox Config. In the Properties tab, you can assign your UDP logger to all of them in one click. You can also click "Edit shared properties..."

each of them has to create its own connection. In the worst case 100 connections at the time.

We are talking about UDP protocol, so even if you have 100 logs in one second, no connection is established at all. Remember, it is "push and forget". I did a project with Arduino, sending a UDP message every 50 ms. No problem at all. UDP messages are actually very fast and efficient. You can actually make them even faster and more efficient if you do multicast UDP (avoiding ARP probe on local subnet), but it seems that Loxone UDP logger can not send multicast messages. I did try

/dev/udp/255.255.255.255/2222

but no success.

Over Websockets its only 1 connection for all.

Congratulations, websockets are very efficient, because they require only 1 TCP/IP connection. But UDP is even better, because it does not require any connection at all. This is how UDP works. Of course, since no connection is established, reliability suffers (see my previous post).

Some value of complex blocks are only available in Websockets. For this try to send the the smartactor informations (color transitions; transition times)

All peripheries (inputs and outputs) and normal blocks (status, virtual status) send normal UDP log, consisting of a timestamp, measurement name, value and whatever else you manually add to the UDP log message. But "smart" blocks actually do send more information.

I do not have smart lights. But I do have "intelligent room controller" (IRC2). This is a UDP log message I get:

2020-06-21 15:18:47;IRC2 Pokojová teplota (Zimní zahrada/Topení): SetOutputs; m_dOut 0.00, dHeat1 0.00, dCool 0.00, Ist: 21.56 -> Soll: 20.00

I did not mention it in my tutorial, because you would need a more sofisticated parser to insert these data into InfluxDB. But you can try it yourself with your smart actors (or smart lights or whatever you have). Create a UDP logger in Lox Config, point it to your own computer and capture UDP packets with Wireshark.

EDIT
Or even better, you can point the UDP logger to your Node Red and capture the logs with UDP input node.

[Gast]

Nice :-)

So I can confirm, that Loxone actually can send multicast UDP messages (my first attempt failed because I was not on the same subnet). So if you want to try multicast UDP, just use
/dev/udp/255.255.255.255/<your port>

Multicast UDP is slightly faster than unicast UDP. Because if the destination IP of the UDP singlecast message sits on the same subnet as the UDP server, the UDP server send a ARP multicast first in order to check whether the destination IP exists. With multicast UDP, there is no such ARP check.

[TomekWaw]

I like the idea and I will test this solution soon.
Now I'm using ioBroker to pass data to InfluxDB and I worked hard with new rights management model to limit number of objects seen by ioBroker.
Managed to filer out most of data except weather data (it's a either a bug or a "feature").
So I still have >2000 useless weather objects in ioBroker.
Perhaps this solution will be more effective and convinient.

[olicat]

So I can confirm, that Loxone actually can send multicast UDP messages

broadcast != multicast

I would be surprised if Loxone actually supports multicast.

Regards, Oliver