DevOps,  Smarthome

Poll data from Xiaomi LYWSD03MMC sensor from a Docker contained script

Xiaomi released a small, square shape temperature/humidity sensor named LYWSD03MMC, which is equipped with a small LCD sensor. Let’s see how we can get and use it’s readings in a home automation system.

The Sensor

The Xiaomi LYWSD03MMC can be considered as a the small sibling of the very popular Xiaomi LYWSDCGQ BLE Sensor (BLE name MJ_HT_V1), which is a bit bigger and round shape, and more importantly costs 3x the price of the new, small LYWSD03MMC.

Here’s GearBest’s promo video of the LYWSD03MMC device. (I’m in no way affiliated with GearBest, I just couldn’t find an official Xiaomi video of the LYWSD03MMC sensor.)

Technology

This sensor uses Bluetooth Low Energy (BLE) to send it’s temperature and humidity readings to a “gateway”. Xiaomi intended to use it with it’s own BLE gateway(s), but the tinkerer community already started finding ways how to read the data from the sensor and get them into DIY home automation systems, like what I use: Home Assistant.

To gather the data from the older model Xiaomi LYWSDCGQ, I use an ESP32 microcontroller board which is equipped with BLE and WiFi chips. I have flashed ESPHome on the ESP32, and Integrated this ESPHome instance to my Home Assistant server. The official guide to this is here, and my own blog has a post about it here. This post is not about the LYWSDCGQ, but I’d like to explain why ESPHome doesn’t (yet?) support the new LYWSD03MMC, and how can we gather data from it anyway.

Here’s a very deep examination of the differences, also from GearBest – I’m not sure why they push content on this sensor so hard :) https://www.gearbest.com/community/SanfL4

First, let’s examine what are the similarities between the the older, round LCD Xiaomi LYWSDCGQ and the newer, smaller, square LCD LYWSD03MMC. They both:

  • Have an LCD screen
  • Use Bluetooth BLE to communicate, but…

The fundamental technological difference between the older, round LCD Xiaomi LYWSDCGQ and the newer, smaller, square LCD LYWSD03MMC is, that the older one is sending the data out to the world through BLE, unencrypted, and broadcasting the data periodically with BLE ADV advertising; which means that you don’t need to “connect to the sensor and poll it” to get the data, but instead it’s enough to listen for the data coming from the BLE MAC address of the LYWSDCGQ sensor. This has the advantage, that getting the data from the LYWSDCGQ doesn’t drain the battery, as we’re not actively connect to it and force read or poll data, but instead we let it to do it’s thing and send data out on it’s own, and we just passively listen for these BLE ADV broadcast packages with ESPHome flashed on ESP32.

Even though the newer, square LYWSD03MMC is still using BLE ADV advertising, but the advertised/broadcasted data are encrypted. This sounds like a good thing at first due to data privacy reasons, but it makes virtually impossible to passively listen for the broadcasted data. Described in this thread of the mitemp_bt component, and more specifically this comment from the main maintainer of mitemp_bt.

It is implied in a comment that the Android app MiTemp can indeed read the sensors data somehow, but it’s not clear if it’s passively listening, or actively polling the LYWSD03MMC.

Goal

I want to get the data of the sensor LYWSD03MMC show in Home Assistant. Now, how do I do that?

Tools

As of February 2020, there is one python script which is meant to be run on a BLE capable Raspberry Pi computer, and needs Python 3.7, and bluepy. https://github.com/JsBergbau/MiTemperature2

This is from the README.md of the JsBergbau/MiTemperature2 Github project: “This sensor doesn’t transmit its values in the advertisment data, like the LYWSDCGQ Bluetooth thermometer. This is more privacy friendly since no one can sniff your temperature readings. On the other side this means you have to establish a bluetooth connection with the device to get the data. When you’re connected no other connection is accepted, meaning if you hold the connection no other can readout your temperature and humidity.
Once you’re connected the LYWSD03MMC it advertises its values about every 6 seconds, so about 10 temperature/humidity readings per minute.

I have tested the script on a Raspberry Pi Zero W, works just fine.

The following test shows that I gave parameters to the LYWSD03MMC.py script, to:

  • -d –device; Specify the BLE MAC address of the target LYWSD03MMC sensor
  • -r –round; Round temperature to one decimal place
  • -b –battery; Get estimated battery level

Output of the script:

pi@raspberrypi:~/lywsd03mmc/MiTemperature2 $ /home/pi/lywsd03mmc/MiTemperature2/LYWSD03MMC.py -d A4:C1:38:62:A6:3F -r -b 3

Trying to connect to A4:C1:38:62:A6:3F
Battery-Level: 99
Temperature: 23.5
Humidity: 35

Temperature: 23.5
Humidity: 35

Temperature: 23.5
Humidity: 35

Battery-Level: 99
Temperature: 23.5
Humidity: 35

As we can see, we get the temperature and humidity data every 6 seconds, and per configured, the battery level on each 3rd reading.

MQTT

Now, how do we get these readings into Home Assistant Core? Since I cannot use ESPHome integration, I can utilise another great tool, a Mosquitto MQTT broker. I won’t get into detail on how this works, there’s tons of information out there, it’s a popular choice among home automation tinkerers.

As it happens, even the JsBergbau/MiTemperature2 Github project for LYWSD03MMC contains a complimentary script sendToMQTT to aid this goal. https://github.com/JsBergbau/MiTemperature2/blob/master/sendToMQTT

The main LYWSD03MMC.py python script can be run with a parameter
–callback sendToMQTT
which passes on the data to the sendToMQTT script, which has to be located beside the LYWSD03MMC.py, in the same directory.

It basically does a mosquitto_pub publish to the MQTT broker, but be careful the MQTT broker connection parameters are hardcoded in the sendToMQTT script, you need to fork it for your specific MQTT broker connection and credentials.

Docker

Due to my recent fascination with Docker, I run most of my applications in Docker containers, including my Home Assistant Core and Mosquitto MQTT broker. This made me think, if I could build this LYWSD03MMC python script with all of it’s prerequisites into a Docker container for portability, and then figure out a way of getting the readings into Home Assistant.

The JsBergbau/MiTemperature2 Github project contains a Dockerfile for building a docker image locally on the Raspberry Pi Zero W https://github.com/JsBergbau/MiTemperature2/tree/master/docker

If you open that Dockerfile, you’ll see that it uses a base image tag xavierrrr/xrrzero:stretchpython3.7 which basically a Debian Stretch with Python 3.7, nothing else. The https://github.com/JsBergbau/MiTemperature2/tree/master/docker README recommends to build this Dockerfile locally on a Raspberry Pi Zero W, the steps are also shared in the README. The local build is recommended because the ENTRYPOINT has a the script running, and also the script parameters hardcoded, so it would not fit the needs of each individual user. But…

I have found, that there is a standalone image tag xavierrrr/xrrzero:stretchmitemperature2 which doesn’t use the xavierrrr/xrrzero:stretchpython3.7 as the FROM base image, but rather has the whole Python3.7 prerequisite written down into it’s own Dockerfile. This would be perfect, as simply pulling the xavierrrr/xrrzero:stretchmitemperature2 docker image would have the whole thing running, but there is a catch. The last line of the xavierrrr/xrrzero:stretchmitemperature2 Dockerfile also has the script running with hardcoded parameters; the parameters of the sensor owned by the author of the script.

What can I do with all of this?

I decided to write my own Dockerfile based on xavierrrr/xrrzero:stretchpython3.7 and subsequently xavierrrr/xrrzero:stretchmitemperature2, but without the last ENTRYPOINT command. That means I will get a Docker container which has all the prerequisites and the LYWSD03MMC.py script packaged into a container, but the script is NOT running periodically or constantly in it.

Why? Because I have multiple LYWSD03MMC sensors. Running the script for 1 MAC address would lock the BLE device of the Raspberry Pi Zero W all the time, and I would get data only for 1 sensor.

What I need to do is a round robin cycle which goes through a list of MAC addresses of my LYWSD03MMC sensors (currently 7 of them).

I would use a docker exec command to run the script with a count parameter:
–count N, -c N Read/Receive N measurements and then exit script

If the -c 1 is used, the script will poll the sensor’s data once. I will call the script with a –callback sendToMQTT parameter (this parameter is explained earlier in this post).

I will have a separate sendToMQTT side-script for each sensor, e.g. sendToMQTT-A4-C1-38-62-A6-3F.sh, this way each of my LYWSD03MMC sensors will have their own MQTT topic to write the readings to.

My Home Assistant will be subscribed to the MQTT topics for each sensor, and show the readings on the dashboard as soon as they come in.

This was the theory…

Now on to the real work. Here are the steps.

Dockerfile

Since the idea is to use the Dockerfile of https://github.com/JsBergbau/MiTemperature2/blob/master/docker/Dockerfile as the base, the best way is to fork it, so I did this: https://github.com/zsiti/MiTemperature2/blob/master/docker/Dockerfile

I just made the change of removing the Vera script, and will use MQTT script instead. Also, the last command, the ENTRYPOINT which would run the script all the time within the container. We remove this:

COPY sendtovera.py sendtovera.py

and this…

ENTRYPOINT while true; do /usr/local/bin/python3.7m LYWSD03MMC.py -d A4:C1:38:7F:E6:DC -r -b 100 --skipidentical 50 -deb --callback sendtovera.py; sleep 2; done

[WIP] Also we have to add somehow the sendToMQTT scripts, maybe using a Docker volume?

Thanks to this we get a Docker container a runnable script within a container.

Install Docker on Pi Zero W

Source: https://github.com/JsBergbau/MiTemperature2/tree/master/docker#pi-zero-docker-installation-on-a-frech-stretchbuster-installation

sudo apt-get install apt-transport-https ca-certificates software-properties-common -y
curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh
sudo usermod -aG docker pi
sudo curl https://download.docker.com/linux/raspbian/gpg

systemctl start docker.service
docker info