I’ve got a set of inexpensive load cells hooked up to log signs whether I’m getting a good night’s sleep. It was an experiment that was both interesting to me and fits within the quite-significant limitations of these cheap little things. I’m going to leave that setup collecting data for a while, in the meantime I want to write down details on the software side before I forget.

I did not try to compensate for temperature or for system warmup, those two together could affect final weight output by as much as half a kilogram. But in the specific purpose of tracking changes on a minute-by-minute basis, those factors could be ignored.

This sensor and HX711 amplifier combination has a recurring issue sending occasional readings that do not reflect what’s actually happening. To minimize effect of these spurious data points, I have taken the following measures:

• The reported weight value is the median weight out of the past minute and a half. Median value filter is more tolerant of spurious data of drastically different values.
• Once I had everything set up, I noted the minimum measured value (empty bed) and the maximum expected value (bed with me in it) and discarded all values outside of that range as “Off Scale Low” and “Off Scale High”. That will throw out some spurious data but still leaves those within the expected range.
• The reported delta value is not the difference between the maximum and minimum values seen within a time window. I track the second-largest and second-smallest values and report that delta instead. This way I can ignore spurious outliers, though it only works as long as spurious data doesn’t happen multiple times within a minute. Fortunately that’s been the case so far. If the problem gets worse I’ll have to devise something else.

And here’s the ESPHome YAML. If you want to copy/paste this code, feel free to do so. But make sure the values of hx711 “dout_pin” and “clk_pin” matches your hardware. The constants used to filter off-scale high/low will also need to be adjusted to fit your setup:

sensor:
  - platform: template
    name: "Delta"
    id: load_cell_delta
    accuracy_decimals: 0
    update_interval: never # updates only from code, no auto-updates
  - platform: template
    name: "Off Scale Low"
    id: load_cell_toolow
    accuracy_decimals: 0
    update_interval: never # updates only from code, no auto-updates
  - platform: template
    name: "Off Scale High"
    id: load_cell_toohigh
    accuracy_decimals: 0
    update_interval: never # updates only from code, no auto-updates
  - platform: hx711
    name: "Filtered"
    dout_pin: D2
    clk_pin: D1
    gain: 128
    update_interval: 0.5s
    filters:
      median:
        window_size: 180
        send_every: 120
    on_raw_value:
      then:
        lambda: |-
          static int load_window = 0;
          static float load_max = 0.0;
          static float load_second_max = 0.0;
          static float load_min = 0.0;
          static float load_second_min = 0.0;

          // Ignore spurious readings that imply negative weight
          if (x > -500000) // Constant experimentally determined for each setup
          {
            id(load_cell_toolow).publish_state(x);
            return;
          }
          // Ignore spurious readings that exceed expected maximum weight
          if (x < -1500000) // Constant experimentally determined for each setup
          {
            id(load_cell_toohigh).publish_state(x);
            return;
          }
          
          // Reached the end of our min/max window, publish observed delta
          if (load_window++ > 120)
          {
            load_window = 0;

            // Use second largest/smallest values, in case the absolute
            // max/min were outliers.
            id(load_cell_delta).publish_state(load_second_max-load_second_min);
          }
          
          if (load_window == 1)
          {
            // Starting a new min/max window
            load_max = x;
            load_second_max = x;
            load_min = x;
            load_second_min = x;
          }
          else
          {
            // Update observations in min/max window
            if (x > load_max)
            {
              load_second_max = load_max;
              load_max = x;
            }
            if (x < load_min)
            {
              load_second_min = load_min;
              load_min = x;
            }
          }

I took an inexpensive HX711-based load cell setup (the type that measures body weight in a bathroom scale) and installed them under my mattress. I wasn’t interested in measuring my weight while I sleep, I was interested in changes that indicate movement while sleeping. Ideally, I would see pauses in the data implying muscle inhibition associated rapid-eye movement (REM) sleep. I would take lack of movement as an indication of restful sleep. Logged to my Home Assistant database, here’s a plot of first night’s data:

This was a relatively restful night where I woke up refreshed. Looking at the plot, I can see when I got into bed and muscle activity gradually reducing as I fell asleep. There are multiple periods of nearly zero movement, implying a nice deep sleep in my cycle. As I started waking up in the morning, the load cell picked up more activity.

There was one unexplained set of data halfway through the night, where the movement activity is low but not as low as restful periods. The movement resembled my “settling down” period. I wonder if I woke up for some reason and had to fall back asleep? If so, I have no memory of this.

For comparison, here’s a graph of a different night’s data:

It was not a restful night of sleep. I have vague memories of waking up in the middle of the night, tossing and turning. I also woke up exhausted which corroborated with this plot of measured weight delta. It took longer before I fell asleep, there were far fewer periods of restful low movement, and I started to stir much earlier before I got out of bed.

I’ll keep the system running for a while, logging information from a time when I’m asleep and unconscious. But that’s about as far as I’m going to go. It would take more sleep science knowledge to analyze this data further and I’m not inclined to do so. Partially because this was a really cheap load cell + HX711 amplifier chip combo delivering unreliable data. Some of these “movements” may just be spurious data from the sensor. I wouldn’t read too much further into it, it’s just a fun project and not a serious health diagnosis tool. But here’s my ESPHome/HX711 code if anyone wants to play with it.

I didn’t expect a lot when I paid less than $10 for a set of load cells from Amazon, and indeed it has some pretty significant limitations. But that’s fine, every instrument has limitations and it’s a matter of making sure an application fit within them. Looking at the limitations of this sensor, I thought I had the perfect project fit: use them to gain some insight on my sleep quality.

Quality of sleep is important and there’s a lot of research behind it. For the home scientist, one of the easiest metrics to measure is the fraction of time we stay still in rapid-eye movement (REM) sleep. Problems disrupting sleep will cut into the amount of time we spend in REM sleep, depriving our brains of an important part of resting. Measuring actual eye movement is difficult, but (healthy) REM sleep also temporarily inhibits our muscles keeping our body still. This is an imperfect correlation: it is possible for muscle movements to happen while in REM sleep (should be small, though) and it is possible to stay still without being in REM sleep. Despite the imperfection, sleep movement is a good proxy.

There are many options to track sleep movement in the consumer medical technology field. Health wearables with accelerometers can do it, but it requires wearing the device while sleeping. Alternatives to wearing something include motion-detection cameras, but I’m not putting a camera in my bedroom. Using a set of cheap load cells seems like a good option, and logging data to my Home Assistant server at home is much better for personal privacy than a cloud-based solution.

I’ve already written my ESPHome YAML lambda tracking maximum/minimum values within a one-minute window. It was originally intended to quantify noise inherent in the system, but it works just as well to pick up changes on sensor readings. So, there will be no additional software work required.

On the hardware side, I have an IKEA bed frame with a series of slats holding up the mattress. I can put my sensors where the slat rests on the frame.

Load on all other slat-frame interface is not measured, which meant the absolute measured values will change depending on where my body is on the bed. Fortunately, the absolute value doesn’t matter because I’m only interested in changes minute-to-minute. Those changes over time are my sleep movement data. This also means I can ignore other problems with this instrument’s absolute values, like system warmup and daily temperature cycle sensitivity.

The bad news is the problem of spurious data will still impact this application. Such erroneous data will indicate movement when no actual movement has occurred. It means these measurements will understate the quality of my sleep by some unknown amount. (I slept better than the data indicated, but by how much?) However, given that the correlation between REM sleep and lack of motion is an imperfect one to begin with, perhaps this error is acceptable. The recorded data is pretty noisy but some patterns are visible.

I dusted off my inexpensive load cell system (read by a HX711 chip) and switched the associated microcontroller from an Arduino Nano to an ESP8266. That ESP8266 was then programmed using ESPHome to upload load cell readings to my Home Assistant server. I configured the ESP8266 to read every half second, but I’ve learned sending that much raw data directly to Home Assistant bogs down the system so that twice-a-second data is filtered to a summary report once a minute.

General Noise

One summary is generated by a small code snippet I wrote tracking the difference between maximum and minimum values seen within that minute. This gives me an idea of the natural level of noise in my particular configuration. If all other variables are unchanged, I saw a fluctuation of roughly 350 sensor counts, mostly within the range from 250 to 450.

The other summary is an average. Since I already had code tracking maximum and minimum, it wouldn’t have been hard to calculate my own average. But rather than adding those 3-5 lines of code, I used ESPHome’s built-in sliding window moving average filter because it was already there. Keeping the system running for a little over 24 hours, here’s the graph it generated:

Spurious Data

The little spikes visible in this graph are caused by the occasional data that does not reflect reality. I saw this in my earlier experiments with the HX711 talking to Node-RED, but that only ran for a few minutes at a time. I had hoped that, by graphing its behavior over a day, I could observe some pattern.

• There was no frequency-based pattern I could detect: they can happen mere minutes apart, and sometimes I can go for hours without one.
• I only have a single day, which is not enough data to say if there’s a time-of-day pattern.
• Visible spikes in the graph were caused by nonsensical values indicating less weight than when the load cell is completely unloaded: negative weight, so to speak. The raw sensor count is usually in the few thousands range when unloaded is approximately -420,000, a big enough difference to visibly throw off the average over 120 readings.
• Even though “negative weight” is the most visible in this graph, there are also unexplained brief flashes of data in the positive weight domain, they just don’t throw off the average value as visibly on this plot.

Temperature Sensitivity

One behavior I never noticed during my short duration Node-RED experiments were the relation between sensor counts and temperature. With a full day’s worth of data plotted, the correlation is clearly visible. From around 7PM to 8AM the next day, temperature dropped from 28.2°C to 21.6°C. (Tracked elsewhere in my Home Assistant network, not on this graph.) During this 6.6°C drop, average sensor count rose from around -426,500 to -420,000. This rounds to approximately 1000 sensor counts per degree Celsius.

Kilogram Test

But what do those sensor counts correspond to? I used my kitchen scale to measure as I poured water into a jar, stopping when they weighed one kilogram together. I placed that on my test setup for two hours. This dropped sensor counts from around -420,000 to -443,000 (plus temperature-induced variation). Using 23,000 sensor counts per kilogram, I can tentatively guess the random noise of ~300 sensor counts correspond to roughly thirteen grams. This is consistent with my earlier observation I need roughly fifteen grams of weight change before it is barely distinguishable from noise.

By the same metric, temperature change for a single degree Celsius changes the reading by roughly 43 grams. Over the course of a day that varies by 6.6 degrees Celsius, that would change weight reading by roughly 280 grams.

System Warmup

I brainstormed on possible reasons for spurious data and thought perhaps they were caused by the JST-XH connectors I used. A small intermittent connection might not be noticed in most of my projects, but load cells work by slight changes in resistance and the HX711 amplifies those changes. Small flaws in a connector that would go unnoticed elsewhere would drastically change behavior here, so I unsoldered the connector and soldered all wires directly to the HX711 board.

That experiment was a bust, direct soldering did not eliminate spurious data. I still don’t know where that’s coming from. But I came out of it with an additional observation: When I disconnect the system for a while to work on it, then turn it back on, there’s a warmup curve visible on the plot. This graph had two such work sessions, and I see a curve of roughly 3000 sensor counts. That’s roughly 130 grams.

Conclusion

Based on these observations, I conclude this specific load cell setup is only dependable down to about half a kilogram before we have to worry about compensating for factors system warmup or ambient temperature. This is consistent with the primary use of these devices: inexpensive bathroom scales for measuring human body weight.

We also need to account for spurious data in some way, for example take multiple readings and average them, possibly ignoring readings that are wildly inconsistent with the rest.

And even if we somehow managed to compensate for environment variables, it’s not possible to reliably measure any changes less than ~20 grams because of fundamental noise in this system.

This isn’t bad for a $10 kit, but its limitations does constrain usefulness. After a bit of thought, I think I have a good project idea to fit this sensor.

A few years ago, I bought a cheap set of load cells to play with. The kind that performs weight measurement in an inexpensive bathroom scale. I got them up and running with the bundled HX711 board, sending data to Node-RED. Using this infrastructure, I performed a silly little (but interesting!) experiment measuring squishing behavior of packing material. I then got distracted with other Node-RED explorations and haven’t done anything with the HX711 load cell setup since. Now I’m going to dust it off (quite literally) and play with it again. This time, instead of Node-RED, I’ll be using the ESPHome + Home Assistant infrastructure.

There are multiple reasons for this switch. After a few experiments with Node-RED, I haven’t found it to be a good fit for the way I think. I like the promise of flow-based programming, and I like Node-RED’s implementation of the idea, but I have yet to find enough of an advantage to justify changing over. Node-RED promised to make prototyping fast, but I found something that got my prototypes up and running even quicker: ESPHome and Home Assistant. In my experiments to date, ESPHome’s YAML-based configuration lets me get simple straightforward components up and running even more quickly than I ever managed under Node-RED. And when I need to venture beyond the simple defaults, I can embed small fragments of C code to do just the special thing I need. This comes to me more naturally than using Node-RED’s counterpart function node with a snippet of JavaScript. It’s also very quick to put together simple UI using Home Assistant, though admittedly with far less control over layout than Node-RED’s dashboard.

But the primary motivation this time around is that I already had an instance of Home Assistant running, so I don’t need to set up logging infrastructure for longer-duration projects. Node-RED is perfectly capable of working with a database, of course, but I’d have to set something up. Home Assistant already has one built-in. By default, it stores data only locally, and only for ten days, making it much more privacy-friendly than internet-based solutions with wildly varying levels of respect for privacy.

Hardware changeover was pretty simple. The HX711 board needed four wires: power, ground, data, and clock. I unsoldered the Arduino Nano I previously installed and replaced it with an ESP8266. It will need to run ESPHome’s HX711 integration, which under the hood used the same PlatformIO library I had used earlier for the Arduino Nano. A few lines of YAML later, load cell data started streaming to my Home Assistant server for me to examine.