Fan Strobe LED Adjustments via ESPHome

After proving my fan blade LED strobe idea worked with a minimalist Arduino sketch, I ported that code over as an ESPHome custom component. I thought it would be good practice for writing ESPHome custom components and gain Home Assistant benefit of adding adjustments in software. For me it was easier to create an analog control with a few lines of YAML and C, than it would be to wire up a potentiometer. (I recognize it may be the reverse for people more comfortable with hardware than software.)

The first adjustment “Fan Speed Control” actually did not require custom component at all: making the fan speed adjustable utilized the built-in fan speed component. In my minimalist sketch the fan is always running at full speed, now I can adjust fan speed and verify that the strobe logic indeed stays in sync with the fan speed. Meaning the strobe keeps these fan blades visually frozen in the same place regardless of their rotational speed.

The first custom output component adjustment “Strobe Duration” changes the duration of LED strobe pulse, from zero to 1000 microseconds. For this LED array, I found values under 100 to be too dim to be useful. The dimmest usable value is around 100, and things seem to work well up to 300. I start detecting motion blur above 300, and things are pretty smeared out above 600.

The next addition is a toggle switch “Strobe Tachometer Toggle”. Because the tachometer signal pulses twice per revolution, I ignore every other pulse. But that means a 50% chance the code will choose to trigger on the wrong pulse, resulting in a visually upside-down image. When this happens, this toggle allows us to flip to trigger on the opposing pulse, flipping the frozen visual right-side up.

The final custom component adjustment “Strobe Delay” adds a delay between triggered by tachometer wire and illumination of LED strobe. This changes the point at which the fan is visually frozen by the strobe light. Dynamically adjusting this value makes it look like the fan blade is slowly rotating position even though it is actually rotating at ~1200RPM. I think it is a fun effect, but to fully take advantage of that effect I will need longer delays, which means finding how I could move that work outside of my interrupt service routine. Inside the ISR I should set up a hardware timer for this delay and turn on LED when timer expires. I can then use the same mechanism to set a timer for LED duration and turn off LED when that timer expires.

Unfortunately, there are only two hardware timers on an ESP8266, and they are both spoken for. One runs WiFi, the other runs Arduino core for things like millis(). To explore this idea further, I will need to move up to an ESP32 which has additional hardware timers exposed to its Arduino core. And if I choose to explore that path, I don’t even need to redo my circuit board: there exists a (mostly) drop-in ESP32 upgrade for anything that runs a Wemos D1 Mini.



YAML Excerpt:

esphome:
  includes:
    - led_strobe_component.h

output:
  - platform: esp8266_pwm
    pin: 14
    id: fan_pwm_output
    frequency: 1000 Hz
  - platform: custom
    type: float
    lambda: |-
      auto led_strobe = new LEDStrobeComponent();
      App.register_component(led_strobe);
      return {led_strobe};
    outputs:
      id: strobe_duration_output
  - platform: custom
    type: float
    lambda: |-
      auto led_strobe_delay = new LEDStrobeDelayComponent();
      App.register_component(led_strobe_delay);
      return {led_strobe_delay};
    outputs:
      id: strobe_delay_output

fan:
  - platform: speed
    output: fan_pwm_output
    id: fan_speed
    name: "Fan Speed Control"

light:
  - platform: monochromatic
    output: strobe_duration_output
    id: strobe_duration
    name: "Strobe Duration"
    gamma_correct: 1.0
  - platform: monochromatic
    output: strobe_delay_output
    id: strobe_delay
    name: "Strobe Delay"
    gamma_correct: 1.0

switch:
  - platform: custom
    lambda: |-
      auto even_odd_flip = new LEDStrobeEvenOddComponent();
      App.register_component(even_odd_flip);
      return {even_odd_flip};
    switches:
      name: "Strobe Tachometer Toggle"

ESPHome custom component file led_strobe_component.h:

#include "esphome.h"

volatile int evenOdd;
volatile int strobeDuration;
volatile int strobeDelay;

IRAM_ATTR void tach_pulse_handler() {
  if (0 == evenOdd) {
    evenOdd = 1;
  } else {
    delayMicroseconds(strobeDelay);
    digitalWrite(13, HIGH);
    delayMicroseconds(strobeDuration);
    digitalWrite(13, LOW);
    evenOdd = 0;
  }
}

class LEDStrobeComponent : public Component, public FloatOutput {
  public:
    void setup() override {
      // LED power transistor starts OFF, which is LOW
      pinMode(13, OUTPUT);
      digitalWrite(13, LOW);

      // Attach interrupt to tachometer wire
      pinMode(12, INPUT_PULLUP);
      evenOdd = 0;
      attachInterrupt(digitalPinToInterrupt(12), tach_pulse_handler, RISING);

      strobeDuration = 200;
    }

    void loop() override {
    }

    void write_state(float state) override {
      // Multiply by 1000 = strobe duration from 0 to 1ms.
      strobeDuration = 1000 * state;
    }
};

class LEDStrobeEvenOddComponent: public Component, public Switch {
  public:
    void write_state(bool state) override {
      evenOdd = !evenOdd;
      publish_state(state);
    }
};

class LEDStrobeDelayComponent: public Component, public FloatOutput {
  public:
    void write_state(float state) override {
      strobeDelay = 1000*state;
    }
};

Disable Sleep on a Laptop Acting as Server

I’ve played with different ways to install and run Home Assistant. At the moment my home instance is running as a virtual machine inside KVM hypervisor. The physical machine is a refurbished Dell Latitude E6230 running Ubuntu Desktop 22.04. Even though it will be running as a server, I installed the desktop edition for access to tools like Virtual Machine Manager. But there’s a downside to installing the desktop edition for server use: I did not want battery-saving features like suspend and sleep.

When I chose to use an old laptop like a server, I had thought its built-in battery would be useful in case of power failure. But I hadn’t tested that hypothesis until now. Roughly twenty minutes after I unplugged the laptop, it went to sleep. D’oh! The machine still reported 95% of battery capacity, but I couldn’t use that capacity as backup power.

The Ubuntu “Settings” user interface was disappointingly useless for this purpose, with no obvious ability to disable sleep when on battery power. Generally speaking, the revamped “Settings” of Ubuntu 22 has been cleaned up and now has fewer settings cluttering up all those menus. I could see this as a well-meaning effort to make Ubuntu less intimidating to beginners, but right now it’s annoying because I can’t do what I want. To the web search engines!

Looking for command-line tools to change Ubuntu power saving settings brought me to many pages with outdated information that no longer applied to Ubuntu 22. My path to success started with this forum thread on Linux.org. It pointed to this page on linux-tips.us. It has a lot of ads, but it also had applicable information: systemd targets. The page listed four potentially applicable targets:

  • suspend.target
  • sleep.target
  • hibernate.target
  • hybrid-sleep.target

Using “systemctl status” I could check which of those were triggered when my laptop went to sleep.

$ systemctl status suspend.target
○ suspend.target - Suspend
     Loaded: loaded (/lib/systemd/system/suspend.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

Jul 21 22:58:32 dellhost systemd[1]: Reached target Suspend.
Jul 21 22:58:32 dellhost systemd[1]: Stopped target Suspend.
$ systemctl status sleep.target
○ sleep.target
     Loaded: masked (Reason: Unit sleep.target is masked.)
     Active: inactive (dead) since Thu 2022-07-21 22:58:32 PDT; 11h ago

Jul 21 22:54:41 dellhost systemd[1]: Reached target Sleep.
Jul 21 22:58:32 dellhost systemd[1]: Stopped target Sleep.
$ systemctl status hibernate.target
○ hibernate.target - System Hibernation
     Loaded: loaded (/lib/systemd/system/hibernate.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)
$ systemctl status hybrid-sleep.target
○ hybrid-sleep.target - Hybrid Suspend+Hibernate
     Loaded: loaded (/lib/systemd/system/hybrid-sleep.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

Looks like my laptop reached the “Sleep” then “Suspend” targets, so I’ll disable those two.

$ sudo systemctl mask sleep.target
Created symlink /etc/systemd/system/sleep.target → /dev/null.
$ sudo systemctl mask suspend.target
Created symlink /etc/systemd/system/suspend.target → /dev/null.

After they were masked, the laptop was willing to use most of its battery capacity instead of just a tiny sliver. This should be good for several hours, but what happens after that? When the battery is almost empty, I want the computer to go into hibernation instead of dying unpredictably and possibly in a bad state. This is why I left hibernation.target alone, but I wanted to do more for battery health. I didn’t want to drain the battery all the way to near-empty, and this thread on AskUbuntu led me to /etc/UPower/UPower.conf which dictates what battery levels will trigger hibernation. I raised the levels so the battery shouldn’t be drained much past 15%.

# Defaults:
# PercentageLow=20
# PercentageCritical=5
# PercentageAction=2
PercentageLow=25
PercentageCritical=20
PercentageAction=15

The UPower service needs to be restarted to pick up those changes.

$ sudo systemctl restart upower.service

Alas, that did not have the effect I hoped it would. Leaving the cord unplugged, the battery dropped straight past 15% and did not go into hibernation. The percentage dropped faster and faster as it went lower, too. Indication that the battery is not in great shape, or at least mismatched with what its management system thought it should be doing.

$ upower -i /org/freedesktop/UPower/devices/battery_BAT0
  native-path:          BAT0
  vendor:               DP-SDI56
  model:                DELL YJNKK18
  serial:               1
  power supply:         yes
  updated:              Fri 22 Jul 2022 03:31:00 PM PDT (9 seconds ago)
  has history:          yes
  has statistics:       yes
  battery
    present:             yes
    rechargeable:        yes
    state:               discharging
    warning-level:       action
    energy:              3.2079 Wh
    energy-empty:        0 Wh
    energy-full:         59.607 Wh
    energy-full-design:  57.72 Wh
    energy-rate:         10.1565 W
    voltage:             9.826 V
    charge-cycles:       N/A
    time to empty:       19.0 minutes
    percentage:          5%
    capacity:            100%
    technology:          lithium-ion
    icon-name:          'battery-caution-symbolic'

I kept it unplugged until it dropped to 2%, at which point the default PercentageAction behavior of PowerOff should have occurred. It did not, so I gave up on this round of testing and plugged the laptop back into its power cord. I’ll have to come back later to figure out why this didn’t work but, hey, at least this old thing was able to run 5 hours and 15 minutes on battery.

And finally: this laptop will be left plugged in most of the time, so it would be nice to limit charging to no more than 80% of capacity to reduce battery wear. I’m OK with 20% reduction in battery runtime. I’m mostly concerned about brief blinks of power of a few minutes. A power failure of 4 hours instead of 5 makes little difference. I have seen “battery charge limit” as an option in the BIOS settings of my newer Dell laptops, but not this old laptop. And unfortunately, it does not appear possible to accomplish this strictly in Ubuntu software without hardware support. That thread did describe an intriguing option, however: dig into the cable to pull out Dell power supply communication wire and hook it up to a switch. When that wire is connected, everything should work as it does today. But when disconnected, some Dell laptops will run on AC power but not charge its battery. I could rig up some sort of external hardware to keep battery level around 75-80%. That would also be a project for another day.

First Run with DS18B20 Temperature Sensor

I have a revised bedside fan (now with integrated LED illumination) up and running, and now it needs a temperature sensor. The goal is so I could automate turning off the fan once things cool down. My previous attempt used a TMP36 sensor that reported results as an analog voltage level. The results made me feel it was not a good fit with ESP8266 ADC peripheral. I don’t blame Espressif, it’s just a matter of different ADC designed for different usage requirements.

As an alternative to TMP36, I wanted to try a sensor that has its own integrated analog circuitry designed for the purpose and reported measurements via a digital communication protocol. Between what I could find in how-to guides and listed by vendors competing on price, I found my next candidate to be Maxim Semiconductor DS18B20 1-Wire Digital Thermometer. Another point in favor of this sensor is that it is frequently sold already packaged in a waterproof enclosure and a length of wire. This helps me place the sensor far enough away to avoid the heat generated by a running ESP8266. The lowest-bidder of the day sold them in a multipack of 15 (*) so that’s what I bought.

When I read through the DS18B20 datasheet, I was a little wary of its “1-Wire” data bus because I wasn’t sure I wanted to spend the time to implement another protocol. But I need not have worried: I only planned to use it with an ESP8266 running ESPHome, and somebody has already integrated the code to use these sensors.

The hardware side was simple: red wire to 3.3V, black wire to ground, and yellow wire to an ESP8266 pin. The only catch is that 1-Wire protocol requires a 4.7kΩ pull-up resistor so the chosen ESP8266 pin must be able to tolerate one. On my first try, I connected the data wire to Wemos D1 Mini pin D8 a.k.a. ESP8266 GPIO15 out of convenience. (It was easy to lay out on my circuit board.) Naturally Murphy’s Law ensured that I chose the pin that, if pulled high, would prevent an ESP8266 from booting. I had to move the data wire (and the pull-up resistor) to another pin before things would run. And now that it’s up and running, I think leveraging an Arduino library to read digital messages of DS18B20 measurement is preferable to using ESP8266 ADC to read TMP36 sensor voltage. Not to mention the convenience of having the sensor already encased and connected to a length of wire. This will be my go-to temperature sensor until I find a reason to try another.


(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Bedside Fan and Light V2

I took apart the Asiahorse Magic-i 120 V2 control hub and remote because I didn’t need them anymore: I could control its trio of fans with my own circuit board. How shall I wield such power? It was fun playing with 3D coordinate mapping with a Pixelblaze, but my most immediate need is a lot less exciting: combination bedside fan and light to replace my first bedside fan project. (Which didn’t have a light.)

For such a simple use, the power of a Pixelblaze is overkill. So, my board was modified to use an ESP8266 (Wemos D1 Mini module) as its brain, running ESPHome for integration with Home Assistant. In the Pixelblaze demo, the fans were always on. Now they will be controlled by that ESP8266 as well.

I’m still not settled enough on the idea to spend the time designing and 3D printing a proper frame, but at least I’ve put a bit more effort into this cardboard creation. I’m reusing a corner of a Newegg shipping box (I think it was the very box used to ship the Asiahorse fan bundle) and I’ve turned it inside out so at least I don’t have to stare at the Newegg logo every time I’m in bed.

Three large holes, one per fan, was cut from that cardboard for airflow. Twelve more holes, four per fan, were drilled into the cardboard for fan mounting screws. The physical assembly came together quickly, but there were a few more hiccups.

First problem was that FastLED, the addressable LED Arduino library used by ESPHome, is not compatible with the latest Arduino 3 framework at the time of this writing. Relevant ESPHome documentation page listed the issues as 1264 and 1322 filed against FastLED Github repository. Until they are resolved, ESPHome offers a workaround for compiling against Arduino framework 2.7.4. Which is what I did to run these WS2812 LEDs. For the first draft, I’m not going to use the addressable nature at all, just a single solid color. I think it would be cool to have a falling waterfall pattern on these lights, but that’ll be experimentation for later.

The second problem is that PWM control of fan speed results in an audible whine, probably an 1kHz whine which is the practical maximum speed for ESP8266 software PWM. The previous fan project removed the audible whine by adding a capacitor to smooth out voltage output. I could do that again, one capacitor per fan, but these fans run quietly enough at full speed I’m willing to skip PWM and have just on/off control.

The final problem is that I still want this fan to be responsive to temperature changes, turn itself off in the middle of the night when it got cool enough. I wasn’t happy with the TMP36 sensor I bought for the previous experiment, so now I’m going to try another sensor: the DS18B20.

Temperature Responsive Cereal Box Fan

This quick and easy project produces a quiet breeze for hot summer nights. I wanted something gentler than the fans already in the house, and I wanted it to automatically turn itself off in the middle of the night once things cooled down enough. It also let me apply lessons I’ve recently learned. Even though I’ve found that the TMP36 sensor isn’t a great fit for a ESP8266, it’s something already on hand for an ESP8266 to tell if it’s cool enough to turn the fan off.

The 3-wire fan is a PC cooling fan with a 200mm diameter, relatively large within that category. I bought it some years ago for my first Luggable PC project, it was just a bit too large for that purpose and sat idle until now. I thought about designing and 3D-printing a stand for this fan, but in the spirit of keeping things simple and quick, I mounted it in an empty cereal box instead. Cutting holes in the box to accommodate the fan took a tiny fraction of the time it would have taken to 3D-print something.

Primary air intake was the top of the box, left open.

I cut a smaller secondary air intake towards the bottom of the box, which also makes it easy to toss my control board in there and feed it power from a salvaged DC power supply. A TMP36 sensor was soldered in the farthest corner in this picture, visible sticking up vertically.

Results

Running ESPHome (YAML excerpt below) this project successfully controls fan speed via ESP8266 PWM. It was also able to read temperature via TMP36 sensor, but values were affected by ESP8266. Located 1/2 of the circuit board away plus the entire height of its legs was not enough distance from the ESP8266 heat island: temperature reading dropped noticeably whenever the fan is turning. Still, it’s enough for me to create a Home Assistant automation to turn off this fan whenever the temperature dropped below a certain threshold. Due to the heating from ESP8266, the temperature value rises a few degrees immediately after the fan was turned off. Thankfully there was no risk of system feedback oscillation, because I did not create an automation to turn the fan on — I do that manually when I’m ready for a light breeze.

This worked well sitting on my bedstand, creating a light cool breeze when I’m ready to fall asleep and turning itself off while I was asleep. But its physical footprint was a problem: it took up space that is ideally used for a bedstand light. The obvious solution was to pull some LEDs into the next version, which is an opportunity to tackle another item on my to-learn list: PC accessories with embedded RGB LEDs.


ESPHome YAML to read TMP36 temperature and fan speed every 5 minutes:

sensor:
  - platform: adc
    pin: A0
    name: "Fan Temperature"
    unit_of_measurement: "°C"
    update_interval: 1s
    accuracy_decimals: 2
    filters:
      - multiply: 100
      - offset: -50
      - sliding_window_moving_average:
          window_size: 450
          send_every: 300
          send_first_at: 15
  - platform: pulse_counter
    pin: 12
    id: fan_rpm_counter
    name: "Fan RPM"
    unit_of_measurement: "RPM"
    accuracy_decimals: 0
    update_interval: 300s
    filters:
      - multiply: 0.5 # 2 pulses per revolution

output:
  - platform: esp8266_pwm
    pin: 14
    id: fan_pwm_output
    frequency: 1000 Hz

fan:
  - platform: speed
    output: fan_pwm_output
    id: fan_speed
    name: "Fan Speed Control"

TMP36 Temperature Sensor + ESP8266 = Not a Great Team

After successfully building a small circuit for 3-pin fan PWM control, I decided to add a temperature sensor. Now I have the option to make it smarter about adjusting speed (or stopping entirely) based on temperature. There are many temperatures sensor options, I decided to start simply and inexpensively with a sensor that returns an analog voltage representing temperature. A batch of TMP36 (*) seemed like a good starting point.

According to the Analog Devices datasheet, TMP36 output pin voltage is 0.75V at 25 degrees Celsius. For every degree of temperature rise, voltage increases 0.01V. For my first draft I wired it to my Wemos D1 Mini module’s analog input pin. But I had to adjust the scaling because a D1 Mini includes a voltage divider to scale input of 0-3.3V down to ESP8266 ADC range of 0-1V. This voltage divider (and math necessary to convert it back) added error to the reading. Since I intend to use this sensor for measuring room temperature, I do not expect to measure above 50 degrees Celsius (122 Farenheit) which corresponded to 1 Volt. Thus, I soldered a wire to connect TMP36 signal directly to ESP8266 module analog input, bypassing the voltage divider.

I noticed that there appears to be a startup time period where the temperature reading is 2-3 degrees too high but, after about 5-10 minutes, it will drop to a steady state temperature and stay there. I’m not sure if this startup behavior is from the TMP36 sensor or from the ESP8266 ADC. Either way, it meant I could not use this sensor in a sleep/read/sleep/read cycle because such a quick read will always result in a too-high value from this startup behavior.

With the initial breadboard test complete, I built a dedicated temperature sensor node with just an ESP8266 with a TMP36. In the interest of compactness, I decided to solder the sensor directly to ESP8266 module pins.

Upon startup, I saw that it reported temperature that was a few degrees too high, but I thought that was just the startup behavior I already noticed. But instead of dropping, it had kept going up. I thought I had a bad TMP36 until I realized it was accurately reading heat generated by a running ESP8266. According to my USB power meter, it consumed less than a third of a Watt, but that’s plenty of heat for a directly-mounted TMP36 to pick up.

If I wanted to measure a room’s air temperature and not temperature of a running ESP8266, I needed to give the sensor some distance. But even then, the readings weren’t reliable.

A little web research taught me that the ESP8266 ADC isn’t very precise nor is it calibrated. For most applications, being off by a few hundredth of a volt is a negligible error, but here every hundredth of a volt represents an entire degree of temperature which is decidedly not negligible. Taking multiple values and averaging them did help with the precision, but not accuracy. Knowing what I know now, in hindsight I should have done this with an ESP32. Those chips (or at least newer units) have their ADCs calibrated at Espressif factory. Though it is more likely that I will try a different temperature sensor in a future project. Either way, right now I have TMP36 on hand with a circuit board suitable for controlling 3-wire PC cooling fans. Time to put them together to do something useful.


Even though it doesn’t work very well, here’s an ESPHome YAML excerpt anyway. This will read a TMP36 every second and report average value every five minutes. TMP36 signal is assumed to have been soldered directly to ESP8266 analog input, bypassing Wemos D1 Mini voltage divider.

sensor:
  - platform: adc
    pin: A0
    name: "Mobile Node Temperature"
    unit_of_measurement: "°C"
    update_interval: 1s
    accuracy_decimals: 2
    filters:
      - multiply: 100
      - offset: -50
      - sliding_window_moving_average:
          window_size: 450
          send_every: 300
          send_first_at: 15

(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

ESP8266 Controlling 4-Wire CPU Cooling Fan

I got curious about how the 4 wires of a CPU cooling fan interfaced with a PC motherboard. After reading the specification, I decided to get hands-on.

I dug up several retired 4-wire CPU fans I had kept. All of these were in-box coolers bundled with various Intel CPUs. And despite the common shape and Intel brand sticker, they were made by three different companies listed at the bottom line of each label: Nidec, Delta, and Foxconn.

I will use an ESP8266 to control these fans running ESPHome, because all relevant code has already been built and ready to go:

  • Tachometer output can be read with the pulse counter peripheral. Though I do have to divide by two (multiply by 0.5) because the spec said there are two pulses per fan revolution.
  • The ESP8266 PWM peripheral is a software implementation with a maximum usable frequency of roughly 1kHz, slower than specified requirement. If this is insufficient, I can upgrade to an ESP32 which has hardware PWM peripheral capable of running 25kHz.
  • Finally, a PWM fan speed control component, so I can change PWM duty cycle from HomeAssistant web UI.

One upside of the PWM MOSFET built into the fan is that I don’t have to wire one up in my test circuit. The fan header pins were wired as follows:

  1. Black wire to circuit ground.
  2. Yellow wire to +12V power supply.
  3. Green wire is tachometer output. Connected to a 1kΩ pull-up resistor and GPIO12. (D6 on a Wemos D1 Mini.)
  4. Blue wire is PWM control input. Connected to a 1kΩ current-limiting resistor and GPIO14. (D5 on Wemos D1 Mini.)

ESPHome YAML excerpt:

sensor:
  - platform: pulse_counter
    pin: 12
    id: fan_rpm_counter
    name: "Fan RPM"
    update_interval: 5s
    filters:
      - multiply: 0.5 # 2 pulses per revolution

output:
  - platform: esp8266_pwm
    pin: 14
    id: fan_pwm_output
    frequency: 1000 Hz

fan:
  - platform: speed
    output: fan_pwm_output
    id: fan_speed
    name: "Fan Speed Control"

Experimental observations:

  • I was not able to turn off any of these fans with a 0% duty cycle. (Emulating pulling PWM pin low.) All three kept spinning.
  • The Nidec fan ignored my PWM signal, presumably because 1 kHz PWM was well outside the specified 25kHz. It acted the same as when the PWM line was left floating.
  • The Delta fan spun slowed linearly down to roughly 35% duty cycle and was roughly 30% of full speed. Below that duty cycle, it remained at 30% of full speed.
  • The Foxconn fan spun down to roughly 25% duty cycle and was roughly 50% of the speed. I thought it was interesting that this fan responded to a wider range of PWM duty cycles but translated that to a narrower range of actual fan speeds. Furthermore, 100% duty cycle was not actually the maximum speed of this fan. Upon initial power up, this fan would spin up to a very high speed (judged by its sound) before settling down to a significantly slower speed that it treated as “100% duty cycle” speed. Was this intended as some sort of “blow out dust” cleaning cycle?
  • These are not closed-loop feedback devices trying to maintain a target speed. If I set 50% duty cycle and started reducing power supply voltage below 12V, the fan controller will not compensate. Fan speed will drop alongside voltage.

Playing with these 4-pin fans were fun, but majority of cooling fans in this market do not have built-in power transistors for PWM control. I went back to learn how to control those fans.

Home Assistant OS in KVM Hypervisor

I encountered some problems running Home Assistant Operating System (HAOS) as a virtual machine on a TrueNAS CORE server, which is based on FreeBSD and its bhyve hypervisor. I wanted to solve these problems and, given my good experience with Home Assistant, I was willing to give it dedicated hardware. A lot of people use a Raspberry Pi, but in these times of hardware scarcity a Raspberry Pi is rarer and more valuable than an old laptop. I pulled out a refurbished Dell Latitude E6230 I had originally intended to use as robot brain. Now it shall be my Home Assistant server, which is a robot brain of sorts. This laptop’s Core i5-3320M CPU launched ten years ago, but as a x86_64 capable CPU designed for power-saving laptop usage, it should suit Home Assistant well.

Using Ubuntu KVM Because Direct Installation Failed Boot

I was willing to run HAOS directly on the machine, but the UEFI boot process failed for reasons I can’t decipher. I couldn’t even copy down an error message due to scrambled text on screen. HAOS 8.0 moved to a new boot procedure as per its release announcement, and the comments thread on that page had lots of users reporting boot problems. [UPDATE: A few days later, HAOS 8.1 was released with several boot fixes.] Undeterred, I tried a different tack: install Ubuntu Desktop 22.04 LTS and run HAOS as a virtual machine under KVM Hypervisor. This is the hypervisor used by the Linux-based TrueNAS SCALE, to which I might migrate in the future. Whether it works with HAOS would be an important data point in that decision.

Even though I expect this computer to run as an unattended server most of the time, I installed Ubuntu Desktop instead of Ubuntu Server for two reasons:

  1. Ubuntu Server has no knowledge of laptop components, so I’d be stuck with default hardware behavior that are problematic. First is that the screen will always stay on, which wastes power. Second is that closing the lid will put the machine to sleep, which defeats the point of a server. With Ubuntu Desktop I’ve found how to solve both problems: edit /etc/systemd/logind.conf and change lid switch behavior to lock, which turns off the screen but leaves the computer running. I don’t know how to do this with Ubuntu Server or Home Assistant OS direct installation.
  2. KVM Hypervisor is a huge piece of software with many settings. Given enough time I’m sure I could learn all of the command line tools I need to get things up and running, but I have a faster option with Ubuntu Desktop: Use Virtual Machine Manager to help me make sense of KVM.

KVM Network Bridge

Home Assistant instructions for installing HAOS as a KVM virtual machine was fairly straightforward except for lack of details on how to set up a network bridge. This is required so HAOS is a peer on my home network, capable of communicating with ESPHome devices. (Equivalent to the network_mode: host option when running Home Assistant Docker container.) HAOS instruction page merely says “Select your bridge” so I had to search elsewhere for details.

A promising search hit was How to use bridged networking with libvirt and KVM on linuxconfig.org. It gave a lot of good background information, but I didn’t care for the actual procedure due to this excerpt: “Notice that you can’t use your main ethernet interface […] we will use an additional interface […] provided by an ethernet to usb adapter attached to my machine.” I don’t want to add another Ethernet adapter to my machine. I know network bridging is possible on the existing adapter, because Docker does it with network_mode:host.

My next stop was Configuring Guest Networking page of KVM documentation. It offered several options corresponding to different scenarios, helping me confirm I wanted “Public Bridge”. This page had a few Linux distribution-specific scripts, including one for Debian. Unfortunately, it wanted me to edit a file /etc/network/interfaces which doesn’t exist on Ubuntu 22.04. Fortunately, that page gave me enough relevant keywords for me to find Network Configuration page of Ubuntu documentation which has a section “Bridging” pointing me to /etc/netplan. I had to change their example to match Ethernet hardware names on my computer, but once done I had a public network bridge upon my existing network adapter.

USB Device Redirection

Even though I’m still running HAOS under a virtual machine hypervisor, ESPHome could access USB hardware thanks to KVM device redirection.

First I plug in my ESP32 development board. Then, I open the Home Assistant virtual machine instance and select “Redirect USB device” under “Virtual Machine” menu.

That will bring up a list of plugged-in USB devices, where I could select the USB to UART bridge device on my ESP32 development board. Once selected, the ESPHome add-on running within this instance of HAOS could see the ESP32 board and flash its firmware via USB. This process is not as direct as it would have been for HAOS running directly on the computer, but it’s far better than what I had to do before.

At the moment, surfacing KVM capability for USB device redirection is not available on TrueNAS SCALE but it is a requested feature. Now that I see the feature working, it has become a must-have for me. Until this is done, I probably won’t bother migrating my TrueNAS server from CORE (FreeBSD/bhyve) to SCALE (Linux/KVM) because I want this feature when I consolidate HAOS back onto my TrueNAS hardware. (And probably send this Dell Latitude E6230 back into the storage closet.)

Start on Boot

And finally, I had to tell KVM to launch Home Assistant automatically upon boot. By checking “Start virtual machine on host boot up” under “Boot Options” setting.

In time I expect that I’ll learn the KVM command lines to accomplish what I’m doing today with Virtual Machine Manager, but today I’m glad VMM helps me get everything up and running quickly.

[UPDATE: virsh autostart is the command line tool to launch a virtual machine upon system startup. Haven’t yet figured out command line procedure for USB redirection.]

Home Assistant OS in TrueNAS CORE Virtual Machine

I started playing with Home Assistant in the form of Home Assistant Core, a docker container, for its low commitment. Once I decided Home Assistant was worthwhile, I moved up to running Home Assistant Operating System as a virtual machine on my TrueNAS CORE server. This move gained features of Home Assistant Supervisor and I found the following subset quite useful:

  • Easy upgrade and rollback for failed upgrades.
  • Add-on integration features, especially for ESPHome.
  • Ability to backup critical data in a compact *.tar file.

The backup situation is a tradeoff. When I ran Home Assistant Core docker container, I could map its data directory to my TrueNAS storage pool. It was a more robust data retention system, as I had configured it for nightly snapshots and regular backups to external media. However, this would take upwards of hundreds of megabytes especially when I’m flooding the Home Assistant database. In contrast, the data backup archive generated by Home Assistant Supervisor is tiny at only a few megabytes. I have ambition to eventually get the best of both worlds: a Home Assistant automation that triggers Supervisor backups, and then store those backup files on my TrueNAS storage pool. This should be possible, as people have created addons to perform automatic backups and upload to Google Drive. But right now I have to do it manually.

Unrelated to the backup situation, there were two significant downsides to running Home Assistant OS on a TrueNAS CORE virtual machine.

  1. The virtual machine does not have access to hardware. If ESPHome add-on could access USB, it could perform first-time firmware upload on ESP32/ESP8266 devices. Without hardware access, I have to perform initial upload some other way which is cumbersome. (Following uploads could be done via WiFi, a huge benefit of ESPHome.)
  2. There are various problems with the FreeBSD bhyve hypervisor running Linux-based operating systems. A category of them (apparently there are more than one) interferes with the ability for a Linux operating system to reboot itself. In practice, this means every time the Home Assistant OS updates itself and reboots, it would shut down but not restart. At one point, I could not perform manual shutdown from TrueNAS interface, so the horrible workaround was to reboot my TrueNAS server. After a few TrueNAS updates, I can now manually shut down and restart the VM. But it is still a big hassle to do this on every Home Assistant OS update.

Due to these problems, I would NOT recommend running Home Assistant OS as a TrueNAS CORE virtual machine. Issue #2 became quite annoying and, when my Home Assistant got stuck trying to reboot for an upgrade to Home Assistant OS 8.0, I decided it was time to try a different setup.

Quick ESP32 PWM Experiment via ESPHome

I’ve mapped out the segments of a small LCD salvaged from an electric blanket controller. I activated these segments with an ESP8266 that alternated between 0V and 3.3V on two GPIO pins. Good for individual segments, but not good enough to drive the whole display. There are eight segment pins and two common pins for 8*2=16 total possible combinations (14 of which are used for the two 7-segment digits) controlled from ten total pins.

Technically speaking, an ESP8266 has enough GPIO pins, but we’d start intruding into the realm of sharing pins between multiple tasks which is more complexity than I wanted to tackle for a quick test. When driving a LCD, we also want to control voltage levels on these pins and ESP8266 lacks hardware PWM peripheral. For these reasons I will use an ESP32. It has more than enough pins, and hardware PWM peripherals to support generating a voltage on all those pins. ESP32 LEDC PWM peripheral is very flexible, and I need to determine how I want to configure that peripheral.

I used ESPHome because it is a great platform for quick experiments like this. I don’t strictly need WiFi here, but easy integration to Home Assistant and ability to update code over the network are great conveniences. Before I found ESPHome, I would wire up a few potentiometers to the circuit board and write code to use ADC to read their positions. A bit of code will then allow me to interactively play with parameters and see their results. But now, with ESPHome in my toolbox, I don’t need to solder potentiometers or write code for ADC. I can get such interactivity from the Home Assistant dashboard.

By default, ESPHome configures LEDC PWM peripheral to run at a frequency of 1kHz. According to Espressif documentation, it can be configured to run as high as 40MHz, though at that point it really isn’t a “PWM” signal anymore with only a fixed 50% duty cycle. Slowing down the frequency increases the number of bits available for duty cycle specification, and I wanted to find a tradeoff that I think will work well for this project. Here is an excerpt of ESPHome configuration YAML I used for this test.

output:
  - platform: ledc
    pin: GPIO23
    id: pwm_23
  - platform: template
    id: pwm_freq
    type: float
    write_action:
      lambda: |-
        int newFreq = (int)(10000000.0*state)+1000;
        id(pwm_23).update_frequency(newFreq);
    
light:
  - platform: monochromatic
    gamma_correct: 1.0
    output: pwm_23
    name: "PWM23"
  - platform: monochromatic
    gamma_correct: 1.0
    output: pwm_freq
    name: "PWM Freq"

This allows me to adjust two variables, each exposed to Home Assistant as a monochromatic dimmable light. Which I can change via a slider on a web page instead of a potentiometer soldered to the board. (The gamma correction value was set to 1.0 because we’re not actually controlling a visible light that requires gamma correction.)

  1. pwm_23 controls the PWM duty cycle.
  2. pwm_freq controls the PWM frequency via ESPHome Template Lambda. Theoretically from 1kHz to 10MHz, though in practice we won’t reach either end as the state never gets all the way down to 0.0 nor all the way up to 1.0.

As I adjusted the frequency, ESPHome automatically calculates the duty cycle bit depth.

[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2491148.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 5
[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2494322.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 5
[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2497869.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 5
[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2501411.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 4
[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2503623.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 4
[15:32:00][D][ledc.output:041]: Calculating resolution bit-depth for frequency 2505428.000000
[15:32:00][D][ledc.output:046]: Resolution calculated as 4

From this snippet, we can see that 2.5MHz is the limit for 5 bits of resolution. 25 = 32 levels, which gives me control of resulting voltage in (3.3V / 32) ~= 0.1V increments. I think that’ll be good enough.


Here are some plots in oscilloscope form: First the starting point of 50% duty cycle at default 1kHz frequency, before and after adding a 100nF capacitor into the mix.

Not nearly good enough to output 1.65V. To make this better, I can increase the capacitance or increase frequency. Increasing capacitance will dull system response (and I need pretty quick response to rapidly cycle between LCD common pins) so I start cranking up the frequency.

At hundreds of kilohertz without capacitor, the resulting wave is more than what this little oscilloscope can capture at this timescale. When the 100nF capacitor is added in, we see a pretty respectable 1.65V signal, might even be good enough. But there’s plenty of room left to go faster.

Getting into the megahertz range, there’s enough natural capacitance in the system (wires, breadboard, etc.) that we see a pretty good waveform even without a real capacitor in line with the output. But with just a 330pF capacitor (much smaller than the 100nF I started with) the resulting voltage looks pretty good. At least, at this time scale. Now I need to move beyond ESPHome for better control of 2.5MHz PWM signals.

Partial Home Assistant Control of Mr. Robot Badge Mk. 2

Once I had Mr Robot Badge Mk. 2 up and running on Arduino, I thought: what about Home Assistant? After all, ESPHome is built on compiling ESP8266 Arduino Core code using PlatformIO command-line tools. The challenging part of ESPHome is always that initial firmware flash, which I’m already set up to do, and after that I could do updates wirelessly.

I started easy: getting badge buttons to be recognized as binary sensors in Home Assistant.

binary_sensor:
  - platform: gpio
    pin:
      number: 0
      mode:
        input: true
        pullup: true
      inverted: true
    name: "Left"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 16
      mode:
        input: true
      inverted: true
    name: "Down"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 15
      mode:
        input: true
        pullup: true
    name: "Right"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 13
      mode:
        input: true
        pullup: true
      inverted: true
    name: "Up"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 12
      mode:
        input: true
        pullup: true
      inverted: true
    name: "A"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 14
      mode:
        input: true
        pullup: true
      inverted: true
    name: "B"
    filters:
      - delayed_on_off: 20ms

Piece of cake! Now on to the more interesting (and more challenging) part: using the LEDs. I followed ESPHome examples to create a custom component. This exposed a single custom output that treated the entire array as a single color (monochromatic) dimmable light.

C header file with custom class declaration:

#include "esphome.h"
#include "Adafruit_IS31FL3741.h"
using namespace esphome;

class MrRobotBadgeFloatOutput : public Component, public FloatOutput
{
    Adafruit_IS31FL3741 ledArray;

    public:
        void setup() override
        {
            ledArray.begin();
            ledArray.setLEDscaling(0x3F); // 25% of max
            ledArray.setGlobalCurrent(0x3F); // 25% of max
            ledArray.enable(true);  // bring out of shutdown
        }

        void write_state(float state) override
        {
            int8_t value = state * 128; // 50% of max
            ledArray.fill(value);
        }
};

YAML:

output:
  - platform: custom
    type: float
    lambda: |-
      auto badge_float = new MrRobotBadgeFloatOutput();
      App.register_component(badge_float);
      return {badge_float};
    outputs:
      id: badge_float

light:
  - platform: monochromatic
    name: "Badge Lights"
    output: badge_float

Tada! I’m back at the same test I had using Arduino, lighting up all the pixels to highlight problematic LEDs.

The next step in this progression is to expose the entire 18×18 array as individually addressable pixels. Doing so would allow using ESPHome’s display rendering engine to draw text and graphics, much as Adafruit GFX does. I found the Addressable Light base class but I got stuck trying to make it happen. I’ve had no luck finding examples on how to implement a custom addressable light, and didn’t get much of anywhere bumping my head in the dark. Digging through existing Addressable Light implementations on GitHub, I’ve found many object classes that may or may not be required to implement my own addressable light.

I imagine there’s an architecture diagram somewhere that describes how these components interact, but my search has come up empty so far and there’s not much in the way of code comments. Most of the online discussions about addressable LED pixels in Home Assistant are centered around WLED, which as of this writing does not support the IS31LF3741 control chip. I’m going to put further exploration on hold until I have more experience with ESPHome custom components to understand their object hierarchy.

Power Control Board for TrueNAS Replication Raspberry Pi

Encouraged by (mostly) success of controlling my Pixel 3a phone’s charging, the next project is to control power for a Raspberry Pi dedicated to data backup for my TrueNAS CORE storage array. (It is a remote target for replication, in TrueNAS parlance.) There were a few reasons for dedicating a Raspberry PI for the task. The first (and somewhat embarrassing) reason was that I couldn’t figure out how to set up a remote replication target using a non-root account. With full root level access wide open, I wasn’t terribly comfortable using that Pi for anything else. The second reason was that I couldn’t figure out how to have a replication target wake up for the replication process and go to sleep after it was done. So in order to keep this process autonomous, I had to leave the replication target running around the clock, and a dedicated Raspberry Pi consumes far less power than a dedicated PC.

Now I want to take a step towards power autonomy and do the easy part first. I have my TrueNAS replications kick off in response to snapshots taken, and by default that takes place daily at midnight. The first and easiest step was then to turn on my Raspberry Pi a few minutes before midnight so it is booted up and ready to receive replication snapshot shortly after midnight. For the moment, I would still have to shut it down manually sometime after replication completes, but I’ll tackle that challenge later.

From an electrical design perspective, this was no different from the Pixel 3a project. I plan to dedicate another buck converter for this task and connect enable pin (via a cable and a 1k resistor) to another GPIO pin on my existing ESP32. This would have been easy enough to implement with a generic perforated prototype circuit board, but I took it as an opportunity to play with a prototype board tailored for Raspberry Pi projects. Aside from the form factor and pre-wired connections to Raspberry Pi GPIO, these prototype kits also usually come with appropriate pin header and standoff hardware for mounting on a Pi. Looking over the various offers, I chose this particular four-pack of blank boards. (*)

Somewhat surprisingly for cheap electronics supply vendors on Amazon, this board is not a direct copy of an existing Adafruit item. Relative to the Adafruit offering, this design is missing the EEPROM provision which I did not need for my project. Roughly two-thirds of the prototype area has pins connected as they are on a breadboard, and the remaining one-third are individual pins with no connection. In comparison the Adafruit board is breadboard-like throughout.

My concern with this design is in its connection to ground. It connects only a single pin, designated #39 in most Pi GPIO diagrams and lower-left in my picture. The many remaining GND pins: 6,9,14,20,25,30, and 34 appear to be unconnected. I’m not sure if I should be worried about this for digital signal integrity or other reasons, but at least it seems to work well enough for today’s simple power supply project. If I encounter problems down the line, I can always solder more grounding wires to see if that’s the cause.

I added a buck converter and a pair of 220uF capacitors: one across input and one across output. Then a JST-XH board-to-wire connector to link back to my ESP32 control board. I needed three wires: +Vin, GND and enable. But I used a four-pin connector just in case I want to surface +5Vout in the future. (Plus, I had more four-pin connectors remaining in my JST-XH assortment pack than three-pin connectors. *)

I thought about mounting the buck converter and capacitors on the underside of this board. There’s enough physical space between the board and the Raspberry Pi to fit them. I decided against it on concern of heat dissipation, and I was glad I did. After this board was installed on top of the Pi, the CPU temperature during replication rose from 65C to 75C presumably due to reduced airflow. If I had mounted components underneath, that probably would have been even worse. Perhaps even high enough to trigger throttling.

I plan to have my ESP32 control board run around the clock, so this particular node doesn’t have the GPIO deep sleep state problem of my earlier project with ESP8266. However, I am still concerned about making sure power stays on, and the potential problems of ensuring so.


(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Home Assistant Mobile App Data Reporting Rate Varies Greatly

I wanted to use Home Assistant and ESPHome to automate periodic charging for a less-used cell phone. When I was working through the hardware, I had thought I would just use a time-based system. “Charge daily for [X] hours” and fine-tune the value of X as I go. This can easily be done, but as I learned more about Home Assistant, I realized an even smarter option is possible: turn off charging once the battery surpasses a threshold.

I had originally dismissed doing any decision based on battery state of charge because I thought I would have to open up the phone and probe battery voltage directly. Or if I want to do it in software, I’d have to write my own Android app to listen for battery status events. But that’s before I installed the Home Assistant mobile app on my Pixel 5a primary phone. When I connected the app to my server, I saw that it reported battery level as one of its entities. Here is a graph for 30 hours:

My normal daily phone usage only occasionally interacted with the app, so most of these reports came while it was running in the background. This is enough to receive updates once every 20-30 minutes. When installed on a little-used phone, however, something inside the Pixel 3a (probably Android’s internal power management algorithms) decided Home Assistant app doesn’t need to run as often. As a result, there were far less frequent reports on battery level. This is within the same 30-hour window as the above graph. (But with slightly different Y-axis because of Home Assistant dashboard scaling.)

While the phone is charging, battery level is reported more frequently, but they can be as much as half an hour apart. It gets worse when the phone is not charging, the middle of this graph has a period where we went almost 14 hours without a single update!

Despite this low frequency, I can write a Home Assistant automation that would still be better than doing it blind on a time-based basis. I wouldn’t be able to stop charging exactly at 80%, but I should still be able to catch it surpassing 80% sometime afterwards and still stop charging before battery is full.

[UPDATE: I’ve found that I could improve update frequency by telling Android 12 that I grant the Home Assistant app more leeway on battery use. In the Android settings menu, under Apps/Home Assistant/Battery the option defaults to “Optimized”. We can change it to “Unrestricted” which has the explanation “Allow battery usage in the background without restrictions. May use more battery.”]

I’m only mildly disappointed the battery level reporting rate is infrequent when running on a less-used Android phone, because it is still good enough for my purpose of keeping the battery in its most effective center band. (Avoiding full charge and also avoiding full discharge.) Besides, these infrequent updates are still more useful than the Home Assistant iOS app, which I installed on my iPad and saw no updates for this entire 30-hour period.

I’ll let my Pixel 3a charging logic run and see how well it works (or not) as I work on the next automation: power for a Raspberry Pi dedicated to NAS replication.

Successful Quick ESPHome Test: M5Stack ESP32 Camera

I don’t really have the proper gear to test and verify my modifications to an USB cable with type C connectors. Flying blind, I knew there was a good chance I would fry something. I dug through my pile of electronics for the cheapest thing I have with an USB-C socket, which turned out to be a M5Stack ESP32 Camera.

I got this particular module as an add-on to the conference badge for Layer One 2019 as documented on Hackaday. It’s been gathering dust ever since, waiting for a project that needed a little camera driven by an ESP32. For conference badge purposes it ran code from this repository, which also pointed to resources that helped me find the M5Stack ESP32Cam product documentation page.

The camera module is an OV2640, which is a very popular for electronics hobbyists and found in various boards like this one from ArduCam. If I want to do more work with ESP32+OV2640 I can find variations on this concept for less than $10 each. But M5Stack is at least a relatively name-brand item here, enough for this module to be explicitly described in ESPHome documentation. (Along with a warning about insufficient cooling in this design!)

Two notes about this ESP32Cam module that might not be present on other ESP32+OV2640 modules:

  1. There is a battery power management IC (IP5306) on board, making this an interesting candidate for projects if I want to run on a single lithium-ion battery cell and if I don’t want to tear apart another USB power bank. I think it handles both charge safety and boost conversion for higher voltage. I don’t know for sure because the only datasheets I’ve found so far are in Simplified Chinese and my reading comprehension isn’t great.
  2. The circuit board included footprints for a few other optional IC components. (BMP280 temperature/pressure/humidity environmental sensor, MPU6050 3-axis accelerometer + 3-axis gyroscope, SPQ2410 microphone.) They are all absent from my particular module, but worth considering if they are ICs that would be useful for a particular project.
  3. There is a red LED next to the camera connected to pin 16. I used it as an ESPHome status light.
status_led:
  pin:
    number: 16

My first attempt to put ESPHome on this module was to compile a *.bin file for installation via https://web.esphome.io. Unfortunately, it doesn’t seem to properly set up the flash memory for booting as the module gets stuck in an endless loop repeating this error:

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371 
ets Jun  8 2016 00:22:57

To work around this problem, I fired up an Ubuntu laptop and ran ESPHome docker container to access a hardware USB port for flashing. This method flashed successfully and the ESP32 was able to get online where I could make future updates over wireless.

A web search indicates an OV2640 has a native sensor resolution of 1632×1232. But the ESPHome camera component running on this module could only handle a maximum of 800×600 resolution. The picture quality was acceptable, but only about 2-3 frames per second gets pushed to Home Assistant. As expected, it is possible to trade resolution for framerate. The lowest resolution of 160×120 is very blurry but at least motion is smooth. If I try resolutions higher than 800×600, at bootup time I would see this error message in debug log:

[E][esp32_camera:095]:   Setup Failed: ESP_ERR_NO_MEM

This isn’t great. But considering its price point of roughly ten bucks for a WiFi-enabled camera module, it’s not terrible. This experiment was a fun detour before I return to my project of automated charging for a Pixel 3a phone.

Two Problems Controlling Buck Converter

My solar power monitor project runs on a ESP8266 microcontroller and an INA219 sensor powered by an old USB power bank. In order to charge it during the day from solar power, I’m trying out a new-to-me buck converter module because it exposed an “Enable” pin that was absent from my usual MP1584 buck converter module. I connected it (via a 1k resistor) to the closest available GPIO pin on my ESP8266 module, which happened to be GPIO0. Configuring ESPHome to use that pin as a switch, I could turn charging on or off from Home Assistant UI. I declared victory but it was premature.

I realized there was a problem when I put the ESP8266 to sleep and noticed charging resumed. This was a surprise. Probing the circuit I found my first problem: there is a pull-up resistor or a voltage divider on board my new buck converter module so that if its enable pin is left floating, it will activate itself as my usual MP1584 module would. This was mildly disappointing, because it meant I might have to unsolder a few resistors to get the behavior I originally wanted, and one of the reasons to buy this module was because I didn’t want to unsolder resisters from my MP1584 buck converter boards. As a short-term hack, I fought the existing circuit by adding a pull-down resistor external to the module. Experimentally it looks like a 10k resistor to ground was enough to do the trick, disabling the buck converter when enable input line is left floating.

But I wasn’t done yet, there’s a second problem to address: When ESP8266 was put back in the circuit, the charging would still resume when I put it into deep sleep. Probing the pin, I saw GPIO0 was at 3.3V while asleep. Reading online resources like this page on Random Nerd Tutorials, I learned the pin needs to be high for ESP8266 to boot. Presumably this means the Wemos D1 Mini module has a pull-up resistor on board for the boot process. Therefore I can’t use GPIO0 for charging control.

I went down the list of still-unused pins by distance to the buck converter on my circuit board. The next closest pin is GPIO2, which I can’t use as I’m already using the blue status LED. After that is GPIO14. It is usually used for SPI communication but I have no SPI peripherals for this project. Looking on the reference chart, it doesn’t seem to get pulled up or down while ESP8266 was asleep. After confirming that behavior with a voltmeter, I switched buck converter enable pin over to GPIO14. It allowed me to control charging from ESPHome and, when the ESP8266 is asleep, the buck converter stays disabled. Finally, the hardware is working as intended! Now I need to figure out the software.

Exploring Low Power ESPHome Nodes

When I investigated adding up power over time into a measure of energy, I found that I have the option of doing it either on board my ESPHome microcontroller or on the Home Assistant server. I’m personally in favor of moving as much computation on the server as I can, and another reason is because keeping the sensor node lightweight gives us the option of putting it to sleep in between sensor readings.

Preliminary measurements put this MP1584EN + ESP8266 + INA219 at a combined average power draw of somewhere around a quarter to a third of a Watt. This is pretty trivial in terms of home power consumption, but not if there is ambition to build nodes that run on battery power. For example, let’s do some simple math with a pair of cheap NiMH rechargeable AA batteries. With a nominal capacity of 2000 mAh and nominal voltage of 1.2V each, that multiplies out (1.2 V * 2 Amps 1 hour * 2 batteries) to 4.8 Watts over an hour. Actual behavior will vary a lot due other variables, but that simple math gives an order of magnitude. Something that constantly draws 0.3 Watt would last somewhere on the order of (4.8 / 0.3) 16 hours, or less than a day, on a pair of rechargeable AA NiMH batteries.

ESPHome has options for putting a node into deep sleep, and the simplest options are based on time like running for X seconds and sleep for Y minutes. For more sophisticated logic, a deep_sleep.enter action is exposed to enter sleep mode. There is also a deep_sleep.prevent action to keep a node awake, and the example is to keep a node awake long enough to upload a code update. This is a problem I’ve tripped over during my MicroPython adventure and it’s nice to see someone has provided a solution in this framework.

The example code reads retained value on a MQTT topic to decide whether to go to sleep or not. I think this is useful, but I also want a locally controlled method for times when MQTT broker is unreachable for any reason. I wanted to dedicate a pin on the ESP8266 for this, with an internal pull-up and an external switch to pull to ground. When the pin is low, the node will go to sleep as programmed. If the pin is high, the node stays awake. I will wire this pin to ground via a jumper so that when the jumper is removed, the node stays awake. And if the jumper is reinstalled, the node goes to sleep.

Such GPIO digital activity can be specified via ESPHome Binary Sensor:

deep_sleep:
  run_duration: 30s
  sleep_duration: 2min
  id: deep_sleep_1

binary_sensor:
  - platform: gpio
    name: "Sleep Jumper"
    id: sleep_jumper
    pin:
      number: 13
      mode:
        input: true
        pullup: true
    on_press:
      then:
        - logger.log: "Preventing deep sleep"
        - deep_sleep.prevent: deep_sleep_1
    on_release:
      then:
        - logger.log: "Entering deep sleep"
        - deep_sleep.enter:
            id: deep_sleep_1
            sleep_duration: 1min

But this is not quite good enough, because on_press only happens if the high-to-low transition happens while the node is awake. If I pull the jumper while the node is asleep, upon wake the pin state is low and my code for high-to-low transition does not run. I needed to check the binary sensor state elsewhere before the sleep timer happens. In the case of this particular project, I also used the analog pin to read battery voltage once every few seconds, so I removed the check from on_press to ADC sensor on_value. (I left on_release code in place so it will still go to sleep when jumper is reinstalled.)

sensor:
  - platform: adc
    pin: A0
    name: "Battery"
    update_interval: 5s
    on_value:
      if:
        condition:
          binary_sensor.is_on: sleep_jumper
        then:
          - logger.log: "Preventing deep sleep"
          - deep_sleep.prevent: deep_sleep_1

This performs a jumper check every time the ADC value is read. This is pretty inelegant code, linking two unrelated topics, but it works for now. It also avoids the problem of digital signal debouncing, which would cause on_press and on_release to both be called in rapid succession unless a delayed_on_off filter is specified.


Ideally, this sensor node would go to sleep immediately after successfully performing a sensor read operation. This should take less than 30 seconds, but the time is variable due to external conditions. (Starting up WiFi, connect to router, connect to Home Assistant, etc.) The naive approach is to call deep_sleep.enter in response to on_value for a sensor, but that was too early. on_value happens immediately after the value is read, before it was submitted to Home Assistant. So when I put it to sleep in on_value, Home Assistant would never receive data. I have to find some other event corresponding to “successfully uploaded value” to trigger sleep, and I haven’t found it yet. The closest so far is the Home Assistant client api.connected condition, but that falls short on two fronts. The first is that it does not differentiate between connecting to Home Assistant (useful) versus ESPHome dashboard (not useful). The second is that it doesn’t say anything about success/failure of sensor value upload. Maybe it’s possible to do something using that condition, in the meantime I wait 30 seconds.

A quick search online found this person’s project also working to prolong battery life for an ESP8266 running ESPHome, and their solution is to use MQTT instead of the Home Assistant communication API. I guess they didn’t find an “after successful send” event, either. Oh well, at least I’m getting data from INA219 between sleep periods, and that data looks pretty good.

Adding Up Power in ESPHome and Home Assistant

Using an INA219 breakout board, I could continuously measure voltage and current passing through a circuit. Data is transmitted by an ESP8266 running ESPHome software and reported to Home Assistant. In order to avoid getting flooded with data, we can use ESPHome sensor filters to aggregate data points. Once we have voltage and current, multiplying them gives us power at a particular instant. The next step is to sum up all of these readings over time to calculate energy produced/consumed. We have two methods to perform this power integration: onboard the microcontroller with ESPHome, or on the Home Assistant server.

ESPHome

The Total Daily Energy component accumulates value from a specified power sensor and integrates a daily tally. (It also needs the Time component to know when midnight rolls around, in order to reset to zero.) The downside of doing this calculation on the controller is that our runny tally must be saved somewhere, or else we would start from zero every time we reset. By default, the tally is saved in flash memory every time a power reading arrives. If power readings are taken at high frequency, this could wear out flash storage very quickly. ESPHome provides two parameters to mitigate wear: we could set min_save_interval to a longer duration in order to reduce the number of writes, or we could set restore to false and skip writing entirely. The former means we lose some amount of data when we reset, the latter means we lose all the data. But your flash memory will thank you!

Home Assistant

Alternatively, we can perform this calculation on Home Assistant server with the unfortunately named integration integration. The first “integration” refers to the math, called Riemann sum integral. The second “integration” is what Home Assistant calls its modules. Hence “integration integration” (which is also very annoying to search for).

Curiously, I found no way in Home Assistant user interface to add this to my instance, I had to go and manually edit configuration.yml as per documentation. After I restarted Home Assistant, a new tally started counting up on my dashboard, but I could not do anything else with the user interface element. I just get an error “This entity does not have a unique ID“.

On the upside, doing this math on the server meant data in progress will be tracked and saved in a real database, kept on a real storage device instead of fragile flash memory. But by default it does not reset at midnight, so the number keeps ticking upwards. Doing more processing with this data is on the to-do list.


Should we do our computation on the microcontroller or on the server? There are certainly advantages to either approach, but right now I lean towards server-side because that lets us put the microcontroller to sleep.

Hello Home Assistant

I have an existing home server set up to run Docker containers, it’s how I’ve been trying out tools like InfluxDB. I added an instance of Home Assistant Core to the list of running containers. When the first screen came up, I was happy to see that it required me to create a username and password before doing anything else. It’s the minimum bar of security, far better than leaving it openly accessible to anyone to probe a known port.

Once I got through initial setup, I was shown the “Overview” dashboard. We can create our own dashboards, but the system starts with this one. It was automatically generated, and by default it is also managed automatically to show everything that pops up. It was populated by a metrological (weather) widget, set to the home location specified during initial setup. I infer this was done so anyone starting fresh with Home Assistant has at least one item to interact with. (Of course, with the focus on local control, Home Assistant has a “Depends on Cloud” label/disclaimer/warning on such features, because it depends on weather data published online.) With weather as starting point, I could add more cards representing devices that already existed on my home network.

The TV was not the only device visible via multiple integrations. My home wireless router was made by Asus, and by default it was visible to the Universal Plug-and-Play Internet Gateway Device integration. However, I disabled that in favor of a more device-specific AsusWRT integration. The latter took a bit more work, as I had to generate a SSH keypair for secure connection between Home Assistant and the router. The public key was pasted into the router’s “Administration” control panel, in the “System” tab, under the “Service” section. I also had to enable SSH (LAN only) and I took the option to change to a nonstandard port.

Once these integrations were added, their associated entities were automatically added to the “Overview” dashboard. This was a lot of data. In fact, I think it is too much data! Thus my first lesson in using Home Assistant is going into the entities list and disabling them. For example, at the moment I don’t see a reason why I needed to know whether my TV is connected via Ethernet or wireless, so I disabled that particular entity. I appreciated the power of having all of these entities at my disposal, but this data overload is also why Home Assistant is not exactly considered beginner friendly.

Anyway, getting my feet wet with Home Assistant was fun, but ESPHome is the reason I’m here.

Notes on Home Assistant Core Docker Compose File

I’m playing with Home Assistant and I started with their Home Assistant Core Docker container image. After a week of use, I understood some of the benefits of going with their full Home Assistant Operating System. If I like Home Assistant enough to keep it around I will likely dig up one of my old computers and make it a dedicated machine. In the meantime, I will continue evaluating Home Assistant by running Home Assistant Core in a container. The documentation even gave us a docker-compose.yml file all ready to go:

version: '3'
services:
  homeassistant:
    container_name: homeassistant
    image: "ghcr.io/home-assistant/home-assistant:stable"
    volumes:
      - /PATH_TO_YOUR_CONFIG:/config
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    privileged: true
    network_mode: host

This is fairly straightforward, but I had wondered about the last two lines.

    privileged: true

First question: why does it need to run in privileged mode? I couldn’t find an answer in Home Assistant documentation. And on the other end, official Docker compose specification just says:

privileged configures the service container to run with elevated privileges. Support and actual impacts are platform-specific.

So the behavior of this flag isn’t even explicitly defined! For the sake of following directions, my first launch of Home Assistant Core image specified true. Once I verified it was up and running, I took down the container and brought it back up without the flag. It seemed to work just fine.

One potential explanation: upon initial startup, Home Assistant needed to create a few directories and files in the mapped volume /config. Perhaps it needed the privileged flag to make sure it had permissions to create those files and set their ownership properly? If so, then I only needed to run with the flag for first execution. If not, then that flag may be completely unnecessary.

    network_mode: host

Second question: why does it need to run in host network mode? Unlike privileged, network mode is much better defined and host means “gives the container raw access to host’s network interface”. I tried running Home Assistant Core with and without this flag. When running without, Home Assistant could no longer automatically detect ESPHome nodes on the network. Apparently auto-discovery requires running in host network mode, and it’s a big part of the convenience of ESPHome. In order to avoid the tedium of getting, tracking, and typing in network addresses, I shall keep this line in my Docker compose file while I play with Home Assistant Core.

Notes on Home Assistant Core vs Home Assistant Operating System

Once I decided to try Home Assistant, the next decision is how to run it. Installation documentation listed many options. Since I’m in the kick-the-tires trial stage, I am not yet ready to dedicate a computer to the task (not even a Raspberry Pi) so I quickly focused on running Home Assistant inside a virtualized environment on my home server. But even then, that left me with two options: run Home Assistant Core in a Docker container, or run Home Assistant Operating System in a virtual machine.

Reading into more details, I was surprised to learn that both cases run Home Assistant Core in a Docker container. The difference is that Home Assistant Operating System also includes a “Supervisor” module that helps manage the Docker instance, doing things like automatic updates (and rollback in case of failure), making backups, and setting up additional Docker instances for Home Assistant add-ons. (ESPHome dashboard is one such addon.) If I opt out of supervisor to run Home Assistant Core on my existing Docker host, I will have to handle my own updates, backups, and add-ons.

Since I already had a backup solution for data used by Docker containers running on my server, I decided to start by running Home Assistant Core directly. After running in this fashion for a week, I’ve learned a few facts in favor of running Home Assistant Operating System on a physical computer:

  • Home Assistant Core updates very frequently, three updates in the first week of playing with it. Thanks to Docker it’s no great hardship to pull a new image and restart, but it’d be nice to have automatic rollback in case of failure.
  • When browsing the wide selection of Home Assistant integrations, there’s usually a little “Add Integration” button that held the promise to automatically set everything up for us. When the thing is an addon that requires running its own Docker container (like the ESPHome dashboard) the promise goes unfulfilled because we’d need the supervisor module for that.
  • When managed by the supervisor, addons like ESPHome can be integrated into the Home Assistant user interface. Versus opening up a separate browser tab when running in a Docker container I manage manually. This also means an addon can integrate with Home Assistant permissions so there’s no need to set up a separate username and password for access control.
  • Some addons like the ESPHome dashboard requires hardware access. In the case of ESPHome, a USB cable is required for flashing initial firmware on an ESP8266/ESP32. Further updates can be done over the network, but that first one needs a cable. Some Docker hosting environments allow routing a physical USB port to the Docker instance, but mine does not.

I could work around these problems so none of them are deal-breakers. But if I like Home Assistant enough to keep it around, I will seriously consider running it on its own physical hardware. Whether that’d be a Raspberry Pi or something else is to be determined.

In the meantime, I will continue running Home Assistant Core in a container. The documentation even gave us a docker-compose.yml file all ready to go, but I was skeptical about running it as-is.