Two Problems Controlling Buck Converter

My solar power monitor project runs on a ESP8266 microcontroller and an INA219 sensor powered by an old USB power bank. In order to charge it during the day from solar power, I’m trying out a new-to-me buck converter module because it exposed an “Enable” pin that was absent from my usual MP1584 buck converter module. I connected it (via a 1k resistor) to the closest available GPIO pin on my ESP8266 module, which happened to be GPIO0. Configuring ESPHome to use that pin as a switch, I could turn charging on or off from Home Assistant UI. I declared victory but it was premature.

I realized there was a problem when I put the ESP8266 to sleep and noticed charging resumed. This was a surprise. Probing the circuit I found my first problem: there is a pull-up resistor or a voltage divider on board my new buck converter module so that if its enable pin is left floating, it will activate itself as my usual MP1584 module would. This was mildly disappointing, because it meant I might have to unsolder a few resistors to get the behavior I originally wanted, and one of the reasons to buy this module was because I didn’t want to unsolder resisters from my MP1584 buck converter boards. As a short-term hack, I fought the existing circuit by adding a pull-down resistor external to the module. Experimentally it looks like a 10k resistor to ground was enough to do the trick, disabling the buck converter when enable input line is left floating.

But I wasn’t done yet, there’s a second problem to address: When ESP8266 was put back in the circuit, the charging would still resume when I put it into deep sleep. Probing the pin, I saw GPIO0 was at 3.3V while asleep. Reading online resources like this page on Random Nerd Tutorials, I learned the pin needs to be high for ESP8266 to boot. Presumably this means the Wemos D1 Mini module has a pull-up resistor on board for the boot process. Therefore I can’t use GPIO0 for charging control.

I went down the list of still-unused pins by distance to the buck converter on my circuit board. The next closest pin is GPIO2, which I can’t use as I’m already using the blue status LED. After that is GPIO14. It is usually used for SPI communication but I have no SPI peripherals for this project. Looking on the reference chart, it doesn’t seem to get pulled up or down while ESP8266 was asleep. After confirming that behavior with a voltmeter, I switched buck converter enable pin over to GPIO14. It allowed me to control charging from ESPHome and, when the ESP8266 is asleep, the buck converter stays disabled. Finally, the hardware is working as intended! Now I need to figure out the software.

ESP8266 ADC Helps Avoid Over-Discharging Battery

I took apart a USB power bank so I could bypass its problematic power output circuit and run a Wemos D1 Mini module directly on the single lithium-ion battery cell. But bypassing the output circuit also means losing its protection against battery over-discharge. This could permanently damage a lithium-ion battery cell, so it is something I have to reimplement.

I can to use the only ADC (analog-to-digital conversion) peripheral on an ESP8266 to monitor battery voltage. The ESP8266 ADC is limited to sensing voltage in the range of zero to one volt, so a voltage divider is necessary to bring the battery’s maximum voltage level of 4.2V down to 1V. The Wemos D1 Mini module has a voltage divider already on board, using 220kΩ and 100kΩ resistors to (almost) bring 3.3V down to 1V. To supplement this, I added another 100kΩ resistor between battery positive and Wemos D1 Mini analog input pin.

For an initial test, I connected the analog input pin to my bench power supply and started adjusting power. It did not quite work as expected, reaching maximum value at a little over 4.1 volts. I suspect one or more of the resistors involved have actual resistance values different than advertised, which is normal with cheap resistors of 15% tolerance.

As a result, I could not sense voltage above 4.1V, which is probably fine for the purpose of over-discharge protection. But I was willing to put in a little extra effort to sense the entire range, and added another 10kΩ resistor in series for a total of 110kΩ between battery positive and the Wemos D1 Mini analog pin. This was enough to compensate for resistor tolerance and allow me to distinguish voltage all the way up to 4.2V.

To translate this divided voltage back to original input voltage, I recorded values from two voltage levels. I recorded what the ESP8266 ADC reported for each, and what I measured with my voltmeter. These data points allow me to use ESPHome Sensor component’s calibrate_linear filter to obtain values good enough to watch for battery over-discharge.

Here are the relevant excerpts from my ESPHome configuration YAML:

  - platform: adc
    pin: A0
    name: "Battery Voltage"
      - calibrate_linear:
          - 0.84052 -> 3.492
          - 0.99707 -> 4.113
        - if:
                id: battery_voltage
                below: 3.0
              - logger.log: "Battery critically low"
              - deep_sleep.enter:

Ideally, I would never reach this point. I should make sure the battery is adequately charged each day. I will need to drop voltage output of solar panel (up to 24V) down to the 5V input voltage expected by an USB power bank’s lithium-ion battery charging circuit, and I want to try a new-to-me buck converter module for the task.

Exploring Low Power ESPHome Nodes

When I investigated adding up power over time into a measure of energy, I found that I have the option of doing it either on board my ESPHome microcontroller or on the Home Assistant server. I’m personally in favor of moving as much computation on the server as I can, and another reason is because keeping the sensor node lightweight gives us the option of putting it to sleep in between sensor readings.

Preliminary measurements put this MP1584EN + ESP8266 + INA219 at a combined average power draw of somewhere around a quarter to a third of a Watt. This is pretty trivial in terms of home power consumption, but not if there is ambition to build nodes that run on battery power. For example, let’s do some simple math with a pair of cheap NiMH rechargeable AA batteries. With a nominal capacity of 2000 mAh and nominal voltage of 1.2V each, that multiplies out (1.2 V * 2 Amps 1 hour * 2 batteries) to 4.8 Watts over an hour. Actual behavior will vary a lot due other variables, but that simple math gives an order of magnitude. Something that constantly draws 0.3 Watt would last somewhere on the order of (4.8 / 0.3) 16 hours, or less than a day, on a pair of rechargeable AA NiMH batteries.

ESPHome has options for putting a node into deep sleep, and the simplest options are based on time like running for X seconds and sleep for Y minutes. For more sophisticated logic, a deep_sleep.enter action is exposed to enter sleep mode. There is also a deep_sleep.prevent action to keep a node awake, and the example is to keep a node awake long enough to upload a code update. This is a problem I’ve tripped over during my MicroPython adventure and it’s nice to see someone has provided a solution in this framework.

The example code reads retained value on a MQTT topic to decide whether to go to sleep or not. I think this is useful, but I also want a locally controlled method for times when MQTT broker is unreachable for any reason. I wanted to dedicate a pin on the ESP8266 for this, with an internal pull-up and an external switch to pull to ground. When the pin is low, the node will go to sleep as programmed. If the pin is high, the node stays awake. I will wire this pin to ground via a jumper so that when the jumper is removed, the node stays awake. And if the jumper is reinstalled, the node goes to sleep.

Such GPIO digital activity can be specified via ESPHome Binary Sensor:

  run_duration: 30s
  sleep_duration: 2min
  id: deep_sleep_1

  - platform: gpio
    name: "Sleep Jumper"
    id: sleep_jumper
      number: 13
        input: true
        pullup: true
        - logger.log: "Preventing deep sleep"
        - deep_sleep.prevent: deep_sleep_1
        - logger.log: "Entering deep sleep"
        - deep_sleep.enter:
            id: deep_sleep_1
            sleep_duration: 1min

But this is not quite good enough, because on_press only happens if the high-to-low transition happens while the node is awake. If I pull the jumper while the node is asleep, upon wake the pin state is low and my code for high-to-low transition does not run. I needed to check the binary sensor state elsewhere before the sleep timer happens. In the case of this particular project, I also used the analog pin to read battery voltage once every few seconds, so I removed the check from on_press to ADC sensor on_value. (I left on_release code in place so it will still go to sleep when jumper is reinstalled.)

  - platform: adc
    pin: A0
    name: "Battery"
    update_interval: 5s
          binary_sensor.is_on: sleep_jumper
          - logger.log: "Preventing deep sleep"
          - deep_sleep.prevent: deep_sleep_1

This performs a jumper check every time the ADC value is read. This is pretty inelegant code, linking two unrelated topics, but it works for now. It also avoids the problem of digital signal debouncing, which would cause on_press and on_release to both be called in rapid succession unless a delayed_on_off filter is specified.

Ideally, this sensor node would go to sleep immediately after successfully performing a sensor read operation. This should take less than 30 seconds, but the time is variable due to external conditions. (Starting up WiFi, connect to router, connect to Home Assistant, etc.) The naive approach is to call deep_sleep.enter in response to on_value for a sensor, but that was too early. on_value happens immediately after the value is read, before it was submitted to Home Assistant. So when I put it to sleep in on_value, Home Assistant would never receive data. I have to find some other event corresponding to “successfully uploaded value” to trigger sleep, and I haven’t found it yet. The closest so far is the Home Assistant client api.connected condition, but that falls short on two fronts. The first is that it does not differentiate between connecting to Home Assistant (useful) versus ESPHome dashboard (not useful). The second is that it doesn’t say anything about success/failure of sensor value upload. Maybe it’s possible to do something using that condition, in the meantime I wait 30 seconds.

A quick search online found this person’s project also working to prolong battery life for an ESP8266 running ESPHome, and their solution is to use MQTT instead of the Home Assistant communication API. I guess they didn’t find an “after successful send” event, either. Oh well, at least I’m getting data from INA219 between sleep periods, and that data looks pretty good.

Adding Up Power in ESPHome and Home Assistant

Using an INA219 breakout board, I could continuously measure voltage and current passing through a circuit. Data is transmitted by an ESP8266 running ESPHome software and reported to Home Assistant. In order to avoid getting flooded with data, we can use ESPHome sensor filters to aggregate data points. Once we have voltage and current, multiplying them gives us power at a particular instant. The next step is to sum up all of these readings over time to calculate energy produced/consumed. We have two methods to perform this power integration: onboard the microcontroller with ESPHome, or on the Home Assistant server.


The Total Daily Energy component accumulates value from a specified power sensor and integrates a daily tally. (It also needs the Time component to know when midnight rolls around, in order to reset to zero.) The downside of doing this calculation on the controller is that our runny tally must be saved somewhere, or else we would start from zero every time we reset. By default, the tally is saved in flash memory every time a power reading arrives. If power readings are taken at high frequency, this could wear out flash storage very quickly. ESPHome provides two parameters to mitigate wear: we could set min_save_interval to a longer duration in order to reduce the number of writes, or we could set restore to false and skip writing entirely. The former means we lose some amount of data when we reset, the latter means we lose all the data. But your flash memory will thank you!

Home Assistant

Alternatively, we can perform this calculation on Home Assistant server with the unfortunately named integration integration. The first “integration” refers to the math, called Riemann sum integral. The second “integration” is what Home Assistant calls its modules. Hence “integration integration” (which is also very annoying to search for).

Curiously, I found no way in Home Assistant user interface to add this to my instance, I had to go and manually edit configuration.yml as per documentation. After I restarted Home Assistant, a new tally started counting up on my dashboard, but I could not do anything else with the user interface element. I just get an error “This entity does not have a unique ID“.

On the upside, doing this math on the server meant data in progress will be tracked and saved in a real database, kept on a real storage device instead of fragile flash memory. But by default it does not reset at midnight, so the number keeps ticking upwards. Doing more processing with this data is on the to-do list.

Should we do our computation on the microcontroller or on the server? There are certainly advantages to either approach, but right now I lean towards server-side because that lets us put the microcontroller to sleep.

ESPHome Sensor Filters Help Manage Flood of Data

I was happy to find that ESPHome made building a sensor node super easy. Once my hardware was soldered together, it took less than ten minutes of software work before I was looking at a flood of voltage and current values coming in from my ESP8266-based power sensor.

It was a flood of data by my decision. Sample code from ESPHome INA219 documentation set data refresh rate at once every sixty seconds. I was hungry for more so I reduced that to a single second. This allowed me to see constantly updating numbers in my Home Assistant dashboard, which is satisfying in a statistics nerd kind of way, but doing so highlighted a few problems. Home Assistant was not designed for this kind of (ab)use. When I click on a chart, it queries from SQLite and took several seconds to plot every single data point on the graph. And since there are far more points than there are pixels on screen, what I get is a crowded mess of lines.

For comparison, InfluxDB and Grafana were designed to handle larger volumes of data and gives us tools for aggregating data. Working with aggregates for analysis and visualization avoids bogging the system down. I’m not sure how to do the same in Home Assistant, or if it’s possible at all, but I do know there are data aggregation tools in ESPHome to filter data before it gets into Home Assistant. These are described in the Sensor Filters documentation. I could still take a reading every second, but I could choose to send just the average value once a minute. Or the maximum value, or the minimum value. Showing average value once a minute, my Home Assistant UI is responsive again. The graph above was for the day I invoked this once-a-minute averaging, and the effect is immediately visible roughly around 10:45PM.

The graph below was from a later day when once-a-minute average was active the entire day:

It is a cleaner graph, but it does not tell the whole story. I had used this INA219 sensor node to measure power consumption of my HP Stream 7 tablet. Measuring once a second, I could see that power use was highly variable with minimum of less than two and a half watts but spikes up to four and a half watts. When showing the average, this information was lost. The average never dropped below 2.4 or rose above 3.2. If I had been planning power capacity for a power supply, this would have been misleading about what I would need. Ideally, I would like to know the minimum and the maximum and the average over a filtered period. If I had been writing my system from scratch, I know roughly what kind of code I would write to accomplish it. Hence the downside of using somebody else’s code: it’s not obvious to me how to do the same thing within this sensor filter framework. I may have to insert my own C code using ESPHome’s Lambda mechanism, something to learn later. [UPDATE: I learned it later and here’s the lambda/template code.] In the meantime I wanted to start adding up instantaneous power figures to calculate energy over time.

Using INA219 Was Super Easy with ESPHome

Once I had ESPHome set up and running, the software side of creating a small wireless voltage and current sensor node with was super easy. I needed to copy sample code for I2C bus component, then sample code for INA219 component, and… that’s it. I started getting voltage, current, and power reports into my Home Assistant dashboard. I am impressed.

It was certainly far less work than the hardware side, which took a bit of soldering. I started with the three modules. From left to right: the INA219 DC sensor board, the MP1584EN DC voltage buck converter, and the ESP8266 in a Wemos D1 Mini form factor.

First the D1 Mini received a small jumper wire connecting D0 to RST, this gives me to option to play with deep sleep.

The MP1584EN was adjusted to output 3.3 volts, then its output was wired directly to the D1 Mini’s 3V3 pin. A small piece of plastic cut from an expired credit card separated them.

The INA219 board was then wired in a similar manner on the other side of D1 mini, with another piece of plastic separating them. For I2C wires I used a white wire for SDA and green wire for SCL lines following Adafruit precedence. Vcc connected to the 3.3V output of MP1584EN in parallel with D1 mini, and ground wires across all three boards. The voltage input for MP1584EN was tapped from Vin- pin of the INA219 board. This means the power consumed by ESP8266 would be included in INA219’s measurements.

A small segment of transparent heat shrink tube packed them all together into a very compact package.

I like the concept of packing everything tightly but I’m squeamish about my particular execution. Some of the wires were a tiny bit longer than they needed to be, and the shrink tube compressed and contorted them to fit. If I do this again, I should plan out wire my lengths for a proper fit.

Like I said earlier, the hardware took far more time than the software, which thanks to ESPHome became a trivial bit of work. I was soon staring at a flood of data, but thankfully ESPHome offers sensor filters to deal with that, too.

Notes on Running ESPHome Dashboard

Once I got Home Assistant running on my home server, I launched an ESPHome container to run alongside and pointed Home Assistant to that container via ESPHome integration. After running it for a while, here are some notes.

Initial Setup Required USB

The primary advantage of this approach is that I have an always-on dashboard for my ESPHome devices, from which I could edit and upload new firmware wirelessly. The primary downside of this approach is that I couldn’t route USB port to this docker instance, so I needed another computer to perform initial firmware flash with USB cable. There are a few options: (1) select “Manual Install” to download a binary file that I would then flash with Advantage: esptool is easy to install. Disadvantage: I have to remember all of the other parameters for flashing. Option (2) copy the configuration YAML file and run a separate instance of ESPHome on the computer with USB port. Advantage: ESPHome took care of all flashing parameters, no need to remember them. Disadvantage: ESPHome not as easy to install. Option (3) select “Manual Install” to download a binary, then use to flash. Advantage: zero setup!

ESPHome /cache Directory

In addition to the required /config directory, we could optionally mount a /cache directory to ESPHome container instance. When present, the directory is used for items that are easily replaceable. For example downloading PlatformIO binaries and intermediate files during compilation. My /config directory is mapped to a ZFS hard drive array. It is regularly backed up so I have a history of my configuration YAML files, but it is not fast. So I mapped /cache to a SSD volume which is fast but not regularly backed up. It also gets quite large, after a few experiments I approached a gigabyte under /cache versus only a few megabytes in /config.

Not Actually Required for Home Assistant

I had thought ESPHome dashboard served as communication node for all my ESP32/ESP8266 boards to talk to Home Assistant. I was wrong! The boards actually get all the code to talk to Home Assistant directly. This meant I don’t strictly need to have Home Assistant Core and ESPHome Dashboard launch together in a common docker-compose.yml file. Home Assistant Core needs to be running, but ESPHome Dashboard could be launched just when I want to wirelessly modify a node.

Add Dashboard Password

But if ESPHome will always be left running as a standalone container, it would be a very good idea to install a minimum bar of security protection. By default, ESPHome dashboard is openly accessible, and I didn’t think it was the best idea. It left open access of all my ESPHome nodes to any port sniffers that might get on my home network. Whether I leave ESPHome running or not, I should at least add a username and password to ESPHome dashboard. This can be done by modifying the container launch commands as per parameters in ESPHome documentation.

version: '3'

    image: esphome/esphome:latest
    command: dashboard --username [my username] --password [my password] /config
    restart: unless-stopped
      - [path to regularly backed-up volume]:/config
      - [path to speedy SSD that isn't backed-up]:/cache
    network_mode: host

This would not be necessary if running as an add-in with Home Assistant Operating System. When installed and managed by the supervisor, access to the ESPHome dashboard becomes part of Home Assistant user control.

All this effort to learn Home Assistant and ESPHome was kicked off when I decided not to write my own code to work with an INA219 voltage+current sensor, hoping it would be easier to use ESPHome instead. And I’m happy to report it absolutely paid off.

Notes on Home Assistant Core Docker Compose File

I’m playing with Home Assistant and I started with their Home Assistant Core Docker container image. After a week of use, I understood some of the benefits of going with their full Home Assistant Operating System. If I like Home Assistant enough to keep it around I will likely dig up one of my old computers and make it a dedicated machine. In the meantime, I will continue evaluating Home Assistant by running Home Assistant Core in a container. The documentation even gave us a docker-compose.yml file all ready to go:

version: '3'
    container_name: homeassistant
    image: ""
      - /PATH_TO_YOUR_CONFIG:/config
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    privileged: true
    network_mode: host

This is fairly straightforward, but I had wondered about the last two lines.

    privileged: true

First question: why does it need to run in privileged mode? I couldn’t find an answer in Home Assistant documentation. And on the other end, official Docker compose specification just says:

privileged configures the service container to run with elevated privileges. Support and actual impacts are platform-specific.

So the behavior of this flag isn’t even explicitly defined! For the sake of following directions, my first launch of Home Assistant Core image specified true. Once I verified it was up and running, I took down the container and brought it back up without the flag. It seemed to work just fine.

One potential explanation: upon initial startup, Home Assistant needed to create a few directories and files in the mapped volume /config. Perhaps it needed the privileged flag to make sure it had permissions to create those files and set their ownership properly? If so, then I only needed to run with the flag for first execution. If not, then that flag may be completely unnecessary.

    network_mode: host

Second question: why does it need to run in host network mode? Unlike privileged, network mode is much better defined and host means “gives the container raw access to host’s network interface”. I tried running Home Assistant Core with and without this flag. When running without, Home Assistant could no longer automatically detect ESPHome nodes on the network. Apparently auto-discovery requires running in host network mode, and it’s a big part of the convenience of ESPHome. In order to avoid the tedium of getting, tracking, and typing in network addresses, I shall keep this line in my Docker compose file while I play with Home Assistant Core.

Notes on Home Assistant Core vs Home Assistant Operating System

Once I decided to try Home Assistant, the next decision is how to run it. Installation documentation listed many options. Since I’m in the kick-the-tires trial stage, I am not yet ready to dedicate a computer to the task (not even a Raspberry Pi) so I quickly focused on running Home Assistant inside a virtualized environment on my home server. But even then, that left me with two options: run Home Assistant Core in a Docker container, or run Home Assistant Operating System in a virtual machine.

Reading into more details, I was surprised to learn that both cases run Home Assistant Core in a Docker container. The difference is that Home Assistant Operating System also includes a “Supervisor” module that helps manage the Docker instance, doing things like automatic updates (and rollback in case of failure), making backups, and setting up additional Docker instances for Home Assistant add-ons. (ESPHome dashboard is one such addon.) If I opt out of supervisor to run Home Assistant Core on my existing Docker host, I will have to handle my own updates, backups, and add-ons.

Since I already had a backup solution for data used by Docker containers running on my server, I decided to start by running Home Assistant Core directly. After running in this fashion for a week, I’ve learned a few facts in favor of running Home Assistant Operating System on a physical computer:

  • Home Assistant Core updates very frequently, three updates in the first week of playing with it. Thanks to Docker it’s no great hardship to pull a new image and restart, but it’d be nice to have automatic rollback in case of failure.
  • When browsing the wide selection of Home Assistant integrations, there’s usually a little “Add Integration” button that held the promise to automatically set everything up for us. When the thing is an addon that requires running its own Docker container (like the ESPHome dashboard) the promise goes unfulfilled because we’d need the supervisor module for that.
  • When managed by the supervisor, addons like ESPHome can be integrated into the Home Assistant user interface. Versus opening up a separate browser tab when running in a Docker container I manage manually. This also means an addon can integrate with Home Assistant permissions so there’s no need to set up a separate username and password for access control.
  • Some addons like the ESPHome dashboard requires hardware access. In the case of ESPHome, a USB cable is required for flashing initial firmware on an ESP8266/ESP32. Further updates can be done over the network, but that first one needs a cable. Some Docker hosting environments allow routing a physical USB port to the Docker instance, but mine does not.

I could work around these problems so none of them are deal-breakers. But if I like Home Assistant enough to keep it around, I will seriously consider running it on its own physical hardware. Whether that’d be a Raspberry Pi or something else is to be determined.

In the meantime, I will continue running Home Assistant Core in a container. The documentation even gave us a docker-compose.yml file all ready to go, but I was skeptical about running it as-is.

Learned About Home Assistant From ESPHome Via WLED

I thought Adafruit’s IO platform was a great service for building network-connected devices. If my current project had been something I wanted to be internet-accessible with responses built on immediate data, then that would be a great choice. However, my current intent is for something locally at home and I wanted the option to query and analyze long term data, so I started looking at Home Assistant instead.

I found out about Home Assistant in a roundabout way. The path started with a tweet from GeekMomProjects:

A cursory look at WLED’s home page told me it is a project superficially similar to Ben Hencke’s Pixelblaze: an ESP8266/ESP32-based platform for building network-connected projects that light up individually addressable LED arrays. The main difference I saw was of network control. A Pixelblaze is well suited for standalone execution, and the network interface is primarily to expose its web-based UI for programming effects. There are ways to control a Pixelblaze over the network, but they are more advanced scenarios. In contrast, WLED’s own interface for standalone effects are dominated by less sophisticated lighting schemes. For anything more sophisticated, WLED has an API for control over the network from other devices.

The Pixelblaze sensor board is a good illustration of this difference: it is consistent with Pixelblaze design to run code that reacts to its environment with the aid of a sensor board. There’s no sensor board peripheral for a WLED: if I want to build a sensor-reactive LED project using WLED, I would build something else with a sensor, and send commands over the network to control WLED lights.

So what would these other network nodes look like? Following some links led me to the ESPHome project. This is a platform for building small network-connected devices using ESP8266/ESP32 as its network gateway, with a library full of templates we can pick up and use. It looks like WLED is an advanced and specialized relative of ESPHome nodes like their adaptation of the FastLED library. I didn’t dig deeper to find exactly how closely related they are. What’s more interesting to me right now is that a lot of other popular electronics devices are available in the ESPHome template library, including the INA219 power monitor I’ve got on my workbench. All ready to go, no coding on my part required.

Using an inexpensive ESP as a small sensor input or output node, and offload processing logic somewhere else? This can work really well for my project depending on that “somewhere else.” If we’re talking about some cloud service, then we’re no better off than Adafruit IO. So I was happy to learn ESPHome is tailored to work with Home Assistant, an automation platform I could run locally.