CL84209 Base Station LCD Segment Map

I have generated a printable lookup chart for the character set of an unknown LCD controller. This chip is embedded inside a black blob along the top edge of an LCD I salvaged from the base station of an AT&T CL84209 cordless phone system. The character set dictates what is rendered in two lines of 15-character alphanumeric text, each character is a dot matrix 5 pixels wide and 7 pixels tall. Below these two lines is a set of custom segmented LCD that can be controlled with 16 bytes, but there appear to be far fewer than 16*8 = 128 segments to control. A segment map will tell us which segments correspond to which bits.

I don’t have the datasheet for this chip, so I don’t know how it numbered its segments. I decided to mostly follow the precedence set by Sanyo LC75853 by numbering both bits and bytes in least-significant-first order. The difference for this map is that while LC75853 started counting from one, I’m going to start counting from zero.

My first attempt at mapping out these segments toggled them on/off alongside the character set data. When displaying characters starting with 0x00, I turned on all the segments whose number has zeroth bit set. For the character set starting with 0x10, I turned on all the segments with the next bit set, etc. In theory I could look at the on/off pattern as it cycled through 0x00 to 0x70 and determine its binary number. I also printed out the bit pattern to Arduino serial console, which looks like this:

0 1 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 10101010 
1 10 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 11001100 
2 100 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 11110000 
3 1000 0 11111111 0 11111111 0 11111111 0 11111111 0 11111111 0 11111111 0 11111111 0 11111111 
4 10000 0 0 11111111 11111111 0 0 11111111 11111111 0 0 11111111 11111111 0 0 11111111 11111111 
5 100000 0 0 0 0 11111111 11111111 11111111 11111111 0 0 0 0 11111111 11111111 11111111 11111111 
6 1000000 0 0 0 0 0 0 0 0 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111 
7 10000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 

In theory this can work, in practice I quickly got tired flipping through images and decoding binary numbers by hand. That was a lot more work than having a quadrature encoder knob to interactively select segments to activate. I had avoided wiring in extra components out of laziness, but between manual binary decoding and soldering, I decided a bit of soldering was the easier path towards this segment map:

This map clearly showed this particular segment allocation used less than 5/8 of available segments. Starting from position zero, we would have five bits for five segments (0, 1, 2, 3, 4) then three bits that were unused. (5, 6, 7) Then five more segments (8, 9, 10, 11, 12) and three unused bits (13, 14, 15). This repeats until the final set of five (120, 121, 122, 123, 124) which were also unused.

I was surprised that segment 9 (“MSG#” title for message count) was a separate segment from 4, the rectangular border around message count. I had expected them to be a single segment.

I was also surprised and annoyed at segment 27, which lights up all three horizontal lines of the hour tens digit for the clock plus the lower left vertical segment. For a clock it made sense to restrict the tens digit to 1 or 2. But if so, why did they bother with segment 26 which isn’t useful for either of those digits? I had hoped maybe I could use it as a generic numeral display and not a clock, by leaving 65 and 66 (AM/PM) inactive. But segment 27 means I could only display 1, 2, 6, and 8. I have not yet thought of an interesting non-clock usage under those restrictions.

Obtaining a full segment map marks the end of investigating this base station LCD, I will now try to do the same for the handset LCD starting with its disassembly.


Source code for this investigation is publicly available on GitHub.

CL84209 Base Station LCD Character Set

After a look at the similarities and differences between two LCDs salvaged from an AT&T CL84209 cordless phone set, I was ready to start controlling one myself to get more data. Earlier probing indicated that I2C communication on the handset was 3.3V, so I’m going to use an ESP8266 because it is also a 3.3V part. The Arduino platform should be a good (enough) starting point, using its Wire library to recreate the I2C control messages I saw sent to the handset LCD.

After a quick test program to verify things worked as expected, I wanted to dump out the character set built inside this LCD controller. Since the character data is sent one byte per character, there are potentially 256 different characters in the font. Dealing with these powers-of-two numbers, I like to keep such information lined up with their hexadecimal values. The LCD can only print up to 15 characters per line, so I couldn’t print these in 16 characters batches. As a next best thing, I dropped down to 8 characters per line along with their starting hexadecimal value. A for() loop cycled through 16 such screens once per second, so I could see the entire character set cycle through.

This is one of the sixteen steps in the cycle. The first line shows characters from 0x70 to 0x77, and the second line from 0x78 to 0x7F. Using workbench lighting, I had trouble trying to photograph the screen without glare. So, I switched to backlighting the LCD using one of my salvaged LED backlights. I took a picture for each of the 16 sections with the intent to edit them together, but the slightly undersized backlight made it difficult. The center section of the screen is noticeably brighter than the edges, which makes it difficult to set a threshold to generate a nice monochrome reference chart. I had to consult Emily Velasco for some photo-editing tips. (I have now added “burn” & “dodge” to my photo editing toolbox.) After some very informative experimentation in GNU Image Manipulation Program I have a reference sheet suitable for printing. Using this chart, I could see that 0x7F is a solid block that activates all pixels of a 5×7 dot matrix, which I will use to as part of an “turn everything on” program to help map out its adjacent segmented region.


Source code for this investigation is publicly available on GitHub.

Partial Home Assistant Control of Mr. Robot Badge Mk. 2

Once I had Mr Robot Badge Mk. 2 up and running on Arduino, I thought: what about Home Assistant? After all, ESPHome is built on compiling ESP8266 Arduino Core code using PlatformIO command-line tools. The challenging part of ESPHome is always that initial firmware flash, which I’m already set up to do, and after that I could do updates wirelessly.

I started easy: getting badge buttons to be recognized as binary sensors in Home Assistant.

binary_sensor:
  - platform: gpio
    pin:
      number: 0
      mode:
        input: true
        pullup: true
      inverted: true
    name: "Left"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 16
      mode:
        input: true
      inverted: true
    name: "Down"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 15
      mode:
        input: true
        pullup: true
    name: "Right"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 13
      mode:
        input: true
        pullup: true
      inverted: true
    name: "Up"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 12
      mode:
        input: true
        pullup: true
      inverted: true
    name: "A"
    filters:
      - delayed_on_off: 20ms
  - platform: gpio
    pin:
      number: 14
      mode:
        input: true
        pullup: true
      inverted: true
    name: "B"
    filters:
      - delayed_on_off: 20ms

Piece of cake! Now on to the more interesting (and more challenging) part: using the LEDs. I followed ESPHome examples to create a custom component. This exposed a single custom output that treated the entire array as a single color (monochromatic) dimmable light.

C header file with custom class declaration:

#include "esphome.h"
#include "Adafruit_IS31FL3741.h"
using namespace esphome;

class MrRobotBadgeFloatOutput : public Component, public FloatOutput
{
    Adafruit_IS31FL3741 ledArray;

    public:
        void setup() override
        {
            ledArray.begin();
            ledArray.setLEDscaling(0x3F); // 25% of max
            ledArray.setGlobalCurrent(0x3F); // 25% of max
            ledArray.enable(true);  // bring out of shutdown
        }

        void write_state(float state) override
        {
            int8_t value = state * 128; // 50% of max
            ledArray.fill(value);
        }
};

YAML:

output:
  - platform: custom
    type: float
    lambda: |-
      auto badge_float = new MrRobotBadgeFloatOutput();
      App.register_component(badge_float);
      return {badge_float};
    outputs:
      id: badge_float

light:
  - platform: monochromatic
    name: "Badge Lights"
    output: badge_float

Tada! I’m back at the same test I had using Arduino, lighting up all the pixels to highlight problematic LEDs.

The next step in this progression is to expose the entire 18×18 array as individually addressable pixels. Doing so would allow using ESPHome’s display rendering engine to draw text and graphics, much as Adafruit GFX does. I found the Addressable Light base class but I got stuck trying to make it happen. I’ve had no luck finding examples on how to implement a custom addressable light, and didn’t get much of anywhere bumping my head in the dark. Digging through existing Addressable Light implementations on GitHub, I’ve found many object classes that may or may not be required to implement my own addressable light.

I imagine there’s an architecture diagram somewhere that describes how these components interact, but my search has come up empty so far and there’s not much in the way of code comments. Most of the online discussions about addressable LED pixels in Home Assistant are centered around WLED, which as of this writing does not support the IS31LF3741 control chip. I’m going to put further exploration on hold until I have more experience with ESPHome custom components to understand their object hierarchy.

Programming Mr Robot Badge Mk. 2 with Arduino

This particular Mr. Robot Badge Mk. 2 was deemed a defective unit with several dark LEDs. I used it as a practice subject for working with surface-mounted electronic devices, bringing two LEDs back into running order though one of them is the wrong color. Is the badge fully repaired? I couldn’t quite tell. The default firmware is big on blinking and flashing patterns, making it difficult to determine if a specific LED is functioning or not. What I needed was a test pattern, something as simple as illuminate all of the LEDs to see if they come up. Fortunately, there was a URL right on the badge that took me to a GitHub repository with sample code and instructions. It used Arduino framework to generate code for this ESP8266, and that’s something I’ve worked with. I think we’re in business.

On the hardware side, I soldered sockets to the unpopulated programmer header and then created a programming cable to connect to my FTDI serial adapter (*). For the software, I cloned the “Starter Pack” repository, followed installation directions, and encountered a build failure:

Arduino\libraries\Brzo_I2C\src\brzo_i2c.c: In function 'brzo_i2c_write':
Arduino\libraries\Brzo_I2C\src\brzo_i2c.c:72:2: error: cannot find a register in class 'RL_REGS' while reloading 'asm'
   72 |  asm volatile (
      |  ^~~
Arduino\libraries\Brzo_I2C\src\brzo_i2c.c:72:2: error: 'asm' operand has impossible constraints
exit status 1
Error compiling for board Generic ESP8266 Module.

This looks like issue #44 in the Brzo library, unfixed at time of this writing. Hmm, this is a problem. Reading the code some more, I learned Brzo is used to create I2C communication routines with the IS31FL3741 driver chip controlling the LED array. Aha, there’s a relatively easy solution. Since the time the badge was created, Adafruit has released a product using the same LED driver chip and corresponding software libraries to go with it. I could remove this custom I2C communication code using Brzo and replace it with Adafruit’s library.

Most of the conversion was straightforward except for the LED pixel coordinate lookup. The IS31Fl3741 chip treats the LEDs as a linear array, and something had to translate the linear index to their X,Y coordinates. The badge example code has a lookup table mapping linear index to X,Y coordinates. To use Adafruit library’s frame buffer, I needed the reverse: a table that converts X,Y coordinates to linear index. I started typing it up by hand before deciding that was stupid: this is the kind of task we use computers for. So I wrote this piece of quick-and-dirty code to cycle through the existing lookup table and print the information back out organized by X,Y coordinates.


      for(uint8_t x = 0; x < 18; x++)
      {
        for(uint8_t y = 0; y < 18; y++)
        {
          reverseLookup[x][y]=-1;
        }
      }
      for(uint8_t i = 0; i < PAGE_0_SZ; i++)
      {
        reverseLookup[page0LUT[i].x][page0LUT[i].y] = i;
      }
      for(uint16_t i = 0; i < PAGE_1_SZ; i++)
      {
        // Unused locations were marked with (-1, -1) but x and y are
        // declared as unsigned which means -1 is actually 0xFF so
        // instead of checking for >0 we check for <18
        if(page1LUT[i].x < 18 && page1LUT[i].y < 18)
        {
          reverseLookup[page1LUT[i].x][page1LUT[i].y] = i+PAGE_0_SZ;
        }
      }
      for(uint8_t y = 0; y < 18; y++)
      {
        Serial.print("{");
        for(uint8_t x = 0; x < 18; x++)
        {
          Serial.print(reverseLookup[x][y]);
          if (x<17)
          {
            Serial.print(",");
          }
        }
        if (y<17)
        {
          Serial.println("},");
        }
        else
        {
          Serial.println("}");
        }
      }

This gave me the numbers I needed in Arduino serial monitor, and I could copy it from there. Some spacing adjustment to make things a little more readable, and we have this:

const uint16_t Lookup[ARRAY_HEIGHT][ARRAY_WIDTH] = {
  {  17,  47,  77, 107, 137, 167, 197, 227, 257,  18,  48,  78, 108, 138, 168, 198, 228, 258},
  {  16,  46,  76, 106, 136, 166, 196, 226, 256,  19,  49,  79, 109, 139, 169, 199, 229, 259},
  {  15,  45,  75, 105, 135, 165, 195, 225, 255,  20,  50,  80, 110, 140, 170, 200, 230, 260},
  {  14,  44,  74, 104, 134, 164, 194, 224, 254,  21,  51,  81, 111, 141, 171, 201, 231, 261},
  {  13,  43,  73, 103, 133, 163, 193, 223, 253,  22,  52,  82, 112, 142, 172, 202, 232, 262},
  {  12,  42,  72, 102, 132, 162, 192, 222, 252,  23,  53,  83, 113, 143, 173, 203, 233, 263},
  {  11,  41,  71, 101, 131, 161, 191, 221, 251,  24,  54,  84, 114, 144, 174, 204, 234, 264},
  {  10,  40,  70, 100, 130, 160, 190, 220, 250,  25,  55,  85, 115, 145, 175, 205, 235, 265},
  {   9,  39,  69,  99, 129, 159, 189, 219, 249,  26,  56,  86, 116, 146, 176, 206, 236, 266},
  {   8,  38,  68,  98, 128, 158, 188, 218, 248,  27,  57,  87, 117, 147, 177, 207, 237, 267},
  {   7,  37,  67,  97, 127, 157, 187, 217, 247,  28,  58,  88, 118, 148, 178, 208, 238, 268},
  {   6,  36,  66,  96, 126, 156, 186, 216, 246,  29,  59,  89, 119, 149, 179, 209, 239, 269},
  {   5,  35,  65,  95, 125, 155, 185, 215, 245, 270, 279, 288, 297, 306, 315, 324, 333, 342},
  {   4,  34,  64,  94, 124, 154, 184, 214, 244, 271, 280, 289, 298, 307, 316, 325, 334, 343},
  {   3,  33,  63,  93, 123, 153, 183, 213, 243, 272, 281, 290, 299, 308, 317, 326, 335, 344},
  {   2,  32,  62,  92, 122, 152, 182, 212, 242, 273, 282, 291, 300, 309, 318, 327, 336, 345},
  {   1,  31,  61,  91, 121, 151, 181, 211, 241, 274, 283, 292, 301, 310, 319, 328, 337, 346},
  {   0,  30,  60,  90, 120, 150, 180, 210, 240, 275, 284, 293, 302, 311, 320, 329, 338, 347}
};

This lookup table got Mr. Robot Badge Mk. 2 up and running without Brzo library! I could write that simple “turn on all LED” test I wanted.

The test exposed two more problematic LEDs. One of them was intermittent: I tapped it and it illuminated for this picture. If it starts flickering again, I’ll give it a dab of solder to see if that helps. The other one is dark and stayed dark through (unscientific) tapping and (scientific) LED test equipment. It looks like I need to find two more surface-mount red LEDs to fully repair this array.

In case anybody else wants to play with Mr. Robot Badge and runs into the same problem, I have collected my changes and updated the README installation instructions in pull request #3 of the badge code sample repository. If that PR rejected, clone my fork directly.


(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Flash Memory Wear Effects of ESPHome Recovery: ESP8266 vs. ESP32

One major difference between controlling charging of a battery and controlling power to a Raspberry Pi is the tolerance for interruptions. Briefly interrupting battery charging is nothing to worry about, we can easily pick up where we left off. But a brief interruption of Raspberry Pi power means it will reset. At the minimum we will lose in-progress work, but consequences can get worse including corruption of the microSD card. If I put an ESPHome node in control of Raspberry Pi power, what happens when that node reboots? I don’t want it to trigger a Raspberry Pi reboot as well.

This was on my mind when I read ESPHome documentation for GPIO Switch: There is a parameter “restore_mode” that allows us to specify how that switch will behave upon bootup. ALWAYS_ON and ALWAYS_OFF are straightforward: the device is hard-coded to flip the switch on/off upon bootup. Neither of these would be acceptable for this case, so I have to use one of the restore options. I added it to my ESP32 configuration and performed an OTA firmware update to trigger a reboot. I was happy to see there was no interruption to the Pi. Or at least if there was, it was short enough that the capacitors I added to my Raspberry Pi power supply was able to bridge the gap.

This is great! But how does the device know the previous state to restore? The most obvious answer is to store information in the onboard flash memory for these devices, but flash memory has a wear life that embedded developers must keep in mind. Especially when dealing with inexpensive components like ESP8266 and ESP32 modules. Their low price point invites use of inexpensive flash with a short wear life. I don’t know how to probe flash memory to judge their life, but I do know ESPHome is an open-source project and I could dig into source code.

ESPHome GPIO Switch page has a link to Core Configuration, where there’s a deprecated flag esp8266_restore_from_flash to dictate whether to store persistent data in flash memory. That gave me the keyword needed to find the Global Variables section on ESPHome Automations page. Where it said there is only 96 bytes available in a mechanism called “RTC memory” and that it would not survive a power-cycle. That didn’t sound very useful but researching further I learned it survives deep sleep and so there’s utility there. Searching in ESPHome GitHub repository, I found the file preferences.cpp for ESP8266 where I believe the implementation lives. It defaults to false which means the default wouldn’t wear out ESP8266 flash memory but at the expense of RTC memory not surviving a power cycle. If we really need that level of recovery and switch esp8266_restore_from_flash to true, we have an additional knob to make trade offs between accuracy and flash memory lifespan using the flash_write_interval parameter.

So that covers ESPHome running on an ESP8266. What about an ESP32? While I see that ESP32 has its own concept of RTC memory, looking in ESPHome source code for ESP32 variant of preferences.cpp I see that it used a different mechanism called NVS. Non-Volatile Storage library is tailored for storing small key-value pairs in flash memory, and was written to minimize wear. This is great. Even better, the API also leaves the door open for different storage mechanisms in future hardware revisions, possibly something with better write durability.

From this, I conclude that ESPHome projects that require restoring states through reboots events are better off running on an ESP32 and its dedicated NVS mechanism. I didn’t have this particular feature in mind when I made the decision to use an ESP32 to build my power-control board, but in hindsight that was the right choice! Armed with confidence in the hardware, I can patch up a few to-do items in my ESPHome-based software.

Initial Logic for Solar Monitor Project

I think I’ve got the hardware portions of my solar power monitor sensor node figured out, so I can write the first version of corresponding software logic. I have set out the following requirements:

  • Over-discharge protection: If battery voltage drops below a threshold, put the system to sleep to protect the battery.
  • Low solar output: When the solar panel isn’t generating any power, put the system to sleep.
  • Battery charging start: when panel power generation rises above a certain level for the first time that day, start charging the repurposed USB power bank battery.
  • Battery charging pause: If cloud cover causes a dip in solar power, pause charging.
  • Battery charging stop: Once battery cell voltage rises to a certain level, stop charging.
  • Sleep override: local hardware method to prevent deep sleep.

The ESPHome documentation for deep sleep described one way to prevent sleep using MQTT, keeping a node awake to receive firmware updates. But I wanted something even lower level hence the jumper and it became useful when I implemented the “low solar output, go to sleep” logic. Apparently INA219 component’s first power value always return zero. Which meant as soon as it booted up, that initial zero reading puts the node immediately back to sleep before it would even get on the network. (Never mind checking MQTT!) The solution is to switch from sampling values once a minute to sampling once a second, and make decisions based on average over a minute.

A different approach would be to go to sleep based on sun’s position in the sky, which can be queried and used in the Sun component. However, I expect this component has dependency on network connection (it needs to know the time, for starters) and would not be reliable if the network goes down. It also doesn’t know if the sun is obscured by clouds, so I think it’s better to use panel power output to decide what to do during the day. But I may explore using the Sun component in a future version to sleep all through the night instead of waking up every few minutes to fruitlessly check power level.

Strictly speaking, I don’t need to worry about stopping battery charging. I can supply 5V all day when panel delivers power, and trust the USB power bank charging circuit to keep the battery from being overcharged. But keeping lithium-ion cells full would shorten their useful life, so in the interest of battery longevity I’ll stop charging before full. On that topic: for optimal battery life I should charge it slowly over the course of the day, but I don’t have control over charging rate used by USB power bank.

One thing I don’t know yet is how the system will handle several rainy days in a row. I assume this panel can still generate enough power to charge an 18650 battery cell, but I might be wrong! I’ll have to wait for a long stretch of rain to come to Southern California, which may be a long wait. After seeing its behavior I can adjust for a future version.

Here’s version 1 of my ESPHome configuration YAML, and I expect to fine tune various hard-coded threshold values over the weeks ahead while I build more projects:

# Blue LED on the ESP8266 module signals connection status.
status_led:
  pin:
    number: 2
    inverted: true

# The goal is to charge once a day, and this flag tracks if we've already done it.
globals:
  - id: never_charged_today
    type: bool
    restore_value: no
    initial_value: "true"

# We can go to deep sleep to conserve battery, but sometimes we don't want to
# actually go to sleep. For example, when we need to upload a firmware update.
# Pin 13 is an input pin with internal pullup. It should be wired to a jumper
# that would ground the pin if jumper is present. Removing the jumper should
# disable going to deep sleep. To enforce this, call try_to_sleep script
# instead of calling deep_sleep.enter directly.
deep_sleep:
  id: deep_sleep_1

binary_sensor:
  - platform: gpio
    name: "Disable Sleep"
    id: sleep_jumper
    pin:
      number: 13
      mode:
        input: true
        pullup: true
    on_release:
      then:
        - logger.log: "Sleep jumper installed"
        - script.execute: try_to_sleep

script:
  - id: try_to_sleep
    then:
      if:
        condition:
          binary_sensor.is_on: sleep_jumper
        then:
          logger.log: "Sleep requested but staying awake due to override jumper"
        else:
          - logger.log: "Sleep requested and permitted by jumper"
          - delay: 5s # Allow sensor values to be sent.
          - deep_sleep.enter:
              id: deep_sleep_1
              sleep_duration: 10min

# This should be wired to a 1k resistor, which then connects to the enable pin
# of a power supply source. When ON, it should deliver power to charge the battery.
switch:
  - platform: gpio
    pin: D5
    id: charge_switch
    name: "Charge Battery"
    restore_mode: RESTORE_DEFAULT_OFF

# An I2C INA219 sensor monitors panel voltage, current, and calculates power.
i2c:
  sda: 4
  scl: 5

sensor:
  - platform: ina219
    address: 0x40
    shunt_resistance: 0.1 ohm
    max_voltage: 24.0V
    max_current: 3.2A
    update_interval: 1s
    current:
      name: "Panel Current"
      id: solar_panel_current
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
    power:
      name: "Panel Power"
      id: solar_panel_power
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
      on_value:
        then:
          # When power is low, put the board to sleep.
          # Note: upon boot, the first reading of current (and therefore power) always
          # seems to be zero, so we need to run moving average filters to ensure we
          # don't shut off immediately on power-up.
          if:
            condition:
              and:
                - sensor.in_range:
                    id: solar_panel_power
                    below: 0.01
                - sensor.in_range:
                    id: solar_panel_voltage
                    below: 3.0
            then:
              - logger.log: "Panel delivering low power, should go to sleep"
              - globals.set:
                  id: never_charged_today
                  value: "true"
              - script.execute: try_to_sleep
    bus_voltage:
      name: "Panel Voltage"
      id: solar_panel_voltage
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
# ESP8266 ADC pin should be wired to a resistor just over 100kOhm to measure
# lithium-ion battery cell voltage. Values under calibrate_linear need to be
# customized for each board (and their resistors.)
  - platform: adc
    pin: A0
    name: "Battery Voltage"
    id: battery_voltage
    update_interval: 1s
    accuracy_decimals: 3
    filters:
      - calibrate_linear:
          - 0.84052 -> 3.492
          - 0.99707 -> 4.113
      - sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
    on_value:
      then:
        - if:
            condition:
              and:
                - lambda: "return id(never_charged_today);"  
                - sensor.in_range:
                    id: solar_panel_power
                    above: 10
            then:
              - logger.log: "Panel has power, start charging for the day"
              - globals.set:
                  id: never_charged_today
                  value: "false"
              - switch.turn_on: charge_switch
        - if:
            condition:
              and:
                - switch.is_on: charge_switch
                - sensor.in_range:
                    id: solar_panel_power
                    below: 5
            then:
              - logger.log: "Charging paused due to low panel output"
              - globals.set:
                  id: never_charged_today
                  value: "true" # Resume charging if power returns
              - switch.turn_off: charge_switch
        # When battery is low enough to trigger this emergency measure, we
        # would not be able to activate charging ourselves. Charging needs to
        # be activated manually (or at least externally)
        - if:
            condition:
              sensor.in_range:
                id: battery_voltage
                below: 3.0
            then:
              - logger.log: "Battery critically low, should sleep to protect battery."
              - script.execute: try_to_sleep
    # We don't need a full charge to last through a day, so turn off charging
    # well before reaching maximum in order to improve battery longevity
    on_value_range:
      above: 4.0
      then:
        - logger.log: "Battery charge is sufficient"
        - switch.turn_off: charge_switch

Two Problems Controlling Buck Converter

My solar power monitor project runs on a ESP8266 microcontroller and an INA219 sensor powered by an old USB power bank. In order to charge it during the day from solar power, I’m trying out a new-to-me buck converter module because it exposed an “Enable” pin that was absent from my usual MP1584 buck converter module. I connected it (via a 1k resistor) to the closest available GPIO pin on my ESP8266 module, which happened to be GPIO0. Configuring ESPHome to use that pin as a switch, I could turn charging on or off from Home Assistant UI. I declared victory but it was premature.

I realized there was a problem when I put the ESP8266 to sleep and noticed charging resumed. This was a surprise. Probing the circuit I found my first problem: there is a pull-up resistor or a voltage divider on board my new buck converter module so that if its enable pin is left floating, it will activate itself as my usual MP1584 module would. This was mildly disappointing, because it meant I might have to unsolder a few resistors to get the behavior I originally wanted, and one of the reasons to buy this module was because I didn’t want to unsolder resisters from my MP1584 buck converter boards. As a short-term hack, I fought the existing circuit by adding a pull-down resistor external to the module. Experimentally it looks like a 10k resistor to ground was enough to do the trick, disabling the buck converter when enable input line is left floating.

But I wasn’t done yet, there’s a second problem to address: When ESP8266 was put back in the circuit, the charging would still resume when I put it into deep sleep. Probing the pin, I saw GPIO0 was at 3.3V while asleep. Reading online resources like this page on Random Nerd Tutorials, I learned the pin needs to be high for ESP8266 to boot. Presumably this means the Wemos D1 Mini module has a pull-up resistor on board for the boot process. Therefore I can’t use GPIO0 for charging control.

I went down the list of still-unused pins by distance to the buck converter on my circuit board. The next closest pin is GPIO2, which I can’t use as I’m already using the blue status LED. After that is GPIO14. It is usually used for SPI communication but I have no SPI peripherals for this project. Looking on the reference chart, it doesn’t seem to get pulled up or down while ESP8266 was asleep. After confirming that behavior with a voltmeter, I switched buck converter enable pin over to GPIO14. It allowed me to control charging from ESPHome and, when the ESP8266 is asleep, the buck converter stays disabled. Finally, the hardware is working as intended! Now I need to figure out the software.

ESP8266 ADC Helps Avoid Over-Discharging Battery

I took apart a USB power bank so I could bypass its problematic power output circuit and run a Wemos D1 Mini module directly on the single lithium-ion battery cell. But bypassing the output circuit also means losing its protection against battery over-discharge. This could permanently damage a lithium-ion battery cell, so it is something I have to reimplement.

I can to use the only ADC (analog-to-digital conversion) peripheral on an ESP8266 to monitor battery voltage. The ESP8266 ADC is limited to sensing voltage in the range of zero to one volt, so a voltage divider is necessary to bring the battery’s maximum voltage level of 4.2V down to 1V. The Wemos D1 Mini module has a voltage divider already on board, using 220kΩ and 100kΩ resistors to (almost) bring 3.3V down to 1V. To supplement this, I added another 100kΩ resistor between battery positive and Wemos D1 Mini analog input pin.

For an initial test, I connected the analog input pin to my bench power supply and started adjusting power. It did not quite work as expected, reaching maximum value at a little over 4.1 volts. I suspect one or more of the resistors involved have actual resistance values different than advertised, which is normal with cheap resistors of 15% tolerance.

As a result, I could not sense voltage above 4.1V, which is probably fine for the purpose of over-discharge protection. But I was willing to put in a little extra effort to sense the entire range, and added another 10kΩ resistor in series for a total of 110kΩ between battery positive and the Wemos D1 Mini analog pin. This was enough to compensate for resistor tolerance and allow me to distinguish voltage all the way up to 4.2V.

To translate this divided voltage back to original input voltage, I recorded values from two voltage levels. I recorded what the ESP8266 ADC reported for each, and what I measured with my voltmeter. These data points allow me to use ESPHome Sensor component’s calibrate_linear filter to obtain values good enough to watch for battery over-discharge.

Here are the relevant excerpts from my ESPHome configuration YAML:

  - platform: adc
    pin: A0
    name: "Battery Voltage"
    filters:
      - calibrate_linear:
          - 0.84052 -> 3.492
          - 0.99707 -> 4.113
    on_value:
      then:
        - if:
            condition:
              sensor.in_range:
                id: battery_voltage
                below: 3.0
            then:
              - logger.log: "Battery critically low"
              - deep_sleep.enter:

Ideally, I would never reach this point. I should make sure the battery is adequately charged each day. I will need to drop voltage output of solar panel (up to 24V) down to the 5V input voltage expected by an USB power bank’s lithium-ion battery charging circuit, and I want to try a new-to-me buck converter module for the task.

Running Wemos D1 Mini ESP8266 On Single Lithium-Ion 18650 Cell

I’ve taken apart a broken USB power bank and the 18650 battery cell within has stayed within nominal range. Its battery and charging circuit looks good, or at least doesn’t do anything obviously bad with that battery. I take it as confirmation of my hypothesis that it’s the 5V boost output circuit that is broken, which is great because I plan to ignore that part.

I couldn’t quite decipher the exact voltage regulator used on board the Wemos D1 Mini clone I bought via Amazon. But from my time running these modules on weak AAs, they seem to behave like LDO (low-dropout regulator) in that they are happy to deliver 3.3V even if the input voltage hovers barely above 3.3V. And if the supply voltage drops even further, that is passed through instead of quitting or making weird noises like as a MP1584 buck converter did. As long as I keep this 18650 cell operating in the healthy range of 3V to 4.2V, I can wire it directly to the “5V” input pin on a Wemos D1 Mini module and it should deliver enough power to run an ESP8266 and INA219.

To mount the USB power bank circuit, I first looked at the existing pin headers hoping they’ll mount directly on a perforated prototype board with 0.1″ pitch. Sadly they are slightly narrower than that (2mm pitch?) and would not fit. However, the battery connectors are close enough to 0.3″ apart that I could solder 0.1″ header pins and mount that to the board. This is a perfect way to tap directly into the battery.

I cut up a small piece of plastic (expired credit card) to serve as insulation between circuit boards and from there it was straightforward to mount this on my prototype board. The battery cell is then soldered to these pins and temporarily secured with tape.

I was all ready to solder these pins directly to the Wemos D1 Mini as well, but then I realized doing so would leave no graceful way to cut power. So I added a pair of jumpers (one for BT+, one for BT-) allowing me to turn things off if needed. Once I finished soldering (and probing to verify I hadn’t shorted anything) I put the jumpers in and saw the blinking LED of an ESP8266 starting up. Success!

Bypassing the broken power output portion of this USB power bank puts this little piece of equipment back to work instead of tossing it in electronic waste. But it also meant I lost over-discharge protection so I will have to implement that myself.

Plotting Solar Panel Voltage and Power

Ever since I bought a cheap solar panel array from Harbor Freight, I’ve wanted to quantify its output. Most solar power instruments were designed for house-sized (or larger) arrays, overkill for my little panel. I had instruments from the world of remote-control hobbies to measure voltage and current, but they were only instantaneous values displayed on screen. I wanted more! This motivated my journey evaluating software tools like InfluxDB database, Grafana visualizer, Home Assistant, and ESPHome. Plus building on what I knew about hardware components like ESP8266 microcontroller, MP1584 buck converter, and INA219 sensor. All with the goal of building my own solar production monitoring system.

I had thought I could make the ESP8266+INA219 sensor node run exclusively on solar power, but I gave up on that and have built a temporary setup powered by old tired alkaline AA batteries. Enough to give me a graph!

This was a fairly sunny day, so cloud cover effects were minimal. The solar panel was connected to a MPPT charger. I could see the power levels (green line) climb smoothly from sunrise, reach a maximum at noon, then smoothly fading off to sunset. This was mostly as expected. The power curve is not symmetric because there are a few things blocking sunlight from the east, reducing morning power.

While the power curve was mostly expected, the voltage levels were not. Until this graph I didn’t know solar panels would jump up so high on voltage before producing meaningful amount of power. (I don’t know if the MPPT charger makes a difference here.) If I had known this, I probably wouldn’t have bothered trying to make a solar powered circuit turn on and off based on voltage. Or if I did, I would have tried anyway and knew why it failed. From this graph I learned that if I am to draw upon solar power, I need to base my decision on power and not on voltage. But in order to read power levels, I need power for my INA219! There’s probably a clever way to circumvent this Catch-22 without batteries, but I’m going to take the easy way out and incorporate a rechargeable battery.

Running ESP8266 on Tired Alkaline AA Batteries

I retreated from trying to run a voltage & current sensor node exclusively on solar power. I’ll add some rechargeable batteries instead, but before I tackle the complexity of incorporating charging circuit (and charging logic) I took a side trip. I’m running the circuit on non-rechargeable alkaline AA batteries for a few days. And not even fresh batteries, these were already-weakened batteries from which I wanted to extract some more utility before I put them in a Joule thief LED circuit.

These batteries have open voltage of only around 1.1V, which is very low by standard of alkalines. (Fresh alkaline AAs measure over 1.5V.) And that open circuit voltage drops very quickly when I put it under strain of powering an ESP8266. So I keep usage brief, with sensor measurement and reporting run of just 30 seconds. (I later reduced it to 15 seconds.) Then I put the ESP8266 to sleep for five minutes. That five minutes give the chemistry inside these very weak batteries a chance to balance out and recover a little power before I hit them with another read+report session.

To measure how those batteries held up under the strain, I was going to wire up another voltage divider to the ESP8266 analog pin A0. But then I saw ESP8266 had an existing provision to report its chip input voltage VCC, no additional wiring necessary, and that capability is exposed in ESPHome. I chose this option out of laziness because it meant I didn’t have to warm up the soldering iron again.

The results: these weakened batteries could still give me a few days of readings before they started fading too much. I could see this in the Home Assistant log of reported VCC values.

For the first day or so, the batteries could deliver enough power that the ESP8266 didn’t see any drop in VCC voltage. But on the second or third day, voltage sags below 2.9V towards the end of my read+report session. This is visible as the lower bound of the VCC graph. During the five-minute sleep period, voltage recovers slightly and this is visible as the upper bound of the VCC graph. As the batteries continue to fade, both the upper bound (how much power the batteries could recover) and the lower bound (how low the voltage sags at the end of the session) decline over time. Eventually, VCC sagged to 1.75V and the ESP8266 could not run far enough to go back to sleep. That’s all I could get out of that particular batch of AAs. I’m happy they could run this circuit for a few days. They taught me some limits of running ESP8266 on low power, and gathered some interesting data along the way.

Start Simple with Alkaline AA Batteries

I thought I’d whip up a circuit to run exclusively on solar power. Growing around solar-powered electronics like desktop calculators, I knew it was a solved problem. But after running into obstacles, I have been humbled by the challenges involved and decided to fall back to battery power. A lot of solar power projects have a rechargeable battery somewhere in the mix, and I’m going to follow that precedence in the hopes of simplified energy management.

But as an intermediate stepping stone, I will adapt my circuit to run on batteries without worrying about the charging circuit just yet. I have the components I need on hand: a pile of alkaline AA batteries and a tray for 5*AA batteries in series.

A fresh AA alkaline battery has an open-circuit voltage just over 1.5V, and four of those in series would deliver more than 6V. Plenty for an ESP8266, but I’m not using fresh batteries for this project. My fresh AA batteries go into devices with motors or other high drain use. Once those devices complain the batteries were too weak, I move them into purely electronic devices with lower amperage demands. (TV remote controls, Hackaday badges, Xbox wireless controllers, etc.) When they are deemed too weak again, they go into my pile of AA batteries awaiting Joule thief LED duty. Open-circuit voltage for veteran batteries in this pile hover around 1.1V, thus I needed five of them in series instead of just four.

These 5-ish volts are too low to activate my modified MP1584 buck converter, which would no longer activate until input voltage of at least 13V. But that’s not a problem, because the Wemos D1 Mini clone board I’m using could run on 5V USB power. These batteries are pretty close to that voltage level, so I bypassed the MP1584 and connected the battery tray to existing “5V” pin on this module and used its onboard voltage regulator (which I didn’t trust to handle solar power directly) to deliver 3.3V to the ESP8266 and INA219. This worked pretty well.

Solar Startup Still Tricky

I have modified a second MP1584 buck converter module so that it would not activate until input voltage surpasses 13V, comfortably above the output voltage of 3.3V. I want to connect input pins to a solar panel so the associated components (ESP8266 WiFi microcontroller and INA219 voltage/current sensor) would be powered exclusively by the sun.

First is a test run with my bench power supply. Gradually increasing supply voltage starting from zero volts. Thanks to the modification, there were no odd behavior or sounds of a MP1584 trying to work with too low of an input voltage, which is great. As I increased voltage past the ~13V threshold, I saw the blue LED of my ESP8266 blink and it booted up as planned. This time, it was able to find INA219 on I2C bus, which is further than I got before.

Feeling optimistic, I connected this circuit to my solar panel at night and hoped I would wake up to find the system running. Sadly I woke up to disappointment, as there were no logged messages from the ESP8266. Probing the circuit with my volt meter, I confirmed a 3.3V supply voltage was present, but for whatever reason the ESP8266 failed to boot that morning. I manually disconnected and reconnected the circuit board, and this time ESP8266 booted up fine (now it has full daylight power) and started reporting values measured by INA219.

I don’t know what happened at sunrise. I hypothesize that when the solar panel output voltage rose past 13V, it has still yet to produce enough power to successfully start an ESP8266. So when MP1584 activated, it could supply 3.3V but not enough amperage to supply an ESP8266 through its boot process, putting it in a glitched state that was neither on nor off and stuck there until I power cycled the system. [UPDATE: Further experimentation found this hypothesis was correct, the panel would reach operating voltage well before generating appreciable power.]

I didn’t have my oscilloscope set up to capture the startup waveform to confirm or disprove this hypothesis. It’s clear there are additional subtleties I don’t know about starting up a circuit on solar power. Do I want to invest the time to learn and experiment with this problem domain? After thinking it over for a bit I decided “nah” and abandoned the idea of running everything exclusively on solar power. I’ll retreat to what I know and incorporate batteries into the system instead. Starting simple with household alkaline AA batteries.

MP1584 Modification Did Not Go as Planned

I wanted a DC voltage/current monitor module to be powered by the same solar panel it is monitoring, which meant it shouldn’t start running until the panel started delivering enough power to run the circuit. I modified a MP1584 buck converter module so it would not activate until the voltage rises close to 12V. I wired this MP1584 module back into a monitoring circuit with an INA219 voltage/current sensor and ESP8266 WiFi microcontroller.

Before I powered it up, I used a meter to verify I hadn’t done anything silly like shorting Vcc to GND. This circuit also had a Vin+ and Vin- plus I2C communication lines SDA and SCL. I verified none of these were shorted to any of the others.

I then connected Vin+ and GND input pins to my bench power supply. I started with zero volts and slowly raised it until close to 12 volts, when I saw the blue led on my ESP8266 board blink signifying startup. I opened ESPHome dashboard so I could see the logged output from this ESP8266, and saw that it powered up and connected to WiFi successfully. However, it failed to find the INA219 as an I2C peripheral.

My first hypothesis was that I made a mistake soldering I2C signal wires SDA and SCL. I probed those connections and verified I hadn’t crossed those wires and there is electrical continuity. But ESP8266 still reported no response from an I2C scan.

The next hypothesis was that I used too thick gauge of wire for I2C signal lines and its higher capacitance had degraded I2C signal. I replaced the 22AWG wire with 30AWG wire, verified they had continuity and I hadn’t accidentally swapped SDA and SCL. But still no response on I2C scan.

While looking at the ESPHome YAML file looking for a configuration error, I smelled the distinctive scent of dying electronics. My first act upon turning back to the workbench was to turn off the power supply. Then I saw the INA219 was very obviously quite dead, a hole burned into the top and charred remains surrounding that hole.

I repeated the same basic probe I performed on this circuit before I powered it up, and this time my meter said Vcc is shorted to GND. Not good! I separated the components and measured them separately. The MP1584 power module did not short Vout (a.k.a. Vcc for the other two boards) to GND. The INA219 sensor module did, which is very likely related to why it died. And mysteriously, the ESP8266 microcontroller module reported the same… how the heck did it continue running with its Vcc shorted to GND?

Dumping both the INA219 sensor board and ESP8266 microcontroller board in my box of fried electronics, I took a closer look at my modified MP1584, suspected killer of electronics.

Investigating MP1584 For Solar Power

I am happy with the performance I’ve seen so far of an INA219 DC voltage/current sensor, and it is one step closer to the goal of a homebuilt power output monitor for my Harbor Freight solar array. All the major pieces are now in place: I have a working INA219, driven by an ESP8266, running code generated by ESPHome, with resulting data collected by Home Assistant.

The next challenge I wanted to tackle was to make this system run exclusively on solar power without a battery. The daytime scenario is easy: solar panel power can feed into MP1584 buck converter to power the circuit. The night scenario is also easy: there’s no power and nothing happens. The hard part comes during the transition between those scenarios: gracefully power up around sunrise, and gracefully shut down around sunset. I don’t expect this exploration to be easy as it will have to deal with all the vague parts of the real analog world. Very different from the digital thinking my brain is familiar with.

But before I go into the real world, I can explore a crude simulation on my workbench. I connected the input wires to my bench power supply to see how the system behaves. From zero to three volts nothing appears to happen, which was expected. From approximately five volts and up, the system is up and running. But between three and five volts, I hear a disconcerting screech from the buck converter module, and the ESP8266 seems to startup erratically. There is a blue LED that is expected to illuminate once, for a fraction of a second, during ESP8266 power-up. But when I hear the screech, I also see the LED blink seemingly randomly. Implying that ESP8266 would try to start up but fail, then try again, and repeat.

It looks like I need to better understand the expected behavior for a MP1584 during this borderline scenario, and see how it aligns with my observations.

Exploring Low Power ESPHome Nodes

When I investigated adding up power over time into a measure of energy, I found that I have the option of doing it either on board my ESPHome microcontroller or on the Home Assistant server. I’m personally in favor of moving as much computation on the server as I can, and another reason is because keeping the sensor node lightweight gives us the option of putting it to sleep in between sensor readings.

Preliminary measurements put this MP1584EN + ESP8266 + INA219 at a combined average power draw of somewhere around a quarter to a third of a Watt. This is pretty trivial in terms of home power consumption, but not if there is ambition to build nodes that run on battery power. For example, let’s do some simple math with a pair of cheap NiMH rechargeable AA batteries. With a nominal capacity of 2000 mAh and nominal voltage of 1.2V each, that multiplies out (1.2 V * 2 Amps 1 hour * 2 batteries) to 4.8 Watts over an hour. Actual behavior will vary a lot due other variables, but that simple math gives an order of magnitude. Something that constantly draws 0.3 Watt would last somewhere on the order of (4.8 / 0.3) 16 hours, or less than a day, on a pair of rechargeable AA NiMH batteries.

ESPHome has options for putting a node into deep sleep, and the simplest options are based on time like running for X seconds and sleep for Y minutes. For more sophisticated logic, a deep_sleep.enter action is exposed to enter sleep mode. There is also a deep_sleep.prevent action to keep a node awake, and the example is to keep a node awake long enough to upload a code update. This is a problem I’ve tripped over during my MicroPython adventure and it’s nice to see someone has provided a solution in this framework.

The example code reads retained value on a MQTT topic to decide whether to go to sleep or not. I think this is useful, but I also want a locally controlled method for times when MQTT broker is unreachable for any reason. I wanted to dedicate a pin on the ESP8266 for this, with an internal pull-up and an external switch to pull to ground. When the pin is low, the node will go to sleep as programmed. If the pin is high, the node stays awake. I will wire this pin to ground via a jumper so that when the jumper is removed, the node stays awake. And if the jumper is reinstalled, the node goes to sleep.

Such GPIO digital activity can be specified via ESPHome Binary Sensor:

deep_sleep:
  run_duration: 30s
  sleep_duration: 2min
  id: deep_sleep_1

binary_sensor:
  - platform: gpio
    name: "Sleep Jumper"
    id: sleep_jumper
    pin:
      number: 13
      mode:
        input: true
        pullup: true
    on_press:
      then:
        - logger.log: "Preventing deep sleep"
        - deep_sleep.prevent: deep_sleep_1
    on_release:
      then:
        - logger.log: "Entering deep sleep"
        - deep_sleep.enter:
            id: deep_sleep_1
            sleep_duration: 1min

But this is not quite good enough, because on_press only happens if the high-to-low transition happens while the node is awake. If I pull the jumper while the node is asleep, upon wake the pin state is low and my code for high-to-low transition does not run. I needed to check the binary sensor state elsewhere before the sleep timer happens. In the case of this particular project, I also used the analog pin to read battery voltage once every few seconds, so I removed the check from on_press to ADC sensor on_value. (I left on_release code in place so it will still go to sleep when jumper is reinstalled.)

sensor:
  - platform: adc
    pin: A0
    name: "Battery"
    update_interval: 5s
    on_value:
      if:
        condition:
          binary_sensor.is_on: sleep_jumper
        then:
          - logger.log: "Preventing deep sleep"
          - deep_sleep.prevent: deep_sleep_1

This performs a jumper check every time the ADC value is read. This is pretty inelegant code, linking two unrelated topics, but it works for now. It also avoids the problem of digital signal debouncing, which would cause on_press and on_release to both be called in rapid succession unless a delayed_on_off filter is specified.


Ideally, this sensor node would go to sleep immediately after successfully performing a sensor read operation. This should take less than 30 seconds, but the time is variable due to external conditions. (Starting up WiFi, connect to router, connect to Home Assistant, etc.) The naive approach is to call deep_sleep.enter in response to on_value for a sensor, but that was too early. on_value happens immediately after the value is read, before it was submitted to Home Assistant. So when I put it to sleep in on_value, Home Assistant would never receive data. I have to find some other event corresponding to “successfully uploaded value” to trigger sleep, and I haven’t found it yet. The closest so far is the Home Assistant client api.connected condition, but that falls short on two fronts. The first is that it does not differentiate between connecting to Home Assistant (useful) versus ESPHome dashboard (not useful). The second is that it doesn’t say anything about success/failure of sensor value upload. Maybe it’s possible to do something using that condition, in the meantime I wait 30 seconds.

A quick search online found this person’s project also working to prolong battery life for an ESP8266 running ESPHome, and their solution is to use MQTT instead of the Home Assistant communication API. I guess they didn’t find an “after successful send” event, either. Oh well, at least I’m getting data from INA219 between sleep periods, and that data looks pretty good.

Adding Up Power in ESPHome and Home Assistant

Using an INA219 breakout board, I could continuously measure voltage and current passing through a circuit. Data is transmitted by an ESP8266 running ESPHome software and reported to Home Assistant. In order to avoid getting flooded with data, we can use ESPHome sensor filters to aggregate data points. Once we have voltage and current, multiplying them gives us power at a particular instant. The next step is to sum up all of these readings over time to calculate energy produced/consumed. We have two methods to perform this power integration: onboard the microcontroller with ESPHome, or on the Home Assistant server.

ESPHome

The Total Daily Energy component accumulates value from a specified power sensor and integrates a daily tally. (It also needs the Time component to know when midnight rolls around, in order to reset to zero.) The downside of doing this calculation on the controller is that our runny tally must be saved somewhere, or else we would start from zero every time we reset. By default, the tally is saved in flash memory every time a power reading arrives. If power readings are taken at high frequency, this could wear out flash storage very quickly. ESPHome provides two parameters to mitigate wear: we could set min_save_interval to a longer duration in order to reduce the number of writes, or we could set restore to false and skip writing entirely. The former means we lose some amount of data when we reset, the latter means we lose all the data. But your flash memory will thank you!

Home Assistant

Alternatively, we can perform this calculation on Home Assistant server with the unfortunately named integration integration. The first “integration” refers to the math, called Riemann sum integral. The second “integration” is what Home Assistant calls its modules. Hence “integration integration” (which is also very annoying to search for).

Curiously, I found no way in Home Assistant user interface to add this to my instance, I had to go and manually edit configuration.yml as per documentation. After I restarted Home Assistant, a new tally started counting up on my dashboard, but I could not do anything else with the user interface element. I just get an error “This entity does not have a unique ID“.

On the upside, doing this math on the server meant data in progress will be tracked and saved in a real database, kept on a real storage device instead of fragile flash memory. But by default it does not reset at midnight, so the number keeps ticking upwards. Doing more processing with this data is on the to-do list.


Should we do our computation on the microcontroller or on the server? There are certainly advantages to either approach, but right now I lean towards server-side because that lets us put the microcontroller to sleep.

ESPHome Sensor Filters Help Manage Flood of Data

I was happy to find that ESPHome made building a sensor node super easy. Once my hardware was soldered together, it took less than ten minutes of software work before I was looking at a flood of voltage and current values coming in from my ESP8266-based power sensor.

It was a flood of data by my decision. Sample code from ESPHome INA219 documentation set data refresh rate at once every sixty seconds. I was hungry for more so I reduced that to a single second. This allowed me to see constantly updating numbers in my Home Assistant dashboard, which is satisfying in a statistics nerd kind of way, but doing so highlighted a few problems. Home Assistant was not designed for this kind of (ab)use. When I click on a chart, it queries from SQLite and took several seconds to plot every single data point on the graph. And since there are far more points than there are pixels on screen, what I get is a crowded mess of lines.

For comparison, InfluxDB and Grafana were designed to handle larger volumes of data and gives us tools for aggregating data. Working with aggregates for analysis and visualization avoids bogging the system down. I’m not sure how to do the same in Home Assistant, or if it’s possible at all, but I do know there are data aggregation tools in ESPHome to filter data before it gets into Home Assistant. These are described in the Sensor Filters documentation. I could still take a reading every second, but I could choose to send just the average value once a minute. Or the maximum value, or the minimum value. Showing average value once a minute, my Home Assistant UI is responsive again. The graph above was for the day I invoked this once-a-minute averaging, and the effect is immediately visible roughly around 10:45PM.

The graph below was from a later day when once-a-minute average was active the entire day:

It is a cleaner graph, but it does not tell the whole story. I had used this INA219 sensor node to measure power consumption of my HP Stream 7 tablet. Measuring once a second, I could see that power use was highly variable with minimum of less than two and a half watts but spikes up to four and a half watts. When showing the average, this information was lost. The average never dropped below 2.4 or rose above 3.2. If I had been planning power capacity for a power supply, this would have been misleading about what I would need. Ideally, I would like to know the minimum and the maximum and the average over a filtered period. If I had been writing my system from scratch, I know roughly what kind of code I would write to accomplish it. Hence the downside of using somebody else’s code: it’s not obvious to me how to do the same thing within this sensor filter framework. I may have to insert my own C code using ESPHome’s Lambda mechanism, something to learn later. [UPDATE: I learned it later and here’s the lambda/template code.] In the meantime I wanted to start adding up instantaneous power figures to calculate energy over time.

Using INA219 Was Super Easy with ESPHome

Once I had ESPHome set up and running, the software side of creating a small wireless voltage and current sensor node with was super easy. I needed to copy sample code for I2C bus component, then sample code for INA219 component, and… that’s it. I started getting voltage, current, and power reports into my Home Assistant dashboard. I am impressed.

It was certainly far less work than the hardware side, which took a bit of soldering. I started with the three modules. From left to right: the INA219 DC sensor board, the MP1584EN DC voltage buck converter, and the ESP8266 in a Wemos D1 Mini form factor.

First the D1 Mini received a small jumper wire connecting D0 to RST, this gives me to option to play with deep sleep.

The MP1584EN was adjusted to output 3.3 volts, then its output was wired directly to the D1 Mini’s 3V3 pin. A small piece of plastic cut from an expired credit card separated them.

The INA219 board was then wired in a similar manner on the other side of D1 mini, with another piece of plastic separating them. For I2C wires I used a white wire for SDA and green wire for SCL lines following Adafruit precedence. Vcc connected to the 3.3V output of MP1584EN in parallel with D1 mini, and ground wires across all three boards. The voltage input for MP1584EN was tapped from Vin- pin of the INA219 board. This means the power consumed by ESP8266 would be included in INA219’s measurements.

A small segment of transparent heat shrink tube packed them all together into a very compact package.

I like the concept of packing everything tightly but I’m squeamish about my particular execution. Some of the wires were a tiny bit longer than they needed to be, and the shrink tube compressed and contorted them to fit. If I do this again, I should plan out wire my lengths for a proper fit.


Like I said earlier, the hardware took far more time than the software, which thanks to ESPHome became a trivial bit of work. I was soon staring at a flood of data, but thankfully ESPHome offers sensor filters to deal with that, too.

Learned About Home Assistant From ESPHome Via WLED

I thought Adafruit’s IO platform was a great service for building network-connected devices. If my current project had been something I wanted to be internet-accessible with responses built on immediate data, then that would be a great choice. However, my current intent is for something locally at home and I wanted the option to query and analyze long term data, so I started looking at Home Assistant instead.

I found out about Home Assistant in a roundabout way. The path started with a tweet from GeekMomProjects:

A cursory look at WLED’s home page told me it is a project superficially similar to Ben Hencke’s Pixelblaze: an ESP8266/ESP32-based platform for building network-connected projects that light up individually addressable LED arrays. The main difference I saw was of network control. A Pixelblaze is well suited for standalone execution, and the network interface is primarily to expose its web-based UI for programming effects. There are ways to control a Pixelblaze over the network, but they are more advanced scenarios. In contrast, WLED’s own interface for standalone effects are dominated by less sophisticated lighting schemes. For anything more sophisticated, WLED has an API for control over the network from other devices.

The Pixelblaze sensor board is a good illustration of this difference: it is consistent with Pixelblaze design to run code that reacts to its environment with the aid of a sensor board. There’s no sensor board peripheral for a WLED: if I want to build a sensor-reactive LED project using WLED, I would build something else with a sensor, and send commands over the network to control WLED lights.

So what would these other network nodes look like? Following some links led me to the ESPHome project. This is a platform for building small network-connected devices using ESP8266/ESP32 as its network gateway, with a library full of templates we can pick up and use. It looks like WLED is an advanced and specialized relative of ESPHome nodes like their adaptation of the FastLED library. I didn’t dig deeper to find exactly how closely related they are. What’s more interesting to me right now is that a lot of other popular electronics devices are available in the ESPHome template library, including the INA219 power monitor I’ve got on my workbench. All ready to go, no coding on my part required.

Using an inexpensive ESP as a small sensor input or output node, and offload processing logic somewhere else? This can work really well for my project depending on that “somewhere else.” If we’re talking about some cloud service, then we’re no better off than Adafruit IO. So I was happy to learn ESPHome is tailored to work with Home Assistant, an automation platform I could run locally.