Solar Power

Roger Cheng

Reduce Battery Charging Rate of 4056 BMS

My solar power monitor uses components I salvaged from a broken USB power bank. After a year and a half of daily charge cycling, the charging circuit has gone out. I replaced it with an inexpensive commodity battery management system (BMS) module based on a 4056 chip. Then I let the module run unmodified for two days to verify everything worked as advertised.

These modules shipped with charging rate configured at one amp. The general rule of thumb is charging rate should stay below 1C, which is 2.6A for these 2.6Ah capacity cells, so the default should be fine. But I wanted to lower the charging rate for several reasons:

  • This battery cell is almost a decade old now, and it is natural for older cells to have reduced capacity along with reduced tolerance for charging speed.
  • Charging at slower gentler rates improves battery longevity, hopefully making this old battery last even longer.
  • There’s no rush: drawing from the solar panel, this thing can charge over entire span of daylight hours when panels are producing power.
  • By reducing the power demand, I can activate this charging circuit even when the solar panels produce little power such as during overcast or rainy days.
  • By slowly charging during daylight hours, it reduces stress on the battery because it would only have to run the microcontroller during early morning and early evening when sunlight is scant.

Due to those reasons, I had ambition to slow the charging rate using original USB power bank charging circuit as well, but I didn’t know how to work with the unmarked chip. Switching to a commodity BMS chip meant I can get more information. I’m not sure exactly whose 4056 chip I have here. But as they seemed to be interchangeable commodities, I just downloaded one of them and learned charging rate is controlled by a resistor between pin 2 (PROG) and ground. The formula is (Charging current) = 1200/(RPROG). Probing this module, I find the charge rate control resistor to be the one labeled R3 and its tiny number says “122”. I understand that to mean 12 * 102 = 1200Ω = 1.2 kΩ. This matches expectation with the formula 1A = 1200/1200.

The beauty of math is infinite, but real-world circuits have limits. What is a practical minimum for charging rate? The datasheet I consulted gave several examples, the lowest is 0.12A rate via a 10kΩ resistor. Conveniently, the just-retired USB power bank circuit board has a 10kΩ resistor (labeled “103” meaning 10 * 103 = 10000Ω) on board. Since it’s not doing anything anymore, I can pull it off and swap it for the default resistor at position R3. My lackluster soldering skill with surface mount devices (SMD) wasn’t pretty, but it’s functional.

Now I have a low-demand charging circuit for my solar power monitor, letting me run it across more weather conditions while gently charging the single old lithium-ion battery cell to extend its operating life. It would be nice if I can do more to extend battery longevity: lithium-ion chemistry batteries are stressed when they are fully charged or fully discharged, so they are best kept partially charged. In this particular project, I handle both in software running on my ESP8266. It goes to deep sleep before voltage drops to a critical level. It also disables the solar panel DC buck converter before the battery is fully charged. It’d be nice if I could reduce software complexity by doing everything onboard the BMS module, but I can’t. Low cutoff voltage is controlled by the mystery chip at location U2, and max charging voltage is fixed at 4.2V for a 4056 chip.

Perhaps in the future I’ll find a charging module that would let me modify those parameters, but for today this is good enough for my solar monitoring project to return to service so I can resume my LEGO nostalgia tour. I was just getting to the good part: LEGO Technic sets!

Roger Cheng

Single Cell Lithium-Ion Battery Management System Module (4056)

My solar panel power monitor project incorporated an old USB power bank for its battery and charger circuit, bypassing its broken USB power output circuit. After a year and a half of daily cycling, the charging circuit has broken down as well. I’m happy I got that extra life out of a USB power bank circuit board that would otherwise have been disposed of. The battery is still going, but I will need a replacement circuit to manage charging it.

With the widespread adoption of lithium-ion battery power, I have many solutions to choose from. The lowest bidder du jour on Amazon was this vendor selling a multipack of 40 BMS modules (*). It arrived in a single 10×4 sheet for the me to break apart as needed, similar to a batch of buck converters I bought earlier. I like this approach much better than loosely packed pieces that may damage each other in shipping.

The main chip in the center of this module had “4056H” printed on top. Many vendors on Amazon/AliExpress/etc. also single-cell BMS modules with this general design, not necessarily with “4056H” on top but all with some variation of “4056” with different prefix/suffix letters. A search for “4056” returned many chips from different companies that seemed to be interchangeable. I assume someone had a successful product that was then copied by many others, but I don’t know who the original was. The same goes for this particular breakout board module design. Looks like both the module and the chip at the center of it have become commodities.

This module also advertised protection against battery over discharge with a 3A current limit and 2.5V voltage limit, but that’s beyond the scope of a 4056 chip. This module must have additional components handling such protection. I see one chip labeled 8205A, which appears to be a dual N-channel MOSFET chip suited for output cutoff controlled by the chip at position U2. I don’t know how to dig deeper because U2 is unmarked on my purchase, but I have learned enough to put this module to work.

The module with its six soldering points is fairly straightforward to incorporate into my project:

  • There are pads on either side of the USB micro-B socket for 5V power input, one of them marked “+”. They are soldered to the existing buck converter dropping solar panel DC power down to 5V.
  • Pins labeled “B+” and “B-” connect to the positive and negative terminals of the 18650 battery cell.
  • Remaining pins labeled “OUT+” and “OUT-” are connected to the ESP8266 microcontroller module.

This configuration successfully recharged the solar monitor battery for two days, verifying everything worked as expected before I proceeded to lower the charging rate from its default of 1 Amp.

(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Roger Cheng

Solar Monitor Battery Charging Failure

I need pause my LEGO nostalgia trip and revisit my solar power monitor project, so a quick recap of the story so far: About a year and a half ago, I built a project to monitor power output of my cheap Harbor Freight solar panels. Power is measured by an Adafruit INA219 sensor breakout board controlled by an ESP8266 microcontroller running software compiled by ESPHome based on my specification YAML. Data is reported to my personal instance of Home Assistant running under KVM hypervisor. I originally wanted to run this device strictly off solar power, but never figured out the analog power bits, so I resigned to incorporating a battery into the device. I had an old USB power bank that could no longer deliver useful USB power, so I took it apart to repurpose the still-working battery cell and charging capability for my solar monitor.

Fast forward to this week: browsing my recent data in Home Assistant, I noticed the solar monitor battery voltage has been erratic for a few days. It usually charges up in the early morning soon after sunrise, then gradually fall through the day until the next morning. But now it no longer charges on a smooth curve in the morning, what’s going on? Looking at charging status LEDs, I can see it changing state erratically once every few seconds. Maybe blinking in charging sequence for a bit, then maybe all four would be illuminated as if the battery was fully charged, then they would go out as if charging power disappeared, or some combination of those states.

Pulling out my voltmeter, I checked the following candidate explanations:

  • Perhaps the buck converter I had used to convert solar panel DC voltage down to 5V is misbehaving? I checked the voltage of enable pin: it is steady at 3.3V. I checked the output voltage: a steady 4.9V with no fluctuation. The buck converter seems to be performing nominally.
  • Perhaps the lithium-ion battery cell is failing? I monitored the battery terminal voltage as the charger cycled through its charging/not-charging states, looking to see if the battery voltage is behaving unexpectedly. Maybe it dropped below critical minimum (3V) or shot up to maximum (4.2V) upon charge. Neither was the case: voltage fluctuated around 3.6V. A little higher when charging and a little lower when discharging, signs of a nominally working battery cell.

If the power input is good and the battery looks good, process of elimination says the problem is the repurposed power bank charging circuit between them. This was not a huge surprise since this piece of electronics was already known to be defective: The reason I took it apart was because the output stage could no longer deliver 5V USB power output. I’m actually rather pleased I got another year and a half of useful life out of the power input charging stage before the whole thing gave out.

Comparing year-and-a-half old picture (left) against current state (right), the most obvious sign of degradation is near the micro-USB power input port, the yellow epoxy towards one corner touching component U9 has visibly darkened. Is it charring from overheating, or some other degradation? That is not clear. What is clear is that I now need something else to manage charging the single 18650 lithium-ion battery cell.

Roger Cheng

Initial Logic for Solar Monitor Project

I think I’ve got the hardware portions of my solar power monitor sensor node figured out, so I can write the first version of corresponding software logic. I have set out the following requirements:

  • Over-discharge protection: If battery voltage drops below a threshold, put the system to sleep to protect the battery.
  • Low solar output: When the solar panel isn’t generating any power, put the system to sleep.
  • Battery charging start: when panel power generation rises above a certain level for the first time that day, start charging the repurposed USB power bank battery.
  • Battery charging pause: If cloud cover causes a dip in solar power, pause charging.
  • Battery charging stop: Once battery cell voltage rises to a certain level, stop charging.
  • Sleep override: local hardware method to prevent deep sleep.

The ESPHome documentation for deep sleep described one way to prevent sleep using MQTT, keeping a node awake to receive firmware updates. But I wanted something even lower level hence the jumper and it became useful when I implemented the “low solar output, go to sleep” logic. Apparently INA219 component’s first power value always return zero. Which meant as soon as it booted up, that initial zero reading puts the node immediately back to sleep before it would even get on the network. (Never mind checking MQTT!) The solution is to switch from sampling values once a minute to sampling once a second, and make decisions based on average over a minute.

A different approach would be to go to sleep based on sun’s position in the sky, which can be queried and used in the Sun component. However, I expect this component has dependency on network connection (it needs to know the time, for starters) and would not be reliable if the network goes down. It also doesn’t know if the sun is obscured by clouds, so I think it’s better to use panel power output to decide what to do during the day. But I may explore using the Sun component in a future version to sleep all through the night instead of waking up every few minutes to fruitlessly check power level.

Strictly speaking, I don’t need to worry about stopping battery charging. I can supply 5V all day when panel delivers power, and trust the USB power bank charging circuit to keep the battery from being overcharged. But keeping lithium-ion cells full would shorten their useful life, so in the interest of battery longevity I’ll stop charging before full. On that topic: for optimal battery life I should charge it slowly over the course of the day, but I don’t have control over charging rate used by USB power bank.

One thing I don’t know yet is how the system will handle several rainy days in a row. I assume this panel can still generate enough power to charge an 18650 battery cell, but I might be wrong! I’ll have to wait for a long stretch of rain to come to Southern California, which may be a long wait. After seeing its behavior I can adjust for a future version.

Here’s version 1 of my ESPHome configuration YAML, and I expect to fine tune various hard-coded threshold values over the weeks ahead while I build more projects:

# Blue LED on the ESP8266 module signals connection status.
status_led:
  pin:
    number: 2
    inverted: true

# The goal is to charge once a day, and this flag tracks if we've already done it.
globals:
  - id: never_charged_today
    type: bool
    restore_value: no
    initial_value: "true"

# We can go to deep sleep to conserve battery, but sometimes we don't want to
# actually go to sleep. For example, when we need to upload a firmware update.
# Pin 13 is an input pin with internal pullup. It should be wired to a jumper
# that would ground the pin if jumper is present. Removing the jumper should
# disable going to deep sleep. To enforce this, call try_to_sleep script
# instead of calling deep_sleep.enter directly.
deep_sleep:
  id: deep_sleep_1

binary_sensor:
  - platform: gpio
    name: "Disable Sleep"
    id: sleep_jumper
    pin:
      number: 13
      mode:
        input: true
        pullup: true
    on_release:
      then:
        - logger.log: "Sleep jumper installed"
        - script.execute: try_to_sleep

script:
  - id: try_to_sleep
    then:
      if:
        condition:
          binary_sensor.is_on: sleep_jumper
        then:
          logger.log: "Sleep requested but staying awake due to override jumper"
        else:
          - logger.log: "Sleep requested and permitted by jumper"
          - delay: 5s # Allow sensor values to be sent.
          - deep_sleep.enter:
              id: deep_sleep_1
              sleep_duration: 10min

# This should be wired to a 1k resistor, which then connects to the enable pin
# of a power supply source. When ON, it should deliver power to charge the battery.
switch:
  - platform: gpio
    pin: D5
    id: charge_switch
    name: "Charge Battery"
    restore_mode: RESTORE_DEFAULT_OFF

# An I2C INA219 sensor monitors panel voltage, current, and calculates power.
i2c:
  sda: 4
  scl: 5

sensor:
  - platform: ina219
    address: 0x40
    shunt_resistance: 0.1 ohm
    max_voltage: 24.0V
    max_current: 3.2A
    update_interval: 1s
    current:
      name: "Panel Current"
      id: solar_panel_current
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
    power:
      name: "Panel Power"
      id: solar_panel_power
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
      on_value:
        then:
          # When power is low, put the board to sleep.
          # Note: upon boot, the first reading of current (and therefore power) always
          # seems to be zero, so we need to run moving average filters to ensure we
          # don't shut off immediately on power-up.
          if:
            condition:
              and:
                - sensor.in_range:
                    id: solar_panel_power
                    below: 0.01
                - sensor.in_range:
                    id: solar_panel_voltage
                    below: 3.0
            then:
              - logger.log: "Panel delivering low power, should go to sleep"
              - globals.set:
                  id: never_charged_today
                  value: "true"
              - script.execute: try_to_sleep
    bus_voltage:
      name: "Panel Voltage"
      id: solar_panel_voltage
      accuracy_decimals: 5
      filters:
        sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
# ESP8266 ADC pin should be wired to a resistor just over 100kOhm to measure
# lithium-ion battery cell voltage. Values under calibrate_linear need to be
# customized for each board (and their resistors.)
  - platform: adc
    pin: A0
    name: "Battery Voltage"
    id: battery_voltage
    update_interval: 1s
    accuracy_decimals: 3
    filters:
      - calibrate_linear:
          - 0.84052 -> 3.492
          - 0.99707 -> 4.113
      - sliding_window_moving_average:
          window_size: 90
          send_every: 60
          send_first_at: 15
    on_value:
      then:
        - if:
            condition:
              and:
                - lambda: "return id(never_charged_today);"  
                - sensor.in_range:
                    id: solar_panel_power
                    above: 10
            then:
              - logger.log: "Panel has power, start charging for the day"
              - globals.set:
                  id: never_charged_today
                  value: "false"
              - switch.turn_on: charge_switch
        - if:
            condition:
              and:
                - switch.is_on: charge_switch
                - sensor.in_range:
                    id: solar_panel_power
                    below: 5
            then:
              - logger.log: "Charging paused due to low panel output"
              - globals.set:
                  id: never_charged_today
                  value: "true" # Resume charging if power returns
              - switch.turn_off: charge_switch
        # When battery is low enough to trigger this emergency measure, we
        # would not be able to activate charging ourselves. Charging needs to
        # be activated manually (or at least externally)
        - if:
            condition:
              sensor.in_range:
                id: battery_voltage
                below: 3.0
            then:
              - logger.log: "Battery critically low, should sleep to protect battery."
              - script.execute: try_to_sleep
    # We don't need a full charge to last through a day, so turn off charging
    # well before reaching maximum in order to improve battery longevity
    on_value_range:
      above: 4.0
      then:
        - logger.log: "Battery charge is sufficient"
        - switch.turn_off: charge_switch
Roger Cheng

Plotting Solar Panel Voltage and Power

Ever since I bought a cheap solar panel array from Harbor Freight, I’ve wanted to quantify its output. Most solar power instruments were designed for house-sized (or larger) arrays, overkill for my little panel. I had instruments from the world of remote-control hobbies to measure voltage and current, but they were only instantaneous values displayed on screen. I wanted more! This motivated my journey evaluating software tools like InfluxDB database, Grafana visualizer, Home Assistant, and ESPHome. Plus building on what I knew about hardware components like ESP8266 microcontroller, MP1584 buck converter, and INA219 sensor. All with the goal of building my own solar production monitoring system.

I had thought I could make the ESP8266+INA219 sensor node run exclusively on solar power, but I gave up on that and have built a temporary setup powered by old tired alkaline AA batteries. Enough to give me a graph!

This was a fairly sunny day, so cloud cover effects were minimal. The solar panel was connected to a MPPT charger. I could see the power levels (green line) climb smoothly from sunrise, reach a maximum at noon, then smoothly fading off to sunset. This was mostly as expected. The power curve is not symmetric because there are a few things blocking sunlight from the east, reducing morning power.

While the power curve was mostly expected, the voltage levels were not. Until this graph I didn’t know solar panels would jump up so high on voltage before producing meaningful amount of power. (I don’t know if the MPPT charger makes a difference here.) If I had known this, I probably wouldn’t have bothered trying to make a solar powered circuit turn on and off based on voltage. Or if I did, I would have tried anyway and knew why it failed. From this graph I learned that if I am to draw upon solar power, I need to base my decision on power and not on voltage. But in order to read power levels, I need power for my INA219! There’s probably a clever way to circumvent this Catch-22 without batteries, but I’m going to take the easy way out and incorporate a rechargeable battery.

Roger Cheng

Solar Startup Still Tricky

I have modified a second MP1584 buck converter module so that it would not activate until input voltage surpasses 13V, comfortably above the output voltage of 3.3V. I want to connect input pins to a solar panel so the associated components (ESP8266 WiFi microcontroller and INA219 voltage/current sensor) would be powered exclusively by the sun.

First is a test run with my bench power supply. Gradually increasing supply voltage starting from zero volts. Thanks to the modification, there were no odd behavior or sounds of a MP1584 trying to work with too low of an input voltage, which is great. As I increased voltage past the ~13V threshold, I saw the blue LED of my ESP8266 blink and it booted up as planned. This time, it was able to find INA219 on I2C bus, which is further than I got before.

Feeling optimistic, I connected this circuit to my solar panel at night and hoped I would wake up to find the system running. Sadly I woke up to disappointment, as there were no logged messages from the ESP8266. Probing the circuit with my volt meter, I confirmed a 3.3V supply voltage was present, but for whatever reason the ESP8266 failed to boot that morning. I manually disconnected and reconnected the circuit board, and this time ESP8266 booted up fine (now it has full daylight power) and started reporting values measured by INA219.

I don’t know what happened at sunrise. I hypothesize that when the solar panel output voltage rose past 13V, it has still yet to produce enough power to successfully start an ESP8266. So when MP1584 activated, it could supply 3.3V but not enough amperage to supply an ESP8266 through its boot process, putting it in a glitched state that was neither on nor off and stuck there until I power cycled the system. [UPDATE: Further experimentation found this hypothesis was correct, the panel would reach operating voltage well before generating appreciable power.]

I didn’t have my oscilloscope set up to capture the startup waveform to confirm or disprove this hypothesis. It’s clear there are additional subtleties I don’t know about starting up a circuit on solar power. Do I want to invest the time to learn and experiment with this problem domain? After thinking it over for a bit I decided “nah” and abandoned the idea of running everything exclusively on solar power. I’ll retreat to what I know and incorporate batteries into the system instead. Starting simple with household alkaline AA batteries.

Roger Cheng

Next Practice Round: MicroPython on ESP8266

I’ve been running through “Hello World” level microcontroller projects, using my little Harbor Freight solar array as a target subject. I just finished a trial run of programming an ESP32 using ESP-IDF platform and while analog input and power-saving deep sleep worked as expected, GPIO output pins did not. I thought I would use an external peripheral to get around the problem but, if I’m doing that, I might as well fall back to the cheaper ESP8266 used in a batch of Wemos D1 Mini clone(*) I had bought.

An earlier experiment with ESP8266 used the Arduino development platform. This time around, I’m trying MicroPython for the following reasons:

  1. With MicroPython I could modify my code without reflashing the entire ESP8266 as I had to do with Arduino.
  2. My Arduino sketch was a fairly short piece of code, but MicroPython promises to get going with even less code.
  3. Last time I looked at MicroPython I rejected it because it didn’t support a hardware peripheral I wanted to use. (MCPWM of ESP32.) This time I’m using just the ADC, which is supported.

I actually wanted to try CircuitPython, Adafruit’s derivative of MicroPython with changes focused on making things even easier for beginners to play with and something I first encountered on their HalloWing product. However, a part of CircuitPython is exposing the microcontroller as a USB mass storage device and letting us drag-and-drop Python source files onto the board for instant execution. This requires native USB support on the microcontroller, which is present on newer ESP32 variations but not the one I have. There used to be an ESP8266 version without the drag-and-drop but Adafruit stopped supporting that configuration some time ago.

Putting CircuitPython back on the “look at later” list, I changed focus to MicroPython. I brought up documentation for putting MicroPython on an ESP8266. In 15 minutes I had an interactive Python command line over serial port, though there were a few speed bumps.

(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Roger Cheng

Switching to ESP32 For Next Exercise

After deciding to move data processing off of the microcontroller, it would make sense to repeat my exercise with an even cheaper microcontroller. But there aren’t a lot of WiFi-capable microcontrollers cheaper than an ESP8266. So I looked at the associated decision to communicate via MQTT instead, because removing requirement for an InfluxDB client library meant opening up potential for other development platforms.

I thought it’d be interesting to step up to ESP8266’s big brother, the ESP32. I could still develop with the Arduino platform with an ESP32 but for the sake of practice I’m switching to Espressif’s ESP-IDF platform. There isn’t an InfluxDB client library for ESP-IDF, but it does have a MQTT library.

ESP32 has more than one ADC channel, and they are more flexible than the single channel on board the ESP8266. However, that is not a motivate at the moment as I don’t have an immediate use for that advantage. I thought it might be interesting to measure current as well as voltage. Unfortunately given how noisy my amateur circuits have proven to be, I doubt I could build a circuit that can pick up the tiny voltage drop across a shunt resistor. Best to delegate that to a dedicated module designed by people who know what they are doing.

One reason I wanted to use an ESP32 is actually the development board. My Wemos D1 Mini clone board used a voltage regulator I could not identify, so I powered it with a separate MP1584EN buck converter module. In contrast, the ESP32 board I have on hand has a regulator clearly marked as an AMS1117. The datasheet for AMS1117 indicated a maximum input voltage of 15V. Since I’m powering my voltage monitor with a lead-acid battery array that has a maximum voltage of 14.4V, in theory I could connect it directly to the voltage input pin on this module.

In practice, connecting ~13V to this dev board gave me an audible pop, a visible spark, and a little cloud of smoke. Uh-oh. I disconnected power and took a closer look. The regulator now has a small crack in its case, surrounded by shiny plastic that had briefly turned liquid and re-solidified. I guess this particular regulator is not genuine AMS1117. It probably works fine converting 5V to 3.3V, but it definitely did not handle a maximum of 15V like real AMS1117 chips are expected to do.

Fortunately, ESP32 development boards are cheap, counterfeit regulators and all. So I chalked this up to lesson learned and pulled another board out of my stockpile. This time voltage regulation is handled by an external MP1584EN buck converter. I still want to play with an ESP32 for its digital output pins.

Roger Cheng

Initial Lessons on ESP8266 Arduino Sketch for InfluxDB

Dipping my toes in building a data monitoring system, I have an ESP8266 Arduino sketch that reads its analog input pin, converts it to original input voltage, and log that information to InfluxDB. Despite the simplicity of the sketch, I’ve already learned some very valuable lessons.

The Good

The Arduino libraries do a very good job of recovering from problems on their own, taking care of it so my Arduino sketch does not. The ESP8266 Arduino WiFi library can reconnect lost WiFi as long as I call WiFiMulti.run() periodically. And the InfluxDB library doesn’t need me to do anything special at all. I call InfluxDbClient.writePoint() whenever I want to write data. If the connection was lost since initial connection, it will be re-established and the data point written with no extra work on my part. I’ve had this sketch up and running as I’ve taken the InfluxDB docker container offline to upgrade to newer versions, or performed firmware updates WiFI access point which takes wireless offline for a few minutes. This sketch recovered and resume logging data, no sweat.

The Bad

ESP8266 ADC (analog-to-digital converter) is pretty darned noisy when asked to measure within a narrow range of its full range of voltages as I am. The full range is 0-22V, and I’m typically only within a narrow band between 12-14V. I tried taking multiple measurements and averaging them, but that didn’t seem to help very much.

This noisiness also made it hard to calibrate readings against voltage values as measured by my multi-meter. It’s not difficult to take a meter reading and calculation a conversion factor for an ADC reading taken at the same time. But if the ADC value can drift even as the actual voltage is held steady, the conversion factor is unreliable. Even worse, since the conversion is done in my Arduino sketch, every time I want to refine this value, I had to hook up a computer and re-upload my Arduino sketch.

Since I expect to add more data sources to this system, I also expected to query by source to see data returned by each. For this first iteration, I tagged points with the MAC address of my ESP8266. This is a pretty good guarantee of uniqueness, but it is not very human-friendly. Or at least not to me, as I’m not in the habit of memorizing MAC addresses of my devices.

The Ugly

As typical of Arduino sketches, this sketch is running loop() as fast as it could. Functionally speaking, this is fine. But it means the ESP8266 is always in a state of high power draw, with the WiFi stack always active and the CPU running as fast as it could. When the objective is merely to record measurements every minute or so, I could be far more energy efficient here.

Addressing these issues (and much more I’m sure) will be topic of future iterations. In the meantime, I have some data points and I want to graph them.

[Source code for this project is publicly accessible on GitHub]

Roger Cheng

Setting Up ESP8266 Arduino Sketch for InfluxDB

I think I’ve got the hardware side sorted out for building an ESP8266 that monitors voltage output of a solar panel, so it’s time to dive into the software side. I want to log this data into InfluxDB as a learning exercise, and the list of client libraries included a pointer to a GitHub repository for an InfluxDB client library for ESP8266 and ESP32 running on their Arduino core.

Arduino is not exactly the most fully featured development environment, so I’ve been doing my Arduino development using PlatformIO plugin of Visual Studio Code. However, this is the first time I’ve had to manually add a third-party library and it’s not the same as Arduino’s Library Manager so I had to go online for a little help like this forum thread. I learned I should go to https://registry.platformio.org/ and search for the desired library. Searching on “InfluxDB” resulted in several results, one of which is the same client library I found earlier but now with instructions on how to install into PlatformIO.

After compiling a simple test sketch, my serial output monitor returned gibberish. The key here is that baud rate must match between my platformio.ini configuration file:

monitor_speed = 115200

And my serial output initialization code in setup() function of my sketch:

  Serial.begin(115200);

Another configuration issue concern information necessary to connect to my home WiFi and my InfluxDB server. This little sketch needs that information to run, but I don’t want to include them in my source code since I intended to upload this to GitHub in a publicly accessible repository. I found a solution on StackOverflow: put my secret information in a separate secrets.h file. After I committed a basic version without any actual information, use the command git update-index --skip-worktree secrets.h to remove it from further Git activity. After that point, I could edit secrets.h and Git would not care to upload that information leaving my local secrets local, which is what I want.

Once all of these setup details were taken care of, I could dive into code and learn some valuable lessons out of the experience.

[Source code for this project is publicly accessible on GitHub]

Roger Cheng

Power Source Selection Jumper

I’m making a Wemos D1 Mini clone (with an ESP8266 at its heart) into a solar panel output voltage monitor. I plan to run it off the solar panel power as well, since it seems silly to involve another power source when it is already hooked up to one. However, having a buck converter supplying 3.3V to the ESP8266 means I need to avoid taking USB power at the same time. Having multiple voltage regulators on the same voltage plane is a bad thing. I don’t want to have dueling regulators when, for example, debugging over USB while it is connected to solar.

I know this is not a new problem, because every battery-powered USB device knows to switch between battery power and USB power. But I’m having trouble finding the right vocabulary to describe exactly my battery-less scenario. Using search terms like “isolating USB power” I usually find people who are trying to avoid ground loops for audio quality, or optical isolators for data, and other similar tasks which are useful but not the problem I’m trying to solve right now.

Momentarily stymied in my research, I switched over to devising my own manual solution. I’m routing the 3.3V output pin of my buck converter through a jumper on the circuit board. When the jumper is installed, the ESP8266 will run on the solar power it is measuring. When the jumper is removed, the module will run on USB power.

But I know myself, and I could not trust myself to remember to install/remove the jumper as the situation changes. Hence the next trick: placement of the jumper. I put it right next to the USB port so that the jumper could not be installed at the same time as the USB cable, ensuring that it is impossible to have both power sources active at the same time.

I think this mostly works, but I’m worried about the jumper pins. They are taller than I had expected and reach pretty close to the USB connector as we can see in this side view. When I plug/unplug the USB cable, I have to carefully avoid accidentally touching those pins. Accidentally shorting those pins would probably not damage the dev board, because electrically it is same as the jumper in place and at that point USB is not plugged in. However, touching the pin could connect voltage supply to ground and that might fry either the buck converter or something on my USB host, neither of which is ideal.

I didn’t like how accident-prone this design is, so I switched to plan B: cut the USB power line.

Roger Cheng

Setting Up ESP8266 Voltage Monitor

I’ve decided to start playing with an ESP8266 in a wireless data monitoring role. I plan to use it to measure power output of my little Harbor Freight solar array and log data into an InfluxDB database. The first draft will only deal with voltage, since I want to start with minimal hardware and the ESP8266 has only a single ADC channel.

In addition to the restriction of a single channel, the ESP8266 has an additional restriction that the voltage on this pin can only be from zero to one volt, a small sub range of its standard 3.3V signal. Since the analog in pin can’t be used for anything else, I learned many ESP8266 development boards went ahead and put a pair of voltage dividing resistors on board. Using my multi-meter I probed the lowest-bidder Wemos D1 Mini clone I purchased on Amazon (*) and found 220KΩ from the A0 pin on the dev board to ADC0 on the ESP8266 module then another 100KΩ to ground.

In a perfect world we would have 230KΩ and 100KΩ to divide 0-3.3V into 0-1V, but this is as close as we can get with commodity resistor values. Assuming going slightly over 1V would not damage the ESP8266, this just means we lose a little bit range if input voltage goes above 3.2V.

I will add to this, because I want to measure output of my solar panel, which has a much higher open circuit voltage. I’ve measured at a little over 20V. Reading online, the commonly quoted maximum voltage is 22V, which seems like a good target to design for. This means ideal additional resistance of 1880KΩ before connecting to A0 input pin. In the interest of keeping things simple with commodity resistor values, I’ll use two 1MΩ resistors in series and sacrifice a bit of resolution.

I want to power my sensor node from the same solar power it is measuring, which again can go up to >20V. The Wemos D1 Mini board has an onboard voltage regulator to take 5V USB power down to 3.3V required by the ESP8266. I can’t tell exactly what module it is, but I will assume it does not handle input voltage up to 22V. Instead I will pull another MP1584EN buck converter from the last batch I bought on Amazon.(*) I will configure it to output 3.3V and connect that to the 3.3V voltage plane on the Wemos D1 Mini.

Which leaves a problem: If I’m getting 3.3V supplied from the solar panel, it is important we do not connect to 5V USB power at the same time. Perhaps a switch is in order?

(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Roger Cheng

Hello Wemos D1 Mini Clone

We can run the Arduino framework on an Espressif ESP8266 chip, but that was not its original purpose. It was originally designed as a WiFi bridge for small devices, handling wireless networking duties on behalf of another microcontroller. But as it turned out, the chip was more than capable to drive the whole show by itself for certain scenarios. Versions 1 and 2 of Pixelblaze, for example, ran on an ESP8266 even though my experience has been with the ESP32-based version 3. I haven’t had a project of my own where the ESP8266 made sense until the current InfluxDB logging project. It is time.

My electronics skill isn’t up to the level of directly using a bare ESP8266 chip. Using a module integrating ESP8266 chip with support circuitry (including a circuit board antenna) is close, but just beyond my comfort level. I went online looking for development boards for such modules and it appears many of them are clones of something called “Wemos D1 Mini“. A brief look at related web pages seem to indicate this rose to prominence alongside a development platform called NodeMCU. I don’t know which came first, the NodeMCU platform or the Wemos D1 Mini module, but they seem to be mentioned side by side in a majority of my search hits. Since I came into this project looking to use the Arduino core on an ESP8266 board, I’ll set aside the NodeMCU angle for now. I clicked “Buy” on a batch of Wemos D1 Mini clones from the lowest bidder of the day. (*)

When they arrived, I was pleasantly surprised to find that I had multiple options on connectors. The package came with three sets: I could have pins appropriate for a breadboard, or sockets appropriate for jumper wires, or passthrough connectors with both pins and sockets. And since none of them were pre-soldered on the board, I actually have a fourth option to “dead-bug” wire a circuit directly without any of those connectors. I think this will prove very useful to fit projects in tight spaces. Some people may gripe about having to heat up a soldering iron before this board is usable, and to them I shall point to this fishing line trick.

For my first ESP8266 project, I’ll stick with traditional pins, followed by mounting it in a perforated prototype board.

(*) Disclosure: As an Amazon Associate I earn from qualifying purchases.

Roger Cheng

Managed InfluxDB Arduino Client Access

While learning how to setup and run InfluxDB 2 in a docker container, I learned some administrative tasks require dropping to the command line. I had to do this by launching bash in a running instance via docker exec -it influxdb /bin/bash. I looked this up because creating less-privileged user accounts for non-administrative access had to be done in the command line, but it turns out InfluxDB manages access levels using authorization tokens and not user accounts. Creating a key with restricted permissions can be done within the UI and does not need the command line.

I wanted restricted access permissions for two main scenarios. The first ability is to add data to a specific table (sorry… um… measurement in InfluxDB terms) but unable to do other things like delete data. This is for my data sources, as I don’t want bugs in my data sources to corrupt or delete the database. The second ability is read-only access for analysis and displaying results. Again, I didn’t want a bug in a data dashboard to damage data.

Once I learned how to create these authorization keys scoped correctly to the minimum access necessary to fulfill a role, the next step is to use them. Looking over InfluxDB’s list of client libraries, I was fascinated to see Arduino listed among the expected list of programming languages. How does an ATMega328 talk to an InfluxDB instance? This got me really curious and clicked that link to a GitHub repository.

On the README.md I got my answer: this library is not for all Arduino boards, just the ESP8266 and ESP32 WiFi-enabled microcontrollers running their respective Arduino cores. Other Arduino platforms like Teensy or the original ATMega328 are excluded. I’ve played with the ESP32 for a bit and never had a real reason to look into its ESP8266 predecessor. This project will be the motivation to play with an ESP8266.

Roger Cheng

InfluxDB Investigation Skipping 1.x, Going Straight To 2.x

After I read enough to gain some working knowledge of how to use InfluxDB, it was time to get hands-on. And the first decision to make is: what version? The InfluxDB project made a major version breaking change not that long ago. And older projects like the Raspberry Pi Power Monitor project is still tied to the 1.x lineage of InfluxDB. Since this is an experiment, I would keep the footprint light by running InfluxDB in a Docker container. So when I looked on InfluxDB’s official Docker repository I was puzzled to see it had only 1.x releases. [Update: 2.x images are now available, more details below.] Looking around for an explanation, I saw the reason was because they did not yet have (1) feature parity and (2) a smooth automatic migration from 1.x to 2.x. This could mean bad things happening to people who periodically pull influxdb:latest from Docker Hub. While this problem was being worked on, InfluxDB 2 Docker images are hosted on quay.io/influxdb/influxdb instead of Docker hub.

I found it curious Docker didn’t have a standard mechanism to hold back people who are not ready to take the plunge for a major semantic version change, but I’m not diving into that rabbit hole right now. I have no dependency on legacy 1.x features, or a legacy 1.x database to migrate, or code using the old SQL-style query language. Therefore I decided to dive in to InfluxDB 2 with the knowledge I would also have to learn its new Flux query language that looks nothing like SQL.

Referencing the document Running InfluxDB 2.0 and Telegraf Using Docker, I quickly got a bare-bones instance of InfluxDB 2 up and running. I didn’t even bother trying to persist data on the first run: it was just to verify that the binaries would execute, and that the network ports were set up correctly so I could get into the administration UI to poke around. On the second run I mapped a volume to /root/.influxdbv2 so my data would live on after the container itself is stopped. Now all I need to do is to get some data to store!

[Update: After InfluxDB Docker Hub was updated to release version 2 binaries, the mapped volume path changed from /root/.influxdbv2 to /var/lib/influxdb2. See the InfluxDB Docker Hub repository for details under the section titled: Upgrading from quay.io-hosted InfluxDB 2.x image. In my case it wasn’t quite as straightforward. The migration was introduced in InfluxDB 2.0.4, but I got all the way up to 2.1.1 from quay.io before I performed this migration. A direct switch to 2.1.1 did not work: it acted as if I had a new InfluxDB instance and didn’t have any of my existing data. Trying to run 2.0.4 would fail with a “migration specification not found” error due to a database schema change introduced in 2.1.0. Fortunately, running 2.1.0 docker image appeared to do the trick, loading up all the existing data. After that, I could run 2.1.1 and still keep all my data.]

Docker compose file I used to run 2.1.1 image hosted on quay.io, data stored in the subdirectory “mapped” which is in the same directory as the docker-compose.yml file:

version: "3.8"

services:
  server:
    image: quay.io/influxdb/influxdb:2.1.1
    restart: unless-stopped
    ports:
      - 8086:8086
    volumes:
      - ./mapped/:/root/.influxdbv2/

Update: here’s the version to use latest Docker hub image instead:

version: "3.8"

services:
  server:
    image: influxdb:latest
    restart: unless-stopped
    ports:
      - 8086:8086
    volumes:
      - ./mapped/:/var/lib/influxdb2/
Roger Cheng

Learning InfluxDB Basics

I’ve decided to learn InfluxDB in a project using it to track some statistics about my little toy solar array. And despite looking at some super-overkill high end solutions, I have to admit that even InfluxDB is more high-end than I would strictly need for a little toy project. It will probably be fine with MySQL, but the learning is the point and the InfluxDB key concepts document was what I needed to start.

It was harder than I had expected to get to that “key concept” document. When someone visits the InfluxDB web site, the big “Get InfluxDB” button highlighted on the home page sends me to their “InfluxDB Cloud” service. Clicking “Developers” and the section titled “InfluxDB fundamentals” is a quick-start guide to… InfluxDB Cloud. They really want me to use their cloud service! I don’t particularly object to a business trying to make money, but their eagerness has overshadowed an actual basic getting started guide. Putting me on their cloud service doesn’t do me any good if I don’t know the basics of InfluxDB!

I know some relational database basics, but because of its time-series focus, InfluxDB has slightly different concepts described using slightly different terminology. Here are the key points, paraphrased from the “key concepts” document, that I used to make the mental transition.

  • Bucket: An InfluxDB instance can host multiple buckets, each of which are independent of the others. I think of these as multiple databases hosted on the same database server.
  • Measurement: This is the one that confused me the most. When I see “measurement” I think of a single quantified piece of data, but in InfluxDB parlance that is called a “Point”. In InfluxDB a “measurement” is a collection of related data. “A measurement acts as a container for tags, fields, and timestamps.” In my mind an InfluxDB “Measurement” is roughly analogous to a table in SQL.
  • Timestamp: Time is obviously very important in a time-series database, and every “Point” has a timestamp. All query operations will typically be time-related in some way, or else why are we bothering with a time-series database? I don’t think timestamps are required to be unique, but since they form the foundation for all queries, they have become de-facto primary keys.
  • Tags: This is where we start venturing further away from SQL. InfluxDB tags are part of a Point and this information is indexed. When we query for data within a time range, we specify the subset of data we want using tags. But even though tags are used in queries for identification, tags are not required to be unique. In fact, there shouldn’t be too many different unique tag values, because that degrades InfluxDB performance. This is a problem called “high series cardinality” and it gets a lot of talk whenever InfluxDB performance is a topic. As I understand it, it is a sign the database layout design was not aligned with strengths of InfluxDB.
  • Fields: Unlike Tags, Fields are not indexed. I think of these as the actual “data” in a Point. This is the data we want to retrieve when we make an InfluxDB query.

Given the above understanding, I’m aiming in the direction of:

  • One bucket for a single measurement.
  • One measurement for all my points.
  • “At 10AM today, solar panel output was 21.4 Watts” is a single point. It has a timestamp of 10AM, it is tagged with “panel output” and a field of “21.4 Watts”

With that goal in mind, I’m ready to fire up my own instance of InfluxDB. But first I have to decide which version to run.

Roger Cheng

Investigating Time Series Data

I’ve had a little solar power system running for a while, based on an inexpensive Harbor Freight photovoltaic array. It’s nowhere near big enough to run a house, but I can do smaller things like charge my cell phone on solar power. It’s for novelty more than anything else. Now I turn my attention to the array again, because I want to start playing with time series data and this little solar array is a good test subject.

The motivation came from reading about a home energy monitoring project on Hackaday. I don’t intend to deploy that specific project, though. The first reason is that particular project isn’t a perfect fit for my little toy solar array, but the real reason is that I wanted to learn the underlying technologies hands-on. Specifically, InfluxDB for storing time-series data and Grafana to graph said data for visualization.

Before I go down the same path, I thought I would do a little reading. Here’s an informative StackExchange thread on storing large time series data. InfluxDB was one of the suggestions and I’m sure the right database for the project would depend on specific requirements. Other tools in the path would affect throughput as well. I expect to use Node-RED for some intermediary processing, and that would introduce bottlenecks of its own. I don’t expect to be anywhere near hundreds of sampling points per second, though, so I should be fine to start.

The original poster of that thread ended up going with HDF5. This is something developed & supported by people who work with supercomputer applications, and again had very different performance requirements than what I need right now. HDF5 came up in a discussion on a revamped ROS logging format (rosbag) and it linked to some people unhappy with HDF5. So there are definitely upsides and downsides to that format. Speaking of ROS2… I don’t know where they eventually landed on the rosbag format. Here is an old design spec absent from the latest main branch, but I don’t know what eventually happened to that information.

While I’m on the subject, out of curiosity I went to look up what CERN scientists use for their data. This led me to something called ROOT, a system with its own file format. Astronomers are another group of scientists who need to track a lot of data, and ASDF exists to serve their needs.

It was fun to poke around at these high-end worlds, but it would be sheer overkill to use tools that support petabytes of data with high detail and precision. I’m pretty sure InfluxDB will be fine for my project, so I will get up to speed on it.

Roger Cheng

Functional and Useful 100W Solar Array

Once the Monoprice PowerCache 220 was connected to the Harbor Freight 100W Solar Kit (Item #63585), we have everything we need to gather a little bit of sun power and make use of it every day. Given the non-optimal solar panel position and the fact we’re close to the winter solstice, this is just about the worst case scenario for solar power. Nevertheless this system has been gathering enough power to keep all the battery-powered electronics in the house charged up. This includes daily use & charge items like cell phones, tablets, and laptop computers. Plus the occasional items like a digital camera.

There is much more we can do to improve performance of this system, but it has met a minimum level of satisfactory performance so we can leave it running as-is for a while and switch gears to other projects. The focus will eventually return to this solar power system and here are two candidate projects for later:

Physical tracking: the panels are currently just sitting indoors vertically set against a south-facing window. It was done as an easy nondestructive way to experiment. When it comes time to improve upon this configuration, we can build a more permanent outdoor installation that angles into the sun. Maybe even motorized sun tracking throughout the day!

Electrical tracking: at the moment, the solar panel output voltage is dictated by the battery being charged. This is convenient and simple to implement but not the most efficient. We can buy (or design and build our own) a “maximum power point tracking” (MPPT) charger that keeps the solar panel voltage at its most efficient level and transform that to the correct battery charging voltage. It costs some power to do this tracking & voltage conversion, but if implemented correctly, the additional power will more than offset the cost.

We’ll add these projects to the bottom of the “to-do” list. For now, behold the glory of electronics being charged by sun power.

PowerCache 220 At Work cropped

Roger Cheng

Solar Charging Plug for Monoprice PowerCache 220

The product description page for Monoprice PowerCache 220 (Item #15278) had pictures showing its solar charging input as two round ports, but there were no connector specifications. Since it was advertised as a solar power cache, it would make sense for them to be MC4 connectors. This turned out not to be the case – they were actually two identical sockets that are electrically wired in parallel.

The device comes with an AC adapter for the charging port, but using household power would miss the point of the exercise. Neither do we want to lose the option to charge from grid power, so we don’t want to just cut off the charging plug. We should find an identical plug for solar charging.

The calipers indicate this connector has an outside diameter of 7.9mm and an inside diameter of 5.5mm. There is a small pin in the center roughly 0.5 mm in diameter. Putting those dimensions into search returned a few candidates and a large number of Lenovo laptop power adapters. A fortunate association, as there is a Kensington “Universal Laptop Power Supply” already on hand. It came with a set of interchangeable plugs to fit different laptop brands. This one was purchased to power a Dell, so the Lenovo plug has sat unused. But not for long!

Kensington Laptop Power

The Lenovo laptop plug matches all the diameters, and is slightly longer. It appeared to fit in the socket nicely. Since the two charging ports are electrically parallel, we could plug the AC adapter into the other charging port. This allowed us to read the voltage on the pins of the Lenovo plug so we know where to solder the positive and negative wire.

The plastic housing of the Lenovo plug was damaged in the soldering process so this plug will probably never fit on the Kensington laptop adapter again, but that’s fine. It now has a new purpose as solar charging plug for Monoprice PowerCache 220.

Lenovo now Solar plug

Roger Cheng

Hunt for AC Inverter Finds Monoprice PowerCache 220

Now that we have a 20 amp-hour 12 volt battery, charged by a solar array delivering up to 125 watt-hours daily. To put that power to use, we’ll need an inverter to convert battery power into household 120 volt AC power. This way the collected solar power can be used for more than just charging USB devices.

An old inverter was dug out of the old equipment collection, but it could only sustain about a minute of work before it would stop, reset, and restart. This isn’t great for the electronics, but what made it intolerable to humans were the cascade of noises that devices emit when charging began. Hearing the symphony roughly once a minute was unacceptable so the search begins for a replacement AC inverter.

The Harbor Freight lineup of AC inverters were obvious candidates. Starting from a basic model just under $20 and going up from there. While investigating options outside of Harbor Freight, one stood out: Monoprice #15278  “PowerCache 220”

It is designed exactly for the task we’re building for. It can accept power from a solar array to charge its 18 amp-hour 12 volt battery. That power can be consumed directly as 12 volts DC, as 5 volt USB power, or as 120 volt AC power.

The PowerCache mostly duplicates the components already in the current solar experiment setup. Buying one might be called wasteful, but for the sake of exploration we’ll call it redundancy. This nearly doubles the battery capacity and allows more ways to put solar power to use. It is also more user-friendly than the current maze of wires and connectors. It is an enclosed unit therefore easily portable. This might come in handy if we ever have a reason to take a little portable power source on the go.

So the search that began as a search for a simple AC inverter ended up with purchase of an integrated unit that included the AC inverter and basically everything else short of the solar panels themselves.

PowerCache 220