I’ve wrapped up version 1.0 of my AS7341 interaction web app project with some ideas for future improvements, but I learned of a big one after I wrote up my project. When an earlier post in my AS7341 series “Sample Code Gave Incomplete Picture of AS7341 SMUX Configuration” was published, there was a comment by [Sebastian] telling me that I’ve overlooked the “Tools & Resources” tab of AMS AS7341 product page.

[Sebastian] is correct! There were several large ZIP file downloads under “Resources” of type “Evaluation Software”. Their descriptions line up with several AMS demos for this sensor. I probably dismissed them as irrelevant as I don’t have the corresponding AMS concept demonstration hardware. But [Sebastian] didn’t make the same mistake. Thanks to his investigation, I’ve been prompted to look inside and found that, in additional to demo-specific resources, there are subdirectories with reference resources including everything I complained was missing:

• Windows application installer, likely for AMS AS7341 GUI software mentioned in calibration Application Note. (I didn’t install on my own computer.)
• Excel spreadsheet also mentioned in calibration Application Note.
• Calibration Application Note along with a few other Application Notes.
• Most importantly: an Application Note on SMUX configuration details!

The gold nugget found within the ZIP file is AMS Application Note AN000666. “SMUX Configuration: How to Configure SMUX for Reading Out Results.” The precise location probably varies from file to file, but for the file I examined (AS7341_EvalSW_ALS_v1-26-3) it was under subdirectory “Documents”/”application notes”/”SMUX”

The key piece of information I had been missing earlier is the concept of mapping AS7341 sensor array to pixel IDs. These pixel IDs are not sequential or regular in any pattern I can decipher, and many pixel IDs are unused. I suspect these ID assignments made sense for reasons important to the engineering team that laid out this implementation on silicon wafers. Between their seemingly random order and the fact roughly half of the IDs were just unused, it was no wonder I failed to reverse-engineer this information from sample code.

But with this Application Note as reference, we now have information in hand to create SMUX configurations to best suit future projects. This is wonderful. Thanks, [Sebastian]! It’s a weight off my shoulders as I proceeded to learn about other mysteries.

With my AS7341+ESP32 assembly all tidied up, alongside my web app project for interacting with them, I think this is a good point to declare version 1.0 complete and move on to something else. Naturally I have more ideas, but today I’m just going to write them down as ideas for later.

Color Accuracy

The most obvious point of improvement is a better translation from detected wavelengths to human-perceived color. I think this would require at least a few days of study (possibly more) before I can be conversant in the topic and maybe understand that Python code sample I found.

Beyond the theoretical math, there are hardware component to better color: the AS7341 has many additional capabilities that I have not used in my little exploratory app. While the eight sensors for specific wavelengths get the attention, the other sensors weren’t there just for fun. They also have a role in color accuracy as per AS7341 application node on color calibration. Those channels provide information on various distortions that may be affecting those eight wavelength sensors.

The clear channel shows the sensor response without a color filter, and the NIR channel shows raw silicon sensor response without even an infrared filter. When any of these sensors return a strong reading, that means enough of their respective types of light are likely “bleeding” into the other color sensors. Flicker detection is likewise also involved because flickering light patterns would impact sensor readings. All of these factors should need to be compensated before feeding into color space conversion.

Temperature Compensation

I haven’t used AS7341’s temperature compensation feature beyond its default behavior of running once upon powerup. Ambient temperature changes would affect sensor behavior, which is true of all sensors. Or to paraphrase what I’ve heard from veteran embedded engineer Elicia White: “Every sensor is a temperature sensor. Some even sense other things.”

Auto Gain Control

A little tangential to the topic of color accuracy, this sensor seems to have some sort of auto gain control to ensure sensors get a good range of values without going too far into saturation. Ideally, I can add an “Auto Gain” checkbox to my app and let the sensor take care of gain control automatically, but that isn’t as easy as it looked at first glance. This feature was not exposed in the Adafruit library and my effort to explore it with twi_nonblock API produced behavior I didn’t understand.

Web App Evolution

Orthogonal to anything I might do to improve AS7341 performance, I might choose to evolve just the web app itself. This first version was written directly in HTML/CSS/JavaScript, the only library I used was Chart.js to plot the bar graph. This process is fine for a simple app but will get more cumbersome for larger projects. So even though this app is fine for its scale, I might use it as “Hello World” exploration of tools that help manage larger projects. Like learning how to use NPM to manage dependencies like Chart.js. Or using TypeScript to tame some of JavaScript’s wild and annoying sides. Or convert it to use an application framework like Angular. That would be sheer overkill for such a small app, but I have big web app project ideas and I need someplace small to start learning.

ESP32 Evolution

Or I might focus on the ESP32 side of things. Top of the list here is using it as a learning project for ESPAsyncWebServer, which has more of the features I might want over the current basic WebServer implementation. Before that, though, I’ll probably switch over to using PlatformIO so I can upload HTML/CSS/JavaScript files to SPIFFS and serve from there, instead of the current unnecessarily cumbersome process of converting them over to hex values in a header file.

Platform Migration

Or a future path would not involve the web app at all. It’s totally possible for a future project idea to be done entirely onboard the ESP32, porting my browser-side JavaScript code to C on the ESP32. It all depends on what motivates me to create enhancements in the future.

Which may be triggered by something like discovering information I had mistakenly overlooked.

It’s been fun playing with my AS7341 interactive UI web app, seeing what the sensor sees and what the app guesses my eyes would see. They’re similar but rarely agree, as should be expected of a quick hack job. Occasionally I would see something wildly inaccurate, and it would take a few seconds for me to realize the problem: one or more AS7341 channels had hit their saturation (ADC_fullscale) values. This “peg the scale” value (a.k.a. “off-scale high”) is no longer accurate. In photography this causes overexposure.

Since the raw server response JSON is printed at the bottom of my app, I could see sensor values that are at their ADC_fullscale value. But this means mentally calculating ADC_fullscale (which isn’t always 65535) and comparing them against those printed numbers. I could do it, but I wouldn’t know to do it unless I suspect something is wrong with the output. I want notification of sensor saturation whenever it happens, even (or especially!) when the result looks reasonable but are actually wrong.

The JavaScript code to detect saturation on a sensor channel was straightforward and can be performed in a loop over values for sensors F1-F8. Though it’s more likely to happen on Clear or NIR first, so I checked those as well. But then I had to show the result in a way that I would notice.

My first modification was to change the header text for server response JSON. Normally displaying “Sensor data OK” but switch to “Sensor saturation (overexposure) detected” as notification. This is an improvement, but I found I don’t always notice when my eyes are glued to the spectrum chart. To make sure I can’t miss it I changed the chart. Every bar was given a white border and, if saturation is detected, that border turns red. This was an improvement, but I might be focused on one channel and miss the fact another channel had saturated. As I understand it, when one channel is saturated, every reading is unreliable. So if this is a global problem, I’ll make the indicator global. The third modification changes every border to red if any channel had saturated. Now I can’t miss it!

With this sensor saturation notification feature, I think my AS7341 interactive web app is at a good place to wrap up for version 1.0. Before I put my sensor away, though, I wanted to make the hardware more compact and portable.

Code for this project is publicly available on GitHub

It’s not going to win any design contests, but I’ve updated my AS7341 web app’s stylesheet so at least its layout is no longer confusing and no longer an embarrassing eyesore. It’s the first of several things I wanted to do for polish. My next challenge is to interpret AS7341 spectrum information into to a color hue a human eye would perceive from looking at the same thing. This is not a trivial conversion, as human color perception has been a long-running area of research. After a few minutes of trying to get my bearings on Wikipedia, I’ve reached the conclusion that doing a good job with my own implementation would take more time than I’m willing to spend on it.

What about somebody else’s implementation? A search for spectrum math library in JavaScript led me to a color picker control named Spectrum, which is not helpful to my current project. Looking on NPM, I found a CIE color conversion library, but I don’t see how to make it perform the type of conversion I seek. Casting my net wider than JavaScript, I found this article titled “Converting a spectrum to a colour” that opens with “This article presents a Python script to map a spectrum of wavelengths to a representation of a colour.” This is exactly what I want! Unfortunately, I struggled to understand the Python code, certainly not enough to convert it to JavaScript for my use. Maybe I can come back later to try again, but in the short term I will try a hack.

AS7341 datasheet tells us which wavelengths each sensors F1-F8 are designed to be sensitive to. Looking online, I found Academo’s “Wavelength to Colour Relationship” page that lets me input a wavelength and translate that to RGB value. Taking a table of RGB values for wavelengths corresponding to AS7341 sensors F1-F8, I added up each column: all the red in one value, all the green, then all the blue.

We see the highest sum for red, followed by green a bit behind, and blue is significantly lower. This is consistent with datasheet telling us the silicon underneath each wavelength filter is naturally more sensitive to red. Since white (and shades of gray) is represented by equal portions of red, green, and blue, getting there required boosting the blue-focused colors a bit to even things out. I didn’t put in rigorous mathematics to make them balanced since I don’t even know if this action makes sense in color science. As a quick hack, I used a spreadsheet and fiddled with numbers via trial and error. I found that if I multiplied 415nm by 1.72, 445nm by 1.6, and 480nm by 1.4, I would get red/green/blue within 1% of each other. From here I can multiply them by F1-F8 readings and calculate each of their contribution to red/green/blue channels and generate a color value.

This is an empirically derived formula with no basis in color science, but it does generate a color value that is vaguely in the right general ballpark. I piped that color into Chart.js to be used as my chart background color, following instructions on their documentation. This is most of what I wanted, and maybe the best I can do without investing the work to understand human color perception. Not great, but good enough for a quick hack project so I can move on to the next feature.

Code for this project is publicly available on GitHub

Thanks to Chart.js documentation, I was quickly up and running with a simple bar chart to visualize AS7341 data. My first draft was done late at night, so this chart corresponded to the spectral distribution of my room’s ceiling light.

With an advertised color temperature of 2700K, its “warm white” showed a strong response in the yellow and orange areas of the spectrum. The next morning, I had a ray of sunshine coming in a window and I set my AS7341 sensor within it.

The first lesson was that sunlight — even just a tiny beam at an oblique angle — is significantly stronger than my ceiling light. Direct exposure will always reach sensor saturation (ADC_fullscale value) no matter what I do. I ended up placing a sheet of printer paper at my sunlight spot and aiming the sensor at that reflected light.

This spectrum has whatever distortion added by a sheet of paper, but it is still very interestingly different from my ceiling light. There is a huge response on NIR sensor, and there isn’t as strong of a peak on orange. My brain sees both of these light sources as white, but the sensor sees very different spectrum between them. Raw sensor data (with clear channel hitting saturation) are as follows:

{
  "415nm": 5749,
  "445nm": 6342,
  "480nm": 9533,
  "515nm": 10746,
  "555nm": 11245,
  "590nm": 12577,
  "630nm": 12633,
  "680nm": 15217,
  "clear": 65535,
  "nir": 34114,
  "settings": {
    "atime": 30,
    "astep": 3596,
    "gain": 64,
    "led_ma": 0,
    "read_time": 674
  }
}

According to AMS AS7341 calibration application note, knowing NIR level is important for properly compensating values of spectral sensors F1-F8. They are sensitive to whatever NIR leaked past their filters, so knowing NIR level is important for precise color accuracy. The clear channel and flicker channels likewise have their own impact on color accuracy. But since I’m just goofing around and not concerned with utmost accuracy, I’m choosing to ignore them and dropping NIR from my visualization.

I will, however, make use of this sunlight spectrum to compensate for the differing sensitivities across spectral sensors F1-F8. Using sunlight as my reference for a light source emitting all wavelengths of light, we confirm AMS AS7341 datasheet information that 415nm is the least sensitive and 680nm is the most sensitive. I can selectively boost sensor values so that F1-F8 would all return the same value under direct sunlight. This is crudely analogous to a camera’s color balance (or “white balance”) features, and I implemented the following normalization options in my app:

• Default normalization curve based on these sunlight values.
• A direct data option skipping the selective boost.
• An option to use the next sensor reading as reference. I can point the sensor at something and activate this option to tell my app: “treat this color as white”.

Each of these options had a corresponding button onscreen. Functional, but the jumble of controls on screen is starting to cause usability problems. I built this app and if I get disoriented, how bad would it be for everyone else? It’s time to put some effort into layout with CSS.

Code for this project is publicly available on GitHub

Getting interactive control over AS7341 sensor parameters helped me better understand their effect on resulting data. Interactive control over sensor integration time (photography analogy: exposure time) made it easy for me to see how the data reacted mostly linearly until they reach their limit. I had known the AS7341 ADCs were 16-bit, so I thought the limit is always 65535. This is wrong: it is actually 65535 OR ADC_fullscale, whichever is lower.

I came across ADC_fullscale in the datasheet but I didn’t understand what that information meant at the time. I had mistakenly thought it placed a limit on integration time, as it is calculated from integration time parameters “atime” and “astep” with the formula (atime+1)*(astep+1). Now I know it does not limit integration time but is actually a cap on ADC values if that formula results in less than 65535. For example, right now I’m running with a fixed “astep” of 3596 which corresponded roughly to 10 milliseconds per “atime”. If I configure “atime” to 9, (atime+1)*(astep+1) = (10)*(3597) = 35970 is the sensor saturation limit ADC_fullscale. Not 65535.

Another thing I learned was that my original plan for “LED stay on” parameter wouldn’t work. I had designed it to be a parameter sent alongside atime, astep, and gain at the beginning of sensor read. It seemed reasonable enough until I tried to design a control to toggle whether we are going to read the sensor continuously. When will the user toggle that to be “OFF”? Odds are, such toggle would happen while we are in the middle of sensor integration. By that time, it was too late to communicate LED should be turned off.

Oh well, mistakes like this happen. That useless “led_stay_on” parameter was removed, and I added code so “led_ma” could be a valid operation by itself without triggering a sensor read. This lets me adjust the LED illumination (usually turning it off) without performing a sensor read. Just another instance where iterative development is useful, updating my design as I go.

Code for this project is publicly available on GitHub

After pleasantly surprised at the fact my web app project is unlikely to leave my old Android phones behind, I got back to my “learn by doing” process. As originally planned, I’m going to leave my first draft “Basic” UI as-is for fallback/debug purposes. I made a copy to serve as the starting point of my “Standard” UI and initial focus is on sensor integration time.

Full Width Control

First change was minor and cosmetic: “Basic” didn’t use any CSS styles, and “Standard” started with just one style to make input sliders 100% of available width. The default width was annoyingly narrow. Making it full width helps me make finer adjustments.

ASTEP Value Fixed at ~10ms

AS7341 sensor integration time is controlled by two parameters: ATIME and ASTEP. As per datasheet, the resulting integration time follows the formula: (ATIME+1)*(ASTEP+1)*2.78 microseconds. My “Basic” UI exposed both parameters directly, but I want to use something more human-friendly for the “Standard” UI. I decided to keep ASTEP fixed at 3596. Per the formula (3596+1)2.78 microseconds = 9999.66 microseconds, or just a tiny bit less than 10 milliseconds. Keeping ASTEP fixed at 3596 means the ATIME range of 0-255 can dictate integration time anywhere from 10ms to 2560ms a.k.a. 2.56 seconds. This covers the entire time range I want for initial experiments. I may adjust this range in the future after playing with the sensor some more.

Read Time is Double Integration Time

Once I had set up the UI to adjust integration in 10ms increments, I took a few readings at various settings and noticed the actual time spent in readAllChannels() is a lot longer than integration time. If I configure for 1 second (1000ms), I end up waiting two seconds. I added a bit of tracking code to my ESP32 Arduino sketch and verified it wasn’t just my imagination: actual read time is over double integration time.

This was puzzling until I remembered readAllChannels() implementation: it configures AS7341 sensor multiplexor (SMUX) to read sensors F1-F4 plus clear and NIR, performs a read, then configures SMUX for sensors F5-F8 (keeping clear and NIR) and perform a second read. So, a one-second sensor integration time meant one second spent for F1-F4 and another second spent on F4-F8. Add in a few tens of milliseconds for processing and communication overhead, and we’ve explained the observation of a little bit over twice integration time.

This is something to keep in mind depending on sensor application. For example, if I want my browser UI to update once a second, I need to set integration time to be less than half a second.

UX Design Decision

Which led to the next question: Should my browser UI show the integration time, because that’s how I’m configuring the sensor? Or should it show double that time, because that is a better estimate of time I should expect to wait for a reading? I decided to go with double for my “Standard” UI: this is a user experience issue, so I should be faithful to what the user will experience.

Code for this project is publicly available on GitHub