This blog is a project diary, where I write down not just the final result, but all the distractions and outright wrong turns taken on the way. I write a much shorter summary (with less noise) for my projects in the README file of their associated GitHub repository. As much as I appreciate markdown, it’s just text and I have to fire up something else for drawings, diagrams, and illustrations. This becomes its own maintenance headache. It’d be nice to have tools built into GitHub markup for such things.

It turns out, I’m not the only one who thought so. I started by looking for a diagram tool to generate images I can link to my README files, preferably one that I might be able to embed into my own web app projects. From there I found Mermaid.js which looked very promising for future project integration. But that’s not the best part: Mermaid.js already have their fans, including people at GitHub. About a year ago, GitHub added support for Mermaid.js charts within their markdown variant, no graphic editor or separate image upload required.

I found more information on how to use this on GitHub documentation site, where I saw Mermaid is one of several supported tools. I have yet to need math formulas or geographic mapping in my markdown, but I have to come back to take a closer look into STL support.

As my first Mermaid test to dip my toes into this pool, I added a little diagram to illustrate the sequence of events in my AS7341 spectral color sensor visualization web app. I started with one of the sample diagrams on their live online editor and edited to convey my information. I then copied that Mermaid markup into my GitHub README.md file, and the diagram is now part of my project documentation there. Everything went through smoothly just as expected and no issues were encountered. Very nice! I’m glad I found this tool and I foresee adding a lot of Mermaid.js diagrams to my project README in the future. Even if I never end up integrating Mermaid.js into my own web app projects.

Artists explore where the mainstream is not. That’s been true for as long as we’ve had artists exploring. Early art worked to develop techniques that capture reality the way we see them with our eyes. And once tools and techniques were perfected for realistic renditions, artists like Picasso and Dali went off to explore art that has no ambition to be realistic.

This evergreen cycle is happening in computer graphics. Early computer graphics were primitive cartoony blocks but eventually evolved into realistic-looking visuals. We’re now to the point where computer generated visual effects can be seamlessly integrated into camera footage and the audience couldn’t tell what was real and what was not. But now that every CGI film looks photorealistic, how does one stand out? The answer is to move away from photorealism and develop novel non-photorealistic rendering techniques.

I saw this in Spider-Man: Into the Spider-Verse, and again in The Mitchells vs. the Machines. I was not surprised that some of the same people were behind both films. Each of these films had their own look, distinct from each other and far from other computer animated films. I remember thinking “this might be interesting to learn more about” and put it in the back of my mind. So when this clip came up as recommended by YouTube, I had to click play and watch it. I’m glad I did.

From this video I learned that the Spider-Verse people weren’t even sure if audience would accept or reject their non-conformity to standards set by computer animation pioneer Pixar. That is, until the first teaser trailer was released and received positively to boost their confidence in their work.

I also learned that they were created via rendering pipelines that have additional stylization passes tacked on to the end of existing photorealistic rendering. However, I don’t know if that’s necessarily a requirement for future exploration in this field, it seems like there’d be room for exploring pipelines that skip some of the photorealistic aspects, but I don’t really know enough to make educated guesses. This is a complex melding of technology and art. It takes some unique talent and experience to pull off. Which is why it made sense (in hindsight) that entire companies exist to consult for non-realistic rendering, with Lollipop Shaders the representative in this video.

As I’m no aspiring filmmaker, I doubt I’ll get anywhere near there, but what about video game engines like Unity 3D? I was curious if anyone has explored applying similar techniques to the Unity rendering pipeline. I looked on Unity’s Asset Store under the category of VFX / Shaders / Fullscreen & Camera Effects. And indeed, there were several offerings. In the vein of Spider-Verse I found a comic book shader. Painterly is more like Mitchells but not in the same way. Shader programmer flockaroo has several art styles on offer, from “notebook drawings” to “Van Gogh”. If I’m ever interested in doing something in Unity and want to avoid the look of default shaders, I have options to buy versus developing my own.

Sometimes a project idea comes up and is hampered by the most unexpected of problems. The visual dimension measuring machine project (I need to come up with a better name) is mechanically speaking a camera bolted to the carriage of a former 3D printer. I wanted to explore how precisely controlled camera movements can help capture precise dimensions of an object in view.

I’ve been working on the former 3D printer, bringing a retired Geeetech A10 back up and running. I then started looking into the webcam side. I wanted something better than the average camera built into the screen bezel of a laptop. Ideally this meant a good optical lens assembly with auto focus capability, and not the more common fixed-focus cameras with mediocre lenses barely better than a pinhole camera.

And… all the good webcams are sold out! I had not known we were in the middle of the Great Webcam Shortage of 2020, but it made sense. A lot of officers workers have switched to working at home, and I’m not the only one dissatisfied with the camera built in to our laptops. Thus the demand for an upgraded camera shot up well past historical norms, and manufacturers are scrambling to meet demand.

So the first draft of my project will have to make do with a webcam I already had on hand, which is probably a good idea for a prototype anyway. I have a HP Webcam HD 4310 I can draft for the purpose. I had been using it to monitor my 3D prints via OctoPi, but that printer is currently offline anyway so the camera is available.

The print on the outside proclaims “1080P HD” and “Auto Focus”. I’m not sure the former is true – it does have the option to output 1080P but the visual quality is rather less than I expected at that resolution. I strongly suspect that 1080P is not the native sensor resolution, but an upscaled image. The alternate explanation is that the 1080P sensor is hampered by poor lens. Either way, it’ll have to do, and at least it does offer auto focus! Let’s find out what’s inside.

When I was researching my Hackaday post about DARPA Subterranean Challenge, I learned that there’s a virtual track to the competition using just digital robots inside Gazebo. I also learned it was not the first time a virtual competition with prize money took place within Gazebo, there was also the NASA Space Robotics Challenge where competitors submit software to control a humanoid robot on a Mars habitat.

What I didn’t know at the time was that the virtual humanoid robot was actually based on a physical robot, the NASA R5. Also called Valkyrie, this robot is the size of a full human adult with a 7-digit price tag putting it quite far out of my reach. This robot was originally built for the 2013 DARPA Robotics Challenge. It appeared the robot had no shortage of ingenious mechanical design (I like the pair of series elastic actuator for that ankle joint.) It was not lacking in sophisticated sensors, and it was not lacking in electric power. What it lacked were the software to tie them all together, and an unfortunate network configuration issue hampered performance on actual day of DARPA competition.

After the competition, Valkyrie visited several research institutions interested in advancing the state of the art in humanoid robotics. I assume some of that research ended up as published papers, though I have not yet gone looking for them. Their experience likely fed into how the NASA Space Robotics Challenge was structured.

That competition was where Valkyrie got its next round of fame, albeit in a digital form inside Gazebo. Competitors were given a simulation environment to perform the list of tasks required. Using a robot simulator meant people don’t need a huge budget and a machine shop to build robots to participate. NASA said the intent is to open up the field to nontraditional sources, to welcome new ideas by new thinkers they termed “citizen inventors”. This proved to be a valid approach, as the winner was an one-person team.

As for the physical robot, I found a code repository seemingly created by NASA to support research institutions that have borrowed Valkyrie, but it feels rather incomplete and has not been updated in several years. Perhaps Valkyrie has been retired and there’s a successor (R6?) underway? A writer at IEEE Spectrum noticed a job listing that implied as such.

(Image source: NASA)