Notes After Node.js Introduction

After I ran through the Docker Getting Started tutorial, I went back into the docker container (the one we built as we progressed through the tutorial) and poked around some more. The tutorial template was an Express application, built on Node.js. Since I had parts of this infrastructure in hand, I thought I will just run with it and use Node.js to build a simple web server. The goal is to create a desktop placeholder for an ESP32 acting as a web server, letting me play and experiment quickly without constantly re-flashing my ESP32.

The other motivation was that I wanted to get better at JavaScript. Not necessarily because of the language itself (I’m not a huge fan) but because of the huge community that has sprung up around it, sharing reusable components. I was impressed by what I saw of Node-RED (built on Node.js) and even more impressed when I realized that was only a small subset of the overall Node.js community. More than a few times I’ve researched a topic and found that there was an available JavaScript tool for doing it. (Like building desktop apps.) I’m tired of looking at this treasure trove from the outside, I want this in my toolbox.

Node.js is available with a Windows installer, but given my recent knowledge, I went instead to their officially published Docker images. Using that to run through Node.js introduction required making a few on-the-fly adaptations to Node.js in a container, but I did not consider that a hinderance. I consider it great container practice! But this only barely scratches the surface of what I can find within the Node.js community. Reading the list of Node.js Frameworks and Tools I realized not only have I not heard about many of these things, I don’t even understand the words used to describe them! But there were a lot of great information in that introduction. On the infrastructure side, the live demo code was made possible by Glitch.com, another online development environment I’ll mentally file away alongside Cloud9 and StackBlitz.

Even though Google’s V8 JavaScript engine is at the heart of both Chrome browser and Node.js server, there are some obviously significant differences between running in a server environment versus running in browser. But sometimes there are concepts from one world that can be useful in the the other, and I was fascinated by people who try to bring these worlds closer together. One example is Browserify which brings some server-side component management features to browser-side code. And Event Emitter is a project working in the other direction, bringing browser-style events to server-side code.

As far as I can tell, the JavaScript language itself was not explicitly designed with a focus on handling asynchronous operations. However, because it evolved in the web world where network latency is an ever-present factor, the ecosystem has developed in that direction out of necessity. The flexibility of the language permitted an evolution into asynchronous callbacks, but “possible” isn’t necessarily “elegant” leading to scenarios like callback hell. To make asynchronous callbacks a little more explicit, JavaScript Promise was introduced to help improve code readability. There’s even a tool to convert old-school callbacks into Promises. But as nice as that was, people saw room for improvement, and they built on top of Promises to create the much more easier-to-read async/await pattern for performing asynchronous operations in JavaScript. Anything that makes JavaScript easier to read is a win in my book.

With such an enormous ecosystem, it’s clear I can spend a lot more time learning Node.js. There are plenty of resources online like Node School to do so, but I wanted to maintain some resemblance of focus on my little rover project. So I went back to the Docker getting started tutorial and researching how to adapt it to my needs. I started looking at a tool called webpack to distill everything into a few static files I can serve from an ESP32, but then I decided I should be able to keep my project simple enough I wouldn’t need webpack. And for serving static files, I learned Express would be overkill. There’s a Node.js module node-static available for serving static files so I’ll start by using that and build up as needed from there. This gives me enough of a starting point on server side so I can start looking at the client side.

Upsetting the NPM apple cart

Decades-old words of wisdom from a computer science pioneer, proven true once again.

A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable.

Leslie Lamport

In a coincidence of perfect timing, my education of NPM yesterday came just in time for me to understand the left-pad incident. The short version is simple enough to understand: unhappy programmer took his ball and went home, causing a lot of other people grief in the process. The bigger picture, though, needed a bit more knowledge to understand.

While going through the NodeSchool.io NPM workshop I had noticed a few things. The workshop used a dummy placeholder registry but there was really no technical or policy reason why every Jane and Jack can’t run the same lesson against the global registry. Up to and including the fact that they can clean up (un-publish) their NPM package when the workshop is over.

I found that fact curious. Such open accessibility felt fragile and I was wondering about the mechanisms to make sure the mechanism is fortified against accidents or abuse. It wouldn’t be something covered in a workshop, so I thought I’d see more details of this protection elsewhere.

Nope, I was wrong.

The left-pad story proved that there wasn’t any mechanism in place at all. A hilariously trivial package was yanked, causing many houses of cards to fall down.

For all the wonders of NPM, there are downsides that had its share of critics. This incident kicked the criticism into high gear. The NPM registry owner received a lot of fire from all sides and have pledged to update their procedure to avoid a repeat in the future. But I’m not sure that’s enough for the famously anti-authoritarian OSS purists. For every “conflict resolution policy” there will be some who see “ruling with an iron fist.”

 

NodeSchool “How to npm” notes

The lesson of the day is “You get what you pay for.”

NodeSchool.io’s workshop to teach NPM (Node.js Package Manager) is titled “How to npm.”  It has the intention of letting students create a package, set up the proper dependency links, publish it, then publish it again to practice sending out updates. Since it would be bad to pollute the global package registry with a bunch of dummy student test packages, the workshop steps publish against a practice registry running locally instead of the real thing.

As of today, there’s a problem with the practice registry. It can accept updates to packages and track the version numbers, but the portion that tracks labels for these versions (“dist-tag” in NPM lingo) is broken and halts the lesson flow with a HTTP 404 error.

The symptoms match a bug in the workshop that was acknowledged, analyzed, and purportedly fixed. However, the fix is incomplete because some people are still seeing the same error in the updated version of the workshop.

Including myself!

I tried to look at the code behind the lesson and try to see if I can come up with a fix for the bug, maybe this can be my first open source contribution! But alas, I’m not yet good enough with Node.js programming to understand what’s going wrong.

Fortunately I’ve already learned a few useful things before I hit this roadblock. I believe all the material useful to somebody consuming packages has been covered. And I don’t expect to publish any packages myself soon.

But if I do, I’ll come back to this workshop again before I commit. Maybe it’ll be fixed by then.

NodeSchool “learnyounode” notes

After making my first pass through the jQuery Learning Center (I’m sure I’ll be back again) I moved on to another free learning resource: NodeSchool is a collection of resources for learning Node.js, including some self-training tutorials.

At the top of the page they point newcomers to the workshop titled “LearnYouNode”. This workshop has the subtitle “Learn You The Node.js For Much Win!” This raised my eyebrows. It wasn’t immediately obvious that the broken English was a running gag of some sort. I don’t know what it is… maybe LOLcat?

Not knowing it was a gag, I thought maybe the tutorial were written by people for whom English was not their primary language. I was worried I would get stuck trying to figure out if I’m misunderstanding broken English or misunderstanding Node.js.

Fortunately, it only took a paragraph of two of tutorial instructions to make it clear the lessons are not, in fact, broken English. Which was very good news because there was very little hand-holding. The audience is presumed to be competent developers off the bat. It actually assumed JavaScript skill level a bit higher than what I have! Fortunately close enough I could get through with some educational web searches.

The overview workshop sped me through some Node.js basics. Even better: After I successfully completed each challenge, it will show me the official answer which inevitably differed from my solution. I learned a lot from the answer key source code which I hope will help my JavaScript skills going forward.

One of the things LearnYouNode introduced me to is NPM. The Node.js Package Manager came up repeatedly in many of my previous explorations, and I knew I had to get a handle on it soon. I’ve only got a taste so far, and there’s a whole other workshop dedicated to the topic. I think that’s a good place to go next.