I chose to experiment with UWP serial device communication with my 3D printer because (1) it sends out a sequence of text immediately upon connection and (2) it was already sitting on the table. Just trying to read that text was an educational exercise, including a side trip through the world of logging.

The next obvious step was to send a command and read the printer’s response. This is where I learned 3D printers like this MatterHackers Pulse XE behaves a little differently from serial devices I’ve worked with before. RoboClaw motor controllers or serial bus servos like the Dynamixel AX-12 or LewanSoul/Hiwonder LX-16A have one behavior in common: They listen for a command in a known format, then they send a response also in a known format. This isn’t what my 3D printer control board does.

It wasn’t obvious to me until in hindsight, but I should have known as soon as I saw it send out information upon connection before receiving any commands. That’s not the only time the printer would send out unprompted information. Sometimes it sends text about SD card status, or to indicate it is busy processing the previous command. Without a 1:1 mapping between command and response, the logic to read and interpret printer response to commands has to be a little more sophisticated than what I’ve needed to write for earlier projects.

Which is a great opportunity to learn how to structure my code to solve problems with the async/await pattern. When I had a strict command/response pattern, it was easy to write code that assumes the information I read is in direct response to the command I sent. Now that data may arrive unprompted, the read and write operations have to be separated into their own asynchronous processing loops. When the read loop receives data, it needs to be able to interpret that possibly in the absence of a corresponding command. But if there is a corresponding command, it needs to pair up the response with the command sent. Which meant I needed a queue of commands awaiting responses and logic to decide when to dequeue them and send a response back to their caller.

Looking at code behavior I can see another potential: that of commands that do not expect a corresponding response. Thankfully I haven’t had to deal with that combination just yet, what I have on hand is adding enough challenge for this beginner. Certainly getting confusing enough I was motivated to extended my logging mechanism to help understand the flow.

[The crude result of this exercise is available publicly on GitHub]

After playing with the Custom Serial Device Access demonstration application to read a 3D printer’s greeting message, I created a blank C# application from the Universal Windows Platform application template in Visual Studio and copy/pasted the minimum bits of code to read that same printer greeting message and send it to text on screen.

The sample application only showed a small selection of text, but I wanted to read the entire message in my test application. This is where I ran into an unexpected behavior. I had set the SerialDevice.ReadTimeout property to various TimeSpan on the scale of a few seconds. Sometimes I would get the timeout behavior I expected, returning with some amount of data less than buffer size. But other times my read operation hangs indefinitely past the timeout period.

I thought I did something wrong with the async/await pattern causing me to await forever, but I cut the code way back to the minimum while still following the precedence of the sample app, and it was still happening unpredictably. Examining the data that was returned, it looked like the same greeting message I saw when I connected via PuTTY serial terminal, nothing to indicate a problem.

Eventually I figured out the factor wasn’t anything in the data I have read, but the data I have not yet read. Specifically, the hanging behavior occurs when there is no further data at all from the serial port waiting to be read. If there was even just one byte, everything is fine: the platform will pull that byte from the serial port, put it in my allocated buffer (I experimented with 1 kilobyte size buffer, 2 KB, 4KB, it didn’t matter) and return to me after the timeout period. But if there are no bytes at all, it hangs waiting.

I suppose this makes some sort of sense, it’s just not what I had expected. The documentation for ReadTimeout mentions that there’s an underlying Win32 data structure SERIAL_TIMEOUTS dictating underlying behavior. A quick glance through that page failed to find anything that corresponds to what I think is happening, which worries me somewhat. Fortunately, there are ways to break out of an await that has waited longer than desired.

[This Hello3DP programming exercise is publicly available on GitHub]

Reading about novel programming patterns and contemplating unexpected hardware platform potential are great… but none of it is real until I sit down and make some code run. Since my motivation centered around controlling external peripherals via serial port, I’ll need a piece of hardware to experiment against. In the interest of reducing variables, I didn’t want to start with one of my own past projects. I expect to run into enough headaches debugging what went wrong, I don’t need to wonder if the problem is my code or my project hardware.

So what do I have sitting around at home that are serial controlled hardware? The easy answer is the brains of a 3D printer. And the most easily accessible item is my MatterHackers Pulse XE printer, which is conveniently sitting on the same table as the computer.

In order to get an idea of what I’m getting into, I connected to the printer via serial port terminal of PuTTY. I saw that a greeting message is sent out via serial as soon as I connected. This is great, because it meant I have some data to read immediately upon connect, no need to worry about sending a command before getting a response.

Moving to the development platform, I loaded up the UWP example program Custom Serial Device Access. After reading the source code to get a rough feel of what the application did, I compiled and ran it. It was able to enumerate the USB serial connection, but when I connect, I did not see the greeting message. Even though the parameters I used for PuTTY (250000 N 8 1) were also used here.

As tempting as it might have been to blame the example program and say it is wrong, I thought it was more likely that one of the other parameters of SerialDevice had to be changed from their default value. Flipping settings one by one to see if they change behavior, I eventually figured out that I needed to set IsDataTerminalReadyEnabled to true in order to receive data from a Pulse XE. I’m lucky it was only a single boolean value I had to change because if I had to change multiple values to a specific combination, there would have been too many possibilities to find by trial and error.

It’s always good to start as simple as possible because I never know what seemingly basic issue would crop up. IsDataTerminalReadyEnabled wasn’t even the only surprise I found, ReadTimeout behavior was also unexpected.

The problem when looking at a list of “Other Resources” links is that it’s hard to know how useful they are until I actually go in and read them. I started my C# async/await journey on this page, which was only a short conceptual overview. At the bottom is a link to a page titled Asynchronous programming with async and await (C#) which sounded just like the page I was already on. A quick look to confirm it’s indeed a different URL, and I opened it up in a new tab to visit later.

I read several different pages sections before I got around to revisiting that opened tab. I had expected a short page with a lot of redundant information, but I quickly realized I had been neglecting a gold mine of information. This page explained the async/await model to me better than any other pages by using one of my favorite mechanisms for teaching: an analogy. The process to prepare a big breakfast was rephrased into asynchronous operations and that’s where it finally clicked.

The most useful clarification was that async doesn’t necessarily mean parallel. In the breakfast analogy it means multiple parts of the breakfast can be cooked but it’s possible to have just one chef in the kitchen doing it all. This broke me out of my previous mental mold, which was preventing some concepts from sinking in because it didn’t make sense in a parallel processing context. This section finally made me understand asynchronous processing is a concept that is frequently related to, but at the root is independent of, parallel processing.

One goal of the async/await pattern was to make it easy for developers to reason about logic flow, but under the hood it actually involves a lot of complexity to make a computer do what a human would hopefully find more intuitive. And like all programming magic, if things should break down the developer needs to know implementation details to debug it. Such detail was summarized in the “What happens in an async method” chart with simple code annotated with a lot of arrows. It was intimating at first, but once past the hump it is very helpful. I later came across an expanded version of this diagram explanation on this page with more details.

Once I had that understanding, I had a much easier time understanding what to do if something fails to go as planned. Cancelling and managing end of tasks is its own section.

I expected to find some example programs to look at, and I did, but looking at source code is only seeing the destination. It was very helpful for me to see the journey to destinations in an example taking a synchronous application and convert it step-by-step to asynchronous operation.

And finally, I return to the asynchronous programming section of UWP documentation. These topics now make a lot more sense than they did before I sat down to learn TAP. And some of the promise of UWP hold intriguing possibilities.

The upside of choosing to learn a Microsoft-backed technology via their developer documentation site is that there’ll be a lot of information. The downside is that it quickly becomes a flood of too much information, especially for things with a lot of surface area touching a lot of products. We end up with a section from each division, each introducing the concepts from their own perspectives. Resulting in a lot of redundant information though, thankfully in this case, I found nothing contradictory.

I started from the C# language side via the async keyword, which led me to a quick conceptual overview with a few short examples. Blitzing through this page got me oriented on the general direction, but it wasn’t detailed enough for me to feel I understood. As a starting point it wasn’t bad, but I needed to keep reading.

Next item I found was from the .NET side, and I was prepared for a long read with the title “Async in depth“. It was indeed more in depth than the previous page, but it was not quite as in depth as I had hoped. Still, it corrected a few misconceptions in my head, such as the fact async behavior does not necessarily mean multiple threads. In I/O bound operations, there may be no dedicated waiting thread at all. It meant in certain situations, this mechanism is better than old-school multithreaded techniques I had known. For example this avoids the situation of building up large numbers of threads sitting around blocked and waiting.

I got more details — and more detailed code walkthroughs — when I broadened my focus from the async keyword itself to TAP or Task-based Asynchronous Pattern, the umbrella under which Microsoft designated not just the async/await pattern itself but also a bunch of other related mechanisms. This section also has a comparison of .NET asynchronous programming patterns, and TAP can interoperate with them. Which is not relevant to me personally but it’s good to see Microsoft’s habit for backwards compatibility is alive and well. This will be useful if I get into TAP and then something even newer and shinier comes along and I need to switch. I hope there aren’t too many more of those, though, as the interop chart will get a lot more complicated and confusing very quickly.

These sections were informative, and I probably could have figured it out from there if I needed to get rolling on a short schedule. But I didn’t have to. I had the luxury of time, and I found the gold mine that is the C# Programming Guide for Asynchronous Programming.

I have decided to sit down and put in the time to learn how to write software for asynchronous events using the relatively new (and certainly new to me) async/await pattern. I had several options for which language/platform to use as my first hands-on experience, and I chose Microsoft’s C#/UWP for multiple reasons. It was something I looked at briefly, there’s a free Community Edition of the Visual Studio software tool to explore (and debug) them, and most importantly, I saw lots of documentation available on Microsoft’s documentation site (formerly MSDN.) While I have no problem learning little things via reading posts on Stack Overflow, for something bigger I wanted a structured approach and I was familiar with Microsoft’s style.

Before I dove in, I was curious why Microsoft saw an incentive to invest in this pattern. This is a significant effort: it meant adding support the .NET platform, adding support in related languages like C#, adding support in tools like Visual Studio. Accompanied by tutorials, documentation, code libraries, and examples. And from what I can tell, their motivation is that using async/await pattern in UWP applications will keep the user interfaces responsive.

Responsive user interfaces have been a desire ever since there were user interfaces, because we want the user to get feedback for their actions. But it’s hard to keep everything running when the code is waiting on something to finish, which is why every user has the experience of clicking a button and watching the entire application freeze up until some mysterious thing is done. On the web the problem is both better and worse: better because the web browser itself usually stays responsive so the user can still click on things, worse because the web developer no longer has control and has to do silly things like warn the user not to click “refresh” or “back” buttons on their browser.

Back to the desktop: application platforms have the concept of an “UI thread” where all the user interaction happens, including events handlers for user actions. It is tempting to do all the work inside the event handler, but that is running on the UI thread. If the work takes time the UI thread is unable to handle other events, leaving the user with no feedback and unable to interact with the application. They are left staring at an hourglass (or spinning “beach ball”, etc) wondering what’s going on.

Until async/await came along the most common recommendation is to split up the work. Keep UI thread event handlers light doing as little as possible. Offload the time-consuming work to somewhere else, commonly another thread. Once the work is done, that worker is supposed to report its results back to the UI. But orchestrating such communications is time consuming to design, hard to get right in code, and even harder to debug when things go wrong. Between lazy developers who don’t bother, and honest mistakes by those who try, we still have a lot of users facing unresponsive frozen apps.

This is where async/await helps UWP and other application platforms: it is a way to design the platform interfaces so that they can tell app developers: “You know what, fine. Go ahead and run your code on the UI thread. But use async/await so we can handle the complicated parts for you.” By encouraging the pattern, UWP could potentially offer both a better developer experience and a better end-user experience. Success would lead to increased revenue, thus the incentive to release bountiful documentation for async/await under their umbrella of Task-based Asynchronous Pattern. (TAP)