Installing TensorFlow: Adventures in Version Matching

One of the big motivations to get a NVIDIA 1060 GPU is to start playing with deep learning neural networks, using tools like Google’s TensorFlow. A quick skim of the installation instructions didn’t look too bad, but once I got hands-on it was clear version mismatches were a recurring issue for running the TensorFlow binaries.

Ubuntu 16.04.4 Operating System

Since I had only a fresh installation of Ubuntu, it was not a big loss (aside from time) to restart from scratch. Goodbye, new hotness Ubuntu 18.04 LTS, we’re leaving you to return to the known quantity Ubuntu 16.04.4 LTS.

NVIDIA Graphics Driver 390

Once Ubuntu 16.04 was up and running with all its latest operating system updates, we proceed to the only part of TensorFlow setup that doesn’t require a specific (old) version: the NVIDIA graphics drivers. Going straight to NVIDIA’s website found this suggestion:

Note that many Linux distributions provide their own packages of the NVIDIA Linux Graphics Driver in the distribution’s native package management format. This may interact better with the rest of your distribution’s framework, and you may want to use this rather than NVIDIA’s official package.

Hmm… OK. Checking Ubuntu’s user support site AskUbuntu for NVIDIA drivers found this set of instructions to install drivers from a graphics drivers project. Staffed by volunteers that repackage NVIDIA’s raw binaries into Ubuntu installer-friendly “personal package archives” (PPA) infrastructure. Following the instructions, I told my installation of Ubuntu about them via:

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update

Then I had to deviate from the directions because sudo apt-get install nvidia-driver-390 returned an error E: Unable to locate package nvidia-drivers-390.  A little searching using the apt search command found the correct name in the PPA:

sudo apt-get install nvidia-390

After driver installation, the graphics driver project page recommends that I test my system by running the Phoronix graphics test suite. It gave my system a good workout and provided some pretty visuals while the tests ran. After the tests were complete, I was happy to upload my system information to their database of installation successes.

NVIDIA CUDA Toolkit 9.0

As of this writing, the TensorFlow binaries released by Google requires NVIDIA CUDA toolkit version 9.0. The latest version of CUDA toolkit is 9.2 but sadly it will not work with TensorFlow binaries, so we have to go find the older version 9.0 in NVIDIA’s past version archives. Again we have to deviate from instructions because sudo apt-get install cuda would install the latest 9.2. We have to be explicit about the older version with sudo apt-get install cuda-9.0.

After the toolkit was installed, follow instructions to compile and run the tools deviceQuery and bandwidthTest to verify installation.

NVIDIA cnDNN SDK 7.0.5

TensorFlow instruction specifies cnDNN v7. The latest version is 7.1.4 as of this writing and based on other experience probably wouldn’t work, so off we go into the old version archives again. There we will find two versions corresponding to CUDA 9.0: 7.0.4 and 7.0.5. Which one to use?

The hint comes from elsewhere in TensorFlow installer documentation talking about an optional component, NVIDIA TensorRT 3.0. The instructions there made a reference to 7.0.5.15-1+cuda9.0 so that swayed the vote on which version to use.

After the SDK was installed, follow instructions to compile and run the tool mnistCUDNN to verify installation.

TensorFlow

After all those pre-requisites were satisfied, it was finally time to install TensorFlow itself! I decided to go with Python 3 for my TensorFlow experimentation, and followed recommendation of virtualenv, before installing the GPU-enabled variant of TensorFlow.

pip3 install --upgrade tensorflow-gpu

Finish Line

After bumping head in the dark against version mismatch error for hours, it felt great to finally cross the finish line. Hello, TensorFlow!

Hello, TensorFlow

 

Installing 2.5″ SSD in Dell Inspiron 15 7000 (7577)

When shopping for the Dell Inspiron 15 7000 (7577) I browsed the various configurations available. It was clear the chassis has provision for two storage devices: One M.2 slot (with NVMe support) that is home to a SSD, plus one 2.5″ drive bay that is sometimes populated by a spinning platter hard drive.

This was an attractive feature of the chassis. Having two drives would allow dual-boot  between Windows and Linux while keeping each operating system completely independent. This is how I typically set up my desktop (and luggable) computers, but historically laptops only had one drive bay so this is a new luxury.

Since I don’t intend to use a spinning platter drive, I ordered the configuration with only a M.2 SSD. And while the chassis has provision for a 2.5″ drive, buying with only a M.2 means I’m not guarantee to receive all the support hardware necessary to use that 2.5″ bay. I held my breath when I first opened up the computer and exhaled a sigh of relief when I saw that all the hardware was present.

Here are the 2.5″ bay hardware that Dell shipped in this chassis:

  • Drive installation bracket, featuring metal plates on either side, each with two shock-isolation grommets. The plates are joined by a thin sheet of plastic that I thought was a cosmetic cover, but is actually part of the structure. Don’t rip it out by its perforations like I almost did!
  • Four screws to fasten that bracket to the system chassis.
  • SATA adapter electrically connecting the 2.5″ drive to the system motherboard.

All of these parts are highly specific to this chassis and would have been a real pain to procure separately. I might have been able to fabricate a drive bracket, and maybe find some screws that worked to hold it to the chassis. But the SATA adapter is well beyond my skills to build my own.

Dell 7577 2.5 inch bay hardware

A special happy surprise were these four hard drive mounting screws. They are  standard M3 fasteners and easy to procure separately. (Some 2.5″ drives even come with a set in their package.) But Dell decided it made sense to keep a set on hand inside the computer ready to go. Sitting in a row here, they serve no structural purpose. They’re just waiting for a 2.5″ drive.

Dell 7577 2.5 inch drive screws

The 2.5″ bay accepts only slim 7mm drives, so full size 9mm drives would not fit. I installed an Intel 530 series SSD as it was the most convenient 7mm drive already available on hand.

Dell 7577 2.5 inch drive installed

Unfortunately this drive did not play well with the laptop. 530 Series were known to be finicky and has caused problems in a few of my other computer projects. (Which is partially why it’s sitting around gathering dust…) And the problems continued inside this Dell. It was so unhappy, in fact, that not only would it stop responding to the computer, it would also occasionally knock the M.2 drive offline. Whatever it was doing, it wasn’t good.

I had an Intel 320 series SSD also sitting around, whose metal case would also work in a 7mm bay but it had the plastic spacer to make it fit snugly in a 9mm bay. Removing the plastic spacer was as simple as removing four screws (though it would have voided the warranty if it hadn’t already expired) but it also meant the drive fell apart. I ended up pulling two screws out of the 350 Series drive. That was good enough to hold the drive together in the laptop’s 7mm bay.

The 320 series was much happier working inside the Dell laptop, so now I could proceed to install the latest Ubuntu LTS (18.04) on that drive. Though I ended up having to erase 18.04 and go back to the much older 16.04.4, that story is coming up next.

 

 

Two Notes of Happiness on New Dell Inspiron 15 7000 (7577)

After waiting for almost a year for GPU prices to return to sanity, I gave up waiting for a discrete card I can install in my Luggable PC. I’ve been waiting to get started playing with CUDA-accelerated TensorFlow training and the best way to get an NVIDIA GPU at the moment is to get it inside a laptop. Since it’s not terribly practical (or price effective) for cryptocurrency miners to build huge racks of laptops for mining, the laptop variants of those chips are easier to come by, and at less crazy prices.

The laptop arrived and everything worked as advertised. But two items are worth calling out because they were details not found on a Dell specification sheet. I had my hopes but there’s no way to know until I open it up!

Dell 7577 Happiness

Happiness #1: Memory

Visible in the upper right corner of the picture are the two memory slots on this laptop chassis. Dell only advertised that the machine will come with 8GB of RAM, they did not specify the arrangement. I had expected them to fill up both slots each with an 4GB memory module because that’s usually cheaper for them. The downside of having two 4GB modules is when it comes to upgrade: with only two memory slots, any upgrade means removing an existing module and losing that capacity.

Fortunately, Dell shipped this computer with a single 8GB module, leaving the other slot open for future upgrade. I don’t have to remove any capacity when I upgrade – just plop a new module into that second slot and I’m good to go. This is great news.

Happiness #2: Hardware for 2.5″ Storage Drive

Visible in the lower left corner of the picture is the 2.5″ drive bay. Some configurations of this laptop are sold with a combination of a 2.5″ spinning disk hard drive augmenting the capacity of a small M.2 SSD. This particular model comes with a 256GB M.2 SSD and no hard drive. I had expected the 2.5″ bay to exist in my chassis, but empty. With not just the 2.5″ drive absent but also missing all the support hardware necessary to install one after purchase.

Fortunately, Dell shipped this computer with all the support hardware in place. This includes the metal bracket along with four screws to secure it to the chassis. It also includes the electronic ribbon cable necessary to connect the drive to the motherboard. Both of these items are specific to this laptop chassis and expensive to obtain if they weren’t already included. It’s good to see them present so I don’t have to hunt.

Extra nice touch from Dell: The M3 screws to fasten a 2.5″ drive to the metal bracket is a standard item and easily obtained elsewhere. Given the absence of a 2.5″ drive, I expected I’d have to find standard fasteners on my own. But I don’t need to! The chassis actually has a place to hold these four screws when not in use, and the computer came with these screws, too. This is a feature I’ve only seen before in a premium engineering laptop from their Precision workstation line, never on their consumer Inspiron line.

With these two thoughtful touches, Dell has made me a happy customer. In the near term I’ll install one of my old 2.5″ SSDs for extra storage capacity (and/or Linux dual-boot) and I’ll keep my eyes on DDR4 memory prices for a future memory upgrade.

Battery: Dell 33YDH

Not exactly a note of happiness, but just one bit of trivia I couldn’t find online: the battery module has a designation 33YDH, useful when shopping for a replacement. All batteries have a cycle life – how many times they can be charged and discharged. Some battery types like lithium-ion also have a calendar life. They will degrade over time no matter how much they are (or aren’t) used. So, in a few years, if I like this laptop enough to want to keep using it, I will need to shop for a replacement 33YDH battery.

Dell 33YDH

 

Bart the Robot Spins His Wheels

Bart NMC round 1The initial test session with Bart the robot was a bust. But after reviewing the manual, we realized that a few jumpers needed to be set. These jumpers mark a control board as the end of the daisy-chain, and the end unit is where the NMC protocol’s roll-call procedure started. Bart probably had a few other NMC modules in his prime, and one of them would have been the end because neither of the current-day modules were configured as such. Since neither knew to act as the end unit, there was no module to respond to the start of the roll-call procedure, and NMC communication fails.

Once we marked our current-day motor control module as the end of the chain, the NMC roll-call was successful and our NMC test program was able to communicate with the boards. We were able to obtain readings from the encoder, and see it respond as we rotated the wheels by hand. That got us excited, and the first few motor commands we sent resulted in motor movement which got us even more excited.

Sadly, that initial enthusiasm was dampened by further experimentation. We tried to understand how the motor commands translated to actual motor movement but our experimentation returned very confusing results. The motors would work well on one move, and when we tweaked what we thought was a single small variable, it does something completely and unpredictably different.

Eventually we fell back to sending the same command over and over, and confirmed that the controller does not do the same thing in response. Same command sent repeatedly would result in wildly different speeds each time. Sometimes it even tries to spin the motor in the opposite direction resulting in an error condition and reset.

And every time we experienced a reset, the board would not consistently reconnect to the computer software. We would frequently have to disconnect and power cycle everything before we could resume experimentation.

Our hypothesis is that these control boards were left on Bart – and not salvaged for his successor – because they were malfunctioning. It’s just as likely something on the board degraded in the past 17 years. Either way, an unpredictable motor control board won’t do Bart any good. These closed-loop motor controllers would have been fantastic if we could get them running, so it was worth a bit of time and money to try them out. But now that we have our answer, it’s time to leave them behind.

Trying to Talk To Bart the Robot

Bart NMC round 1The motivation to learn about stepper motors came from Bart the robot. The previous owners had pretty much cleaned out Bart components, but the locomotion subsystem is mostly intact. This consists of two wheel assemblies, each of which is driven by a beefy-looking stepper motor with a quadrature encoder attached to the other end of the shaft. This assembly is attached to a control board of some sort, implying closed-loop control. The board looks superficially intact so the obvious first step is to see if we can make them run.

The control board is made by J.R.Kerr LLC, a company that has since been acquired, but apparently there’s some sort of tech support obligation still in effect. Contacting the new owners confirmed the product is long out of production and no longer on the actively supported product list, but they suggested we look at documentation one of its still-supported relatives “PIC-SERVO 3PH” to get a rough idea. From such documentation we learned the product line communicate to each other via a protocol called “NMC” over RS-485 hardware. The official adapter “SSA-485” costs more than what we’d want to spend just for exploration, but there are vendors on Amazon selling RS-485 adapter for impulse-buy money.

The adapter takes care of the hardware. For the software, a NMC Test Utility was available from the software downloads page. As the only person with a Windows machine, my laptop was roped into test duty.

The first exploration session was a failure – there was no response from the control boards at all. But there were no smoke and no fire, which is good. Probing the board confirmed components were receiving power, but there was no communication and certainly no motor motion.

 

Learning How To Use Pololu Stepper Driver Modules

My first experience with stepper motors is with this very inexpensive Amazon offering. I’ve since learned that these stepper motors are termed “unipolar” which incurs some trade-offs. From the price tag I knew they were cheap, and from the description I knew they were easy to control from a simple program. What I did not know about is the fairly significant headwinds if one wishes to get beyond the basics.

The simple driver module that goes with these simple motors only works for straightforward on/off control. When I tried to modulate the power to be somewhere between on and off, mysterious electromagnetic stuff started happening causing erratic motor behavior. At the time I decided to postpone solving the issue and to look into it later. Well, now is later and I’m going to solve my problem by ignoring unipolar motors entirely. Because it’s more productive to look at the bipolar stepper motors used by pretty much every halfway decent piece of hardware.

The motors themselves are more expensive, and the drivers are as well. Fortunately economies of scale meant “more expensive” is still only a few dollars. Pololu sells a line of stepper motor driver modules that are popular with the 3D printing crowd. (Or at least that’s where I learned of them.) The module’s physical form factor and pinout has become something of a de-facto industry standard. And a bipolar stepper motor for experimentation is equally easy to obtain as pretty much any stepper motor salvaged from consumer electronics will be a bipolar motor. For the purposes of my experiment, this motor came from a dead inkjet printer’s paper-feed mechanism.

Hooking up the electronics is a fairly straightforward exercise in reading data sheet and following instructions. The only thing I neglected was a capacitor across the motor input pins, something pointed out to me when I brought this experimental rig to a local maker meet. Fortunately I had been playing with a small enough motor that the absence of said capacitor didn’t fry everything.

All I needed to do was generate two data signals: direction and step. This is apparently a fairly common interface, even industrial-type stepper motor controllers accept similar inputs, so a Pololu is a great way to start. I created a small program to run on an 8-bit PIC microcontroller to generate these pulses, and the motor is off and running. It was super easy to get started, and this setup is enough for me to play around and build some basic understanding of stepper motor behavior. How they trade torque for speed, and how they respond to higher voltage/amperage. It’s a good foundation for designing future robotics projects.

Pololu ExperimentComponents on the breadboard, from left to right:

  1. Breadboard Power Supply
  2. Pololu A4983 Stepper Driver
  3. PIC16F18345 with program to generate step/direction based on potentiometer value.
  4. LEDs hooked up in parallel with step and direction signals.
  5. Potentiometer

Microchip’s New XC8 Compiler Appears Incompatible With MCC Boilerplate

For the purposes of experimenting with a Pololu stepper motor driver, I wanted to generate pulses from a PIC running a simple program. This meant downloading and installing the latest tools from Microchip for 8-bit PIC development: The MPLAB X IDE, the XC8 compiler, and the MPLAB Code Configurator (MCC).

With these tools set up, I pulled down my previous PIC stepper motor project intending to use it as a starting point. The stepper driver portion will be different, but I wanted the analog dial capability to adjust speed and direction. Before I start modifying code, though, I hit “Build” to make sure everything still compiled.

It did not.

In file included from mcc_generated_files/mcc.c:74:
In file included from mcc_generated_files/mcc.h:52:
mcc_generated_files/interrupt_manager.h:110:6: error: variable has incomplete type 'void'
void interrupt INTERRUPT_InterruptManager(void);
^
mcc_generated_files/interrupt_manager.h:110:15: error: expected ';' after top level declarator
void interrupt INTERRUPT_InterruptManager(void);
^
;
2 errors generated.
(908) exit status = 1

Since “void” is a very fundamental type for C programs, this error tells me this isn’t the matter of a minor typo, something is wrong at a very basic level. As a test, I created a new MPLAB project with a bare skeleton generated by MCC. This “Hello World” program failed to compile in the same way, indicating the problem is a system configuration issue.

A little digging around found the issue was introduced with XC8 compiler version 2.0, introduced just a few months ago in May. It moves C language support up to C99, keeping up with industry standards. However, this change also broke compatibility with old code. Not just old code sitting on a hobbyist’s Github page, but also old boilerplate code generated by MCC.

I expect that Microchip will eventually update these templates in a future MCC update, but for now, PIC programmers that use MCC for rapid prototyping will need to change their project settings to fall back to the older C90 standards. See xc8 2.0 migration document here for more details on this workaround.

Project Properties

A Gentle Introduction To Surface Mount Soldering

In my electronics projects to date, I’ve avoided surface mount devices (SMD) as much as I could. They require custom circuit boards because, given the absence of through-hole legs, they don’t work on prototyping breadboards. They’re small, which makes them difficult to handle without specialized tools. Tools like microscopes to see them, fine-tipped tweezers to handle them, and specialized fine tips soldering irons to solder the tiny connections.

That avoidance came to a crashing end at Layer One, where I had to face SMD head on or be left out of the fun on the Layer One badge add-on kit. The tools were provided at the event, as well as some guidance, so I got over the very beginner parts of the learning curve. It doesn’t make me an expert by any means, that would require more practice.

In the spirit of keeping the momentum going, I decided to check out a beginner-friendly SMD soldering electronics kit. The “I Can Surface Mount Solder” kit was designed by someone who also wanted a gentle introduction to SMD and decided to design a circuit for the purpose. All information is open source so I can make my own. And catering to lazy people like myself, the designer has also put kits up for sale on the maker marketplace Tindie.

There’s a volume discount for buying ten or more, with no increase in shipping, so I decided to buy ten and bring them to share at my local hobbyist meetup. I knew I wasn’t the only one who wanted to practice SMD with something simple. Before the event I had one taker for a kit besides myself. During the meet, a third one was put together by a SGVHAK regular and two more were put together by people who have never attended a SGVHAK meet before. They came because they read the meeting information on Meetup.com and wanted to try SMD soldering. I count this as a publicity win.

The kit itself was far easier to put together than the LayerOne LED add-on kit. The SMD components were about the largest sizes available. So they could be seen by the naked eye and while we still needed tweezers to handle them, we could solder them with regular-sized soldering tips. The only real technical challenge was determining the appropriate orientation of the lone red LED, something that took us a while to figure out. Fortunately we all determined the direction correctly before soldering.

At the end of the night, we had five little pulsing heartbeat pendants and five people who had the satisfaction of a successful SMD soldering project.

I Heart SMD

Charred Liner Needs To Be Replaced in Monoprice Maker Ultimate (Wanhao Duplicator i6)

I’ve had my Monoprice Maker Ultimate for about a year and a half now. It has been the workhorse behind many, many projects in that time. Including some fairly major projects Luggable PC (both Mark I and Mark II) and Sawppy the Rover. The major projects usually demanded around-the-clock printing for weeks on end, and the only real problem it has given me was the 24V relay that died. Twice.

Towards the end of getting Sawppy to version 1.0, I had been printing in PETG on my Maker Select, leaving the Maker Ultimate mostly unused in the home stretch. After I reached a pausing point for Sawppy, I came back to the Ultimate for a few quick prints because it was still loaded with inexpensive PLA…. and the print failed halfway from insufficient extrusion.

I had thought it was a clogged nozzle which wouldn’t be a big deal, but after clearing the nozzle with the 0.4mm drill bit and running the cleaning filament, the problem persisted. Fortunately, I recognize the symptoms from a hard-learned lesson on the Maker Select – the PTFE liner tube is damaged and needed to be replaced.

This particular liner tube wasn’t abused with high temperature like it was on a Maker Select trying to print PETG fast. But internet consensus seems to be that the liner tube is accepted as a wear-and-tear item that eventually requires replacement even under ideal usage. So – probably not indicative of anything wrong here, it’s just time.

Removing the jammed nozzle the printer immediately unveiled a charred tube.

Liner Charred

It took some heat and persuasion to remove the old tube, which stretched in the process of removal. We can see there was quite a bit of cruft welding the tube to the nozzle.

Liner Removed

Interestingly, there are two distinct and separate areas of browning. The print tip was expected. The middle charred section would be right around the length of the heating block and makes sense as one of the hottest sections this tube had to endure. It’s a bit of a surprise that we still have a little white section between them, though.

Anyway, it’s clear this tube has put in a long and productive career guiding filament into the nozzle, but it’s time for a replacement which brought the printer back up and running.

Embedding an Instagram Post with BBCode Without Plugin

Embedding an Instagram post is trivial on a WordPress blog like this one: copy the full Instagram URL (like https://www.instagram.com/p/BfryG0VnUmF/) and paste it into the visual editor window. Behind the scenes, that URL is parsed to create an embed as shown here.

There are similar plugins to add an tag to a BBCode-based web forum. But what if a forum does not have such direct support installed? This was the case for the web forum set up as community driven support for JPL’s Open Source Rover.

On every Instagram post, there’s an “Embed” option that will bring up a chunk of HTML (which links to some JavaScript) to create an embed. However, a BBCode based web forum does not allow embedding arbitrary HTML like that.

Time to read the manual which in this case is Instagram’s developer resources page about embedding. They prefer that people use the fancy methods like that chunk of HTML we can’t use. But way down towards the bottom, they do describe how to use the /media/ endpoint to pull down just an image file with no active components.

Instagram Rover L

This is simple enough to use within the BBCode [IMG] tag. Then we can surround that image tag with a [URL] tag to turn it into a link to the Instagram post.

[URL=https://www.instagram.com/p/BfryG0VnUmF/][IMG]https://instagram.com/p/BfryG0VnUmF/media/?size=m[/IMG][/URL]

It’s not as fancy as the full embed code, but it does get the basic point across and provides an easy way to access the original Instagram post. Good enough for a SGVHAK Rover post on the JPL OSR web forum.

Github Seems To Have Stopped Showing STL Changes

A few years ago Github courted the 3D printing crowd by offering a 3D model viewer to see STL files, and then added a feature to visualize differences between revisions of STL files.

039e6170-1c8b-11e3-8020-b3157840fcf6

One of the reasons I put Sawppy STLs on Github is for people to see parts in their browser without having to install any software. I thought it would also be cool for people to see parts as they evolved. When I first started playing with putting STL files on Github, I thought this was a great way to track changes across major revisions.

Unfortunately, the revision visualization module seems to be gone. The 3D model viewer is still there so the primary motivation for putting STLs on Github is still good. But when I try to view file changes, the changes are no longer shown. The official help documentation still talks about the feature, it just doesn’t seem to work.

I liked seeing STL diffs visually and it makes me sad the feature is now inaccessible.

Road to Sawppy is Paved with Plastic

Today our Sawppy storyline on this blog has caught up to Sawppy version 1.0. The mechanical design for the six-wheel chassis has matured enough that it is a sufficiently functional platform for future projects. We still have mechanical tasks to do ahead of us: the camera mast still needs work, and Sawppy needs a robot arm like the real rovers. But the mechanical work will take a pause, so refinements in electrical design and software can catch up.

As part of declaring version 1.0, the assembly process has been documented on Github in the hopes that other people will build their own Sawppy. I know there’s interest but I don’t know how that interest will translate into action. It would be very rewarding for me to see other rovers running around.

The version 1.0 milestone also marks a time for housecleaning. I had been keeping all iterations of parts I’ve designed and printed on this project in a big bucket of fail. This is occasionally useful when I need to refer back to what I did for comparison to see if I’m actually improving the design. It was also useful to dig up for illustrating various posts on this blog as I tell Sawppy’s story. Now that I’ve completed documentation on Github and told the story of Sawppy evolution on this blog, it’s time to discard them.

But before we do that, a big group picture of all the retired parts with Sawppy the Rover version 1.0.

Sawppy with Parts

Sawppy the Rover Receives WiFi Upgrade, Increases Range

Sawppy is now back up and running with all its 3D printed parts recreated in MatterHackers PETG plastic. While the pieces could be replaced piecemeal, I decided to take everything apart and reassemble the whole thing so I could take pictures along the way and document the assembly process. Since I was basically rebuilding the rover from scratch anyway, I performed another upgrade: the compact Netgear WiFi router previously installed has been replaced with larger dual-band unit, the Asus RT-AC1200. Test drives have proven the new router gives Sawppy significantly more range!

Sawppy can now rove a decent distance from its handler. Far enough that it’s no longer easy to see exactly which way the rover is pointed, and we need to occasionally refer to Sawppy’s on-board camera video feed to see what’s going on. In the picture below, John is holding his phone showing the video feed while Emily is on the driving controls. This picture was taken as Emily drove Sawppy back towards us. In the relatively quiet RF environment of this industrial park, Sawppy can drive about three times further away than the distance shown in the picture before wireless communication suffers occasional data dropouts.

This range test proved that Sawppy can get far enough away that driving it around becomes a team activity: one to monitor the situation and another on the controls. This is more than enough range for most of our purposes.

What’s still unknown is Sawppy’s tolerance of noisy RF environments. That test will come, eventually…

Sawppy Pilots

Reclaiming Bearings From 3D Printed Parts: Round 2

Now that PETG is up and running smoothly on my printer, it was put to work reprinting all Sawppy parts so I could replace the existing PLA parts with PETG parts. Retiring parts means throwing away things like heat-set inserts, but the bearings could be recovered. I originally designed a few holes into the parts to make bearing removal easier, but I’ve since realized those holes are unnecessary. Plastic — both PLA and PETG — are soft enough for the bearings to flex and move inside their assigned positions. This flex allows us to work bearings loose by twisting a shaft inside the bearing.

Bearing Extraction

I tried describing this with words but my vocabulary really isn’t up to the task. Here’s a video to illustrate the technique.

This works well enough I think those bearing removal assist holes I added before are now unnecessary. I’ll probably remove the holes on my next revision of rover parts so it looks less like a piece of Swiss cheese.

Titan Aero Upgrade for Monoprice Maker Select (Wanhao Duplicator i3)

In order to improve PETG printing performance, my open-box Monoprice Maker Select is receiving a hardware upgrade. The print head assembly (filament extruder and hot end) is being replaced with an E3D Titan Aero, a combination all-metal hot end and geared extruder.

For this first pass, the goal is to be as simple and nondestructive as practical so I could revert if things don’t work out. If this works, I can make things nicer later. Obviously, the first step is to remove my existing print head, leaving just the metal X-axis carriage assembly. Since I’m trying to be nondestructive, the goal is to fit into this space in the U-shaped metal and bolt onto existing holes.

Stock extruder hot end removed

To test for fit, I laid out parts for assembly. Some people are squeamish about using the print surface as a work surface, preferring to leave it as pristine as possible. I have no such qualms.

Titan Aero parts laid out

A few quick tests confirmed there is indeed space within the U-shaped metal to accommodate a Titan Aero. The hole for the actual nozzle doesn’t line up, though, which means the Titan Aero nozzle will have to dangle off to the side of the metal bracket. This wish for non-destructiveness will extract a price in the form of a small reduction in print volume. I decided the tradeoff is worthwhile for now. I designed & printed a simple adapter to mount the whole works on the existing metal bracket. The Titan Aero kit does not include a stepper motor, so I reused the existing extruder motor.

When I was just eyeballing the parts, I thought I could use the existing heater cartridge and thermister. The advantage of this approach is reduced wiring work and we wouldn’t have to change print controller configuration. Sadly, the heater cartridge is a tiny fraction of a millimeter too large to fit and thermister is an entirely different shape. So some wiring tasks and controller configuration changes had to be made. Since the long-term plan is to build a better chassis using these parts, I kept most of the wires for the heater and thermister with the hope the wires will be better routed in that future dream chassis. In the short-term, the wires are just coiled on top of the print assembly.

The final modification was to the cooling fan — when it was powered up for the first time, I heard how loud it was and said “Oh hell no.” I replaced it with a 40mm Noctua fan which doesn’t move as much air but is far quieter. If the reduced air volume causes heat creep issues I’ll revisit this fan replacement, but for now I’m grateful for the silence.

Once the upgrade was hacked together, the printer can now easily extrudes PETG at decent print speed with 0.3mm layer height. I was initially worried about the adapter bracket holding up under the heat (it was printed in PLA) but the Noctua cooling fan seemed to be doing its job and things never get hot enough for the bracket to be a problem so I’m happy to leave well enough alone. I’ve got a rover to reprint in PETG.

Titan Aero nondestructive install

Hot End Upgrade Options for Monoprice Maker Select (Wanhao Duplicator i3)

The Sawppy rover project has reached a point where I need PETG for more heat-tolerant rover parts, and the stock hardware on a Maker Select isn’t good enough to deliver the prints I needed at the speed I wanted. The working hypothesis is that the stock hot end couldn’t melt PETG at high enough volume to print 0.3mm layer height at a decent speed. Technically Monoprice did not lie when they said the printer could print PETG. It just couldn’t do so at an acceptable pace for my project.

The recommended solution for melting PETG faster is to go to an all-metal hot end. Searching internet forums found two leading candidates. The first is from Micro-Swiss, which offers a drop-in replacement kit to turn the default hot end to an all-metal hot end.

The second leading contender is from E3D, which sells the Titan Aero. It’s an all-metal hot end with an integrated extruder, unlike the Micro-Swiss kit which replaces a few key heating components in the stock hardware leaving most of it intact. The Titan Aero option costs more than twice as much as Micro-Swiss upgrade kit and requires more work to install.

If I was happy with the stock extruder on this printer, the Micro-Swiss option would have been the one I chose. But I was not happy with the stock extruder! It’s been a cause of headaches since day one with inconsistent extrusion caused by slipping filament and who knows what else. Upgrading the stock electronics to a Panucatt Azteeg X5 Mini solved a few other problems with the side effect of making extruder issues much more apparent.

Maker Select Underextrusion

There are various hacks to work around problems with the stock extruder, but now that I’m presented with an option to upgrade the extruder at the same time as the all-metal hot end upgrade that I want, it’s easy to take that step up to a Titan Aero.

Problems Printing PETG With Monoprice Maker Select (Wanhao Duplicator i3)

The first experiment in PETG was printing a servo coupler. It was small, printed at 0.1mm layer height. After the success of that initial experiment, I set the printer to work on a Sawppy rover wheel overnight at 0.3mm layer height. It did not turn out well.

First Complex PETG

Little bits of extraneous PETG strings all over the place! Stringing is usually credited to poor retraction settings, but that’s not the whole story here. Once this print gets above the first 20mm and no longer printing the center hub, it no longer performs any retracts – the wheel is printed in a continuous motion without retracts.

What these strings actually demonstrate is not poor retraction, but very poor layer adhesion. As the print head circles the perimeter laying down filament, it’s not all sticking and instead dragging along little bits of PETG causing these strings. It’s not very visible from this camera angle, but there are visible gaps between layers. And the layers came apart with only minor physical handling.

The layers came apart more easily in the middle sections. This was puzzling – what problem would be worse in the middle of the night but magically recover by morning? The answer: ambient air temperature. Apparently PETG needs more time than PLA to properly bond with the previous layer, and when cooled too quickly it won’t bond. I can’t change the weather on command, but I could turn off the print cooling fan.

Turning off print cooling helped somewhat, but it was not the whole solution. PETG melts less easily than PLA, which is a desired feature when it comes to rover parts that don’t deform under heat. But that attribute also creates printing headaches. The 0.1mm layer height print bonded well but the 0.3mm print did not, leading to the hypothesis that the print nozzle couldn’t melt PETG fast enough to deliver triple the volume of plastic.

To test this hypothesis, the print speed was cut to 1/3 of previous speed. The test object worked well, but this print speed is not acceptable. It would turn a rover wheel from an 8-hour print project to an all-day 24 hour print!

Another test is to turn up the heat on the nozzle, hopefully a hotter nozzle will melt PETG more quickly. This worked… briefly. It got too hot for the liner and it deformed, jamming the print path.

Damaged PTFE liner

The liner was original so perhaps it was just time for a replacement anyway. But when the replacement liner also jammed up within a few prints, I knew this was not going to work.

Given these data points, the hypothesis of “hot end couldn’t melt PETG fast enough” has merit. We know slowing down works, but is unacceptably slow. We know heating up works, until the liner quits.

I was not willing to accept the slowdown, so the alternative is to upgrade the hardware.

First Simple PETG Print is a Success

Once the Maker Select reached “good enough” status it was back to work printing PLA parts for Sawppy in parallel with my Maker Ultimate. This allowed me to iterate through designs much more quickly and was instrumental in getting Sawppy built in time for its first public appearance at JPL’s IT Expo.

A few problems surfaced at this event, but the one that prompted a complete reprint of Sawppy was PLA deformation under Southern California summer heat. This is where the current 3D printer story line rejoins the rover construction story line. With this experience of plastic deformation, I now have motivation to try using a different material. There are a few options, and PETG presented the best tradeoff between temperature tolerance, ease of printing, and cost.

The first object to be reprinted in PETG were the steering servo couplers. This proved to be a weak point that needed to be addressed. The design was printed at 0.1mm layer height so the sideways hole for the M3 thread heat-set insert would have clear definition. (This turned out to be unnecessary – later couplers were printed at 0.3mm layer height and functioned adequately.)

I knew PETG had different requirement for printing, starting with print nozzle temperature. I started with my PLA print profile and dialed up the heat. In order to test layer bond in the print, and also to get a feel of PETG failure mode, I put the result in a vise and cranked the handle. I was happy to see PETG deformed rather than shattered as PLA would. Examination of the deformed object showed layer bonding is good. This is a good start for printing PETG.

PETG crush test

Bolt Test Print on Monoprice Maker Select (Wanhao Duplicator i3)

After upgrading the control electronics of my Monoprice Maker Select to an Azteeg X5 Mini (which is a major change) there were a handful of issues to chase down. Some documented recently on this blog, others too minor to be worth writing about. Once the biggest problems were resolved, the printer was in a decently usable state. Not perfect, but acceptable. Or so I thought… time for a test.

The test print object is a bolt with its corresponding nut. There’s no practical reason to 3D print my own fasteners – buying them would be cheaper, faster, and stronger. The purpose of this exercise is to test dimensional accuracy. While we could print a calibration cube and measure its dimensions, it’s not as satisfying as fitting one precision part into another. A successful test would allow threading the printed nut onto the printed bolt. Also, we’d end up with a simple little fidget toy.

A good reference for dimensional accuracy is this page in the Slic3r manual. Most of the information on this page areapplicable to 3D printing in general and not exclusive to Slic3r users.

The 3D data for test print was pulled off McMaster-Carr’s web site which has CAD data for much of its merchandise. Here is the bolt, and here is the matching nut. Several iterations were printed to fine-tune settings. In this picture, the bolt on the right was printed at 0.3mm layer height. This proved too coarse to properly recreate the thread and did not work. The bolt on the left is printed at 0.1mm layer height, which was able to recreate the thread profile with enough accuracy. But that by itself was not enough – it also needed an XY compensation parameter of -0.2mm before the nut will smoothly install on the bolt, shown on the left side of this picture.

Requiring a dimensional adjustment of 0.2mm is not great, as that is half the width of our 0.4mm print nozzle. In theory we should be able to do better, but for now this is good enough to resume printing Sawppy parts.

Bolt Test

Rovers Gonna Rove at SGVLUG/SGVHAK BBQ

The annual summer BBQ for San Gabriel Valley Linux User’s Group (SGVLUG + offshoot subgroups like SGVHAK) was this past weekend, so obviously a few projects from the group made a showing too. This included our SGVHAK rover and my own follow-up Sawppy the Rover.

Sawppy joined the socializing first, hanging out next to some of the seats attracting attention from members who haven’t seen the motorized rover model before. (And some who have but just wanted another look.)

SGVLUG BBQ 1

Later in the evening, SGVHAK rover joined on the party. The rovers would occasionally run around sometimes at the same time and sometimes into each other. They sort of got into the way of people at times, getting underfoot like a big cat might. Fortunately nobody tripped and fell over a rover.

SGVLUG BBQ 2

This was Sawppy’s first public outing since conversion to PETG, a conversion motivated by earlier PLA parts that deformed under heat. This weekend turned out to be a great test since the entire region is in the middle of a record-breaking heat wave. And this time, Sawppy did not melt under the scorching sun.

Sawppy also got a few different pilots, as anyone who expressed interest was offered a chance to take the helm. Partially to share the toy, but also to get some time seeing Sawppy driven by people who don’t drive the same way I do. The intent was to probe for unknown areas of weakness – after all, this type of activity is how we exposed the weakness in SGVHAK rover’s original steering mechanism.

Fortunately no such flaws were found today. Sawppy survived the evening driving by multiple people (and running over things, and running into things) without any apparent damage. This is a very good endorsement of Sawppy’s design maturity.