Neato Robot ROS Package Expects Specific Response But Responses Actually Differ Between Neato Units

Neato mini USB cable connection to laptop

I got the outdated neato_robot ROS package mostly working just by adding a timeout to its serial communications. But this only masked the symptom of an unknown problem with no understanding of why it failed. To understand what happened, I removed the timeout and add the standard Python debugging library to see where it had hung.

import pdb; pdb.set_trace()

I found the hang was getMotors() in neato_driver.py. It is waiting for my Neato to return all the motor parameters specified in the list xv11_motor_info. This list appears to reflect data returned by author’s Neato robot vacuum, but my Neato returns a much shorter list with only a partial overlap. Hence getMotors() waits forever for data that will never come. This is a downside of writing ROS code without full information from hardware maker: We could write code that works on our own Neato, but we would have no idea how responses differ across different robot vacuums, or how to write code to accommodate those variations.

Turning attention back to this code,  self.state[] is supposed to be filled with responses to the kind of data listed in xv11_motor_info.  Once I added a timeout, though, getMotors() breaks out of its for loop with incomplete data in self.state[].  How would this missing information manifest? What behavior does it change for the robot?

Answer: it doesn’t affect behavior at all. At the end of getMotors()we see that it only really cared about two parameters: LeftWheel_PositionInMM and RightWheel_PositionInMM. Remainder parameters are actually ignored. Happily, the partial overlap between author’s Neato and my Neato does include these two critical parameters, and that’s why I was able to obtain /odom data running on my Neato after adding a timeout. (Side note: I have only looked to see there is data – I have not yet checked to see if /odom data reflects actual robot odometry.)

Next I need to see if there are other similar problems in this code. I changed xv11_motor_info list of parameters to match those returned by my Neato. Now getMotors() will work as originally intended and cycle through all the parameters returned by my Neato (even though it only needs two of them.) If this change to neato_robot package still hangs without a timeout, I know there are similar problems hiding elsewhere in this package. If my modification allow it to run without a timeout, I’ll know there aren’t any others I need to go hunt for.

Experiment result: success! There are no other hangs requiring a timeout to break out of their loop. This was encouraging, so I removed import pdb.

Unfortunately, that removal caused the code to hang again. Unlike the now-understood problem, adding a timeout does not restore functionality. Removal of debugger package isn’t supposed to affect behavior, but when it does, it usually implies a threading or related timing issue in the code. This one will be annoying as the hang only manifests without Python’s debugging library, which meant I’d have to track it down without debugger support.

Rover Mr. Blue Now Up And Running On SGVHAK Rover Code

Mr. Blue is alive

While I’m making my way learning how to write proper modules for Robot Operating System, the rest of my SGVHAK compatriots have not been twiddling their thumbs waiting. Just like I went and built Sawppy as my idea of a cool rover project, Dave designed and built Mr. Blue as his idea of a cool rover.

All the blue printed plastic parts (which gave this rover its name) was designed in Dave’s preferred CAD software OpenSCAD. These pieces connect thin wall aluminum tubing that is very strong yet quite affordable. Making this rover a great exploration into a construction method quite different from Sawppy, where I designed 3D-printed plastic components in Onshape CAD and they connected Misumi aluminum extrusion beams.

Mr. Blue also expresses Dave’s ability to design and build electronics circuits. Where my Sawppy used serial bus servos, Dave has custom controller boards driving all the motors. There are two types on board: one drives a DC motor with feedback provided by optical quadrature encoders, the other drives a commodity servo motor but with precise position feedback via high-resolution magnetic encoders. In both cases, they feed into a 3D-printed gearbox for additional mechanical reduction.

Originally, the plan was to get Mr. Blue up and running on ROS, but Dave’s progress at construction was faster than my progress at learning ROS. As an interim solution, I added support for Dave’s motor control boards to my SGVHAK Rover software project. Now we have the same basic software running three rovers built with three different motor systems:

We have quite a rover family going!

Neato Robot ROS Package Runs After Adding Serial Communication Timeout

Neato mini USB cable connection to laptop

The great thing about ROS is that it is a popular and open platform for everyone to work with, resulting in a large ecosystem of modules that cover a majority of robot hardware. But being so open to everyone does have a few downsides, because not everyone has the same resources to develop and maintain their ROS modules. This means some modules fall out of date faster than others, or works with its author’s version of hardware but not another version.

Such turned out to be the case for the existing neato_robot ROS package, which was created years ago and while the author kept updated it through four ROS releases, that maintenance effort stopped and so this code is now five releases behind latest ROS. That in itself is not necessarily a problem – Open Source Robotics Foundation tries to keep ROS backwards compatible as much as they can – but it does mean potential for hidden gotchas.

When I followed instructions for installation and running, the first error message was about configuring the serial port which was fine. But after that… nothing. No information published to /odom or /base_scan, and no response to command sent to /cmd_vel.

Digging into the source code, the first thing to catch my attention was that the code opened a serial port for communication but did not set provision for timeout. Any problem in serial communication will cause it to hang, which is exactly what it is doing now. Adding a timeout is a quick and easy way to test if this is actually related.

Existing:

self.port = serial.Serial(port,115200)

Modified:

self.port = serial.Serial(port,115200,timeout=0.01)

And that was indeed helpful, restoring nominal functionality to the ROS node. I could now drive the robot vacuum by sending commands to /cmd_vel and I can see a stream of data published to /odom and /base_scan.

I briefly contemplated being lazy and stopping there, but I thought I should try to better understand the failure. Onward to debugging!

Existing Neato Robot ROS Package Will Need Some Updating

Neato mini USB cable connection to laptopNow that we’ve got a pretty good handle on getting old Neato robot vacuums up and running, including talking to their brains via USB, it’s time to dig into the software side and explore its potential. I could adapt my SGVHAK rover code to control a Neato, but I think it’s more interesting to get my Neato up and running on ROS.

Unsurprisingly, there already exists a neato_robot ROS package for Neato. Equally unsurprisingly, it was maintained up to ROS Hydro and since abandoned. ROS releases are named alphabetically, meaning this code is three releases behind my installation of ROS Kinetic and a whopping five releases behind the current state of the art ROS Melodic.

But hey, it’s all open source, if I want to get it to function all I need to do is roll up my sleeves and get to work. Since it’s out of date, I think it’d be a bad idea to grab the pre built package. Instead I followed links to its source code repository (last updated October 2014) and cloned it into my local catkin workspace.

And continuing on the trend of not being surprised, this code did not work immediately. Publishing events to /cmd_vel topic did not elicit robot chassis movement, and subscribing to /odom or /base_scan received no data. Fortunately this node was written in Python, which I hope would be easier to debug than a ROS node written in C++. But either way, it’s time to dig in and figure out what it is doing and how that differs from what it is supposed to be doing.

Sawppy Documentation Updates

It’s been a few months since I declared Sawppy version 1.0 complete and documented. A project like this is never done and there’s always more to do, but I had reached a good milestone in mechanical design supported by “good enough” electronics driven by software borrowed from another rover project. I released it open source to the world, curious what would happen next.

backyard sawppy 1600

I was very happy to see people go beyond just talking about building their own Sawppy, and have gone ahead and actually done so. In the course of their builds, people have reached out to me with questions about the information I’ve published, which I welcome. It’s always good to have extra sets of eyes to look over what I’ve written. Sawppy documentation is my best effort, but it’s very easy for me to overlook gaps in information because it’s all in my head and my brain doesn’t necessarily notice when something crucial is missing. Having other people try it out and letting me know of their experience is the best real world test, and I add clarifications in response.

And this past week the Sawppy project documentation crossed another threshold: its first external contribution pull request! I really appreciate the work of [TechMav75]. Not just point out gaps in my documentation, but actually rolling up sleeves and helping to patch those gaps based on their experience building a Sawppy. Thank you!

As for software, SGVHAK Rover’s adapted software was always intended as a placeholder, not the long term software solution for Sawppy. Indeed the first three Sawppy builds I’m aware of didn’t use that software, either. Two builders have written their own rover software, and a third has wired up their Sawppy for standard remote control. Nevertheless, some Sawppy builders want to use what works today so I should put in the time to write more detailed instructions on how to adapt that code to drive Sawppy. In the immediate present, user feedback has alerted me to an embarrassing bug that I’ve since fixed. It’s not my first public embarrassment, it won’t be the last.

Thank you everyone who has shown interest in my project. Constructive feedback and questions are always welcome, use whichever venue you are most comfortable with:

  • I encourage asking questions in public so my public answer will be useful to others as well. Go to Sawppy’s Hackaday.io page, click on “Discussion“, and add your comment.
  • If you rather not comment in public, you can send me a private message. From Sawppy’s Hackaday.io page, click on my name up top to reach my Hackaday.io profile, and click “Send a private message”.
  • File an issue on Sawppy’s Github repository.
  • And finally, my favorite: not just file an issue, but also send me your proposed solution via a pull request! Let’s work together to make Sawppy better.

(Cross-posted to Hackaday.io)

Trying To Charge Neato XV-21 Without a Charging Dock

When I found my Neato XV-21 in a thrift store, it wouldn’t power on and it didn’t have its charging dock. I originally thought I would buy it just for parts but it’s been a long and interesting adventure poking at how these robot vacuums worked. I made enough progress to catch [Emily]’s interest, who saw another Neato robot vacuum in a thrift store and bought it. Hers had the advantage of a charging dock so we could open one up to see how it worked.

Armed with that knowledge, I thought I’d try to build my own replacement charging dock for my Neato XV-21. I bought two 24V DC power supplies similar to that inside a Neato charging dock at the W6TRW swap meet. Since I didn’t have a good way to make contact with a Neato’s charging strips, I opened up my robot vacuum and rigged up a test configuration using clip leads on my crowded workbench.

Neato charging experiments on workbench

The first candidate was a power supply with HP branding.

Neato charging experiments HP supplyThe HP power supply label said 24V @ 1.5A, which is slightly lower than 24V @ 1.67A listed on the supply inside a Neato dock, but I thought it was worth a try. First I verified the open circuit voltage was 24V, then I clipped it to the vacuum. The Neato woke up as if it saw a power source, but two seconds later it acted as if the power source went away. The output voltage on the power supply had dropped to under 1V and remained there. I thought maybe I burned out something, but if I unplug the power supply and plug it back in, its output open circuit voltage was back up to 24V. This behavior indicated no permanent damage was done, but that I had tripped some sort of internal protection mechanism.

I then did what, in hindsight, I should have done as my first step: tried charging my vacuum with a bench power supply. I set the dials to 24V and 1.67A, like the label on Neato charging dock’s power supply, and hooked up the wires. My bench power supply immediately hit the current limit I had set, and voltage has sagged to a little over 14V. Since that’s within the range of NiMH battery voltage, it appears the Neato had connected its DC input directly into its batteries.

NiMH batteries are fairly durable so this isn’t necessarily a horrible thing to do in and of itself. But it does mean the Neato will happily soak up more than the rated power of its power supply. Which explains the HP power supply behavior: it is an over-current protection mechanism that is great for safety in case of short circuits but inconvenient for trying to charge a Neato.

Neato charging experiments VeriFone supplyThe second salvaged power supply had VeriFone branding and claimed it is capable of 24V @ 1.7A. This is better than the unit inside a Neato, but now we know this is only part of the story. The real question is how it behaves when connected to something that draws more than its rated 1.7A. Does this also have an over-current protection mechanism?

The answer is yes. Again its open circuit voltage is 24V as advertised, but once wired to a Neato, the voltage collapses to around 12. After collapse, it quickly builds up to about 15, then the voltage collapses again. The Neato is very optimistic, showing itself to be charging, but there is negligible actual progress in charging the battery.

Neato charging experiments Dell supplyBut now that I know the current capacity is the limiting concern, maybe another power supply with higher amperage limits will work even if its maximum voltage is lower than 24 Volts. So I pulled out an old salvaged Dell power supply that only claims to deliver 19.5 V but could do so at 3.34 A. I verified its open circuit voltage was over 19 V, connected it to a Neato, which again woke up thinking there’s a charger. But once power started flowing, output voltage collapsed to 7.5V and remained there until a power cycle like the HP power supply did.

Out of three DC power adapters that should have worked on paper, zero actually worked for charging my Neato vacuum because the Neato drew far more than their rated amperage. So for the immediate future, my Neato charging will be done with my bench power supply. To make this easier, I made an easy (if not particularly pretty) modification to my Neato vacuum. Two holes were drilled on each side of a charging contact. Something I now have the confidence to do, because I’ve taken apart the charging contacts and saw there were no critical parts at risk.

Neato charging experiments drilled holes around contacts

With these holes drilled, it was easy to connect my bench power supply’s alligator clip leads for charging.

Neato charging experiments drilled holes around contacts for alligator clips

Sawppy Field Repair: Servo Coupler

Every time I encounter problems at a public Sawppy appearance, it is a lesson. It might be a lesson in how I might be able to improve Sawppy’s design, or it might be a lesson in how to better prepare a kit for field repairs a.k.a. Sawppy First Aid Kit. When a fuse burnt out at Caltech Science for March 2019, that was easy to address: add replacement fuses to the kit.

The broken servo coupler encountered at SCaLE 17x was a little tougher. It wasn’t enough to keep an extra 3D-printed coupler handy, because a little block of plastic isn’t very useful by itself. It requires a heat set insert to accommodate a set screw holding against an 8mm shaft, and it requires four more screws to fasten against the servo horn. Fiddling with a lot of tiny screws would not be very practical when trying to make field repairs away from my workbench. This is especially true when I have a literal ‘field’ to deal with: on grassy grounds where a lost screw is very difficult to recover.

Sawppy servo coupler FRU

The answer to minimizing chance of lost screw is to prepare a replacement assembly consisting of:

  • 3D-printed coupler
  • Heat-set insert already installed.
  • Set screw already installed inside the heat-set insert.
  • Servo horn that comes with a serial bus servo.
  • 4 small screws already fastening the coupler to the servo horn.

With such an assembly ready to go, the only fastener I risk losing during replacement is the screw holding the servo horn to the servo output spline.

The first downside of this approach is that I only have as many servo horns as serial bus servos. Keeping a replacement unit ready in Sawppy’s field repair kit means I have an extra serial bus servo with no horn to use it. Perhaps I could purchase extra servo horns, or maybe this is just a hint I should also have an extra replacement servo in the bag.

The second downside of this approach is that, in case of steering couplers, a replacement would not be identical. Steering trim would be slightly different with a replacement coupler due to natural variation from unit to unit. When I perform this replacement, I would also have to adjust Sawppy’s steering trim for proper operation. But in an emergency? It’ll probably be close enough.

(Cross-posted to Hackaday.io)

Trying To Make Two Good Neato XV Battery Packs From Four Bad Packs

Neato XV battery pack untouched

A Neato robot vacuum in their initial XV series product line is powered by a pair of identical 6-cell NiMH battery packs. When I picked up my XV-21 from a local thrift store it did not power up, a fault which I’ve isolated to its failed battery packs which I’ve since replaced to get the whole system back up and running with help of [Emily]’s loan of Neato charging dock. When I evaluated my battery replacement options earlier, one was to buy new cells and rebuild the pair of packs myself. I rejected that option because new cells actually would have cost more than pre-built replacement packs.

But since then Emily found a second thrift store Neato, a XV-12 with its own failing battery pack. This makes a total of four identical 6-cell NiMH battery packs. What are the chances we have sufficient good-enough NiMH cells in this pile for one set of healthy batteries? It costs nothing but a bit of time, well within the spirit of the kind of projects we tackle at SGVHAK meetups, and so it’s worth a shot.

First, the XV-12 battery packs were trickle charged overnight to get a sense of their capabilities, just like I did for XV-21 batteries earlier. Fortunately, the self-discharge profile looked promising.

Pack A: 7.93V self discharged to 7.64V after a few days.

Pack B: 6.59V self discharged to 5.98V after a few days.

Judging on voltage level alone, pack A is in better shape than pack B. The latter shows signs of having one completely dead cell. They’re certainly in far better shape than XV-21 battery pack. Out of 12 cells, only 1 held itself at ~1.3V after a week. The rest all self-discharged to a level ranging from 0.9V to flat zero after a few days.

So we disassembled pack B and deployed a volt meter to verify there is one cell that could only deliver around 0.1V. This cell was marked with an X and removed from the pack. Since we don’t have a battery spot welder available, we took care to make sure we keep the tabs on this pack.

The only not-dead cell from the XV-21 pack was marked with a check and removed from its pack. And again we took care to keep the battery tabs, this time making sure it stays with the ‘good’ cell.

Neato XV battery pack cell replacement

With the battery tabs intact, it was easy to solder a new pack together.

Neato XV battery pack soldering

A dab of hot glue helps the cobbled-together pack stay intact for installation into vacuum

Neato XV battery pack installed

When we turned on the robot vacuum, it no longer displayed a battery issue error screen, which is a great sign. We then left the robot sitting on its charger for about half an hour, then pressed the big red button to start a vacuum cycle. The vacuum suction motor turned on (Yay!) the brush motor turned on (Yay!) the robot started to move (Yay!) and then it went dark. (Noooo!) When we tried turning it back on, the error screen returned.

Neato XV battery pack still unhappy

Our little cell-swapping experiment did not result in a battery pack capable of running a Neato. It might find a future life powering low drain electronics projects, but it wasn’t enough to run a robot vacuum’s high drain motors. Emily ended up buying new battery packs as well to restore her XV-12 back to running condition.

Neato Robot Vacuum Is Certainly No Substitute For Manual Vacuuming

So now that we have the entire Neato robot vacuum system including charging dock, it’s time to see how did they perform at the job they were built to do. Not that their performance was important, as I bought my Neato intending to scavenge it for parts and it was only a happy accident to end up with a fully functioning robot vacuum. But before I start having fun with robot experiments, I should at least see how it works in its original capacity. Mostly just from curiosity, but if I’m going to show up at places with a modified robot vacuum, I also expect to be asked by people if it’s worthwhile to buy one to vacuum their home. Those who paid full price for their robot vacuums will have very different expectations from people like Emily and I who picked up our robot vacuums for cheap at a thrift store.

The common wisdom with Roomba robot vacuums is that they still miss a lot of surface area due to their random walk nature. A Neato, with its laser distance scanner, is supposed to provide full ground coverage. In reality, all open spaces are indeed very efficiently covered. However, the laser scanner meant a Neato is less effective at cleaning edges and corners. Because they’re smart enough to not run into edges and corners, a Neato’s cleaning brush never get close to those areas. We can see this most clearly in this picture of a dusty doorway after a Neato vacuuming pass, showing the smooth path it took swerving around a door frame instead of digging in and cleaning out those corners. (Note: my house is not quite as disgusting as the photo implies: the contrast has been exaggerated via Photoshop.)

Neato swerved around door frame high contrast

And naturally a little robot vacuum would not be able to move furniture out of the way. For example, floor under the dining table is not vacuumed, because the dining chairs legs all around the table blocked access. But it turned out the dining table itself was an obstacle. Just as my earlier experiment with Neato scanner had problem seeing certain furniture features, an actual Neato was unable to see the ramp-like shape of my dining table legs and managed to launch itself into an awkward angle and got stuck.

Neato stuck on dining table.jpg

As a home of a tinkerer, my house had many other features unfriendly to robot vacuums. My Neato kept getting itself tangled up in power cords for various AC-powered electronics gadgets, and of course there are piles of stuff all around the house in odd shapes, some of which share the “important features are out of sight of laser scanner” problem with office chair and dining table. There may be homes where a Neato would be a productive little housekeeping worker, but I’m afraid my home is just too much of a hazardous environment for this little Neato to be effective.

Which is great! I now feel less guilty about relieving it of vacuum duty and put it back to work for the reason I bought it: as a chassis for robotics projects. But it was fun to see a Neato in action doing its job. It was enlightening to see its own mapping and routing software at work, a benchmark to compare against for my own code driving this chassis. It is a really endearing little robot, with friendly messages on its LCD screen and my favorite part: the way it cuddles its charging dock. And now that one Neato is back up to full running condition, [Emily] and I will team up and try to get them both running.

Examining Neato XV-12 Charging Dock

When [Emily] found her Neato vacuum in a thrift store, it had an advantage over mine in that hers still have the company of its charging dock. This is our first look at a Neato robot vacuum charging dock and a chance to determine how one worked. We wanted to have some idea of what to expect when we put it to work charging newly installed replacement batteries.

Neato charging dock front

The charging dock is designed to sit against a wall. The two metal strips are obviously for supplying power, as they line up with the two metal wires at the back of a Neato vacuum. When the dock is plugged in, a volt meter reports 24V DC between those two plates, top plate positive and bottom plate ground. Each of the plate is mounted on a piece of spring-loaded plastic that allows approximately 3-5mm of horizontal movement. A Neato vacuum can press its wires against these plates to draw power.

Above the plates is a black plastic window, we expect something behind that window to communicate with the Neato so a hungry robot vacuum knows where to go to feed itself. How does it work? We hypothesized there are infrared emitters and receivers behind that panel, functioning like a consumer electronics remote control, to talk to a Neato vacuum.

Neato charging dock back.jpg

Neato charging power adapterThe orange tab on top looked very inviting as a way to open the dock. A bit of fiddling later, the dock was open. It was surprisingly simple inside. There was an AC power supply delivering 24V DC. It has a standard power cable on the input side, which can be routed to exit either side of the dock. This way a user can swap as needed to point towards the nearest power outlet, and possibly swap for a longer standard power cable if necessary to reach an outlet. The output wires of the power supply lead to the two metal plates, and that’s it.

Surprisingly, there’s nothing visible behind the black plastic window. The IR emitters and receivers we expected were absent, as were any circuit boards with components to communicate with the vacuum. So this charger dock location beacon must work passively. Now we’re really interested in finding out more. How does it work?

The black plastic window were held in place with a few clips. They stood between us and knowledge and were quickly dispatched. We were afraid the black plastic might be glued in place, but fortunately that was not the case and it popped off for us to see underneath.

Neato charging dock mystery panel demystified

We see a pattern laid out with two types of surfaces. The white segments are highly reflective much like the stripes on high visibility orange safety vests. The black segments are presumed to provide a contrast against the white parts. We found out earlier that a Neato lidar data stream returns both distance and intensity of reflections it saw. The distance is useful for navigation, but using just distance information the charger would be an unremarkable flat surface. This is where intensity comes into the picture: these surfaces behind the black plastic window will create a distinct pattern in reflection intensity, something a Neato robot vacuum can seek to find its charging dock.

Disassembling this passive system tells us two things:

  1. The engineers are Neato are quite clever
  2. We now know enough to try creating our own charging docks. Userful when we have Neato vacuums found at thrift stores without their charger.

Before we tackle new projects, though, let’s see how a full Neato system works in practice.

BeagleBone Blue And Robot Control Library Drives eduMIP

My motivation to learn about the BeagleBone Blue came from my rover Sawppy driving by the BeagleBoard foundation booth at SCaLE 17x. While this board might not the best fit for a six wheel drive four wheel steering rocker bogie mars rover model, it has a great deal of potential for other projects.

But what motivated the BeagleBone Blue? When brainstorming about what I could do with something cool, it’s always instructive to learn a little bit about where it came from. A little research usually pays off because the better my idea aligns with its original intent, the better my chances are of a successful project.

I found that I could thank the Coordinated Robotics Lab at University of California, San Diego for this creation. As teaching tool for one of the courses at UCSD, they created the Robotics Cape add-on for a BeagleBone Black. It is filled with goodies useful for robot projects they could cover in class. More importantly, with quadrature input to go along with DC motor output and a 9-axis IMU on top of other sensors, this board is designed for robots that react to their environment. Not just simple automata that flail their limbs.

The signature robot chassis for this brain is the eduMIP. MIP stands for Mobile Inverted Pendulum and the “edu” prefix makes it clear it’s about teaching the principles behind such systems and invite exploration and experimentation. Not just a little self-balancing Segway-like toy, but one where we can dig into and modify its internals. I like where they are coming from.

eduMIP 1600
Photo of eduMIP by Renaissance Robotics.

BeagleBone Blue, then, is an offering to make robots like an eduMIP easier to build. By merging a BeagleBone Black with the Robotics Cape into a single board, removing components that aren’t as useful for a mobile robot (such as the Ethernet port) we arrive at a BeagleBone Blue.

Of course, the brawn of a robotics chassis isn’t much use without the smarts to make it all work together. Befitting university coursework nature and BeagleBoard Foundation’s standard procedure, its peripherals software now called Robot Control Library are documented and source code available on Github.

I could buy an eduMIP of my own to help me explore the BeagleBone Blue, and at $50 it is quite affordable. But I think I want to spend some time with the BeagleBone Blue itself before I spend more money.

AltoEdge Infinity USB Foot Pedal Dates Back Before Windows 7

This SGVHAK teardown project came courtesy of an electronics waste bin. A nondescript box with a USB cable, it has three moving parts on top of a heavy base. The center piece takes up majority of width, and two far smaller pieces sitting on either side. Each piece can be pressed down and we can feel a tactile click of a switch. It has a respectable heft and doesn’t look damaged or even worn. It feels rather beefy and unlikely to physically break.

Infinity Foot Pedal IN-USB-2

A label on the bottom of the device lets us know it is version 14 of the Infinity IN-USB-2 foot pedal. Which explains its mass and durability: this box was designed to sit under a desk and be stepped on. A box sitting out of sight explained its raised side pedals allowing its user to find them by feel.

Infinity Foot Pedal IN-USB-2 v14 label

A few screws on the bottom held a plate in place, easily removed. We see a few springs for the pedals, and two pieces of metal that gave the device its heft.

Infinity Foot Pedal IN-USB-2 bottom panel removed

There were a few visible plastic clips holding individual pedals in place, but they were only the first line of defense – unclipping them allowed individual pedal to move a little further but did not release them. There were also a few hinge pins that could be removed, but again it allowed additional movement but did not release.

The two shiny metal weights were held by tenacious stretchy glue. We could pry them up far enough to see they weren’t obviously hiding screws, but we were wary to apply addition force as it threatened to break apart the plastic housing.

Without an obvious way forward for nondestructive disassembly, we decided to pause and reassemble the pedal to see if it can be useful intact before we risk destroying it. My computer was running Ubuntu at the time, which gave us a starting point with the dmesg tool to see what kind of greeting it has to say to my computer.

[ 459.086214] usb 1-4.4.3: new low-speed USB device number 17 using xhci_hcd
[ 459.192673] usb 1-4.4.3: New USB device found, idVendor=05f3, idProduct=00ff
[ 459.192679] usb 1-4.4.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 459.192683] usb 1-4.4.3: Product: VEC USB Footpedal
[ 459.192687] usb 1-4.4.3: Manufacturer: VEC
[ 459.196360] input: VEC VEC USB Footpedal as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4.3/1-4.4.3:1.0/0003:05F3:00FF.0012/input/input28
[ 459.196980] hid-generic 0003:05F3:00FF.0012: input,hiddev1,hidraw9: USB HID v1.00 Device [VEC VEC USB Footpedal] on usb-0000:00:14.0-4.4.3/input0

So far everything looks in line with the manufacturer’s name we found earlier. It also tells us the device conforms to USB HID (Universal Serial Bus Human Interface Device) specification. The final line also hinted us to a newly visible device under the path/dev/hidraw9.

$ ls -l /dev/hidraw9
crw------- 1 root root 240, 9 Mar 17 13:32 /dev/hidraw9

This path is owned by root, so further experimentation requires taking ownership of that path to see what we can do with it.

$ sudo chown $USER /dev/hidraw9

Now we can try treating it as a file with the cat command. Every time we press or release a pedal we get some kind of visual feedback but we don’t understand it.

cat dev hidraw9

We then tried treating it as a serial port using minicom but that didn’t get us much further. It vaguely resembles the garbage that might occur if a baud rate setting is incorrect, but changing baud rate in minicom didn’t do anything. Probably because it’s not a serial port!

Since the device was classified as a USB HID v1.00 Device, the next thought was to try communicating with it via some sort of HID API for developers. But USB HID is not a trivial thing and after a half hour of following and reading links to documentation I was no closer to talking to the pedal in a “proper” way. So I tabled that approach and returned to treating it as a file. It’s pretty trivial using Python’s file APIs to open it up for reading.

>>> hr9 = open('/dev/hidraw9','r')

Reading a few bytes at a time, we figured out the device sends two bytes upon every action. First byte is a bitfield indicating pedal status, and the second is always zero. The leftmost pedal corresponds to the least significant bit 0x1, then center pedal 0x2 and right pedal 0x4. So if both right and center pedals were both pressed, it would give 0x6. Here’s a simple Python loop that reads two bytes at a time and outputs to command line.

>>> while True:
... hr9.read(2)

The output if I press and release the left, then repeat for center and right pedal.

'\x01\x00'
'\x00\x00'
'\x02\x00'
'\x00\x00'
'\x04\x00'
'\x00\x00'

Not every action will trigger data events. There’s a small time window where separate events are collapsed together for a single notification. If I’m quick enough on the press and on release, I can push the right and left pedals simultaneously for a single 0x05 report, then release simultaneously for a 0x00 report, without any intermedia reports of 0x04 or 0x01.

'\x05\x00'
'\x00\x00'

This is a very promising set of experiments indicating that, if it should be necessary, we can write code to make use of this pedal in Linux without digging through all of HID API.

With that knowledge under our belts, experimentation then moved to Windows 10, which immediately recognized it as a USB HID and even shows us the name. However, it doesn’t do much without further help.

VEC USB Footpedal device is ready

Searching for answers on the web, we learned this device was designed for people transcribing audio recordings into text. The pedals allow them to control sound playback (pause, play, rewind, etc.) without taking their typing hands off the keyboard. I’m sure this is a productivity boon for its target audience, but that wasn’t us. Fortunately, the manufacturer has also released a piece of software call Pedalware which will allow this pedal to be used outside its designed scenario, like emulating keyboard keys or mouse buttons. I thought it sounded interesting enough to try.

And this is where we started getting a hint why this device has been retired… this piece of hardware’s associated software is old. Pedalware’s installer demands Windows 7 or earlier and refused to run under Windows 10.

Pedalware needs Windows 7 or earlier

At this point, Windows 10 backwards compatibility module kicked in and offered the option of running in compatibility mode. I accepted.

Pedalware needs Windows Vista compatibility mode

That was enough to get Pedalware up and running on my Windows 10 computer. Now I can assign an arbitrary keyboard or mouse action to each of three pedals.

Pedalware up and running

This worked fine in everyday web browsing and productivity applications. It is, however, too slow for gaming purposes. The aforementioned time window seen under Linux, which collapsed multiple events into a single event, manifests here as well resulting in foot click actions getting lost in high-speed gaming action.

But that’s fine, the device was never intended to be a gaming peripheral. The real problem comes from its driver software becoming unreliable as a computer goes into low-power standby. When the computer resumes, the pedal doesn’t always come back into action. And once it gets stuck, the only way to get it back is a full reboot.

This was a sign of the times when this device was designed. I remember when many peripherals would not gracefully handle a computer going to sleep, which meant I typically leave my computer running in the Windows XP/Vista/7 days. Computers have gotten more power efficient over these years but it’s still better to put them to sleep. Also, modern USB peripherals are much better about resuming from sleep.

But this pedal does not, and that’s probably why it was retired. Fortunately, my work does not require a predictably functional foot pedal, so I’ll keep it around and try using it on the occasions when it works.

Cree Dimmable LED Bulb Teardown

I brought an old LED light bulb to a SGVHAK meetup for an educational dissection. This bulb has illuminated my front porch for several years, hooked up to a light-sensitive fixture that turns on the bulb when dark, and turn it off when the sun is up. However, when illuminated this bulb has started flickering. At first it was a mild pulse that I didn’t mind very much as I don’t usually need the light myself anyway. But after a while, the blinking started getting annoying and even the bright pulses were too dark for the light to serve its intended purpose. This bulb was retired, and now we take it apart to see what’s inside.

Looking inside the cooling vents, we can see there are two circuit boards mounted at right angles to each other. Obviously there will be LEDs soldered to these boards, with a power supply at its base. Since all the LEDs pulsed together, we expect there to be a power supply failure and hope we might be able to see what component caused the problem.

Here’s the bulb intact, before teardown began.

Cree LED bulb teardown 1 - intact

Here’s the label on the bulb. This was not a bargain basement device, it was a dimmable bulb and it was also designed to be usable in damp environments as the front porch is occasionally exposed to rain. It was also exposed to outdoors wildlife, including some insects who have climbed inside and sadly died there, too.

Cree LED bulb teardown 6 - label

It was nicely sealed with no obvious way to take apart the plastic housing nicely, so out came a beefy cutter and we start cutting from top vents.

Cree LED bulb teardown 2 - first cut

Once the top vents were clipped open, we could perform some literal debugging of our device by pouring the dead carcasses out. They look like honeybees.

Cree LED bulb teardown 3 - pour out the bugs

Aside from debugging, the opened top also lets us see more details inside. Sadly, there were no visible mechanisms for easy release, so cutting continues at waist-level vents.

Cree LED bulb teardown 4 - second tier cut

Once those portions were cut away, we could see more of internals. There were fewer LED surface mount packages than we had expected.

Cree LED bulb teardown 5 - second tier removed

At this point I ran out of convenient places to cut with a hand tool, so cutting moved on to a band saw.

Cree LED bulb teardown 7 - band saw

Once the band saw cut through all around the base of transparent plastic bulb exterior, we were able to free its internals for a closer look. Surprisingly, the circuit boards connect to each other and to the base with tiny spring-loaded connectors rather than a direct soldered joint. One hypothesis is the bulb was not only designed for humid environments, it was designed to sustain vibration as well. Another hypothesis was that humid environments also imply a larger temperature swing, where spring-loaded connectors can accommodate thermal expansion/contraction better than soldered joints.

Cree LED bulb teardown 8 - disassembled

Here is the main board’s topside. Nothing appeared obviously damaged. The big electrolytic capacitor immediately drew our attention, as that is the type of component most likely to fail with age. However, the usual signs were absent. No leaking electrolyte, no bulging of the body, and no breakage in the top. With the help of our multi meter, we could tell the capacitor has neither failed open or failed short. We also measured its capacitance, which won’t be a reliable number as the capacitor is still installed on the board: we’d be measuring capacitance of the capacitor as well as the board it is installed on. But the number was roughly in the ballpark of the rating printed on its side, so it looks clear on all counts.

Cree LED bulb teardown 9 - main board front

Bottom side of the main showed no such obvious attention-getters. The small light colored surface mount component at the base might be a safety fuse, and it tested OK for continuity.

Cree LED bulb teardown A - main board back

We explore the fewer-than-expected LED modules by trying to power up a single LED using a bench top power supply. At first it stayed dark and we thought maybe the LEDs were at fault instead of the power supply. But then we realised we weren’t giving it enough power: we were surprised it took over 24 volts before a single module would illuminate.

Cree LED bulb teardown B - LED needs 24V

An explanation surfaced once we adjusted camera settings to see individual light sources inside the package: there are actually ten LEDs in each package, in a three + four + three configuration. This explains how it could be so bright with so few surface mounted modules!

Cree LED bulb teardown D - ten LED per package

At this point we’ve verified all the discrete components we understood and could test, we’ve removed dead bug remains that might have caused problems, and we cleaned up all the electrical connectors. Maybe that’s enough to bring this bulb back to life?

Cree LED bulb teardown C - back on 110V AC

Sadly, the answer was no. We hooked it up to AC power and plugged it in: it is still dim and blinky. Obviously we failed to understand how this particular bulb works – the power supply circuitry was far more complex and sophisticated than we had expected. But still, it was fun to look inside a premium (for its day) LED bulb.

 

Sawppy at Caltech Science for March 2019

Today Sawppy joined the fun at Science for March 2019, organized by the Caltech Postdoc Association. Since Sawppy didn’t exist yet during 2018’s event, this was Sawppy’s first visit. When Sawppy arrived at South Campus Gate, a quick check of campus directory oriented us to Beckman Mall where the event will be held.

Caltech Science for March 2019 - Sawppy at Campus Directory

Part of the journey included crossing picturesque Millikan Pond.

Caltech Science for March 2019 - Sawppy to cross Millikan bridge

As with last year’s event, the rover started the day sitting on a table for display. Of course, last year we had only a single rover. This year we have three, two of which were in running condition and could be driven for demonstration today.

Caltech Science for March 2019 - Rovers on SGVHAK table

Sawppy attracted a crowd as it usually does, and were driven around by enthusiastic children. Some of them weren’t as gentle with the control as they should be, and about halfway through the day, operator roughness by one of the kids burned out the fuse. I kept telling myself I should have spare fuses in Sawppy’s first aid kit that I keep in my backpack, but I never did put one in. I had to hack a workaround today but by the end of the night I definitely remembered to put extra fuses in the bag.

Caltech Science for March 2019 - Sawppy with burnt fuse

Group rover outings are always fun. Today’s special activity is a group climb on a grassy slope at the north end of Beckmann Hall. SGVHAK rover gave Sawppy a head start, but with faster and more power motors, combined with better traction tires, it was no contest. SGVHAK easily outclimbed Sawppy, but the important part was that we pleased the crowd with this little demonstration of rover climbing capabilities.

Caltech Science for March 2019 - Sawppy and SGVHAK Rover climbing

(Cross-posted to Hackaday.io)

Sawppy and SGVHAK Rover Will Be At Caltech Science for March

Shortly after SGVHAK Rover was completed last March, it also made an appearance at California Institute of Technology’s Science for March event organized by the Caltech Postdoc Association. The rover was well received and we’re going to do it again for this year’s event taking place tomorrow. This event is free and you can register here.

Rover Brick Demo

We were but young rover herders at last year’s appearance, and there were some durability issues that we think we’ve addressed. I’ve taken some of those lessons to heart with my own Sawppy rover, which was only a dream at last year’s event but this year will be one of several rovers joining in the festivities.

During our outing at the Downtown Los Angeles (DTLA) Mini Maker Faire, we saw familiar faces from our local friends at Caltech and also nearby Pasadena City College (PCC) and we hope to see them again this year.

The weather forecast is sunny and clear, it should be a good time!

Window Shopping BeagleBone Blue

Sawppy was a great ice breaker as I roamed through the expo hall of SCaLE 17x. It was certainly the right audience to appreciate such a project, even though there were few companies with products directly relevant to a hobbyist Mars rover. One notable exception, however is the BeagleBoard Foundation booth. As Sawppy drove by, the reception was: “Is that a Raspberry Pi? Yes it is. That should be a BeagleBone Blue!”

Beaglebone Blue 1600
BeagleBone Blue picture from Make.

With this prompt, I looked into BBBlue in more detail. At $80 it is significantly more expensive than a bare Raspberry Pi, but it incorporates a lot of robotics-related features that a Pi would require several HATs to reach parity.

All BeagleBoards offer a few advantages over a Raspberry Pi, which the BBBlue inherits:

  • Integrated flash storage, Pi requires a separate microSD card.
  • Onboard LEDs for diagnosis information.
  • Onboard buttons for user interaction – including a power button! It’s always personally grated me a Raspberry Pi has no graceful shutdown button.

Above and beyond standard BeagleBoards, the Blue adds:

  • A voltage regulator, which I know well is an extra component on a Pi.
  • On top of that, BBBlue can also handle charging a 2S LiPo battery! Being able to leave the battery inside a robot would be a huge convenience. And people who don’t own smart battery chargers wouldn’t need to buy one if all they do is use their battery with a BBBlue.
  • 8 PWM headers for RC-style servo motors.
  • 4 H-bridge to control 4 DC motors.
  • 4 Quadrature encoder inputs to know what those motors are up to.
  • 9-axis IMU (XYZ accelaration + XYZ rotation)
  • Barometer

Sadly, a BBBlue is not a great fit for Sawppy because it uses serial bus servos making all the hardware control features (8 PWM header, 4 motor control, 4 quadrature input) redundant. But I can definitely think of a few projects that would make good use of a BeagleBone Blue. It is promising enough for me to order one to play with.

Sawppy Wiring Schematic Tests Drive Digi-Key Scheme-It

When I declared Sawppy the Rover has reached version 1.0 and posted instructions online, I was fully aware of the fact that the instructions would be incomplete. Not out of neglect or malice, but out of the fact all of Sawppy is in my head and there will be places where I decided something was obvious enough not to require documentation – and learn I was wrong in that judgement.

Such is the case now. One member of RSSC accepted my invitation to build a Sawppy after my presentation in January, and submitted the feedback that I need to post a schematic of how I’ve wired up Sawppy. I previously submitted this answer as response to a comment on the Hackaday.io project page, but more detail was needed.

The minimum electronics components for a bare-bones Sawppy has [10 * LewanSoul LX-16A] wired in parallel with each other and to [1 * LewanSoul “BusLinker” a.k.a. “Debug Board”] to translate generic USB serial to servo half-duplex serial. That translator board is connected to [1 * Raspberry Pi] via USB.

I thought this was a good opportunity to try Scheme-It from Digit-Key. It is a web-based electronics schematic editor that purports to let people quickly sketch up electronics schematics for sharing. The electrical wiring for a bare-bones Sawppy version 1.0 should be a nice easy exercise for this tool.

The terms and conditions for using Scheme-It is fairly typical for a web-based application. A Digi-Key account is required for login, something I already had from earlier purchases for electronics experiments. In exchange for free use of the software, a user also has to grant Digit-Key a license to the schematic. Not a big deal in this case, as I wanted Sawppy to be shared as widely as possible. And finally, Digi-Key reserves the right to take down this service at any time, deleting all of my data. This is irritating but not unexpected. If this is important to me I better make a copy for my own storage.

Since my sample schematic is fairly simple, it only took about an hour to go from absolute beginner to the schematic I wanted to create. I could (and did) choose to share the project file publicly via a URL, though it appears accessing the project requires logging in to Scheme-It, which required a Digi-Key account as previously mentioned. I could also export to PDF or PNG formats. The PDF export feature was unsatisfactory. Many labels were moved out-of-place making the schematic illegible. In contrast the PNG export looks OK so I’ve posted this schematic PNG to Sawppy’s Hackaday.io project page, as well as Sawppy’s assembly instructions hosted on Github.

Sawppy V1 Schematic

(Cross-posted to Hackaday.io)

New Batteries for Thrift Store Neato Vacuums

The Neato XV-21 I found at a thrift store has bee joined by a XV-12 found by [Emily]. Both Neato robot vacuums found at thrift stores have degraded batteries. This was not a surprise as robot vacuums work their battery packs hard. Not just frequent charge and discharge cycles, but with a heavy power draw under use to run vacuum motors. This means both of these vacuums were retired by their previous owners when the battery no longer hold enough charge to perform its duties.

Neato XV-12 battery unhappy

Fortunately, this common failure also meant there’s a robust aftermarket for replacement batteries. Curiously, the economics of the markets are such that whole replacement packs can be ordered online for less than ordering individual cells and rebuilding the packs myself.

With previous experiments, I have gained confidence I can verify functionality of individual components using test mode accessible via USB port. And since this XV-12 was found with an official Neato charging dock, it’s time to install replacement batteries and test the full system.

The replacement batteries claim a capacity of 4000mAh which, on paper, is an increase from the original battery’s label capacity of 3800mAh. However, battery manufacturers play pretty loosely with these ratings so I expect the difference of 200mAh to be fairly insignificant in practice. When I took apart the original pack, I saw a thermister for monitoring temperature, an overcurrent protection fuse, and an overheat fuse. I assume the replacement pack has a thermister because the Neato computer can read it, but there’s no immediate way to tell if the overcurrent or overheat protection also exists on the new pack.

Neato XV-12 batteries old and new

With new battery packs installed, it’s time to put the robot up against the charging dock and verify the charging system works as expected. But before we do that, let’s take a closer look at this charging dock.

Cleaning Up a Thrift Store Neato XV-12

When I looked over my Neato XV-21, I thought it was far too clean to be a high mileage appliance. There were several possibilities:

  1. The previous owner was meticulous about keeping things clean (supported by the plastic protective wrap.)
  2. The Neato was barely used.
  3. A Neato is very good at keeping itself clean.

Thanks to a second Neato found in a thrift shop, this time a XV-12, we now know #3 is false. This one is well used, and very much looked it! The previous owner didn’t bother cleaning the vacuum before dropping it off at the thrift store. I have several loops of hair tangled up in the roller brush. The largest loop has even snagged what looked like furniture padding foam.

Neato XV-12 dirty brush roller

The largest loop has also tightened enough to damage the roller brush. Once I cut that loop of hair and debris off the roller brush, I see a slot cut into the rubbery blade.

Neato dirty brush roller ripped

There was also debris tangled on the motor shaft driving this roller that had to be cleaned off before the roller brush could function properly again. The cavity for the roller brush showed extensive wear from a life as a working household vacuum (above), whereas the XV-21 showed barely any wear (below).

Neato wear pattern

Those were problems found on the suction side of the vacuum. What about the exhaust side? The vacuum is generated by a fan which sat downstream from the dust bin filter, so its cleaniness would be a clue at how effective the filter was. Everything seen here are dirt particles that have made their way past the filter.

Neato XV-12 dirty vacuum motor

But even though this vacuum lived a harder life, its battery was actually in better condition and could run the vacuum computer for several minutes before fading out. As a result I was able to query all components individually using its USB port without having to hack a battery into this vacuum. All signs indicate that this robot vacuum is likely fully functional except for its battery.

And fortunately, with the arrival of replacement battery packs, we don’t need to drill holes and hack external battery packs for a full system test. We can install the new batteries into this XV-12.

 

Sawppy at SCaLE 17x: The Trouble with Rovers

My primary obligation for Southern California Linux Expo (SCaLE 17x) was to co-present The Trouble with Rovers with Lan Dang on Saturday afternoon. Which meant when Sawppy’s coupler broke Friday evening I had to scramble to fix it for Saturday. A rush repair job is always going to leave some details to be desired but it was sufficient to resume operation.

Sawppy will obviously be one of the visual aids present at our talk, but that doesn’t mean it gets to spend the rest of the day just sitting around. No sir, as soon as Sawppy arrived on location, it immediately started working as a roving billboard for both itself and the talk.

Sawppy Scale 17x Sat 1 - Roving billboard

It was not explicitly coordinated beforehand, but the SGVHAK rover was also equipped with advertising for our talk. Both of our rovers were out and about, pulling roving billboard duty, and occasionally our paths crossed in the hallways of SCaLE.

Sawppy Scale 17x Sat 2 - Two roving billboards

I was happy with the turnout for our rover session. While we did lose a few people who left partway through the talk, it was more than made up for by the enthusiastic people who followed along and came up to ask questions after. I had a lot of fun explaining details on what we did for both rovers, minutiae that we trimmed from the talk proper but was still interesting to our smaller and more technical audience afterwards.

Sawppy Scale 17x Sat 3 - Session

Sawppy continued to roam around Pasadena Convention Center, spreading word of my rover project to people who are excited about the possibility of building their own. Some people thought it would be their motivation to finally buy their own 3D printer, others have all the tools on hand and it’s just a matter of prioritization and finding project time. I was most gratified by a group of students from California State University San Bernardino who thinks it would be a great group project. They are exactly the target audience!

And it’s fun as always to see young children’s faces light up when they see Sawppy, some of whom were eager to take control. My favorite was this 6-year old who loved to drive Sawppy over his own toes over and over.

Sawppy Scale 17x Sat 4 - Self foot runover

(Cross-posted to Hackaday.io)