SGVHAK Rover Interface Avoids Confirmation Dialog

Shutdown Reboot

Our SGVHAK Rover’s brain prefers to be shut down gracefully rather than having its power removed at an arbitrary time. If power cuts out in the middle of a system write operation, it risks corrupting our system storage drive. While our control software is written to be fairly platform-agnostic, we’re primarily running it on Raspberry Pi which lacks a hardware shutdown button. So we need to create an UI to initiate a system shutdown via software.

The easiest thing to do would be to add a “Shutdown” button. And since it is a rather drastic event, have a “Are you sure?” confirmation dialog. This historically common pattern is falling out of favor with user interface designers. Computer users today are constantly inundated with confirmation dialog making them less effective. If our user has built up a habit of dismissing confirmation dialog without thinking, a confirmation dialog is no confirmation at all.

So how do we enforce confirmation of action without resorting to the overplayed confirmation dialog? We have to design our UI to prevent an accidental shutdown from a wayward finger press. To accomplish this goal, our shutdown procedure is designed so user must make a deliberate series of actions, none of which is “Yes/No’ on an ineffectual dialog box.

First – our user must enter a “System Power” menu. If they entered by mistake, any of the three active buttons will take then back to main menu. There’s no way to accidentally shutdown with a single mistaken tap.

Second – they must select “Shutdown” or “Reboot”, forcing a deliberate choice to be made before our UI activates “OK” button. If an incorrect selection is made, it can be corrected without accidentally triggering the action because both options are on the far left of the screen and “OK” is on the far right. Bottom line: Even with two accidental presses, our system will not shut down or reboot.

Third – With the “OK” button now activate after two deliberate actions, the user can tap it to begin the shutdown (or reboot) process.

SGVHAK Rover Steering Trim Adjustment

One of the changes we made to our SGVHAK Rover relative to the baseline design was a decision to omit absolute encoders for our corner steering motors. The baseline design used absolute encoders so it knows every wheel’s steering angle upon system startup, but we decided to sacrifice that ability in exchange for cost savings (~$50/ea * 4 corners = ~$200.) Since there is no free lunch, this decision also means we have additional work to do upon our system startup.

Our control software launches with the assumption that all steerable wheels are pointed straight front-back, which is of course not always true. We could manually push the wheels to straight front-back orientation before turning on power, but that’s not great for the gearbox. Also, the shaft coupler used to connect our motor gearbox to our steering mechanism is liable to slip. (This is considered a bug even though it has a potential feature: in theory sharp jolts to the system would slip the coupler instead of breaking something else in the system. In practice, we found that it didn’t slip enough to avoid breaking a gearbox.)

Given both of these scenarios, we need a software-based solution to adjust steering trim on the fly. Once our RoboClaw motor controller is told where zero degree position is, it is quite capable of holding a commanded angle until the coupler slips or the system reboots, whichever comes first.

Steering Trim

Steering trim adjustment took the form of a multi-stage HTML form designed for our specific workflow. First our user chooses a wheel to work with, and they are directed to do so because every other UI element on the page is inactive until one wheel is chosen.

Once a wheel is chosen, the remaining UI activates and wheel selection UI deactivates to guarantee we’re working on a single wheel at a time. In this stage, steering angle can be adjusted in 1- and 5-degree increments. When we’re satisfied our wheel has been returned to zero degrees deflection, we can select “Accept Current Angle as New Zero” to commit. We also have the option to abort adjustment and retain previous zero position. Either way, wheel selection activates to allow user to select another wheel, and the rest of the page deactivates until a wheel is chosen.

We have no scenarios where multiple steering motors need to be trimmed at the same time, so this user experience specifically focus on one wheel at a time makes the process straightforward. It also becomes impossible to make mistakes caused by user thinking they’re adjusting wheel when they’re actually adjusting another, which is a good thing for positive user experience.

SGVHAK Rover Control With Cartesian Coordinates

Having learned some valuable lessons from trying to drive our SGVHAK rover using an interface based on polar coordinates, we returned to the drawing board to create an alternative. Thankfully, the abstracted command structure meant we could experiment with front-end user interface without any changes to the back-end rover calculations.

Our first-hand experience taught us polar coordinate math are more trouble than it was worth, so our next attempt will stay in the Cartesian coordinate space. Furthermore, we’ll rearrange the layout so a position going full speed ahead would never be adjacent to a position going full speed reverse. (This was the case for polar coordinate system when doing full deflection turns.) Third, we’ll have to make sure low speed maneuvering has just as much room to fine-tune its angle as going high speed. And lastly, when someone releases their finger out of caution, we need to stop the rover but we don’t need to reset steering angles.

It turned out to be very easy to satisfy all of the above desires: map X-axis left-right dimension to steering angle, and Y-axis up-down dimension to speed.

Cartesian Pad

Full speed ahead is now along the top edge of the blue rectangle, which is far from the bottom edge representing full speed reverse. There is no longer any risk of suddenly jerking between them.

At any given speed, we now have the full width for designating steering angle, we are no longer frustrated by a tiny range of adjustments when moving slow speed versus high.

And finally, when the user releases their finger, the dot will snap to vertical center (stop moving) but will remain at its most recent horizontal position, preserving the steering angle most recently used by our rover driver. This makes fine position adjustments much easier by allowing slow incremental inching forward/reverse movements.

Most users who tried both UI design prefer the Cartesian coordinate system. It was used for most of our driving at SCaLE 16X.

SGVHAK Rover Control With Polar Coordinates

Once an abstraction layer was defined for communication between user interface and rover chassis calculations, I could start trying out various ideas for building a rover driving UI. The baseline rover UI presents two touchscreen controls mimicking joysticks on a traditional remote control vehicle: one moves up/down for controlling forward/back movement, and the other moves left/right for controlling steering. Driving the baseline rover using its UI is a two-thumb affair.

That is a perfectly valid and functional UI, but my ambitions grew beyond copying it. I wanted to explore ideas around single-finger operation. This decision held firm even after the person who created the UI for baseline rover warned us that it could be tricky to implement – single-point operation was already tried before going with the two-thumb approach. I may yet bow to their wisdom, but I wanted to give it a shot first!

Polar Pad

The first implementation of a one-finger driving control is based around polar coordinates. It seemed like an obvious match to rover mechanics: polar coordinates on a two-dimensional plane is an angle and a distance, which maps directly to rover control’s angle and velocity. The user puts their finger down on the red central control dot, and they can drag it around inside the blue circle to drive the rover. Move the dot up to drive forward, rotate the dot left to steer left, and so forth.

As I got into it, though, problems started to mount.

The first and most obvious problem is that our underlying infrastructure – HTML, <canvas> tag, and JavaScript input events – use cartesian coordinates. In order to implement polar coordinate control we need trigonometry math to convert between the two systems. Errors creep in as we perform multiple operations and conversions between coordinate spaces.

The second problem was a consequence of mapping upper semicircle to forward velocity and lower semicircle to moving backwards. This became problematic when the user wishes to steer at maximum angle.  Turning far left means moving straight left, but at that point small movements can flip between upper and lower semicircle, causing the rover to jerk suddenly between forward and backward movement.

To mitigate this, a band of dead space between the semicircles was introduced. The user is no longer able to use the full 180 degrees of a semicircle for movement, they are restricted to the middle 140 degrees with the far left 20 and far right 20 degrees blocked off. This was effective to prevent sudden transitions between full speed forward/backwards motion but it wasted screen real estate.

The third problem of polar coordinates was exposed when trying to maneuver the rover at low-speed around obstacles. When our finger is near the center of the circle, small left-right movement translates into large changes in steering angle. It was more difficult than it should be to make small steering changes at low speed.

The last problem is makes the above problem worse. When somebody runs into a problem, their first instinct is to lift their finger off the touchscreen. In order to retreat to a known safe condition, our control dot pops back to the center position representing wheels stopped and all steerable wheels pointing straight front-back. This is fine, but when people are frustrated by problem #3 of low-speed maneuverability, this behavior snapping to straight front-back erases whatever angle they were previously driving at, making low-speed maneuvers even more frustrating.

And what happens when a rover driver is trying to perform delicate maneuvers but find themselves fighting a bad UI that gets in the way and cause frustration? The rover ends up running into things and risk breaking gearboxes.

Now we have experimental evidence polar coordinate driving doesn’t work as originally thought, let’s take that lesson and create a new UI.

 

SGVHAK Rover Control Abstraction

Once we were confident our SGVHAK rover chassis control code looked pretty good, we were willing to trust that it could drive a rover without sending conflicting commands that make our rover try to twist itself into a pretzel. Now our focus turns to creating an interface for user control input.

A quick rough draft had been in place to help test rover chassis Ackermann math, letting a rover driver send a steering angle and travel speed to rover’s control code. But it was very tied to the current rover implementation: angle range was limited to angle range of what we have today, and speed was specified in raw quadrature pulses per second (QPPS) that we send straight down to RoboClaw motion controller‘s API.

Since we have ambitions for future rover variants, this direct dependency is not good. Future rovers may have a wider (or narrower) range of valid steering angles. They may also use something other than RoboClaw motor controllers. The answer is a typical software solution: abstraction!

This required changing the interface between our HTML UI front-end and the Flask-based Python code running on the rover. Instead of a fixed number angle or a fixed number of encoder pulses, user now send driving commands in terms of two percentages ranging from negative 100% to positive 100%.

For steering, -100% tells a rover to steer as far left as it could go, 0% is straight forward/back travel, and 100% is to steer as far right as it could go.

For velocity, -100% is travelling backwards as fast as it could, 0% is stop, and 100% is damn the torpedoes, full speed ahead.

Now when we adapt the code for future rover variants, we could focus on rover-side configuration and remain confident user-side HTML interface will automatically adapt to whatever capabilities are present on a particular rover variant.

This also allows us to experiment with user-side HTML without worrying about having to make matching changes in rover-side Python code. We can try different control schemes, as long as it ends up sending a velocity and steering angle percentage to the rover.

The first and crudest implementation of this concept is shown here: two HTML slider objects that move from -100% to 100%. The user can move the slider and tap “Send command” to rover. As the most basic HTML implementation possible, this is a useful test tool to isolate whether a particular control problem is in HTML front-end or in Python back-end, but it is obviously not a great driving interface.

That’s coming up next.

Velocity Angle

Visualizing SGVHAK Rover Chassis Calculations

When we were putting together code to calculate Ackermann steering angles for SGVHAK rover‘s wheels, initial testing was done via Python command line using some fixed test values to match against results pre-calculated by hand. Once we built some confidence our math was on the right track, it was time to start testing against more varied inputs.

There was just one minor problem – at this point in rover development, our software is a tiny bit ahead of the hardware which means we can’t test on hardware that hasn’t yet been assembled. Even if we could, it wouldn’t be a great idea to jump from a small fixed set of test values to actual hardware. As an intermediate step, we created a bit of code to visualize results of roverchass.py calculations: the “Chassis Configuration” screen will show what our control code thinks all six wheels should be doing. (This shows the intent, which is different from hardware telemetry of what’s actually happening.)

SGVHAK Rover Chassis Config

Each of the six wheels got its own blue box, laid out roughly analogous to their physical position on the rover chassis. Originally wheel steering & speed was displayed as text: number of degrees a wheel should be steered, and a second number reflecting the intended velocity for each wheel. This was accurate information but it took some effort to interpret. To make this information easier to understand, numerical text was changed to a visual representation.

By using HTML’s <canvas> element, we could draw some simple graphics. Each wheel is represented by a black rectangular outline. A wheel’s steering angle is now represented by tilting its rectangle to that angle. Inside a wheel’s rectangle is a green fill representing desired velocity. When a wheel is supposed to be stopped, there is no green fill and its black rectangle is empty. When a wheel is supposed to be rolling forward, green rectangle fills from the middle upwards. Faster velocity is represented by more green in the upper half. For a wheel that should be rolling backwards, green fills downwards instead.

This visualization of roverchassis.py calculation results were far easier for humans to understand. This made it very quick to validate that roverchassis is behaving roughly correctly. There’s a place for precise engineering calculations, but there’s also place for a quick glance to check “yeah, that looks about right.”

SGVHAK Rover Software: Chassis Calculations in Python

SGVHAK Rover Chassis ConfigOne of the biggest changes in our new rover controller code is the goal of managing all ten rover chassis motors together, starting to think of the rover as a single unit instead of ten individual things.

This logic is centered in roverchassis.py, which is be responsible for calculating Ackermann angles for all steerable wheels in the rover, as well as calculating the desired travel velocity for each wheel. In order to make this code as flexible as we can, we offload details of rover chassis into configuration files and make our Python math as generic as possible running calculations based on configuration file values. This meant we could support rovers with different geometries without editing source code. This feature began with a desire to have flexible wheelbase and track dimensions of our rover chassis, but eventually extended to supporting similar but different configurations such as two-wheel differential drive.

Another goal was to support different types of motors and motor controllers. We appreciated all the features in the RoboClaw motor controller we started with, but we also thought about trading off some of these features for lower cost. In preparation for this flexibility, we kept roverchassis.py focused on general Ackermann-related math and kept motor-specific code in a separate class. In the first draft, this meant pulling all RoboClaw related code into roboclaw_wrapper.py which translates general concepts from roverchassis.py into specific commands in Ion Motion Control’s public domain API roboclaw.py. Later on we validated this flexibility by steering a single wheel with a RC servo followed by adding support for LewanSoul serial bus servos.

This was all possible using a flexible syntax in config_roverchassis.json that allows us to adjust wheel dimensions as well as switch out individual wheel travel or steering motors for different motor controllers.

 

 

A Fresh Start for SGVHAK Rover UI

SGVHAK Rover Chassis ConfigWhen our SGVHAK rover project started moving beyond individual units of our custom wheel gearbox design and towards a multiple wheel chassis, it was happening on both the hardware and software fronts. Our Flask-based control UI has worked well up to this point, allowing us to tune RoboClaw motor controller parameters to suit our wheel design. Building this first draft UI taught us a lot, and as we learned we also realized how it reflected our ignorance at the beginning of this project.

And so we bid goodbye to our first draft rover UI RogerPiBot to start a new project SGVHAK_Rover. This way we retain all the testing & tuning capabilities of the existing software, leaving that as-is. Freeing us to create a new user experience.

The major changes are:

  1. Simple touch-friendly controls instead of HTML forms with a lot of fields. The form was a very powerful tool for tuning parameters from a desktop workstation, but we’re getting out of the engineering test phase. All interfaces will now be tailored towards operation from a small touchscreen.
  2. Rover as a unit instead of wheel as a unit. We need to start treating all six travel motors and four steering motors aboard our chassis as a single unit, as the user expects the software to do. Target audience for this software will not be interested in poke and prod at motors individually.
  3. Configuration files instead of source code edits. The new software is built on the assumption that will be people who want to use it but is not familiar with Python or code in particular. Part of this meant configuration information will now be stored in human-readable JSON files at the root directory instead of asking people to go in and edit Python source code.

There were various other changes big and small. One of the larger changes was separate from the rover itself: we chose to switch away from using Bootstrap purely for the sake of HTML creation exercise. This time around we’re using a different library called Materialize to help create our responsive UI.

Unfriendly UI For Two Wheel Test Chassis

Our first attempt at rover software was a wonderfully informative hands-on adventure on creating a path from a cell phone’s web browser HTML all the way to serial bus commands sent to a RoboClaw motor controller. Fortunately or unfortunately, these lessons also put a finite lifespan to the software as we start to discover basic mistakes that we didn’t know we were making until we got a little smarter.

Before we leave the first draft behind, however, it had one final role to play. This point in software development timeline coincided with our rover wheel development’s completion. The actual six-wheeled rover chassis is still in the future, but our wheel development is mature enough for us to build a test chassis. Two wheel units were assembled and integrated into a bare-bones test chassis.

This chassis needed a brain and a controller.

To enable motor control via differential drive, we hacked together a minimal user interface for sending commands to our test chassis. Not much effort was spent on user-friendliness and its layout is very much an engineering number cruncher’s idea of an interface. (Also referred to as “bad UI.”) It was just enough to let us move the robot around and uncover some finishing issues in the wheel design, but not much more than that.

The robot moving under its own power attracted attention from other people in the work space. Naturally some asked if they could try driving the robot but one look at the interface and the response was “Um… maybe I’ll wait for a later version” and the phone was handed back.

No offense taken, it’s another vote in favor of moving on. We’ll close out the “RogerPiBot” project at this point and start a new rover control project incorporating all the lessons we’ve learned.

Raw Differential Drive

Basic Motor Test for SGVHAK Rover

Our first opportunity to deploy the new Bootstrap-based touch-friendly HTML UI was a basic but critical motor test. A RoboClaw motor controller utilizes encoder feedback to know what the motor is doing. Based on this feedback, logic within the controller adjusts its electrical output to deliver either a commanded velocity or a commanded position. For any of that to work, an encoder must be installed properly. Our rover’s travelling wheel motors incorporated a pair of LED+photo diode units as optical quadrature encoders. Since we have a pair of sensors, there are two possible ways to wire them to RoboClaw input: one of which is correct and other is backwards.

To verify wiring, we need a basic motor test that will instruct RoboClaw to drive its motor at some power level, which is something it can do without encoder input. Then the test will query for encoder count and verify positive motor motion increments encoder count. If positive motion decrements encoder count, we’ll need to swap some wires.

Which wires need swapping? That depends on which way our motor is turning. We want positive motor rotation to correspond to forward rover motion, which means the left and right sides of the rover will be wired in mirror image. For rover’s right side wheels, positive motion will be clockwise (when viewed from rover’s right) and rover left wheels will be counter-clockwise. This desire to see the wheel motion in context of rover chassis is why we wanted this motor test to be usable on board the rover itself via the Raspberry Pi touchscreen.

If positive motor motion is rotating in the direction we want, and we need to swap wires, then we swap the encoder wires to maintain motor direction. If we need to swap wires and our motor is spinning in the wrong direction, we leave the encoder wires alone and swap our motor control wires instead. If the direction is wrong, and encoder is in sync, we’ll need to swap both sets of wires.

Basic Motor Test

HTML UI work described in this post began with this commit visible on Github.

RoboClaw HTML UI Work Assisted By API Stub

It was great when RoboClaw control UI project picked up help from somebody who knew what they were doing with web design. But it exposed a problem: the code is dependent on an actual RoboClaw motor controller connected to the machine. This was fine when development work focused on how to talk to the controller and hear what it has to say in return. But now that we have a parallel track working on improving the UI appearance, the RoboClaw dependency is unnecessary.

import roboclaw_stubTo remove this dependency, we introduced a RoboClaw API stub which could stand in for a physical RoboClaw when iterating through UI work. To do this in languages like C++, we would have to declare implementation of an interface and write a body. This makes sense when we expect all implementations of an interface to have roughly equivalent levels of functionality. For a test stub like this, though, that is not true. We only needed the small subset necessary to allow development of the specific relevant UI component. In languages like C++ this means most of the interface implementation would be tedious fillers that do nothing but raise a “Not Implemented” error.

Fortunately we are working in Python, which does not have such rigorous enforcement of interface implementations. We could implement just the methods we actually need to unblock UI development, and stop. There’s no need to recreate everything and we could leave out all superfluous “Not Implemented” declarations.

This is one arena where “duck typing” works in our favor, letting us get away with skipping all formalities of statically typed languages. There are downsides to dynamic binding but today it is a win.

Make RoboClaw HTML Control UI Scale Across Different Screen Types with Bootstrap

The basic HTML UI we were using to learn how to work with a RoboClaw motion controller was bare-bones web 1.0 in style. This worked very well when it only had to let us experiment to understand the controller while it’s sitting on a test stand next to the development workstation computer.

Once we got ready to move beyond tethered desktop computer scenarios, though, we had a problem. We had two touchscreen scenarios: one for UI directly on the robot via the Raspberry Pi Foundation touchscreen, the second scenario is remote control using a cell phone or tablet’s web browser.

Because the UI content was written in bare HTML, it does not scale to touchscreen devices the way we expect all modern web sites to do. In order to make this happen, we had to bring our HTML from the relative stone ages to conform to current practices.

Fortunately, this project received help from someone who has recently completed training on web design and development. [Amy] volunteered to jump in and make the controller HTML presentable and usable on touchscreen devices. Thanks to the open source nature of the web, most of the hard work has already been done by others and we just need to pull in one of the many freely available libraries to help build an UI that can respond to changes in the web browser environment.

For this experiment, [Amy] pulled in Bootstrap then began retrofitting the control test UI so Bootstrap can perform its magic across the different types of screens.

 

Link in Bootstrap

Refining Motor Parameters Applicable to SGVHAK Rover

Once we had a basic HTML UI for tuning RoboClaw PID parameters, we started exploring remaining surface area of RoboClaw API. In the arena of motor control, we have three categories of motion to choose from:

  1. PWM or duty cycle: This is the most basic type of control, where the motor is given a particular amount of power regardless of its load. If the motor is unloaded, it turns faster. If there is load, the power level does not change so motor slows down (or stalls). Since the quadrature encoder is not involved in this mode, it is particularly useful when diagnosing problems and we want to eliminate encoder error as a potential problem. There’s also no PID turning required for this mode to work, so it is also useful when trying to verify whether a problem is caused by bad PID gains or not.
  2. Velocity: In this mode, quadrature encoder activity matters. RoboClaw monitors encoder pulses and adjusts motor power to maintain the commanded velocity. If there is load of the motor, its slowdown will be apparent via encoder pulses and RoboClaw adds power to maintain commanded velocity. And conversely, when motor load is reduced and it starts speeding up, RoboClaw will see this in the encoder pulses and cut power to maintain commanded velocity. This is the mode we’ll use to control the wheel motors’ rolling travel. It requires a properly functioning encoder, at a high enough resolution, plus reasonably tuned velocity PID parameters.
  3. RoverTestConfig

    Position: In this mode, RoboClaw is told to move to a specific position and hold there. If external forces move the motor off its commanded position, RoboClaw will increase power to return to position and hold there. We will be using this mode to command motors in charge of wheel steering angle. In addition to the encoder requirement and velocity PID, there is a second set of position PID and both sets need to be tuned for a particular configuration.

Getting our bearings with RoboClaw motor control, we would first explore a tuning API from the command line then incorporate it into our HTML-based UI so we could rapidly iterate through parameter values.

The many iterations of this exploratory and tuning adventure (including all the mis-steps along the way) are publicly visible on Github.

Web-Based RoboClaw Control For Test and Configuration

The first step in the journey to a HTML-based rover UI is to prove we can write something in Flask to allow a web browser to send command to a RoboClaw motion controller. The next step is to build something useful. Early on in SGVHAK rover project, that meant a technical tool to configure and test motor control parameters.

The goal of this tool is to simplify tasks done earlier via typing RoboClaw API commands into the interactive Python shell. As part of proving the browser interface concept, we’ll also try to make the interface usable from a touchscreen device. More point-and-click, less typing on a keyboard. We’re not putting too much effort into making the user experience pretty and polished, though. This is an engineering tuning and testing tool, not an end-user tool.

The advantages of using a web framework immediately paid off in the ease of building sets of motor controls. It allowed rapidly iterating through ideas to find the feature set that would be useful and discarding features that are not. There’s little wasted effort as we’re not worried about making things look good and are OK with primitive web 1.0 forms.

One downside is not the fault of a web framework, it is more a consequence of trying to fit a square peg in a round hole. Web apps are fundamentally focused on supporting large numbers of users, and here we’re really only concerned about controlling a single resource: the RoboClaw. Flask is built around some patterns intended to help developers write code that scales to high number of users & sessions. But in the context of this single-user, single-session usage, such patterns sometimes got in the way.

Another downside is a web-based interface inherits all the problems of the web. Inputs sent by a client is inherently untrustworthy and must be properly validated before processing. The web framework can help handle some of that with automatic protection of cross-site scripting attacks, but the developer is ultimately responsible for sanitizing the input before processing. In the context of an engineering tool with short expected useful life, it’s a problem we can set aside for now.

RoverTestConfig

The proof of concept code discussed in this blog post is available on GitHub: testconfig.py under https://github.com/Roger-random/RogerPiBot

SGVHAK Rover Control With Flask, a Web Server.

Recent blog posts outlined the hardware side of SGVHAK rover project, now we’ll rewind and tell the software side. The first drop of information for the SGVHAK rover team included only information on hardware, the baseline rover software was promised and yet to come. We didn’t think too much of it until the Microsoft Windows-based Ion Studio tool failed to auto-tune the PID parameters for our rover wheel.

The effort to debug rover wheel control led to the Python API released by Ion Motion Control to communicate with Roboclaw motion control board. As a platform-independent tool, Python was always the ultimate intent since our target platform for rover brain is a Raspberry Pi. The failure of Ion Studio just advanced the timeline.

The manual PID tuning session was done entirely in the Python interactive shell calling into the Python API, from updating PID values to commanding motor motion. While flexible and effective, all the typing became very repetitive very quickly.  Now that we’re past the first PID tuning work session, we have a decent idea what APIs are most useful for this project.

We didn’t set out to design a new rover UI, originally we just wanted to ease building the machine and tuning parameters. This nuts-and-bolts goal is why investigation started with a text-based system. An advantage of a simple command line based system is that it can be used wirelessly via SSH so that we won’t have to be tethered to the robot while we tune. This is why we started by looking into Python’s curses library to create text-based displays.

It didn’t take long, though, before ambitions grew. The ultimate UI for the rover won’t be text-based, why learn a whole new text UI framework just so we’d have to learn a graphical UI framework later? That thought was just the first step. The next step looked at Qt, which is a cross-platform GUI framework we already know to work across PC, Macintosh, and Pi. But wireless networked access to a Qt GUI takes some work. Far more than it would take to put up some HTML to be accessed over the network. So… why not do that instead?

That’s how we arrived at today’s investigation: for a simple rover UI, try using HTML. This allows us to have the exact same UI over the network as well as locally. (http://localhost) We’re learning something new anyway, and learning a Python web framework is more likely to be useful for future projects than learning a text UI framework. To kick off this effort, we ask two questions:

  1. What framework to use? There are a lot of different Python web frameworks running around on the internet. While learning Ruby on Rails, I came across references to its Python counterpart Django. They are both far heavier than we need for this project, though. A search for something more lightweight pointed to Flask, backed up by at least one Quora discussion.
  2. Does it work with Roboclaw API? It took a little fiddling with user groups to gain access to the USB serial port, but once that was done the answer is: Yes it does.

With a successful smoke test, we’re going ahead with building a HTML-based rover UI.

HelloFlask Success

Launching a Flask Web App on Startup

flaskThe first few experiments with Flask has been successful. A simple user interface can be created in HTML and accessed from local as well as remote clients over HTTP. One of the downsides bit us today: a web app isn’t as easy to launch as a native local application. When running locally, we still need to get two parts up and running: the server-side code and the web browser pointing to http://localhost.

raspberry-pi-logoThis is especially problematic when the desire is to launch the app on power-up of the Raspberry Pi 3 that will run the server side. “Run on startup” is one of those tasks that have many different approaches across different flavors of Linux. Each with their advantages and disadvantages, benefits and gotchas.

The first attempt is to write a shell script that launches the Flask web server and the Chromium web browser. This works when executed interactively at the command line logged in as the default user pi. But it didn’t work when executed as a cron @reboot job, probably something to do with the fact it would run as root.

The next effort led to Raspberry Pi documentation, which pointed to the file /etc/rc.local. We seem to have better luck here – the Flask web server does launch but Chromium web browser still did not.

Further hunting eventually found this page for setting up a Raspberry Pi running Chromium web browser as a web kiosk. Not quite the project at hand, but close enough for the answer to be applicable. This meant leaving Flask startup in /etc/rc.local and editing the file /home/pi/.config/lxsession/LXDE-pi/autostart to have it launch chromium-browser with the parameter --kiosk http://localhost:5000/

Upon power-up, this Pi will now launch Flask web server and Chromium web browser to point at that server running on the same device. It works, but it’s not clear why these hoops were necessary. It’s pretty bewildering for a beginner to try to understand the why behind all these different startup mechanisms. A few hours of reading (sometimes contradictory) documentation did not make the situation any more clear. Sadly, time constraints dictate that we can’t get to the bottom of this issue today. Perhaps later.

Exploring Flask, a Python Web Framework

flaskToday’s exploration centers around Flask, a framework for building web sites with the Python programming language. The motivation was a new project that will be discussed separately, the important detail for now is that it needed to interact with some existing Python libraries and desired a user interface accessible from multiple different (and separate) devices. It seemed to make sense to turn the central device into a web server and expose the user interface to its clients using HTML.

There had been previous explorations into writing web server software using Ruby on Rails, which is not the best fit for this project. The first issue is the language – we need to talk to a Python library and there’s no need to jump through language translation layers for a simple project. The second issue is complexity – Rails is a full MVC framework for building an app on a database and we don’t need that here, either.

If we wanted the Python equivalent to Ruby on Rails, the popular choice of the day is Django. But since we don’t, Flask is the lighter-weight framework we’ll play with today. Sometime in the future, when a project is right for Django, we’ll explore using the right tool for the job.

Flask is a thin layer that’s a short step above having to call Python’s HTTP and URL classes directly, and can be very quick to get started.  A simple Hello World type of app requires only a single Python file and a few lines of code, no database configuration and project generation necessary. The developer builds up from this simple foundation. This is great for keeping things simple and easy to work with. For the most part, features that a developer would want can be added piecemeal.

The downside is that Flask also skips out on a lot of web practices that aren’t explicitly features, but are very nice to have by default. For example, the default Ruby on Rails generator for HTML <form> includes a token to prevent cross-site request forgery. This is not explicitly a feature a developer would think of adding, and could easily get overlooked especially when the forms are generated manually. Fortunately most of the Flask add-ons to generate forms – like this one – automatically includes CSRF protection.

Some caution would be warranted if the Flask application is going to be a public web site, but for a local network project prototype it should be good enough.

Twitter API Newbie Wades Into The Pool

Twitter DeveloperTwitter trends are easy to follow by searching for the hashtag. The default view given by Twitter shows the most popular tweets with that tag, and there are options to see the most recent tweets first, or browse them by photo, etc. But what if someone wants the tweets organized another way? Fortunately Twitter knows they can’t cover everybody’s desires and offers a developer API for those with creative ideas on tweets.

Twitter’s API is a “RESTful” design. It follows many of the ideas of REpresentational State Transfer (REST) but not strictly adhering to all of it. Functionally, it is built around HTTP GET and POST requests and returns information in the text-based JavaScript Object Notation (JSON) format. The details of those HTTP requests – and the JSON that it retrieves – are documented on Twitter’s developer site.

Before the aspiring Twitter app author can get anywhere, though, one needs to register with Twitter for access to these APIs. Once an app is registered, it is assigned a set of secret access keys and tokens. This allows Twitter to monitor usage and limit/disable Twitter access by individual applications.

Once registered, the code experimentation can begin. As a RESTful API communication via JSON, any web-friendly development framework can be used. (And if it is not web friendly… why would it have anything to do with Twitter?) As a starting point, Twitter has a non-exhaustive list of libraries that cater to different programming environments.

Out of this list, Python-Twitter was chosen as the first step into this world. Python language is fun and easy to experiment with. Python-Twitter installs easily via Python’s pip package manager. And yes – it also happens to be the first entry on the list.

A few commands into Python interactive console was rewarded with immediate results, though sadly also a few issues. Text of some tweets were truncated. Perhaps this has something to do with the recent increase in Twitter’s character limit? Whatever the root cause, it is tracked as an open issue in Python-Twitter’s repository.

The first few search queries returned only 15 results, even when there were definitely more than 15 answers. This turned out to be the default limit which is easily increased up to 100 results per search query. Going any further, unfortunately, is not possible with the standard search API. As of a few months ago, the only option to do more is to subscribe to the very expensive Twitter Enterprise API. This is out of the question for a hobbyist! Fortunately people have already complained and this past November Twitter responded with a premium search API to bridge the gap. Even though it is free, access is restricted so a registered developer has to apply for access.

Even if access is obtained, premium search is a new feature accessible via a new endpoint that is not yet supported by any of the Python libraries on Twitter’s list. Searching with a unique part of the API URL (“30day”) found this Python search tweet API that might be worth a look. If it doesn’t fit the bill for whatever reason, it might be time to build a custom solution.

Fusion 360 Lines Created Via API Are Not Limited To Sketch Plane

Cube on Sketch

Today’s performance tip helps avoid creating unnecessary planes and sketches when creating objects using the Fusion 360 API. First, some background: when sketching lines in Fusion 360 must be done in the context of a sketch object. This sketch object is in turn created to be aligned with a plane. It may be one of the default planes that intersect the origin (XZ plane, XY plane, or YZ plane) or a construction plane.

When sketching with the interactive UI, the user’s sketch entities are constrained to the plane of the sketch. For example, if a sketch was created on the XY plane, then all sketch entities would have a Z coordinate of zero. This makes sense to keep the data organized and easy to understand by the human looking at a 2D computer monitor.

When creating entities with the API, though, they are apparently not constrained. The first clue came from looking at the SketchLines.addByTwoPoints API – its two parameters are Point3D objects. A little experimentation confirmed the API would gladly accept 3D coordinates that lie outside the sketch plane. The example screenshot on this page is the result of drawing an one-centimeter cube on a sketch aligned to the XY plane. Only four of the eight corners (and four of the 12 edges) lie on the plane, but the API happily accepted all the information for a 3D cube.

This is very useful when creating geometries via code. Humans need to see points on a sketch stay in a plane for them to be workable on a 2D monitor screen, otherwise things get too confusing. But code doesn’t care about appearance on a screen. So go ahead – let code create complex 3D entities on a single sketch. Skipping the unnecessary planes and sketches will generate fewer API objects. Reducing object management overhead and allowing faster performance.

Accelerate Fusion 360 API Object Creation With DirectDesignType

64Cylinders

Exploring the capabilities of the Fusion 360 API will involve experiments creating objects via script. Since it’s all code, it’s easy to repeat a process many times, a natural allure of using scripts to automate repetitive tasks. The downside is that Fusion 360 (as of time of writing) has problems resulting in slow response.

The sample code below creates an array of small cylinders one at a time. (This is not the most efficient method to create an array of cylinders but merely an illustration of the problem.) The size of the array can be adjusted by changing the value of arraySize before running the script. For a 4×4 array, the results are almost instantaneous. On a modern Core i5, slowdowns can be observed in an 8×8 array. The first few cylinders are fast, but as additional cylinders appear, each new cylinder takes longer than the last.

Fusion 360 performs a lot of bookkeeping overhead to track these operations. While the majority are outside our control, we do have control over the most expensive overhead: the timeline. It is useful when we are working with hand-created designs, but unnecessary for script – if we want to change anything, we’d update the script and run it again. In the user interface, we can click on the gear in the lower right hand side and select “Do not capture Design History”. To do the same thing in script, we can set the type of the design object to DirectDesignType. (Uncomment line in sample code below.)

This mitigates but does not solve the problem. While it is now practical to generate far larger number of objects, there is still a noticeable slowdown as the object counts go up. According to this forum post, this issue has been logged as a defect for the development team. Perhaps a future release of Fusion 360 will create the last object as quickly as it created the first.

import adsk.core, adsk.fusion, adsk.cam, traceback

arraySize = 8 # Will extrude arraySize^2 objects

def run(context):
  ui = None
  try:
    app = adsk.core.Application.get()
    ui = app.userInterface

    app.documents.add(adsk.core.DocumentTypes.FusionDesignDocumentType)

    # adsk.fusion.Design.cast(app.activeProduct).designType = adsk.fusion.DesignTypes.DirectDesignType # Uncomment to turn off tracking timeline

    rootComp = adsk.fusion.Design.cast(app.activeProduct).rootComponent
    sketch = rootComp.sketches.add(rootComp.xYConstructionPlane)
    extrudeDistance = adsk.core.ValueInput.createByReal(1)

    rangeX = range(1-arraySize, arraySize, 2)
    rangeY = range(1-arraySize, arraySize, 2)

    for nowX in rangeX:
      for nowY in rangeY:
        sketch.sketchCurves.sketchCircles.addByCenterRadius(
          adsk.core.Point3D.create(nowX, nowY, 0), 0.5)

        rootComp.features.extrudeFeatures.addSimple(
          sketch.profiles[-1],
          extrudeDistance,
          adsk.fusion.FeatureOperations.NewBodyFeatureOperation)

        adsk.doEvents()

except:
  if ui:
    ui.messageBox('Failed:\n{}'.format(traceback.format_exc()))

This code is also available on Github.