SGVHAK Rover Steering Trim Adjustment

One of the changes we made to our SGVHAK Rover relative to the baseline design was a decision to omit absolute encoders for our corner steering motors. The baseline design used absolute encoders so it knows every wheel’s steering angle upon system startup, but we decided to sacrifice that ability in exchange for cost savings (~$50/ea * 4 corners = ~$200.) Since there is no free lunch, this decision also means we have additional work to do upon our system startup.

Our control software launches with the assumption that all steerable wheels are pointed straight front-back, which is of course not always true. We could manually push the wheels to straight front-back orientation before turning on power, but that’s not great for the gearbox. Also, the shaft coupler used to connect our motor gearbox to our steering mechanism is liable to slip. (This is considered a bug even though it has a potential feature: in theory sharp jolts to the system would slip the coupler instead of breaking something else in the system. In practice, we found that it didn’t slip enough to avoid breaking a gearbox.)

Given both of these scenarios, we need a software-based solution to adjust steering trim on the fly. Once our RoboClaw motor controller is told where zero degree position is, it is quite capable of holding a commanded angle until the coupler slips or the system reboots, whichever comes first.

Steering Trim

Steering trim adjustment took the form of a multi-stage HTML form designed for our specific workflow. First our user chooses a wheel to work with, and they are directed to do so because every other UI element on the page is inactive until one wheel is chosen.

Once a wheel is chosen, the remaining UI activates and wheel selection UI deactivates to guarantee we’re working on a single wheel at a time. In this stage, steering angle can be adjusted in 1- and 5-degree increments. When we’re satisfied our wheel has been returned to zero degrees deflection, we can select “Accept Current Angle as New Zero” to commit. We also have the option to abort adjustment and retain previous zero position. Either way, wheel selection activates to allow user to select another wheel, and the rest of the page deactivates until a wheel is chosen.

We have no scenarios where multiple steering motors need to be trimmed at the same time, so this user experience specifically focus on one wheel at a time makes the process straightforward. It also becomes impossible to make mistakes caused by user thinking they’re adjusting wheel when they’re actually adjusting another, which is a good thing for positive user experience.

SGVHAK Rover Control With Cartesian Coordinates

Having learned some valuable lessons from trying to drive our SGVHAK rover using an interface based on polar coordinates, we returned to the drawing board to create an alternative. Thankfully, the abstracted command structure meant we could experiment with front-end user interface without any changes to the back-end rover calculations.

Our first-hand experience taught us polar coordinate math are more trouble than it was worth, so our next attempt will stay in the Cartesian coordinate space. Furthermore, we’ll rearrange the layout so a position going full speed ahead would never be adjacent to a position going full speed reverse. (This was the case for polar coordinate system when doing full deflection turns.) Third, we’ll have to make sure low speed maneuvering has just as much room to fine-tune its angle as going high speed. And lastly, when someone releases their finger out of caution, we need to stop the rover but we don’t need to reset steering angles.

It turned out to be very easy to satisfy all of the above desires: map X-axis left-right dimension to steering angle, and Y-axis up-down dimension to speed.

Cartesian Pad

Full speed ahead is now along the top edge of the blue rectangle, which is far from the bottom edge representing full speed reverse. There is no longer any risk of suddenly jerking between them.

At any given speed, we now have the full width for designating steering angle, we are no longer frustrated by a tiny range of adjustments when moving slow speed versus high.

And finally, when the user releases their finger, the dot will snap to vertical center (stop moving) but will remain at its most recent horizontal position, preserving the steering angle most recently used by our rover driver. This makes fine position adjustments much easier by allowing slow incremental inching forward/reverse movements.

Most users who tried both UI design prefer the Cartesian coordinate system. It was used for most of our driving at SCaLE 16X.

SGVHAK Rover Control With Polar Coordinates

Once an abstraction layer was defined for communication between user interface and rover chassis calculations, I could start trying out various ideas for building a rover driving UI. The baseline rover UI presents two touchscreen controls mimicking joysticks on a traditional remote control vehicle: one moves up/down for controlling forward/back movement, and the other moves left/right for controlling steering. Driving the baseline rover using its UI is a two-thumb affair.

That is a perfectly valid and functional UI, but my ambitions grew beyond copying it. I wanted to explore ideas around single-finger operation. This decision held firm even after the person who created the UI for baseline rover warned us that it could be tricky to implement – single-point operation was already tried before going with the two-thumb approach. I may yet bow to their wisdom, but I wanted to give it a shot first!

Polar Pad

The first implementation of a one-finger driving control is based around polar coordinates. It seemed like an obvious match to rover mechanics: polar coordinates on a two-dimensional plane is an angle and a distance, which maps directly to rover control’s angle and velocity. The user puts their finger down on the red central control dot, and they can drag it around inside the blue circle to drive the rover. Move the dot up to drive forward, rotate the dot left to steer left, and so forth.

As I got into it, though, problems started to mount.

The first and most obvious problem is that our underlying infrastructure – HTML, <canvas> tag, and JavaScript input events – use cartesian coordinates. In order to implement polar coordinate control we need trigonometry math to convert between the two systems. Errors creep in as we perform multiple operations and conversions between coordinate spaces.

The second problem was a consequence of mapping upper semicircle to forward velocity and lower semicircle to moving backwards. This became problematic when the user wishes to steer at maximum angle.  Turning far left means moving straight left, but at that point small movements can flip between upper and lower semicircle, causing the rover to jerk suddenly between forward and backward movement.

To mitigate this, a band of dead space between the semicircles was introduced. The user is no longer able to use the full 180 degrees of a semicircle for movement, they are restricted to the middle 140 degrees with the far left 20 and far right 20 degrees blocked off. This was effective to prevent sudden transitions between full speed forward/backwards motion but it wasted screen real estate.

The third problem of polar coordinates was exposed when trying to maneuver the rover at low-speed around obstacles. When our finger is near the center of the circle, small left-right movement translates into large changes in steering angle. It was more difficult than it should be to make small steering changes at low speed.

The last problem is makes the above problem worse. When somebody runs into a problem, their first instinct is to lift their finger off the touchscreen. In order to retreat to a known safe condition, our control dot pops back to the center position representing wheels stopped and all steerable wheels pointing straight front-back. This is fine, but when people are frustrated by problem #3 of low-speed maneuverability, this behavior snapping to straight front-back erases whatever angle they were previously driving at, making low-speed maneuvers even more frustrating.

And what happens when a rover driver is trying to perform delicate maneuvers but find themselves fighting a bad UI that gets in the way and cause frustration? The rover ends up running into things and risk breaking gearboxes.

Now we have experimental evidence polar coordinate driving doesn’t work as originally thought, let’s take that lesson and create a new UI.

 

SGVHAK Rover Control Abstraction

Once we were confident our SGVHAK rover chassis control code looked pretty good, we were willing to trust that it could drive a rover without sending conflicting commands that make our rover try to twist itself into a pretzel. Now our focus turns to creating an interface for user control input.

A quick rough draft had been in place to help test rover chassis Ackermann math, letting a rover driver send a steering angle and travel speed to rover’s control code. But it was very tied to the current rover implementation: angle range was limited to angle range of what we have today, and speed was specified in raw quadrature pulses per second (QPPS) that we send straight down to RoboClaw motion controller‘s API.

Since we have ambitions for future rover variants, this direct dependency is not good. Future rovers may have a wider (or narrower) range of valid steering angles. They may also use something other than RoboClaw motor controllers. The answer is a typical software solution: abstraction!

This required changing the interface between our HTML UI front-end and the Flask-based Python code running on the rover. Instead of a fixed number angle or a fixed number of encoder pulses, user now send driving commands in terms of two percentages ranging from negative 100% to positive 100%.

For steering, -100% tells a rover to steer as far left as it could go, 0% is straight forward/back travel, and 100% is to steer as far right as it could go.

For velocity, -100% is travelling backwards as fast as it could, 0% is stop, and 100% is damn the torpedoes, full speed ahead.

Now when we adapt the code for future rover variants, we could focus on rover-side configuration and remain confident user-side HTML interface will automatically adapt to whatever capabilities are present on a particular rover variant.

This also allows us to experiment with user-side HTML without worrying about having to make matching changes in rover-side Python code. We can try different control schemes, as long as it ends up sending a velocity and steering angle percentage to the rover.

The first and crudest implementation of this concept is shown here: two HTML slider objects that move from -100% to 100%. The user can move the slider and tap “Send command” to rover. As the most basic HTML implementation possible, this is a useful test tool to isolate whether a particular control problem is in HTML front-end or in Python back-end, but it is obviously not a great driving interface.

That’s coming up next.

Velocity Angle

Visualizing SGVHAK Rover Chassis Calculations

When we were putting together code to calculate Ackermann steering angles for SGVHAK rover‘s wheels, initial testing was done via Python command line using some fixed test values to match against results pre-calculated by hand. Once we built some confidence our math was on the right track, it was time to start testing against more varied inputs.

There was just one minor problem – at this point in rover development, our software is a tiny bit ahead of the hardware which means we can’t test on hardware that hasn’t yet been assembled. Even if we could, it wouldn’t be a great idea to jump from a small fixed set of test values to actual hardware. As an intermediate step, we created a bit of code to visualize results of roverchass.py calculations: the “Chassis Configuration” screen will show what our control code thinks all six wheels should be doing. (This shows the intent, which is different from hardware telemetry of what’s actually happening.)

SGVHAK Rover Chassis Config

Each of the six wheels got its own blue box, laid out roughly analogous to their physical position on the rover chassis. Originally wheel steering & speed was displayed as text: number of degrees a wheel should be steered, and a second number reflecting the intended velocity for each wheel. This was accurate information but it took some effort to interpret. To make this information easier to understand, numerical text was changed to a visual representation.

By using HTML’s <canvas> element, we could draw some simple graphics. Each wheel is represented by a black rectangular outline. A wheel’s steering angle is now represented by tilting its rectangle to that angle. Inside a wheel’s rectangle is a green fill representing desired velocity. When a wheel is supposed to be stopped, there is no green fill and its black rectangle is empty. When a wheel is supposed to be rolling forward, green rectangle fills from the middle upwards. Faster velocity is represented by more green in the upper half. For a wheel that should be rolling backwards, green fills downwards instead.

This visualization of roverchassis.py calculation results were far easier for humans to understand. This made it very quick to validate that roverchassis is behaving roughly correctly. There’s a place for precise engineering calculations, but there’s also place for a quick glance to check “yeah, that looks about right.”

SGVHAK Rover Software: Chassis Calculations in Python

SGVHAK Rover Chassis ConfigOne of the biggest changes in our new rover controller code is the goal of managing all ten rover chassis motors together, starting to think of the rover as a single unit instead of ten individual things.

This logic is centered in roverchassis.py, which is be responsible for calculating Ackermann angles for all steerable wheels in the rover, as well as calculating the desired travel velocity for each wheel. In order to make this code as flexible as we can, we offload details of rover chassis into configuration files and make our Python math as generic as possible running calculations based on configuration file values. This meant we could support rovers with different geometries without editing source code. This feature began with a desire to have flexible wheelbase and track dimensions of our rover chassis, but eventually extended to supporting similar but different configurations such as two-wheel differential drive.

Another goal was to support different types of motors and motor controllers. We appreciated all the features in the RoboClaw motor controller we started with, but we also thought about trading off some of these features for lower cost. In preparation for this flexibility, we kept roverchassis.py focused on general Ackermann-related math and kept motor-specific code in a separate class. In the first draft, this meant pulling all RoboClaw related code into roboclaw_wrapper.py which translates general concepts from roverchassis.py into specific commands in Ion Motion Control’s public domain API roboclaw.py. Later on we validated this flexibility by steering a single wheel with a RC servo followed by adding support for LewanSoul serial bus servos.

This was all possible using a flexible syntax in config_roverchassis.json that allows us to adjust wheel dimensions as well as switch out individual wheel travel or steering motors for different motor controllers.

 

 

A Fresh Start for SGVHAK Rover UI

SGVHAK Rover Chassis ConfigWhen our SGVHAK rover project started moving beyond individual units of our custom wheel gearbox design and towards a multiple wheel chassis, it was happening on both the hardware and software fronts. Our Flask-based control UI has worked well up to this point, allowing us to tune RoboClaw motor controller parameters to suit our wheel design. Building this first draft UI taught us a lot, and as we learned we also realized how it reflected our ignorance at the beginning of this project.

And so we bid goodbye to our first draft rover UI RogerPiBot to start a new project SGVHAK_Rover. This way we retain all the testing & tuning capabilities of the existing software, leaving that as-is. Freeing us to create a new user experience.

The major changes are:

  1. Simple touch-friendly controls instead of HTML forms with a lot of fields. The form was a very powerful tool for tuning parameters from a desktop workstation, but we’re getting out of the engineering test phase. All interfaces will now be tailored towards operation from a small touchscreen.
  2. Rover as a unit instead of wheel as a unit. We need to start treating all six travel motors and four steering motors aboard our chassis as a single unit, as the user expects the software to do. Target audience for this software will not be interested in poke and prod at motors individually.
  3. Configuration files instead of source code edits. The new software is built on the assumption that will be people who want to use it but is not familiar with Python or code in particular. Part of this meant configuration information will now be stored in human-readable JSON files at the root directory instead of asking people to go in and edit Python source code.

There were various other changes big and small. One of the larger changes was separate from the rover itself: we chose to switch away from using Bootstrap purely for the sake of HTML creation exercise. This time around we’re using a different library called Materialize to help create our responsive UI.

Unfriendly UI For Two Wheel Test Chassis

Our first attempt at rover software was a wonderfully informative hands-on adventure on creating a path from a cell phone’s web browser HTML all the way to serial bus commands sent to a RoboClaw motor controller. Fortunately or unfortunately, these lessons also put a finite lifespan to the software as we start to discover basic mistakes that we didn’t know we were making until we got a little smarter.

Before we leave the first draft behind, however, it had one final role to play. This point in software development timeline coincided with our rover wheel development’s completion. The actual six-wheeled rover chassis is still in the future, but our wheel development is mature enough for us to build a test chassis. Two wheel units were assembled and integrated into a bare-bones test chassis.

This chassis needed a brain and a controller.

To enable motor control via differential drive, we hacked together a minimal user interface for sending commands to our test chassis. Not much effort was spent on user-friendliness and its layout is very much an engineering number cruncher’s idea of an interface. (Also referred to as “bad UI.”) It was just enough to let us move the robot around and uncover some finishing issues in the wheel design, but not much more than that.

The robot moving under its own power attracted attention from other people in the work space. Naturally some asked if they could try driving the robot but one look at the interface and the response was “Um… maybe I’ll wait for a later version” and the phone was handed back.

No offense taken, it’s another vote in favor of moving on. We’ll close out the “RogerPiBot” project at this point and start a new rover control project incorporating all the lessons we’ve learned.

Raw Differential Drive

RoboClaw HTML UI Work Assisted By API Stub

It was great when RoboClaw control UI project picked up help from somebody who knew what they were doing with web design. But it exposed a problem: the code is dependent on an actual RoboClaw motor controller connected to the machine. This was fine when development work focused on how to talk to the controller and hear what it has to say in return. But now that we have a parallel track working on improving the UI appearance, the RoboClaw dependency is unnecessary.

import roboclaw_stubTo remove this dependency, we introduced a RoboClaw API stub which could stand in for a physical RoboClaw when iterating through UI work. To do this in languages like C++, we would have to declare implementation of an interface and write a body. This makes sense when we expect all implementations of an interface to have roughly equivalent levels of functionality. For a test stub like this, though, that is not true. We only needed the small subset necessary to allow development of the specific relevant UI component. In languages like C++ this means most of the interface implementation would be tedious fillers that do nothing but raise a “Not Implemented” error.

Fortunately we are working in Python, which does not have such rigorous enforcement of interface implementations. We could implement just the methods we actually need to unblock UI development, and stop. There’s no need to recreate everything and we could leave out all superfluous “Not Implemented” declarations.

This is one arena where “duck typing” works in our favor, letting us get away with skipping all formalities of statically typed languages. There are downsides to dynamic binding but today it is a win.

Refining Motor Parameters Applicable to SGVHAK Rover

Once we had a basic HTML UI for tuning RoboClaw PID parameters, we started exploring remaining surface area of RoboClaw API. In the arena of motor control, we have three categories of motion to choose from:

  1. PWM or duty cycle: This is the most basic type of control, where the motor is given a particular amount of power regardless of its load. If the motor is unloaded, it turns faster. If there is load, the power level does not change so motor slows down (or stalls). Since the quadrature encoder is not involved in this mode, it is particularly useful when diagnosing problems and we want to eliminate encoder error as a potential problem. There’s also no PID turning required for this mode to work, so it is also useful when trying to verify whether a problem is caused by bad PID gains or not.
  2. Velocity: In this mode, quadrature encoder activity matters. RoboClaw monitors encoder pulses and adjusts motor power to maintain the commanded velocity. If there is load of the motor, its slowdown will be apparent via encoder pulses and RoboClaw adds power to maintain commanded velocity. And conversely, when motor load is reduced and it starts speeding up, RoboClaw will see this in the encoder pulses and cut power to maintain commanded velocity. This is the mode we’ll use to control the wheel motors’ rolling travel. It requires a properly functioning encoder, at a high enough resolution, plus reasonably tuned velocity PID parameters.
  3. RoverTestConfig

    Position: In this mode, RoboClaw is told to move to a specific position and hold there. If external forces move the motor off its commanded position, RoboClaw will increase power to return to position and hold there. We will be using this mode to command motors in charge of wheel steering angle. In addition to the encoder requirement and velocity PID, there is a second set of position PID and both sets need to be tuned for a particular configuration.

Getting our bearings with RoboClaw motor control, we would first explore a tuning API from the command line then incorporate it into our HTML-based UI so we could rapidly iterate through parameter values.

The many iterations of this exploratory and tuning adventure (including all the mis-steps along the way) are publicly visible on Github.

Web-Based RoboClaw Control For Test and Configuration

The first step in the journey to a HTML-based rover UI is to prove we can write something in Flask to allow a web browser to send command to a RoboClaw motion controller. The next step is to build something useful. Early on in SGVHAK rover project, that meant a technical tool to configure and test motor control parameters.

The goal of this tool is to simplify tasks done earlier via typing RoboClaw API commands into the interactive Python shell. As part of proving the browser interface concept, we’ll also try to make the interface usable from a touchscreen device. More point-and-click, less typing on a keyboard. We’re not putting too much effort into making the user experience pretty and polished, though. This is an engineering tuning and testing tool, not an end-user tool.

The advantages of using a web framework immediately paid off in the ease of building sets of motor controls. It allowed rapidly iterating through ideas to find the feature set that would be useful and discarding features that are not. There’s little wasted effort as we’re not worried about making things look good and are OK with primitive web 1.0 forms.

One downside is not the fault of a web framework, it is more a consequence of trying to fit a square peg in a round hole. Web apps are fundamentally focused on supporting large numbers of users, and here we’re really only concerned about controlling a single resource: the RoboClaw. Flask is built around some patterns intended to help developers write code that scales to high number of users & sessions. But in the context of this single-user, single-session usage, such patterns sometimes got in the way.

Another downside is a web-based interface inherits all the problems of the web. Inputs sent by a client is inherently untrustworthy and must be properly validated before processing. The web framework can help handle some of that with automatic protection of cross-site scripting attacks, but the developer is ultimately responsible for sanitizing the input before processing. In the context of an engineering tool with short expected useful life, it’s a problem we can set aside for now.

RoverTestConfig

The proof of concept code discussed in this blog post is available on GitHub: testconfig.py under https://github.com/Roger-random/RogerPiBot

SGVHAK Rover Control With Flask, a Web Server.

Recent blog posts outlined the hardware side of SGVHAK rover project, now we’ll rewind and tell the software side. The first drop of information for the SGVHAK rover team included only information on hardware, the baseline rover software was promised and yet to come. We didn’t think too much of it until the Microsoft Windows-based Ion Studio tool failed to auto-tune the PID parameters for our rover wheel.

The effort to debug rover wheel control led to the Python API released by Ion Motion Control to communicate with Roboclaw motion control board. As a platform-independent tool, Python was always the ultimate intent since our target platform for rover brain is a Raspberry Pi. The failure of Ion Studio just advanced the timeline.

The manual PID tuning session was done entirely in the Python interactive shell calling into the Python API, from updating PID values to commanding motor motion. While flexible and effective, all the typing became very repetitive very quickly.  Now that we’re past the first PID tuning work session, we have a decent idea what APIs are most useful for this project.

We didn’t set out to design a new rover UI, originally we just wanted to ease building the machine and tuning parameters. This nuts-and-bolts goal is why investigation started with a text-based system. An advantage of a simple command line based system is that it can be used wirelessly via SSH so that we won’t have to be tethered to the robot while we tune. This is why we started by looking into Python’s curses library to create text-based displays.

It didn’t take long, though, before ambitions grew. The ultimate UI for the rover won’t be text-based, why learn a whole new text UI framework just so we’d have to learn a graphical UI framework later? That thought was just the first step. The next step looked at Qt, which is a cross-platform GUI framework we already know to work across PC, Macintosh, and Pi. But wireless networked access to a Qt GUI takes some work. Far more than it would take to put up some HTML to be accessed over the network. So… why not do that instead?

That’s how we arrived at today’s investigation: for a simple rover UI, try using HTML. This allows us to have the exact same UI over the network as well as locally. (http://localhost) We’re learning something new anyway, and learning a Python web framework is more likely to be useful for future projects than learning a text UI framework. To kick off this effort, we ask two questions:

  1. What framework to use? There are a lot of different Python web frameworks running around on the internet. While learning Ruby on Rails, I came across references to its Python counterpart Django. They are both far heavier than we need for this project, though. A search for something more lightweight pointed to Flask, backed up by at least one Quora discussion.
  2. Does it work with Roboclaw API? It took a little fiddling with user groups to gain access to the USB serial port, but once that was done the answer is: Yes it does.

With a successful smoke test, we’re going ahead with building a HTML-based rover UI.

HelloFlask Success

Launching a Flask Web App on Startup

flaskThe first few experiments with Flask has been successful. A simple user interface can be created in HTML and accessed from local as well as remote clients over HTTP. One of the downsides bit us today: a web app isn’t as easy to launch as a native local application. When running locally, we still need to get two parts up and running: the server-side code and the web browser pointing to http://localhost.

raspberry-pi-logoThis is especially problematic when the desire is to launch the app on power-up of the Raspberry Pi 3 that will run the server side. “Run on startup” is one of those tasks that have many different approaches across different flavors of Linux. Each with their advantages and disadvantages, benefits and gotchas.

The first attempt is to write a shell script that launches the Flask web server and the Chromium web browser. This works when executed interactively at the command line logged in as the default user pi. But it didn’t work when executed as a cron @reboot job, probably something to do with the fact it would run as root.

The next effort led to Raspberry Pi documentation, which pointed to the file /etc/rc.local. We seem to have better luck here – the Flask web server does launch but Chromium web browser still did not.

Further hunting eventually found this page for setting up a Raspberry Pi running Chromium web browser as a web kiosk. Not quite the project at hand, but close enough for the answer to be applicable. This meant leaving Flask startup in /etc/rc.local and editing the file /home/pi/.config/lxsession/LXDE-pi/autostart to have it launch chromium-browser with the parameter --kiosk http://localhost:5000/

Upon power-up, this Pi will now launch Flask web server and Chromium web browser to point at that server running on the same device. It works, but it’s not clear why these hoops were necessary. It’s pretty bewildering for a beginner to try to understand the why behind all these different startup mechanisms. A few hours of reading (sometimes contradictory) documentation did not make the situation any more clear. Sadly, time constraints dictate that we can’t get to the bottom of this issue today. Perhaps later.

Exploring Flask, a Python Web Framework

flaskToday’s exploration centers around Flask, a framework for building web sites with the Python programming language. The motivation was a new project that will be discussed separately, the important detail for now is that it needed to interact with some existing Python libraries and desired a user interface accessible from multiple different (and separate) devices. It seemed to make sense to turn the central device into a web server and expose the user interface to its clients using HTML.

There had been previous explorations into writing web server software using Ruby on Rails, which is not the best fit for this project. The first issue is the language – we need to talk to a Python library and there’s no need to jump through language translation layers for a simple project. The second issue is complexity – Rails is a full MVC framework for building an app on a database and we don’t need that here, either.

If we wanted the Python equivalent to Ruby on Rails, the popular choice of the day is Django. But since we don’t, Flask is the lighter-weight framework we’ll play with today. Sometime in the future, when a project is right for Django, we’ll explore using the right tool for the job.

Flask is a thin layer that’s a short step above having to call Python’s HTTP and URL classes directly, and can be very quick to get started.  A simple Hello World type of app requires only a single Python file and a few lines of code, no database configuration and project generation necessary. The developer builds up from this simple foundation. This is great for keeping things simple and easy to work with. For the most part, features that a developer would want can be added piecemeal.

The downside is that Flask also skips out on a lot of web practices that aren’t explicitly features, but are very nice to have by default. For example, the default Ruby on Rails generator for HTML <form> includes a token to prevent cross-site request forgery. This is not explicitly a feature a developer would think of adding, and could easily get overlooked especially when the forms are generated manually. Fortunately most of the Flask add-ons to generate forms – like this one – automatically includes CSRF protection.

Some caution would be warranted if the Flask application is going to be a public web site, but for a local network project prototype it should be good enough.

Twitter API Newbie Wades Into The Pool

Twitter DeveloperTwitter trends are easy to follow by searching for the hashtag. The default view given by Twitter shows the most popular tweets with that tag, and there are options to see the most recent tweets first, or browse them by photo, etc. But what if someone wants the tweets organized another way? Fortunately Twitter knows they can’t cover everybody’s desires and offers a developer API for those with creative ideas on tweets.

Twitter’s API is a “RESTful” design. It follows many of the ideas of REpresentational State Transfer (REST) but not strictly adhering to all of it. Functionally, it is built around HTTP GET and POST requests and returns information in the text-based JavaScript Object Notation (JSON) format. The details of those HTTP requests – and the JSON that it retrieves – are documented on Twitter’s developer site.

Before the aspiring Twitter app author can get anywhere, though, one needs to register with Twitter for access to these APIs. Once an app is registered, it is assigned a set of secret access keys and tokens. This allows Twitter to monitor usage and limit/disable Twitter access by individual applications.

Once registered, the code experimentation can begin. As a RESTful API communication via JSON, any web-friendly development framework can be used. (And if it is not web friendly… why would it have anything to do with Twitter?) As a starting point, Twitter has a non-exhaustive list of libraries that cater to different programming environments.

Out of this list, Python-Twitter was chosen as the first step into this world. Python language is fun and easy to experiment with. Python-Twitter installs easily via Python’s pip package manager. And yes – it also happens to be the first entry on the list.

A few commands into Python interactive console was rewarded with immediate results, though sadly also a few issues. Text of some tweets were truncated. Perhaps this has something to do with the recent increase in Twitter’s character limit? Whatever the root cause, it is tracked as an open issue in Python-Twitter’s repository.

The first few search queries returned only 15 results, even when there were definitely more than 15 answers. This turned out to be the default limit which is easily increased up to 100 results per search query. Going any further, unfortunately, is not possible with the standard search API. As of a few months ago, the only option to do more is to subscribe to the very expensive Twitter Enterprise API. This is out of the question for a hobbyist! Fortunately people have already complained and this past November Twitter responded with a premium search API to bridge the gap. Even though it is free, access is restricted so a registered developer has to apply for access.

Even if access is obtained, premium search is a new feature accessible via a new endpoint that is not yet supported by any of the Python libraries on Twitter’s list. Searching with a unique part of the API URL (“30day”) found this Python search tweet API that might be worth a look. If it doesn’t fit the bill for whatever reason, it might be time to build a custom solution.

Fusion 360 Lines Created Via API Are Not Limited To Sketch Plane

Cube on Sketch

Today’s performance tip helps avoid creating unnecessary planes and sketches when creating objects using the Fusion 360 API. First, some background: when sketching lines in Fusion 360 must be done in the context of a sketch object. This sketch object is in turn created to be aligned with a plane. It may be one of the default planes that intersect the origin (XZ plane, XY plane, or YZ plane) or a construction plane.

When sketching with the interactive UI, the user’s sketch entities are constrained to the plane of the sketch. For example, if a sketch was created on the XY plane, then all sketch entities would have a Z coordinate of zero. This makes sense to keep the data organized and easy to understand by the human looking at a 2D computer monitor.

When creating entities with the API, though, they are apparently not constrained. The first clue came from looking at the SketchLines.addByTwoPoints API – its two parameters are Point3D objects. A little experimentation confirmed the API would gladly accept 3D coordinates that lie outside the sketch plane. The example screenshot on this page is the result of drawing an one-centimeter cube on a sketch aligned to the XY plane. Only four of the eight corners (and four of the 12 edges) lie on the plane, but the API happily accepted all the information for a 3D cube.

This is very useful when creating geometries via code. Humans need to see points on a sketch stay in a plane for them to be workable on a 2D monitor screen, otherwise things get too confusing. But code doesn’t care about appearance on a screen. So go ahead – let code create complex 3D entities on a single sketch. Skipping the unnecessary planes and sketches will generate fewer API objects. Reducing object management overhead and allowing faster performance.

Accelerate Fusion 360 API Object Creation With DirectDesignType

64Cylinders

Exploring the capabilities of the Fusion 360 API will involve experiments creating objects via script. Since it’s all code, it’s easy to repeat a process many times, a natural allure of using scripts to automate repetitive tasks. The downside is that Fusion 360 (as of time of writing) has problems resulting in slow response.

The sample code below creates an array of small cylinders one at a time. (This is not the most efficient method to create an array of cylinders but merely an illustration of the problem.) The size of the array can be adjusted by changing the value of arraySize before running the script. For a 4×4 array, the results are almost instantaneous. On a modern Core i5, slowdowns can be observed in an 8×8 array. The first few cylinders are fast, but as additional cylinders appear, each new cylinder takes longer than the last.

Fusion 360 performs a lot of bookkeeping overhead to track these operations. While the majority are outside our control, we do have control over the most expensive overhead: the timeline. It is useful when we are working with hand-created designs, but unnecessary for script – if we want to change anything, we’d update the script and run it again. In the user interface, we can click on the gear in the lower right hand side and select “Do not capture Design History”. To do the same thing in script, we can set the type of the design object to DirectDesignType. (Uncomment line in sample code below.)

This mitigates but does not solve the problem. While it is now practical to generate far larger number of objects, there is still a noticeable slowdown as the object counts go up. According to this forum post, this issue has been logged as a defect for the development team. Perhaps a future release of Fusion 360 will create the last object as quickly as it created the first.

import adsk.core, adsk.fusion, adsk.cam, traceback

arraySize = 8 # Will extrude arraySize^2 objects

def run(context):
  ui = None
  try:
    app = adsk.core.Application.get()
    ui = app.userInterface

    app.documents.add(adsk.core.DocumentTypes.FusionDesignDocumentType)

    # adsk.fusion.Design.cast(app.activeProduct).designType = adsk.fusion.DesignTypes.DirectDesignType # Uncomment to turn off tracking timeline

    rootComp = adsk.fusion.Design.cast(app.activeProduct).rootComponent
    sketch = rootComp.sketches.add(rootComp.xYConstructionPlane)
    extrudeDistance = adsk.core.ValueInput.createByReal(1)

    rangeX = range(1-arraySize, arraySize, 2)
    rangeY = range(1-arraySize, arraySize, 2)

    for nowX in rangeX:
      for nowY in rangeY:
        sketch.sketchCurves.sketchCircles.addByCenterRadius(
          adsk.core.Point3D.create(nowX, nowY, 0), 0.5)

        rootComp.features.extrudeFeatures.addSimple(
          sketch.profiles[-1],
          extrudeDistance,
          adsk.fusion.FeatureOperations.NewBodyFeatureOperation)

        adsk.doEvents()

except:
  if ui:
    ui.messageBox('Failed:\n{}'.format(traceback.format_exc()))

This code is also available on Github.

Fusion 360 Script Engine Uses Python Version 3

One of the differences between Python 2 and 3 was changing print from a statement to a function. This means a Python beginner like myself immediately runs into the breaking change trying to print “Hello World!” to the console. (print "Hello World!" versus print("Hello World!")) It sets the tone pretty clearly up front: when working with Python, developers need to know which version they’re writing for.

Which is why I was very puzzled when I searched for this information in the Autodesk documentation and came up empty-handed. The most obvious place in my mind would be the “Python Specific Issues” page of the user’s manual, but it wasn’t mentioned there. Searching various other combinations of terms – on and off Autodesk’s web site – failed to find a straightforward answer. Given the relatively young age of Fusion 360’s Python scripting support, I would expect them to use Python 3 but I wanted confirmation.

Well, if I can’t find it in documentation, there’s always looking at the code itself. And a tiny baby step beyond the simple boilerplate script generated by Fusion 360. It’s not quite printing “Hello World” but it’s almost that simple.

First I imported the Python sys module for querying system parameters.

import sys

Then I changed the boilerplate message box output string to include the version number.

ui.messageBox("We're running on Python " + str(sys.version_info.major), "Version detection")

Executing the script confirms the scripting engine is running Python 3.

F360Python3

Once the web search engines index this post, people who have this question in the future will know the answer just by reading the title of this post on the search results. They won’t even need to click on the link to read this page.

(This really simple bit of code is almost not worth committing to Github… but it’s a slow day so it was pushed up to be publicly available.)

Qt Quick with PyQt5 on Raspberry Pi

QtLogoThe prime motivation for me to go through Qt licensing documentation and installing Qt Creator IDE was to explore the new UI infrastructure introduced in Qt 5 under the umbrella of “Qt Quick“. As far as I can tell, this is an entirely different system for creating user interface of a Qt application. Built with modern ideas such as OpenGL graphics acceleration for animation effects and UI layout declared with a text-based markup language QML (probably stands for Qt Markup Language.)

Up to this point my experience with building graphics user interface in Qt was with the QWidget-based infrastructure, which has a long lineage in past editions of Qt. Qt Quick is new for Qt5 and seem to share nothing in common with QWidget other than both a part of Qt5. Now that I’ve had a bit of QWidget UI work under my belt I wanted to see what Qt Quick has to offer. And this starts with a smoke test to make sure I could run Qt Quick in the environments I care about: Python and Raspberry Pi.

Step 1: Qt Creator IDE Default Boilerplate.

Once the Qt Creator IDE was up and running, I followed the Qt Quick tutorial to create a bare bones boilerplate Qt Quick application. Even without any changes to the startup boilerplate, it reported error messages complaining of missing modules. Reading the error message, I looked at the output of apt list qml-module-qtquick* and installed the ones that sound right. (From memory:qml-module-qtquick2qml-module-qtquick-controls2, qml-module-qtquick-templates2, and qml-module-qtquick-layouts)

QML CPP

Once the boilerplate successfully launched, I switched languages…

Step 2: PyQt5

The next goal is to get it up and running on Python via PyQt5. The PyQt5 documentation claimed support for QML but the example on the introductory page doesn’t quite line up with the Qt Creator boilerplate code. Looking at the Qt Creator boilerplate main.cpp for reference, I translated the application launch code into main.py. This required sudo apt install python3-pyqt5.qtquick in addition to the python3-pyqt5 I already had. (If there are additional dependencies I forgot about, look for them in the output of apt list python3-pyqt5*)

QML PyQt

Once that was done, the application launched successfully on my Ubuntu desktop machine, albeit with visual appearance very different from the C++ version. That’s good enough for now, so I pushed these changes up to Github and switched platforms…

Step 3: Raspberry Pi (Ubuntu mate)

I pulled the project git repository to my Raspberry Pi running Ubuntu Mate and tried to run the project. After installing the required packages, I got stuck. My QML’s import QtQuick 2.7 failed with error module "QtQuick" version 2.7 is not installed The obvious implication is that the version of QtQuick in qml-module-qtquick2 was too old, but I couldn’t figure out how to verify version number is indeed the problem or if it’s a configuration issue elsewhere in the system.

Searching on the web, I found somebody on stackoverflow.com stuck in the same place. As of this writing, no solution had been posted. I wish I was good enough to figure out what’s going on and contribute intelligently to the discussion!

I don’t have a full grasp of what goes on in the world of repositories ran by various Debian-based distributions, but I could see URLs flying by on-screen and I remembered that Ubuntu Mate pulled from different repositories than Raspbian. I switched to Raspbian to give that a shot…

Step 4: Raspberry Pi (Raspbian Stretch)

After repeating the process on the latest Raspbian, the Qt Quick QML test application launches. Hooray! Whether it was some configuration issue or out of date binaries we don’t know yet for sure, but it does run.

That’s the good news. Now the bad news: it launches with the error:

JIT is disabled for QML. Property bindings and animations will be very slow. Visit https://wiki.qt.io/V4 to learn about possible solutions for your platform.

And indeed, the transition between “First” and “Second” tabs were slow. Looking on the page that it pointed to, it looks like the V4 JavaScript engine used by Qt for QML applications does not have JIT compilation for Raspberry Pi’s ARM chip. That’s a shame.

For now, this excludes Qt Quick as a candidate for writing modern responsive user interfaces for Raspberry Pi applications. If I want to stick with Qt and Python, I’m better off writing Qt interfaces in the old school QWidget style. We’ll keep an eye on this – maybe they’ll add JIT support for Raspberry Pi in the future.


(The source code related to this blog post are publicly available on Github.)

First Use of Python Threads is Quickly Followed By First Crash… in Qt

QtLogoBased on experience, I fully expected my first venture into Python multi-threading to run into problems. I just didn’t know exactly what that problem would look like. I had hoped that my first Python threading bug would be friendlier to understand and debug, just as the Python programming language has been friendlier for a beginner to write.

Sadly, no such luck. After I started using my GPIO class, with the debounce routine powered by threading.Timer, the crash is a very unfriendly Segmentation Fault. Surprisingly, it’s not the full “Segmentation Fault (core dumped)” so there was no core for me to debug with. No matter, it’s my program and I can relaunch it under gdb and try to dissect each instance of the crash.

The symptoms are consistent with a multi-thread timing issue:

  • It is unpredictable. I could use my program for several minutes without a problem, or the program might die within a few seconds of launch.
  • When something does go wrong, the actual point of failure that crashed in gdb is deep in the bowels of the system far from the actual problem.
    • In one case, a Python thread worker without any of my code on the stack.
    • In another case, internal system corruption error “Error in `python3': corrupted size vs. prev_size
  • The state of the rest of the system is also unpredictable. Using gdb’s “info threads” command I could get an overlook of the rest of the program, and it looks different every time.

I thought my timer-related code was simple: it checks the state of the input pin, and if the pin level has changed (after the debounce wait) also change the appearance of the visual indicator. The visual indication change was done by changing the stylesheet on the QWidget on screen. One of the error messages gave me my first clue that was my bug.

Could not parse stylesheet of object 0x21f6f58

Going back to look at the state of the other threads with this in mind, I saw they all had something in common: the main UI thread at the point of the crash is working on some part of visual rendering.

So we have a candidate explanation: I had been updating the QWidget stylesheet in the timer callback, which executes on a thread that is not the main Qt UI thread. So when this occurs when the UI thread is in the middle or rendering the QWidget, it would see the data structure change out from under it by the timer thread. This is not good and could explain the crash.

To test the hypothesis, I modified my program to perform stylesheet updates from the timer callback thread and do it frequently. Tens of times per second versus once every few seconds in my normal execution. The program now crashes consistently within a few seconds of launch which I saw as good confidence builder for my hypothesis.

The next thing we need is a fix. I need to make sure the stylesheet update occurs on the UI thread, and it seems simplest to post a message to the UI thread event queue. I can accomplish this by emitting a signal from the timer thread and have that go to a slot on the UI thread via a Queued Connection. The slot executes on the UI thread and can safely update the stylesheet there.

After this modification, the test program is now only emitting a signal from the timer callback thread to queue a stylesheet update, instead of performing the actual update itself on the timer thread. And now the test program no longer crashes.

I started debugging this believing I ran into a problem deep in the bowels of Python implementation internals. It turns out my clumsy use of threads had confused Qt.

Lesson learned, moving on to the next lesson…

(The project discussed in this blog post is publicly available on GitHub. The threading fix is in this commit.)