Author Kurt Raschke

Open standards are a force multiplier for civic software

In software engineering, software modularity and reusability are considered best practices. Unfortunately, in the civic software world, these principles are often ignored, because governments and public bodies fail to use open standards and interfaces for their data.

When governments and other public bodies adopt open standards, everyone wins. Consider, for example, the Open311 standard. When a city implements an Open311 endpoint, its citizens suddenly have the option of using any software which has been developed to support the Open311 standard. There’s no need for civic hackers in Chicago to develop one iPhone app, only for another group of civic hackers in New York to implement a substantially similar app, just because the two cities use different APIs.

For those developers, the use of open standards acts as a force multiplier: they don’t have to know anything about the cities where their apps are being used, because they all adhere to the same standards. Civic software is, for the most part, the domain of non-profits and individuals working on their own time. Development resources are far from unlimited, and, simply put, we have neither the time nor money to spend on what might be characterized as niche applications which only have use in a limited geographic area, or with one government’s proprietary API.

Closer to home, I’ve watched over the past few years as developers have expended needless effort building transit apps for the Washington, D.C. metropolitan area, simply to accommodate local transit authorities’ refusal to publish clean, high-quality data in standard formats.

Arlington County’s Mobility Lab Transit Tech initiative has developed two applications which rely on data from transit authorities. One is a package for driving real-time transit signs, and the other is Transit Near Me, a mobile webapp for mapping transit options.

I want to emphasize that I don’t mean to minimize the work of the Mobility Lab developers—in the end, they did what they needed to in order to be able to ship a working product, given the data they had access to.

Having said that, though, these applications are not all that different from similar transit apps which have already been built. The Mobility Lab’s real-time sign, for example, is (in terms of basic design concepts) not all that different from the OneBusAway sign mode.

Granted, the Mobility Lab’s real-time sign looks more polished, and includes support for transit modes like bike sharing, but imagine if instead of building a new piece of software from the ground up, the Mobility Lab developers had worked to polish the OneBusAway sign mode and add support for other transit modes?

Had they done so, every city which uses OneBusAway would have been able to benefit immediately from the improvements.

But, there’s a problem. OneBusAway consumes real-time transit information in the GTFS-realtime and SIRI VM formats. Out of the agencies in the region, only Montgomery County Ride On and VRE provide GTFS-realtime data. The other agencies which provide real-time data use proprietary formats which are incompatible with GTFS-realtime. Without detouring too deeply into technical territory, WMATA’s proprietary API actually provides all of the information that would be necessary to construct a GTFS-realtime feed for Metrobus, were it not for the fact that the API uses route, stop, and trip identifiers which are completely different from those in the static GTFS schedule.

The same goes for Transit Near Me; it is, in essence, a mobile version of the OpenTripPlanner system map. Cities around the world have adopted OpenTripPlanner; wouldn’t they also benefit from an interactive system map optimized for mobile devices?

OpenTripPlanner is designed to consume clean, well-constructed GTFS feeds, while instead Transit Near Me must include various work-arounds for idiosyncrasies in WMATA’s data: bad shapes which must be replaced with data from shapefiles, stop IDs which are only available in the API and not the GTFS feed, etc.

I should emphasize again that this isn’t just about the Mobility Lab; their work happens to highlight the problem particularly well, but they’re not the only developers to get caught up in this maelstrom:

What’s the solution? Civic hackers need to stand together with each other, and stand up for good software engineering principles. I doubt that any one developer alone will be able to convince WMATA to get their data in order (goodness knows I’ve tried). But if we stand together and recognize that reinventing the wheel over and over again is not a productive use of our time, we may be able to convince data providers to embrace open standards. When we do, it will have benefits not just locally, but for people around the world who benefit from the work of civic hackers.

No mainline Linux kernel support? Be wary.

A few months ago, I bought a GlobalScale D2Plug—a solid-state, small-form-factor ARM plug computer. It comes out of the box with an old Ubuntu distribution, but by following some instructions I found online, I managed to get Debian stable installed on an SD card and working well.

More recently, I wanted to set up the D2Plug in a new application, for which I intended to upgrade to Debian unstable. It was then that I began to understand what was really going on with the D2Plug. As I discovered, support for the D2Plug’s hardware simply never made it into the mainline kernel.

As such, you’re stuck with an old kernel version, and dependence on Globalscale’s source tree.

I would not characterize the D2Plug as an expensive paperweight just yet. There is a kernel fork which aims to provide a modern kernel for the CuBox, a platform similar to the D2Plug, so there’s still some hope.

However, without a modern kernel, the D2Plug will indeed become an expensive paperweight soon. As distributions move to the 3.x kernel series (as many have already), it will become harder and harder to run modern software on the D2Plug. It is for this reason that I was ultimately unable to upgrade the D2Plug to Debian unstable.

Though there is a considerable amount of hype surrounding the Raspberry Pi (a similar, though considerably lower-specification ARM board), you may be surprised to find that it, too, lacks mainline kernel support as of this writing. Granted, the Raspberry Pi comes with a suitable OS image, and other distributions, like Raspbian, are now available, but it’s nothing like the x86 world where you can download a generic installer and have it work out-of-the-box.

Of course, some plug computers and other ARM devices do have mainline kernel support; this isn’t a condemnation of every ARM device out there. It’s also possible that the D2Plug will eventually get mainline kernel support, although that seems unlikely; the last attempt appears to have been in May 2011 when Greg Kroah-Hartman asked “What’s with the reliance on the 2.6.32 kernel tree? That is quite old now, has anyone done any forward porting of the code yet?”

Considering the Raspberry Pi’s popularity, it seems much more likely that it will get mainline kernel support, and I look forward to being able to run stock ARM builds of Linux distributions on the Pi, in which case I’d be much more likely to buy one. So long as it is dependent on custom builds, the risk is that it, too, could end up stuck on old distributions like the D2Plug is.

For San Francisco, and governments everywhere, technology startups are the perfect partners

A recent article in the New York Times describes SMART Muni, “an Apple iPad app that uses Global Positioning System technology to track all of the city’s buses in real time, allowing transit managers and passengers to monitor problems and delays”. It sounds like the perfect success story: civic coders taking open data (Muni tracks its buses and trains with NextBus, which provides an XML data feed) and using that data to improve operations and create real value for the agency.

Unfortunately, it’s not a success story: the app has never been used in production. As the article explains, “Muni hopes to put the app to good use some day, but the agency is $29 million over budget and cannot afford to buy the iPads required to run the software…[nor] is the city willing to invest $100,000 to run a pilot program.”

The costs involved here—a few hundred dollars each for some iPads, perhaps a few thousand dollars to fund a stipend for a civic coder, even $100,000 for a pilot—pale in comparison to the costs associated with the big-name IT consulting firms that governments are used to dealing with.

In addition, startups, teams of civic coders, and open source projects can often deliver a working prototype or even a completed project much faster than conventional development teams. As the New York Times describes, “a small team of volunteers took just 10 days last summer to create [the app].”

Unfortunately, the City of San Francisco is out of touch with the realities of technology: “‘Start-ups fail at a high rate,’ said Jay Nath, chief innovation officer of San Francisco. ‘As stewards of taxpayer dollars, we need to be thoughtful of using that money wisely and not absorbing too much risk.’” Nath is right about one thing: start-ups do fail at an alarming rate. But that’s not the risk you might think it is, because startups aren’t like conventional development projects.

Unlike conventional projects, startups fail fast. Instead of wasting years and millions of dollars, when a startup has an idea that isn’t going anywhere, it winds up quickly. Maybe it was a bad (or even outright infeasible) idea to begin with, or the startup had the wrong team, or they tried to do too much at once. Maybe their idea’s been superseded by a newer, even better technology. Whatever the reason may be, the startup doesn’t just grind away for years, running up a million-dollar bill. Instead, they admit that they can’t deliver, and get out gracefully.

Consider, for example, the FBI’s Virtual Case File, a five-year, $170-million development effort that never actually delivered any working software. Imagine if the VCF project had failed after three or six months, not five years. Imagine if it had spent less than a million dollars before failing, not $170 million. Of course, the project still wouldn’t be done—but we’d have known that something was wrong up front, instead of finding out five years later, after millions of taxpayer dollars had been wasted on a doomed development effort.

More importantly, startups do have the agility necessary to keep up with the ever-changing technology marketplace. A development effort that takes five or ten years is bound to deliver a product that is obsolete as soon as it arrives, unless major changes are made along the way.

The conventional development practices used by many government agencies and their contractors don’t incorporate that kind of agility. Specifications and requirements are written early in the project’s life, perhaps even before a development team has been selected (if the project must be put out for bids). Even if the requirements are found to be lacking—or flat-out wrong—development marches on. In the end, the team will deliver a product that meets the requirements (thus satisfying the bean-counters) but which is already out-of-date and which doesn’t actually do what users need it to do.

I alluded to these problems in my recent coverage of WMATA’s initiative to install real-time information displays at bus stops. By only considering bids from vendors with “standard, proven products” and “successful existing and fully operational implementations, in multiple transit agencies”, they potentially shut out innovative startups (or even teams of civic coders, like the Mobility Lab).

It’s entirely possible that the first team to tackle a thorny problem may fail—but rather than casting them as “failures that burn holes in the city’s budget”, we’ve got to communicate to governments and taxpayers alike that not all failures are the same. There’s a big difference between a project that runs for years, spends millions of dollars, and has nothing to show for it in the end, and a project that fails after just a few months, has spent well less than a million dollars, and can identify what went wrong, so the next project will be more successful.

When it comes to technology, the best way for governments to be good ‘stewards of taxpayer dollars’ is to adopt successful development practices: small, agile, competent teams, that build inexpensive, flexible products, and fail quickly if they can’t get the job done. The old way—forking over millions and millions to high-priced contractors until they finally declare defeat, then taking it up in a years-long legal battle—just doesn’t look like good stewardship anymore. Sure, established companies may have a long track record that startups don’t, but what’s it a record of? We don’t need any more million-dollar failures. We need smart civic coders developing next-generation solutions like SMART Muni, and we need governments to accept, embrace, and support them.

Rendering issue in Basic Maths with Google Chrome

Basic Maths, the WordPress theme I use on this site, seems to have developed a rendering problem in Google Chrome. The problem is only present in Chrome, and not Safari, so it’s not a WebKit issue, and Gecko in Firefox doesn’t exhibit the problem either.

You can see the issue on any individual post page, such as this one from the demo site:

Basic Maths post page rendering in Chrome 17.0.932.0 on Mac OS X 10.6.8

Basic Maths post page rendering in Firefox 5.0.1 on Mac OS X 10.6.8

Notice how the borders at the top and bottom of the meta box overrun the first sidebar column in Chrome.

Not being a CSS wizard, I’m not in a good position to derive a minimal test case from the theme’s CSS, and I can’t say precisely when the issue first appeared in Chrome, either (although I will say that I run the Chrome dev version). I know this makes for a bad bug report, but I figure it’s better than nothing to document the issue publicly. The issue doesn’t seem to be specific to certain Basic Maths installations, either; both this site and the demo site, as well as others I’ve come across online, are similarly affected in Chrome.

Air and Space