William Lachance

Up until today, I was under the impression that the options for someone wanting raw Canadian road data were quite limited. Yahoo and Google maps are great, but their terms of use prevent creative and novel use (it's against their terms of use create a derived work). OpenStreetMap is free of these restrictions, but its coverage can be kind of spotty in outlying areas. What's a geohacker to do?

Little did I know that the Canadian government makes a complete survey of this information available under an extremely non-restrictive license (basically all you need to do is provide attribution). One better, they've come to an arrangement with the OpenStreetMap project that allows them to import all this data (under the creative commons attribution license), which will (as long as some care is taken) bring OpenStreetMap up to the level of Yahoo or Google maps. I'm positively giddy about the novel applications this should make possible.

That being said, integration of the data into OpenStreetMap will probably take some time. However, if you're writing an application which consumes OSM data (like I am), there's no need to wait. All I had to do to get an extremely accurate and complete OSM file for the region of the Halifax Regional Municipality was download the Nova Scotia road network in KML and use gpsbabel to simplify and trim said network down to my region of interest, outputting the result in OpenStreetMap format:

gpsbabel -i kml -f RoadSegment.kml -x simplify,error=0.01k \
-x polygon,file=restrictpoly.arc -o osm -F hrm.osm

The result? Pure city road network win. Here's a quick visualization courtesy of a cheesy 100 line script I knocked up using PyGame:

The empty space in the middle is the Bedford basin. The dense structure towards the center is the Halifax peninsula, connected via two bridges to the city of Dartmouth. The cluster to the north is Bedford.

I'd like to emphasize that, internally, this is a complete map, with street names and everything. Apparently GeoBase also provides a data set with address information which could be useful for writing a free geocoder, but I haven't had the chance to look at that yet.

If all that was nerd speak to you, just wait. I'll be bringing this topic down to earth in a way that any conscious being can appreciate in a few weeks...

I guess I deserve every one of them for being such a smart ass.

Date: Thu, 1 Nov 2007 10:07:57 -0300
From: William Lachance <wrlach@gmail.com>
To: Air Canada <confirmation@aircanada.com>
Subject: Re: Air Canada - Booking confirmation

On 10/31/07, Air Canada <confirmation@aircanada.com> wrote:
>
>
>
>  ****** PLEASE DO NOT REPLY TO THIS E-MAIL ******

Ok.

-- 
William Lachance
wrlach@gmail.com

I just posted an article to advogato on the proper use of assertions. Perhaps I'm wrongheaded, but I truly believe that the software I use on a regular basis would be of much higher quality if people actually read and used the advice contained therein. If you want to comment, but don't have an advogato account, you may do so here.

apenwarr: Your bit of economic theory is interesting, but I'm still pretty certain the key driver here is supply and demand, not price optimization. See, for example, the testimony by Paul Sankey before the U.S. congress on the subject: "Gouging is an Idiotic Explanation".

The facts are simple: domestic demand in the U.S. for gasoline (refined crude oil) is about 22 million barrels per day, while supply is only about 17 million barrels per day (the difference is made up with imports from overseas). Most of the blame for this expensive discrepancy can be placed on the difficulties currently experienced by the U.S. refining industry (extended maintenance, tighter product specifications, safety concerns), rather than some kind of deliberate conspiracy by an oil oligopoly to restrict supply and/or fix the price.

The marketplace for energy is simply too complex, with too many stakeholders, for it to be treated as a monopoly. If only they could, I'm sure those U.S. refineries would very much like to be processing and reselling those 5 million barrels of gasoline that are currently being imported (who wouldn't given the current high prices?). Why invoke conspiracy and collusion when an alternative, well-sourced explanation works just as well?

apenwarr: The lack of informed discussion on Global Warming is frustrating. Even granting the fact that temperature has been increasing over the last few decades, I haven't yet heard a convincing explanation on why this is necessarily due to carbon emissions as opposed to any number of other things. It seems like a probable explanation (CO2 is a greenhouse gas, after all, and the amount of it in the atmosphere has increased very substantially), but you need more than correlation to prove causality. Perhaps there's a convincing scientific explanation out there, but I read all the way through The Weather Makers (allegedly one of the best books available on the subject for the layman) without getting any answers on this.

Regarding gasoline prices, as I understand it the recent surge in the price has more to do with lack of refinery capacity in the United States than anything else. The high prices you mention in the late 70s were due to OPEC's oil embargo. However, this isn't to say that we might not be creeping up to a peak in supply of crude oil. Most of the major exporting nations have already hit their production peak and global supply has been flat at about 85 million barrels per day since 2005. Hand-waving about economics will not make this issue go away.

After announcing Pathfinder a week or so ago, it occurred to me that most people might not know what it's good for (or at least the subtleties thereof). I certainly wouldn't have before I started working on this project, and the information on its more esoteric features is sort of dispersed across the Internet (if available at all). I figured I might as well try to help this situation (as well as Pathfinder's prospects for world domination?) by writing a bit about what it does and in what circumstances it might be useful.

Let's start at the beginning: certificates. An X.509 certificate is a token asserting identity (and various properties corresponding to that identity). At minimum, such a certificate contains an RSA public key, a subject name (who this certificate represents), and an authority name (who's vouching for the authenticity and/or trustworthiness of this certificate by signing it with its RSA key). That's really only the beginning though: an X.509 certificate can contain lots of other stuff corresponding to the identity of its holder: more on that later.

When an SSL connection is established (the most common example of this is connecting to a website over https in Mozilla or IE), these certificates are exchanged between server and client. In most software these days, only the client does any kind of validation of the certificate provided by the server: specifically, it will check the certificate to see that it signed by a certificate in a local "trusted" store (Mozilla and IE provide a trusted store composed of various certificate authorities by default). If so, we proceed with a Diffie-Hellman key exchange using the RSA keys in the client and server certificates and then proceed exchanging data. If not, well... that's up to the application. Web browsers usually pop up a warning that most people click through and ignore (an oft-discussed issue in usability circles: see here, for example).

Although uncommon, it is within the perogative of the server side of the connection to insist that the client provide a certificate when the connection is established, and to reject the client if this certificate is not within its trust store. This is the sort of thing you'd use if you're really paranoid about who you want to allow to (for example) connect to your super-secure web site.

When I talk about verifying signatures, I'm really referring to something broader called "path validation". That is to say, given a sequence of certificates 1..N (where 1 is ultimately trusted), is each certificate signed by the previous one? In the basic cases we've been talking about, there's only two certificates in the path under consideration, and we're only talking about verifying signatures. Validation can get much more complicated and paranoid than this, for those who have higher expectations regarding security. RFC3280 has all the details, but to name just a few things:

• Certificate revocation lists (CRLs): A certificate is supposed to provide a set of CRL distribution points, which tells you where to find a certificate revocation list which indicates (signed by the signer of the certificate) whether or not a certificate has been revoked. This gives you added confidence that a certificate hasn't been compromised.
• Chains of trust: A certificate may be signed by a CA certificate which is not in the trust store, but which is in turn signed by a CA certificate which is in the trust store. This information may be looked up by chasing the AIA (authority information access) field of the certificate which commonly points to an LDAP and/or HTTP URI.
• Bridged trust: Two certificate authorities can elect to cross-sign each other over a PKI bridge. This further reduces the extent to which it is necessary to ultimately trust CA certificates: you can just elect to trust the CA certificates in (for example) your organization, and the appropriate software can validate a path that goes from the client certificate to your CA through a series of "bridge" certificates (the trust in which can easily be revoked, if required).
• Policies constraints: You can insist that a certificate conform to certain user-defined policies in validation.

Needless to say, writing the code to do all of this correctly is rather difficult and, moreover, goes rather beyond the scope of a library like OpenSSL when it comes to chasing down CRL and chains of trust over the Internet. Pathfinder centralizes handling all of these things in a single daemon (accessible from the callback that libnss and OpenSSL use when initializing an SSL connection), making it easier to write and deploy software in a Linux/Unix environment which imposes the stricter requirements on certificate validity that I describe above. At least at first, I only expect it to be useful in certain niche cases with complicated requirements (most likely mail and web servers that are expected to traffic in sensitive information), but perhaps eventually we might see this sort of thing in wider deployment, making the Internet a safer place for everyone. One can dream, anyway...

Oh yeah, did I mention that Pathfinder uses D-Bus? Look: shiny thing!

What I've been working on

For the past while, I've been telling people that I've been working on some "security software" using WvStreams. Unfortunately, I couldn't say exactly what at the time. But that's all over now. Allow me to introduce PathFinder, the next-generation solution for certificate path validation. To my knowledge, Pathfinder is the first open source product which can perform RFC3280-compliant validation and discovery, downloading certificates and CRLs as necessary.

Best of all, getting access to this functionality is a simple matter of sending the certificate and a few parameters over D-Bus, so it should be easy for 3rd party applications to adopt. Sample code for using PathFinder from OpenSSL and Netscape Security Services is provided in the source distribution. The opportunities that this presents are exciting (at least one is already in the pipeline).

Special thanks to Carillon for sponsoring this project and the associated improvements to WvStreams' certificate and CRL APIs.

A note on WvStreams

WvStreams 4.4 is available now.

Over the past few years, some of us have become quite enamoured with WvStreams and how easy it makes it to write high-performance network applications in C++. But with the closure of the Montreal office of NITI^[1] in December 2006 and the associated departure of most of its original developers, I was uncertain about its future.

Who would have thought that, slightly more than half a year later, we'd see two releases, a vibrant community form around the library, and several exciting new projects? WvStreams certainly isn't the solution to every problem, but it does seem to be finding its niche. I'm hoping that it will eventually evolve into something like the Twisted of the C++ programming world.

^[1] The original corporate patron of WvStreams.

After hearing about Zero Footprint several times in the press (they've inked deals with both Air Canada and the city of Toronto), I decided to check out their site the other day. After considerable digging, I could find absolutely no documentation on the economic or scientific model they were using for calculating an individual's carbon footprint. A message to info@zerofootprint.net has received no response (so far). Am I the only one who sees a serious problem here? How can anyone have confidence in their projections? Because they have a scientific advisory board with big names on it? For all we know, they're just making their numbers up.

Just read through the study that the Canadian Centre for Policy Alternatives published on "gas price gouging". As usual, the blamed party is the oil companies, who apparently are pricing gasoline in excess of the cost of production. What's missing in the report is any analysis of SUPPLY. You don't need to be an energy economist to realize that people will consume more of a commodity if the price goes down. The world is consuming pretty much all of global oil production right now, this stuff is available on a global market, so what other signal can the market give to Canadians to moderate their use of oil other than increasing the price? Shortages?

Ok, so maybe you're a bit of a socialist (as I am) and think that the government should sometimes step in to regulate a market. We have a good amount of the resource right here in Canada, so we could just keep more for ourselves, right? This principle was put into practice with regard to energy in the 1980 with the National Energy Program. Mechanisms were put into place to restrict oil exports to the United States and the rest of the world and force depressed prices here in Canada. Eastern Canada got cheap gasoline at a steep economic cost to Western Canada. More than twenty years later, the NEP is still bitterly remembered in Calgary (capital of Alberta, Canada's oil-rich Western Province). Do we really want to go back there again?

I don't think so, but that's not really the point. There are clearly much bigger issues here beyond the prices people pay at the pump and "greedy oil companies" (the needless dependance of Canada's primarily urban population on the automobile being the big one). When are we going to get a decent public discussion going in Canada on energy issues?

Avery just posted a pretty interesting article comparing the cultural institutions of the United States with those of France. The conclusion that he draws is that the states is an "unstable" society while France is a "flow" society. While I haven't spent enough time in Europe to really comment on the latter (although I suspect he may be somewhat overstating the case), I think his analysis of North America is essentially spot on. However, I'd like to add something to it:

Unstable societies require the continual input of greater and greater amounts of high quality energy to avoid collapse.

What else will allow the rapid development of dispersed urban centers (with correspondingly expensive cost of transportation and infrastructure)? What else will compensate for a high bankruptcy and failure rate in business? What else will compensate for the routine and accepted displacement of unskilled labor due to relentless increases in "productivity"?

Much has been made in Canada recently about the development of the tar sands in the province of Alberta, but I think people are missing the real story in all the hullabaloo about that province's spectacular growth. Twenty years ago, you couldn't find anyone willing to spit on the tar sands. Today, the province is experiencing an economic boom based on an extraction method with an incredibly low EROI (5:1 versus 17:1 or better for oil produced by conventional means). This begs the question: why are the (multinational!) oil companies bothering with the tar sands if there are more lucrative opportunities for energy extraction out there? Why put so much capital and expertise into something which provides such a poor return on investment?

Perhaps I'm missing something (if so, please comment), but this indicates to me that we're experiencing real problems in finding the sorts of high quality sources of energy which sustain our current social and economic systems.

edit (May 31 2007): After rereading some stuff, I just realized that my postulate on unstable societies almost certainly came from Thomas Homer Dixon's The Upside of Down. How easily we forget where our ideas come from...

A little late, but I've been meaning to talk about this for a while:

Although it's been years and years since I've actually lived in Ottawa, I still go back fairly frequently to visit. When I do, I invariably get an update on municipal policy from my mother. The most recent story of note was the cancellation of the O-Train project after the recent municipal elections, when a conservative mayor was elected.

Urbanities blamed the "idiots from suburbia" for this. And indeed, most of the votes for Larry O'Brien (Ottawa's new conservative mayor) did come from the suburbs. But ultimately, this never really struck me as being a particularly constructive point of view: people who live outside of the metro area never struck me as being dumber, less reasonable, or less well informed than those who live inside it. They may have made a different choice as far as living arrangements than I have (or will, if I can help it), but their reasons for doing at least understandable to me.

Even if they weren't understandable, it would be rather pompous and self-righteous of me to tell other people how to go about living their lives. The suburban dream's realization may ultimately be something of a nightmare, but calling people idiots and their aspirations stupid isn't likely to endear them to your point of view. If we want to create a better world, wouldn't it be preferable to enact policies and build infrastructure that would enable these people to realize those aspirations?

And, come to think of it, wouldn't a light rail system linking the suburbs with the center of the city be exactly this sort of thing? Even if you didn't use it, it would mean a shorter commute to work because of reduced traffic congestion, an overall reduction in smog and CO2 emissions, an increase in property values, and a whole host of other things which should play exactly to the needs of these people. What's really going on here?

In fact, it's pretty simple. In order to settle a political score with one of his enemies, John Baird, minister of the environment in Canada's "new" government, inappropriately leaked information and withheld funding during the campaign. This cast a negative light on the project from which it never recovered. It was canceled in December 2006, ultimately costing the city seventy five million dollars in fines:

http://thetyee.ca/News/2007/01/23/LightRail/
http://www.clivedoucet.com/articles/gm030707greenerwash.htm

Changing the North American mentality just isn't going to happen overnight. Not without a crisis, anyway. But corrupt politicians using their power inappropriately? At the very least we can draw attention to them and try to ensure that they (and their "new" government) don't get re-elected.

Got the urge to upgrade my emacs installation a while back, was pleasantly surprised to find that installing/running the latest version was as easy as:

cd $HOME/src
wget ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-22.0.97.tar.gz
tar zxvf emacs-22.0.97.tar.gz
cd emacs-22.0.97
./configure --prefix=$HOME/emacs22
make
make install
PATH=$HOME/emacs22/bin:$PATH emacs

Seems like a nice overall improvement over previous versions so far and it hasn't crashed on me once. Of particular note is the default inclusion of RMS's wonderful GNU Emacs LISP Reference Manual in info, which has been a godsend in my previous disconnected-by-default state of being. I know that's something one is supposed to be able to discover/download/install seperately, but after almost 10 years of using emacs variants, it's only within the last year that the zen of the program has begun to come together in my head.

Heck, a year ago I didn't even *bother* with M-x info (a bounty of well-organized, easily navigable information), assuming that it was just legacy garbage that had been obsoleted by the interwebs and google. Never mind Emacs Lisp, which I assumed was the domain of the terminally insane. Now it's become pretty clear how one could conceivably have a computing session which never left emacs, and why one might want to do so.

Strange how vintage software and idioms seem to excite me more than anything else these days, at least when it comes down to tools which I use for actual productive work. For the more superficial things in life, I'm quite happy with my Gaim, Evolution and Mozilla, thank you very much (except maybe for the latter's habit of leaking pixmaps and taking up all of my system's memory). Perhaps I'll write more on that epiphany later.

Why oh why do most daemons fork into the background by default? For example:

Usage:
  gnome-screensaver [OPTION...]

Help Options:
  -?, --help               Show help options
  --help-all               Show all help options
  --help-gtk               Show GTK+ Options

Application Options:
  --version                Version of this application
  --no-daemon              Don't become a daemon
  --debug                  Enable debugging code
  --display=DISPLAY        X display to use

If I'm going to run a daemon manually, chances are that I'm trying to debug it. When you're debugging things, you want to be able to kill it quickly if need be (not to mention easily see what it's spewing out...). I certainly don't want it to fork into the background, daemonize itself, and start writing to the syslog (in the case of system-level daemons).

How much work is it for the writer of an initscript (or session manager, what have you) to add an argument to the program's arglist? Practically zero. Meanwhile I (the helpful person debugging your daemon) is doomed to waste valuable time figuring out how to get the behaviour that I need. This makes no sense!

New Laptop

Months after replacing its hard disk, my trusty old Thinkpad finally gave out last month (blown solder on the motherboard). Figuring that it wasn't worth sinking any more money into the poor old sinking ship, I decided to get a new laptop (normally I'd wait and just continue using my desktop, but present circumstances necessitate frequent travel).

Following pcolijn's earlier advice, I decided to go the Dell route and bought a wonderful D620. It's a pretty nifty machine and (depending on how you configure it) doesn't require any binary drivers on Linux. An install of Debian unstable went exceedingly smoothly. Apparently some people were complaining about the quality of the LCD panel a while back, but I noticed no issues whatsoever.

If anyone wants spare parts for a Thinkpad r32, let me know...

AlumNit News

I have made the executive decision that the official mascot shall be a rooster.

Nutrition

ncm seems to have a nose for interesting articles and books. The article he just linked to on nutrition is well worth reading. Not surprised that the author served as the executive editor of Harper's: his essay reads like one of the better articles from that magazine.

Decided to start syndicating my Advogato entries from LiveJournal, instead of keeping two seperate diaries. I'm filtering syndication entries by tag, so the technical community there will be spared my musings on politics, philosophy, and day-to-day life (although I reserve the right to syndicate that stuff too in certain situations).

--

In open source news, I've just released version 4.3 of WvStreams: the infamous C++ network library behind WvDial, Nitix, and other fine pieces of software.

One new feature included in this release which I think merits a look is Peter Zion's WvStreamsDebugger: a command line client for manipulating all the sockets/workers a particular network application might have on its mainloop at any given time. This kind of command-line goodness can be a real godsend when debugging issues with a piece of software on a customer's system. I'm sure the idea could be easily reapplied to other frameworks and scenarios, so check it out!