Wednesday, January 17, 2007

The Pinocchio Problem

I only permit myself about 4 hours a month for blogging. That's been my rough budget for the past year or so. Normally I have all sorts of topics I'd like to write about, things I'm kicking around, and it's not too hard to pick one. But for the past month, I've only had one thing on my mind, and I've been going nuts trying to find a way to voice it — you know, to present it in a nice, concise way so you can gulp it down. And so far I'm failing. I think that means I don't understand it very well. Maybe trying to write it down will help.

It's about designing software. See, it seems like there's a good way to design software. A best way, even. And nobody does it. Well, a few people do, but even in those rare instances, I think it's accidental half the time.

I've been thinking about this problem on and off for quite a while, and wouldn't you know it, suddenly 18 years have gone by and I still can't quite articulate this... well, design principle, if that's what it is. But for the past month, I feel like I've been getting closer. Maybe you can help! I'll tell you what I know, and you can tell me what you know, and maybe we'll figure out something extraordinary.

By way of setting context for today's blog, I'll summarize everything I've ever written to date in one sentence: I think most software is crap. Well, that's not quite right. It's fairer to say that I think all software is crap. Yeah. There you have it, Stevey in a Nutshell: software is crap.

Even so, I think some software systems are better than others: the producer of the crap in question swallowed some pennies, maybe, so their crap is shiny in places. Once in a while someone will even swallow a ruby, and their crap is both beautiful and valuable, viewed from a certain, ah, distance. But a turd is still a turd, no matter how many precious stones someone ate to make it.

Lovely metaphor, eh? Made it myself!

Whenever I reflect on the software systems I like best — the ones that feel like old friends, like nice places to live — I see that they have some properties in common. I'll tell you what they are, at least the ones I've noticed, in a minute. Promise. But it's not the whole story. Once you add these properties to a software system, if you do them right, then you usually get a system that's as good as we can make them today. But they're still crap!

The real problem comes when I start thinking about what would happen if we could build software systems that aren't crappy. That thought exercise raises all sorts of interesting questions, and I don't know the answers to any of them. But I'll throw them out there too, maybe, as long as I don't go over my 4-hour time limit. Life beckons, and all that.

Favorite Systems

The big realization I had, sometime in the last month or so, is that all of the common properties of my favorite software systems can be derived from a single root cause: one property, or design principle, that if present will cause software to take on the right characteristics automatically.

What are my favorite software systems? Here are a few of the very best: Unix. Windows XP. Mac OS/X. Emacs. Microsoft Excel. Firefox. Ruby on Rails. Python. Ruby. Scheme. Common Lisp. LP Muds. The Java Virtual Machine.

A few more that just barely make the cut, for now: Microsoft Word. OmniGraffle Pro. JavaScript. Perforce.

Some that I think would make the cut if I learned how to use them effectively: The GIMP. Mathematica. VIM. Lua. Internet Explorer.

Most popular software systems out there don't make the cut. Most of them are quite useful, but I think they lack the essential properties of ideal software design. Examples: IntelliJ. Eclipse. Visual Studio. Java. C++. Perl. Nethack. Microsoft PowerPoint. All Nintendo and PlayStation console games. Nearly all PC games, with the notable exceptions of Doom and Quake. Most web applications, including highly useful ones like Amazon.com or Google Maps.

I won't keep you in suspense. I think the most important principle in all of software design is this: Systems should never reboot.

If you design a system so that it never needs to reboot, then you will eventually, even if it's by a very roundabout path, arrive at a system that will live forever.

All the systems I've listed need to reboot occasionally, which means their practical lifespan is anywhere from a dozen to a hundred more years. Some of them are getting up there — lots of them are in their twenties and thirties now, and there are even a few in their early forties. But they're all still a far cry from immortal.

I think the second most important design principle, really a corollary to the first, is that systems must be able to grow without rebooting. A system that can't grow over time is static, so it really isn't a system at all; it's a function. It might be a very complex function with lots of possible inputs and outputs. It might be a very useful function, and it might live for a long time. But functions are always either replaced or subsumed by systems that can grow. And I've come to believe, over nearly two decades of thinking about this, that systems that can grow and change without rebooting can live forever.

Essential Features

Here are some of the properties shared by the best software systems in the world today. Not all systems have all the properties I'll list here; I think very few of them have the full superset. I think you'll see that the more of the essential properties a system has, the more powerful, important, and long-lived it is.

Note: most of these are features for programmers. Features for non-technical end-users don't contribute to a system's lifespan. In the fullness of time, I believe programming fluency will become as ubiquitous as literacy, so it won't matter.

First: every great system has a command shell. It is always an integral part of the system. It's been there since the system was born. The designer of the system couldn't imagine life without a command shell. The command shell is a full interface to the system: anything you can do with the system in some other way can also be done in the command shell. Great command shells are a big topic in their own right (most of the essential properties of living systems are, come to think of it.) A rough sketch: a great command shell always has a command language, an interactive help facility, a scripting language, an extension system, a command-history facility, and a rich command-line editor. A truly great command shell is an example of a living system in its own right: it can survive its parent system and migrate elsewhere.

All existing command shells are crap, but they are an essential component of building the best software that can be built today.

Emacs can be thought of as the ultimate command shell: what happens when command shells are taken to their logical extreme, or at least as far as anyone has taken the idea to date.

Command shells aren't the only common feature of the greatest software systems in the world, though, so we'll have to leave them for now.

Great systems also have advice. There's no universally accepted name for this feature. Sometimes it's called hooks, or filters, or aspect-oriented programming. As far as I know, Lisp had it first, and it's called advice in Lisp. Advice is a mini-framework that provides before, around, and after hooks by which you can programmatically modify the behavior of some action or function call in the system. Not all advice systems are created equal. The more scope that is given to an advice system — that is, the more reach it has in the system it's advising — the more powerful the parent system will be.

Ruby on Rails has a minor advice system, which Rails calls "filters". It is (only) capable of advising controller actions, which are special functions that render page requests. You can't put a before-, after-, or around-filter on any old API function in the system, although Ruby itself makes this possible to some extent through its metaprogramming facilities. But it's insufficient for advice to be merely theoretically possible. Advice must be built into the system from the ground up, and must be exposed as a first-class, well-documented programmer interface. Advice is very, very powerful. Even the simple action-filtering system in Rails offers amazing flexibility; it's hard to imagine writing a Rails app without it.

Emacs has a sophisticated advice system. Common Lisp has, arguably, the most powerful advice system in the world, which contributes in no small part to the power of the language. Aspect-Oriented Programming is a herculean attempt to bring advice to the Java language, but due to fundamental limitations of Java, it has to be implemented as a language extension with its own compiler and other language tools, which has severely hampered adoption. Another impediment is that Java programmers prefer to write dead systems, and any hint of a breath of life in the system bothers them greatly.

To be sure, it bothers me too. Living software is a little scary. It's no surprise that most programmers prefer writing marionettes instead of real people; marionettes are a lot easier to manage. But I think living software is more interesting, and more useful, and quite frankly it's an inevitability in any event, so we might as well strive to understand it better. To me, that means building it.

Moving right along, world-class software systems always have an extension language and a plug-in system — a way for programmers to extend the base functionality of the application. Sometimes plugins are called "mods". It's a way for your users to grow the system in ways the designer didn't anticipate.

Microsoft Excel has an excellent mod system. It's quite a remarkable programming framework, verging on being a platform in its own right. Like all the best mod systems, it's tiered, with the easy entry point being Excel macros, working its way up through a full COM interface that can be scripted using VB or even Python or Ruby: any language with COM bindings.

One of the killer features that made Doom and Quake so popular was their mod systems. Doom had some basic facilities for scripting your own levels, sprites, and even game logic. Quake had QuakeC, which is still, I think, the gold standard for PC game scripting, but my info is a little dated, since for the past five years or so I've played mostly console games, which are sadly all deader than a coffin-nail.

The very best plug-in systems are powerful enough to build the entire application in its own plug-in system. This has been the core philosophy behind both Emacs and Eclipse. There's a minimal bootstrap layer, which as we will see functions as the system's hardware, and the rest of the system, to the greatest extent possible (as dictated by performance, usually), is written in the extension language.

Firefox has a plugin system. It's a real piece of crap, but it has one, and one thing you'll quickly discover if you build a plug-in system is that there will always be a few crazed programmers who learn to use it and push it to its limits. This may fool you into thinking you have a good plug-in system, but in reality it has to be both easy to use and possible to use without rebooting the system; Firefox breaks both of these cardinal rules, so it's in an unstable state: either it'll get fixed, or something better will come along and everyone will switch to that.

What's really amazing to me is that there isn't even a special bypass for Firefox extension developers. Their development cycle is amazingly painful, as they have to manually re-install the plugin (using the GUI) on every change. With some symlink trickery you can get around about half the steps, but the community actually frowns on this! They evidently feel that if the user is going to experience the installation pain one time, then the programmer must experience it every single time a line of code changes, as a perpetual reminder that Firefox has a crappy plug-in system that the programmer can't actually do anything about. Fun!

Firefox was given a gift by a programmer named Aaron Boodman, now at Google. The gift was GreaseMonkey, which provides an alternate way of writing Firefox extensions. Unlike Firefox's regular plug-in system, GreaseMonkey extensions can be installed and updated without rebooting Firefox, and they're relatively easy to write. This gift has given Firefox new legs. I doubt most Firefox developers (let alone the Firefox user community at large) fully appreciate the importance of GreaseMonkey to the long-term survival of Firefox.

Interestingly, GreaseMonkey is implemented as a Firefox plugin that offers its own plugin system. This is a common pattern in plug-in systems: some plugins will grow large and configurable enough to be viewed as standalone applications. Emacs has many such plugins: the advice package is a good example. For that matter, the Rails filtering system is also implemented in the style of an optional system extension. Plugins, like other software systems, also have a lifespan that's determined by how well they incorporate the features I'm discussing today. They eventually need command shells, advice, extension languages, and so on. However, plugins usually get these things for free by virtue of having been built within a living system.

Building extensible systems is much harder than the alternative. It's commonly estimated at being three to five times harder than building the equivalent non-extensible version of the system. Adding a plug-in system is much easier up front. Adding extensibility to an existing legacy system is fantastically difficult, and requires massive refactoring of the system. The refactoring needed is rarely of the pretty little automatable type currently so fashionable in Java circles. It typically requires effort of the same order as a full rewrite of the system, although as with all refactoring, the risk can be mitigated by putting thorough unit tests in place first.

Many software systems these days are accessible only remotely, over a network, e.g. via a browser and HTTP. Some of the bigger companies that have built such systems, including Yahoo!, Amazon.com and eBay, have begun to realize that programmer extensibility is a critical feature for the long-term survival of their systems. They have begun selectively opening up access to their internal systems, usually via web service interfaces. This provides a certain level of extensibility: in particular, it enables independent software developers to create their own interfaces to the system. Over time, I think a key differentiator in the web-application space will be the quality of the programmer access to the systems backing the web apps.

Plugin systems have security issues. They have usability issues. They have namespace issues. They have dependency-graph issues. They have backwards-compatibility issues. Plugin systems are damn hard to do at all, let alone do well. And they are absolutely essential for ensuring the long-term survival of software systems.

What other features do great software systems have?

I think one important element, at least today, is that they either have to be a killer app, or they need one. Every world-class software system is, by its very nature, a platform. If you have a command-shell, and an extension language with advice, and a plug-in architecture, then you've already got the makings of a platform. But you need to give users some reason for using it in the first place. So the GIMP is for editing images, Eclipse is for editing Java code, Emacs is for editing plain text, Rails is for building web apps, Firefox is for browsing, Python is for scripting, Lua is for embedding, Mathematica is for math, and so on. Each of them can be stretched into other domains, but they're strongest when they're being used in their niche.

The most generic software systems, I think, are operating systems and programming languages, and even they have a primary purpose. OSes are primarily for resource management, and the most successful programming languages have usually carved out some niche: you use C++ when you need speed, Perl when you need Unix system administration, and Java when you need an especially fat API to impress a client. JavaScript had a lock on browser programming, Lisp has been mostly carried by Emacs all these years, and Ruby has taken off mostly due to Rails.

So software systems need a niche. Someday there may be a generic software system so powerful and well-designed that it's the best system to use for literally everything. Maybe you'll be the one to build it.

The last big feature I'll enumerate today, and it's just as important as the rest, is that great software systems are introspective. You can poke around and examine them at runtime, and ideally they poke around and examine themselves as well. At the very least they should have some basic health monitoring in place. Even in a relatively small system with lots of static checking, you still need monitoring on things like input and output queues, and for large systems, you need to monitor just about everything, including the monitoring systems. (If you don't have meta-monitoring, one bad thing that can happen is that your system gets into a state where all the health checks are returning OK, but the system is in fact totally wedged.)

Introspection can (and should) take many different forms, not just health monitoring. System administration tools and diagnostics are one kind of introspection; single-step debugging is another; profiling is yet another. Dynamic linking is still another: the system needs to be able to (for instance) run bytecode verifiers to make sure the code being loaded passes basic security checks. Introspection usually comes with a performance penalty, so many programmers avoid it at runtime, or they make do with whatever minimal introspection facilities the underlying system provides (e.g. RTTI or Java's reflection facility). If you trade off introspection for speed, you're carving years or even decades off the lifespan of your software system. If you're a consultant, or you just want to get it working so you can move on to something else, then maybe it doesn't matter. But I think most programmers prefer to work on systems that will last a long time.

There's a long tail of other features shared by the best software systems, but at this juncture I think it's best to talk a bit about how they all derive from not rebooting, at which point you should be able to identify the other common features easily enough.

Rebooting is Dying

I can't do this topic justice today, partly because it's a big topic, and partly because I just don't understand it very well. So the best I can do is sketch the outlines of it, and hopefully you'll get the picture.

First, let's get some philosophical perspectives out of the way. I take a very broad view of software: most of it isn't man-made. It's only the man-made stuff that's crap. There's a lot of pretty good naturally-occurring software out there. I think that the workings of our brains can mostly be considered software, and the workings of our bodies are definitely software. (Heck, we can even see the assembly-language instructions in our DNA, although we haven't quite figured out the full code yet.) So people are carrying around at least two software systems: body and mind.

I also think of stable ecosystems as being software systems, and for that matter, so are stable governments. So, too, are organizations of people, e.g. companies like the one you work for. Unless I grossly misunderstood Turing's position, software is anything that can produce computation, and computation only requires a machine with some simple mechanisms for changing the machine's state in deterministic ways, including branches, jumps, and reading/writing of persistent state, plus some set of instructions (a program) for playing out these operations.

I'm assuming my definition of software here is self-evident enough that I don't need to defend the position that not all software is man-made.

So my first argument against rebooting is that in nature it doesn't happen. Or, more accurately, when it does happen it's pretty catastrophic. If you don't like the way a person works, you don't kill them, fix their DNA, and then regrow them. If you don't like the way a government works, you don't shut it down, figure out what's wrong, and start it back up again. Why, then, do we almost always develop software that way?

My next argument against rebooting is from plain old software, the kind programmers make today. Take off your programmer's hat for a moment, and think about rebooting as an end user. Do you like having to reboot your software? Of course not. It's inconvenient.

Do you remember way back, maybe ten years ago, when Microsoft made the decision to eliminate all reboots from Windows NT? I've never worked at Microsoft and I heard this thirdhand, so I don't know exactly how it went down, but my understanding is that some exec got pissed off that seemingly every configuration change required a system reboot, and he or she orchestrated a massive effort to remove them all. When I heard about it, they were down to just five scenarios where a full restart was required. Just think of the impact that change had on the U.S. economy — millions of people saving 5 to 30 minutes a day on downtime from Windows reboots. It's just staggering. If only they could have eliminated the blue screens, it would have been pretty darn good software, wouldn't it?

Linux has gone through some growing pains here, and it's finally getting better, but I would guess it still has many more reboot scenarios than Windows does today.

However, I think the reboot problem may be much deeper than a little inconvenience. Viewed from a radical, yet possibly defensible perspective, a reboot is a murder.

How so? Well, one of the biggest unsolved open questions of all time is the question of consciousness: what the heck is it? What does it mean to be conscious? What exactly does it mean to be self-aware? Are pets conscious? Insects? Venus fly-traps? Where do you draw the line? At various times in history, the question of consciousness has gone in and out of fashion, but for the past few decades it's been very much in, and philosophers and cognitive scientists are working together to try to figure it out. As far as I can tell, the prevailing viewpoint (or leading theory, or whatever) is something like this: consciousness is recursive self-awareness, and it's a gradient (as opposed to a simple on/off) that's apparently a function of how many inputs and/or recursive levels of self-awareness your brain can process simultaneously. So pets are conscious, just not as much as people are. At least that's what I've gathered from reading various books, papers and essays about it written over the past 30 years or so.

Also, a particular consciousness (i.e. a specific person) is localized, focused uniquely at any given time in a manner similar to the way your vision can only be focused on a specific location. That means if you clone it, you've created a separate person. At least that seems to be what the big thought leaders like Daniel Dennett are saying. I.e. if you were somehow able to upload your brain's software to some other blank brain and "turn it on", you'd have a person just like you, but it wouldn't be you, and that person would immediately begin to diverge from you by virtue of having different experiences.

Well, if any of this is true (and weenie academics are welcome to come poke holes in minutiae of my exposition, with the understanding that pedantry is unlikely to change the big picture here very much), then software can be conscious. It's debatable whether any software in the world today is conscious enough to present any ethical problems with shutting it off, but you have to wonder how far off it really is.

For instance, I used to think of my dog Cino (a shih-tzu) as a relatively simple state machine: pee, poo, sleep, eat, play, lick nose for hours. As I got to know him, I gradually discerned many more states, hundreds of them, maybe even thousands, but I'm still not entirely convinced that his behavior isn't deterministic. I love him to pieces, so I'll give him the benefit of the doubt and assume he has a free will. But it's clear that you can predict his behavior at least 90% of the time without much effort. I imagine the same is true for me!

A dog seems more complex than a mouse, which is in turn probably a lot more complex than a cockroach, and at some point down the chain (maybe at ants? flatworms? single-celled organisms?) it seems like there's going to be a point where the animal is completely deterministic.

At least it seemed that way to me at first, but I now think otherwise. I personally believe that all creatures can exhibit some nondeterminism, precisely to the extent that they are composed of software rather than hardware. Your hardware is definitely deterministic. Anything Cino has done since he was born, without needing to be taught, is either hardware or firmware (i.e. built-in software that can't be unlearned or changed). Routines for going to pee (and knowing when it's time), sneezing and other reflexes, processing sensory information, digesting food, and thousands of others are all hardcoded in either his brain firmware or his body firmware, and it works predictably; any observable differences are attributable to his software responding to the current environment.

In other words, I think that both consciousness and free will (i.e. nondeterminism) are software properties.

Nobody's going to shed a tear over the deliberate slaying of an amoeba. Well, most people won't. Similarly, I don't think it makes sense to get worked up when your "Hello, World!" process commits ritual suicide by exiting main() after emitting its one message to the world. But I think we've established that each invocation of your "Hello, World" program is creating a separate instance of a minute consciousness.

Well... sort of. A "Hello, World" program, which has no loops or branches, can't exhibit any nondeterminism (unless it's imposed externally, e.g. by a random hardware error), so you can think of it as pure hardware implemented in software. But somewhere between Hello, World and Hal 9000 lies a minimal software program that can be considered to possess rudimentary consciousness, at which point turning it off is tantamount to killing it.

Do we have any software that complicated today? Complex enough and self-aware enough to be considered conscious from an ethical standpoint? Probably not. But I think we'll get there someday, and by then I sure hope we're not developing software the way we do today, with a compile/reboot cycle.

I think of most of the software we build today as being like setting up dominoes. It's Domino Design. You design it very carefully, and it runs once, during which time the dominoes all fall in a nice, predictable order, and if necessary you pick them all up again. The end result can be much fancier than you can achieve with dominoes — think of all the frames in the movie Toy Story, for instance. Not much different, but definitely fancier.

Even really complex pieces of software like search engines or e-commerce systems are generally approached using Domino Design. If you program primarily in C++ or Java, you're almost certainly guilty of it.

Rebooting a domino system is unavoidable. The notion of rebooting it for every change, every upgrade, is hardwired into the way we think about these systems. As for me, I don't think dominoes are very interesting. I'd personally rather play with an amoeba. And I'd think twice before squishing it and finding myself a new amoeba, if, for instance, it refused to learn to run a maze or something.

DWIM and QWAN

My thinking in this section is a bit fuzzy, so I'll keep it short. DWIM is a cute acronym invented, I think, by Larry Wall. It means "Do What I Mean", and the Perl community bandies it about quite a bit.

The idea behind DWIM is that perfect software is like a grateful djinn in a bottle who is granting you a wish. I'm not talking about those cheap-ass wishes you get from fountains or shooting stars, either. Everyone knows you have to phrase penny-wishes in legalese because the Wish Fairy gives you literally what you asked for, not what you meant. Just like all the software you use. In contrast, the DWIM genie is on your side, and is tolerant of grammatical and logical mistakes that might otherwise result in getting the wrong thing. ("A guy walks into a bar with a miniature piano and a twelve-inch pianist...")

DWIM is ostensibly what every programmer is trying to build into their software, but it's almost always done by guesswork, making assumptions about your end-users. Sometimes it's done by collaborative filtering or some other algorithmic approach, but that's error-prone as well. The only way to make DWIM appear more than fleetingly and accidentally is to create truly intelligent software — not just conscious, but intelligent, and hopefully also wise and perceptive and (gulp) friendly.

But that's where it starts to get a little worrisome, since everyone knows that "intelligent" doesn't necessarily imply sympathy to your cause, especially given that your cause may well be stupid.

So do we want DWIM or no? On the one hand, we want our software to be so good that it predicts your every desire and responds to all your needs, so you can live that hedonistic lifestyle you've always craved. On the other hand, it appears that the only way to do that is to create software that could easily be smarter than we are, at which point we worry about being marginalized at best, or enslaved and/or exterminated at worst.

I think this explains, at least in part, why most of our industry is trying to achieve pseudo-DWIM by building bigger and bigger dead systems. (And what better language for building big, dead systems than Java?) Dead systems are safely deterministic and controllable.

Let's be slightly less ambitious for a moment, and assume for the sake of argument that building conscious, intelligent software is unlikely to happen in our lifetime. Is it still possible to build software that's better than the run-of-the-mill crap most of us are churning out today?

There's another genie named QWAN, invented by the architect Christopher Alexander; it stands for "Quality Without a Name", and (much like the elusive thesis of this essay) it's something he's known about most of his life, and has been trying to both articulate it and find a way to produce it reliably for the entire time. In short, QWAN is an all-but intangible "I know it when I see it" property of certain spaces and/or structures that appears to be (a) most achievable via organic growth, and (b) mysteriously dependent on some hardware or firmware in the human brain, since it triggers warm fuzzies for just about everyone, but nobody really knows why.

At some point in the murky past, some software professionals realized that software can also have QWAN, and in the interim it's become clear that it's just as difficult to pin down in software as in physical architecture.

However, I'll assert that QWAN is most apparent in exactly the systems I listed as my favorites earlier in this essay, and it's mostly absent in the systems I said lacked the essential properties of living software. (I then enumerated a few of the big ones and did some vague hand-waving about the rest of them, in case you've forgotten.)

People have observed that Emacs has QWAN: a nice, organic, comfortable rightness that fits like a pair of old jeans, or a snug warm chair in a library by a fire. It's very right-brain stuff we're talking about here, all touchy and feely and sensitive: exactly the kind of thing that programmers are often so lousy at, so it's no wonder we don't know the recipe for building it into our software. But unlike with UI design, software QWAN can only come from the programmer, who is playing the roles of interior decorator, head chef, and ergonomic consultant for all the programmer-users of said software.

That's why I think most software is crap. I'm not talking about the end-user experience, I'm talking about the programmer experience (if it even offers one). Design is an art, not a science, and most of us aren't born artists. So it's actually pretty remarkable when QWAN evidences itself even a little in a software system.

Note, incidentally, that the end-user experience and programmer experience offered by a software system are almost totally orthogonal. Visual Studio actually offers a pretty slick end-user experience, where the end-user in question is a programmer trying to write some C++ or C# code (as opposed to trying to build tools to help write C++ or C# code, or to help automate away other common tasks.) Emacs offers a completely hostile end-user experience and a nearly unparalleled programmer experience. Eclipse is somewhere in the middle in both dimensions, but is definitely weighted more towards the end-user programmer than the programmer-user.

Finally, there are some aspects to QWAN that probably can't be captured in any recipe. Just adding my ingredients (a command shell, an extension language and advice system, etc.) will improve your software system, but it's no guarantee that QWAN will appear. Some of it just boils down to taste. And since tastes differ, and QWAN is nowhere near as smart as its cousin DWIM, one person's QWAN may be another's software hell.

I still think we should try to build it.

The Role of Type Systems

OK. I've finally built up enough context to be able to explain my position on type systems.

In short, type systems are for building hardware. Every software system has a machine beneath it — a stack of machines, actually, a whole heap of them, from the von Neumann machine with its CPUs and registers and buses down through the semiconductors and doping barriers to the atoms and quarks and their quantum-mechanical underpinnings.

But I take a broader view of hardware. Hardware is anything that produces computations deterministically and is so inflexible that you must either break it or replace it wholesale in order to change it. So it's not just the physical hardware made of atoms; in my view the hardware of a machine also encompasses any software that cannot be changed without a system reboot.

The notion of a type system is closely tied to the notions of "compile time" and "static checking". I tend to take a very broad view of type systems; I consider all constraint systems imposed on computation to be types of type systems, including things like scoping and visibility constraints (public/private/protected/friend/etc.), security constraints, even the fundamental computing architecture of the program as defined by its complete set of function signatures. My view of "static" typing actually includes some stuff that happens at runtime.

But most people, when they talk about static type systems, are talking about the kinds of validations that occur at compile time (with the help of type annotations that are part of the language syntax, and optionally type inference that's run on the IR built from the AST) and which evaporate at runtime.

Static type systems have several benefits over deferring the constraint checking to runtime. For one thing, they can result in faster code, because if you do your checking up front, and you can guarantee the system won't change at runtime, then you don't need to do the checks at runtime. For another, static constraints can often be examined and reported on by tools other than the compiler (e.g. your IDE), which can make it easier to follow the program flow.

And that's all just ducky.

There's nothing wrong with static type systems. You just have to realize that when you use them, you're building hardware, not software.

A static type system has absolutely no runtime effect on a correct program: exactly the same machine code is generated regardless of how many types you chose to model. Case in point: C++ code is no more efficient than C code, and C lacks any but the most primitive type system. C isn't dynamically typed, but it's not particularly statically typed either. It's possible to write fast, robust programs in straight C. (Emacs and the Linux kernel are two fine examples.)

The proper use of a static type system is to freeze software into hardware. Whenever when a particular set of deterministic operations (e.g. OpenGL rendering primitives) becomes ubiquitous enough and stable enough that it's worth trading off some flexibility to create a high-performance machine layer out of the API, we move it into hardware. But it's mostly a performance optimization (and in very small part, a standardization move, I suppose) that incurs a dramatic penalty in flexibility.

Because most programmers nowadays prefer to build marionettes rather than scary real children, most programming is oriented towards building layer upon layer of hardware. C++ and Java programmers (and I'd be willing to bet, C# programmers) are by default baking every line of code they write into hardware by modeling everything in the type system up front.

It's possible to write loosely-typed C++ and Java code using just arrays, linked lists, hashes, trees, and functions, but it's strongly deprecated (or at least frowned upon) in both camps, where strong typing and heavy-duty modeling have been growing increasingly fashionable for over a decade, to the detriment of productivity (which translates to software schedule) and flexibility (which also translates to schedule, via the ability to incorporate new features).

The devotees of the Hindley-Milner type system (which originated in academia and has only rarely escaped into the industry) like to believe that the H-M type system is far better than the Java or C++ type systems, because (among other reasons) it has fewer holes in it.

The reason H-M isn't used in the industry, folks, is that it doesn't have any holes. If you're trying to build software, but you believe (as most do) that you do it by repeatedly building and discarding hardware in a long, painful cycle until the hardware finally does what you want, then you need escape hatches: ways to tell the type system to shut the hell up, that you know what you're doing, that your program is in fact producing the desired computation. Type casts, narrowing and widening conversions, friend functions to bypass the standard class protections, stuffing minilanguages into strings and parsing them out by hand, there are dozens of ways to bypass the type systems in Java and C++, and programmers use them all the time, because (little do they know) they're actually trying to build software, not hardware.

H-M is very pretty, in a totally useless formal mathematical sense. It handles a few computation constructs very nicely; the pattern matching dispatch found in Haskell, SML and OCaml is particularly handy. Unsurprisingly, it handles some other common and highly desirable constructs awkwardly at best, but they explain those scenarios away by saying that you're mistaken, you don't actually want them. You know, things like, oh, setting variables. You don't want to do that, trust them on this. (OCaml lets you do it, but they look down their noses at such impurities.)

It's true that with some effort you can build beautiful, lithe marionettes with Haskell. But they will forever remain little wooden Pinocchio, never to be granted their wish for boyhood by the Good Fairy. Haskell has little or no notion of dynamic code loading or runtime reflection; it's such an alien concept to them that it's difficult even to discuss why they might be useful with a Haskell fan: their world-view just doesn't incorporate the idea of software that grows while it's alive and breathing.

Looking around at the somewhat shabby offerings available in the land of dynamic typing, it's not hard to see why so many programmers view dynamic languages as toys. The main crop of popular dynamic languages includes Perl, PHP, Python, Visual Basic, JavaScript, Ruby, Tcl, and Lua. With all due respect to the language designers, they all apparently flunked out of compiler school before starting work on their scripting languages. Case in point: they all screwed up lexical scoping. Ironically, having their scripting languages propelled into the limelight has turned all of them (Larry, Guido, Matz, Brendan, etc.) into world-class language designers, which makes them, Salieri-like, able to appreciate the weaknesses of their languages (which can now change only with great difficulty) in a way that most programmers will never know.

Scheme has great promise, but suffers from the fatal flaw that it can never grow, not in the way industrial-strength languages and platforms need to grow to sustain massive programmer populations. It has to stay small to keep its niche, which is in CS education.

Common Lisp is in many ways really ideal: it's a dynamically typed language with optional type annotations (i.e. you build software, then selectively turn it into hardware), lots of great tools and documentation, all of the essential features of living software I've enumerated in this essay, and a fair bit of QWAN thrown in for good measure. However, it has stopped growing, and programmers can sense momentum like a shark senses blood. Common Lisp has all of the exciting energy of a debate between William F. Buckley, Jr. and Charleton Heston. (I watched one once, and I'd swear they both slept through half of it.)

Plus Lisp lacks a C-like syntax. We will never be able to make real progress in computing and language design in our industry until C syntax is wholly eradicated: a task that could take fifty years. The only way to make progress in the meantime is to separate the model (the AST) and the presentation (the syntax) in programming languages, allow skinnable syntax, and let the C-like-syntax lovers continue to use it until they're all dead. Common Lisp could be enhanced to permit alternate syntaxes, but since it's not growing anymore, it's not going to happen. I would look elsewhere for the Next Big Thing.

Conclusion

It's 3:30am; I've overshot my monthly budget by over 2 hours. You might actually get a succinct blog out of me next month! In any case, I think I've got everything I wanted off my chest.

I think we want to build software that does what you mean. It will change society in ways that are difficult to predict, since DWIM can only really come from truly intelligent software. Nevertheless, I think it's what we want.

I doubt it will happen any time soon, because most programmers are relentlessly focused on building hardware, through diligent overapplication of their type systems to what would otherwise be perfectly good software.

In the interim between now and the emergence of DWIM from its magic lamp, I think we should build living software. These systems may only be as alive as a tree, but that should be sufficient to awaken QWAN in them, and all systems that have any QWAN at all are truly wonderful to use — at least as a programmer; their end-user quality varies widely.

Living software has a command shell, since you need a way to talk to it like a grown-up. It has an extension language, since you need a way to help it grow. It has an advice system, since you need a way to train and tailor it. It has a niche, since it needs users in order to thrive. It has a plug-in architecture, so you can dress it up for your party. And it is self-aware to the maximum extent possible given the external performance constraints. These features must be seamlessly and elegantly integrated, each subsystem implemented with the same care and attention to detail as the system as a whole.

And you shouldn't need to murder it every time you grow it. If you treat your software like a living thing, then eventually it just might become one.

If we ever wake up a real AI, I think it should be named Pinocchio.

92 Comments:

Blogger Philippa Cowderoy said...

Yeah, dynamic loading in haskell is so alien that people went out of their way to write hs-plugins and a number of projects are built around it as a core (notably yi and lambdabot).

Funnily enough, we've got mutable variables when we want them, too. Perhaps you'd like to find out what the serious haskell hackers are actually doing before starting the bashing?

Incidentally, QuakeC has been miles from the gold standard for game scripting for a very, very long time - and doesn't exhibit any of the properties you advocate other than being a scripting language.

6:18 AM, January 17, 2007  
Blogger Unknown said...

I would say that not only is software crap because it is dead rather than living and immortal, it is also crap because it is almost totally illegible even to other programmers. So even if someone were to come along and want to move a piece of software towards the goal of immortality, it is usually nearly impossible to do it because it's so hard to understand what the authors meant when they wrote it.

And just as software tends to be dead because programmers prefer it that way and not because it has to be, it also tends to be illegible simply because most programmers either have no idea how to write legible maintainable code, or if they do know how, they just don't bother. Or both.

One way or another, this field we're in has a long way to go.

6:53 AM, January 17, 2007  
Blogger Michiel de Mare said...

Thank you, this was one of the most inspiring articles I've read recently! Keep blogging and ignore the flames!

7:04 AM, January 17, 2007  
Blogger Logan Capaldo said...

Just to reiterate what Philippa said, here's a paper discussing the dynamic, plugin oriented nature of lambdabot and Yi.

http://www.cse.unsw.edu.au/~dons/papers/SC05.html

7:28 AM, January 17, 2007  
Blogger Unknown said...

How does smalltalk compare to the problem you are describing ?

8:11 AM, January 17, 2007  
Blogger Rodrigo said...

Have you ever tried Plan 9 and Acme? The way everything can be exposed with a file system interface in a namespace that can be customized per process is awesome.

8:12 AM, January 17, 2007  
Blogger Unknown said...

I want to say one word to you. Just one word - Erlang

Shame no one uses it.

9:21 AM, January 17, 2007  
Blogger mem said...

Steve,

An interesting post, though freedom was not always equated with non-determinism (free will was not always considered the power of contrary choice).

Perhaps as creators we might be willing instead to plan the obsolescence of a software package, rather than let it happen. You might think of it as designing bad design with an eye toward its removal.

Granted this won't work with the cross-cutting concerns that AOP tries to fix (they're too systemic to permit bad design), but in situations where we can compartmentalize pieces of the algorithm and accept worse results in the interim.

I am not always the best reader, so perhaps I've missed part of the intended point; but even Pinocchio presumably died. You complain legitimately that C-syntax is so pervasive it will take 50 years to get away from it (though I consider death by parentheses just as bad). Isn't that long-livedness precisely what you're arguing for in good software?

It seems to me that bad software can have the same features as good software; what we're really in need of is a better means of evolving our software and deprecating the old stuff.

9:24 AM, January 17, 2007  
Blogger Si said...

Steve,

"The only way to make progress in the meantime is to separate the model (the AST) and the presentation (the syntax) in programming languages, allow skinnable syntax, and let the C-like-syntax lovers continue to use it until they're all dead. Common Lisp could be enhanced to permit alternate syntaxes, but since it's not growing anymore, it's not going to happen. I would look elsewhere for the Next Big Thing."

Are you tracking the progress of the .Net CLR and associated (largely independent and interchangable) languages?

There are some exciting things on the horizon for C# developers, closures and lambda expressions to name two. There is also lots of movement in dynamic languages on the .Net platform and an official research port of OCaml, F#.

We even have a powerful .Netted command shell.

9:27 AM, January 17, 2007  
Blogger Joel Franusic said...

I see that Erlang has already been mentioned.

As I'm reading your article, I haven't finished yet, I keep thinking "Erlang, Erlang!"

I got really interested in Erlang when I found out that the most feature rich Jabber server out there (ejabberd) is written by a very small group of programmers. Not only that, it is "fault-tolerant" which, as I understand it, means "no reboots". The best introduction to Erlang I've seen so far is Erlang: The Movie. Watch the video. Hello Joe!

9:34 AM, January 17, 2007  
Blogger Matthew said...

There's another genie named QWAN, invented by the architect Christopher Alexander

You might check out Zen and the Art of Motorcycle Maintenance, if you haven't already read it. Pirsig suggests that this odd, undefinable Quality arises from the relationship between the builder and the object being built.

I dunno who came first, Pirsig or Alexander, but my guess would be Pirsig.

10:18 AM, January 17, 2007  
Blogger dm said...

"The best introduction to Erlang I've seen so far is Erlang: The Movie."

comedy gold. i was scratching my head for at least the first half trying to determine whether this was for real. Hello Joe, indeed. I especially loved Joe (or was it Mike?) reading from the cue cards at the end. You would have thought the guy would have the strengths of his language down cold by now...

anyway, steve, what I'm convinced software development inherently lacks is a clear cut objective function. the constraints have always been pretty well defined, but what you are trying to maximize subject to those constraints is not, since it is often driven by the tastes of particular individuals. ultimately the software developer intermediates between hardware engineer and whimsical end user. even if you're talking systems for programmers, you still end up constrained by the ultimate applications these guys will work on.

by your definition, perhaps the only good software is the software community itself, especially academia and the open source movement. the community never reboots, it's extensible and while it will never be perfect it strives toward perfection. i really think software/systems can be more or less than your axioms, though.

lastly, don't you think C will survive for as long as it is a useful approximation of the native language of the hardware?

11:09 AM, January 17, 2007  
Blogger dasil003 said...

I think the real problem is bumping up against the limits of human intelligence. We are able to build complex systems through abstraction, but as we know, all abstractions are leaky. Static typing and 'hardware'-like programs are just means of simplifying the overall system so that it can be understood and debugged.

Certainly there are those programmers out there who are smart enough to incrementally move us towards this utopia you describe, but I think the real breakthrough will come with better AI. I suppose it is possible to build a software system incrementally that is able to manage itself as it grows to alleviate the problem of programmer understanding, but such software could only come about in a research environment, and it would need a tremendous amount of funding and some way to bridge the gap to career programmers to find its niche.

However the good news is that we probably only need one such operating system to gain critical mass before it can start gobbling up other software to fulfill its ultimate goal of world domination.

11:14 AM, January 17, 2007  
Blogger Evil Otto said...

Some of the systems that came to mind while I was reading this have already been mentioned - erlang/otp and smalltalk especially, with strong concepts of a running image and loading things into that image. Regarding smalltalk, I recall reading somewhere that the main distributed smalltalk image (squeak maybe?) has evolved over the past 2 decades and is under serious threat of murder, if you will, because people want to take it apart and make it simpler. Another system where the main concept is that of an image that grows and evolves is forth.

Predictably, most of the comments thus far (this one included) are people defending the percieved slams against their favorite languages, or pointing out the ones you missed. But I think maybe your larger point is that we as system designers need to be more than just programmers, we need to be (and behave as) philosophers, and perhaps in a way, parents.

11:15 AM, January 17, 2007  
Blogger lahosken said...

Christopher Alexander, the "Quality Without a Name" guy is still writing. And lately when he writes about something that possesses, uhm, goodness, he calls that goodness "life". No, really. http://www.natureoforder.com/

Also, swallowing pennies so that you can produce shiny poop is probably not the easiest way. I once dumped some glitter in my toilet bowl. You'd think it would all go away with one flush, but it didn't. But each time a, uhm, solid appeared in that bowl, some of the glitter stuck to it. If you really want to ingest something, an artist made some tablet-capsule-pill thingies that were full of glitter. Yes, for just that purpose. Sheesh, artists.

11:32 AM, January 17, 2007  
Blogger alex said...

"Linux has gone through some growing pains here, and it's finally getting better, but I would guess it still has many more reboot scenarios than Windows does today."

What are these Linux reboot scenarios you speak of? *nix machines are notorious for ridiculously long uptimes.

11:54 AM, January 17, 2007  
Blogger Unknown said...

A related idea occured to me this morning, (typing this comment I discovered that dasil003 said something very similar):

The creation of software (these days) is, in most cases an attempt to force the semantics of a particular circumstance into syntax, or "hardware" as you call it.

I'm a mere CS student and neither a language scientist nor a philosopher, but it seems to me that this fundamental step, formulating semantics in terms of syntax, is a kind of barrier that gets increasingly harder to break as the problems we're trying to express become more difficult.
Probably up to a point where problems just become impossible to express in terms of syntax.

This doesn't have much to do with CS anymore though and seems like a rather general problem. Thought I'd share it anyway.

12:01 PM, January 17, 2007  
Blogger Unknown said...

Windows XP still has many many more reboots that linux. I've used both in parallel for many years and can safely report that the only reboot necessary on linux is kernel updates. Any other software update can be done without a reboot. However, the individual pieces do need to be restarted, so it's not perfect.

One problem is that people are so used to rebooting with Windows, when the encounter an odd situation on a linux workstation they immediately resort to their old habit: reboot. It's highly likely that all they really needed to do was restart their network service, or X windows, or application.

12:02 PM, January 17, 2007  
Blogger Thunderfist said...

You may have missed the true nature of the problem with the simple assumption that 'programming' needs to be fixed. It seems to me that programming is the problem and no better syntax, system, style of dynamic loading, or the other features you mentioned will ever solve the problem.

The only truly new approach to AI that is comming along now it is comming from the robotics people, not the AI people. AI has hit a dead end of philosophizing bout consiousness.

Anyway, if we want truly intelligent system, we need to solve the same problems as biological systems solve, and extend to higher level systems. This implies non-programming-based systems.

ANyway here is more info in this line of thought and type of system...

http://www.cpjava.net/NewCognition.doc

12:19 PM, January 17, 2007  
Blogger Tim Hoolihan said...

Though not as clean as having it as part of the language, I found an interesting implementation of dynamic scoping for Ruby:

http://chneukirchen.org/blog/archive/2005/04/dynamic-variables-in-ruby.html

12:55 PM, January 17, 2007  
Blogger Todd DeLuca said...

Thanks for the very thought provoking article.

Developing a living systems without killing (rebooting) it seems like it would require careful management of state and rules so the system doesn't get confused and die as the rules or state are being changed, and/or robust handling of errors, so the system regresses to a functioning state. However any loss of state which is not easily recoverable might be considered a reboot, even if the system is still up and running to some degree.

Branching is a useful because it allows murderous development of the software system while still allowing a living copy to remain intact. Is branching indicative of systems which are not truly robust enough to be "living"?

BTW, development of living organisms through the process of evolution involves a lot of rebooting -- old version dies, new versions sometimes live.

1:01 PM, January 17, 2007  
Blogger Pierre Clouthier said...

I couldn't print your essay with Firefox, I had to switch to IE (yuch).

Might want to consider making it printable with Firefox.

Why print? I don't have time to read it at work, I want to read it at home on the can or at breakfast.

1:10 PM, January 17, 2007  
Blogger Johnny Gambas said...

Very good article as usual. Always inspiring. Thanks.

1:17 PM, January 17, 2007  
Blogger Josh Staiger said...

You mention Unix as one of your "close-to-living systems", but if we look inside Unix we see a lot of rebooting: cd, ls, grep, awk. A million tiny processes springing to life and dying off: each unmodifiable in its lifetime. (probably not a coincidence that if we look inside a human being we see something similar)

The key point is that each one is small (do one thing well), each is intended to have a short lifespan, and each co-operates well with the other.

In fact the greps of the world look conspicuously like our fundamental unit of abstraction: the function.

Contrast this with Visual Studio, which is huge, has a long lifespan, and doesn't play particularly well with anyone outside VisualStudioLand.

I think the general principle here is not that we shouldn't write dead things, but that we should try to make our units of deadness as small, simple, and modular as possible so it's possible to assemble them into a larger system that is *alive* and malleable at the same time.

They should either *be* functions (like in Emacs) or very close to functions (like in Unix) and we should be able to invoke them or replace them directly if we want.

1:20 PM, January 17, 2007  
Blogger Tim Hoolihan said...

Sorry, first url was cut off...
Dynamic Scoping In Ruby

1:32 PM, January 17, 2007  
Blogger Dan Nugent said...

Sounds like you want every software project be it's own programming language/environment... and that sounds like a horrible idea.

1:45 PM, January 17, 2007  
Blogger Doug Cutting said...

I think DWIM is older than Perl. I first ran into the term in Interlisp, where it was a feature authored by Warren Teitelman.

1:59 PM, January 17, 2007  
Blogger Joel said...

"If you don't like the way a government works, you don't shut it down, figure out what's wrong, and start it back up again."

Unless you're Mao or Bush.

3:41 PM, January 17, 2007  
Blogger timb said...

"So my first argument against rebooting is that in nature it doesn't happen. Or, more accurately, when it does happen it's pretty catastrophic."

i like to think sleeping is analagous to rebooting.

3:55 PM, January 17, 2007  
Blogger bonecrusher said...

Others have mentioned this, but the best example of software that fits your description is Smalltalk. Smalltalk is living code, in a way that no other language can come close.

Squeek is a weird, buggy, undocumented mess and for the life of me, I can't figure out how to make a real application in it. But I keep firing it up just to let it blow my mind. Even though I have no idea how to create a freakin' dialog box in it, I can tell that this is how we should be writing software.

5:08 PM, January 17, 2007  
Blogger Dotan Dimet said...

Gimp, but not Photoshop? Because it's more extensible?
The idea of plugins that have their own plugins reminded me of Wordpress and some of it's more complicated themes.
A good example of software in the style you talk of is Dave Winer's Userland Frontier (Radio, Manilla or whatever). An object database with a scripting language that just kept growing functionality, with methods getting converted to hardware ("kernel methods") after starting as scripts in the database.

5:10 PM, January 17, 2007  
Blogger __Vali said...

Thank you !

IMHO, you should gather together all your posts and publish them in a book (O’Reilly one ?!); if a blogger’s posts are to be to be released into a book (and “why not?”) then yours are, definitively, worth to happen.

__vali

5:12 PM, January 17, 2007  
Blogger Tonio said...

If you don't like the way a person works, you don't kill them, fix their DNA, and then regrow them. ... Why, then, do we almost always develop software that way?

Because we can.

The way I look at computer software is this: it has to run on both computer hardware and users. Unusable software that "crashes" users is just as bad as -- or worse than -- buggy software that crashes computers.

The same kind of thinking usefully generalizes to your all-encompassing definition of software -- government regulations need to not only be "beautiful", they need to make sense to courts and be enforceable by society and seem just to people.

DNA is beautiful because it works without rebooting and is easy (even fun) to use. GIMP is ugly because it's ugly and hard to use, never mind how good its plugin architecture is.

5:46 PM, January 17, 2007  
Blogger Blake said...

Ruby on Rails has actually embraced a full pattern of 'advice' in the upcoming 1.2 release. A method called 'alias_method_chain' provides layering additional functionality on top of an existing method implementation and then makes the original implementation available. Example

Class Foo
def bar
print "ooF"
end
end

Class Override < Foo
def bar_with_advice
print "Add to the chain..."
bar_without_advice
end

alias_method_chain :bar, :advice
end

Anyway, its essentially sugar for what you can do with method aliasing with out of the box Ruby. But it does give you hooks for extending any method, as its defined in the Module class...

9:05 PM, January 17, 2007  
Blogger tom said...

I posted the link for the guys in the #lua channel, and one of them had this to say:

"I think if he knew lua better he wouldn't have made that crack about it getting lexical scoping wrong; it's the only one in the list that gets it right."

I can't vouch for its validity, but thats the word on the street.

9:14 PM, January 17, 2007  
Blogger Klondike said...

Fantastic. I feel like I really got this. It also filled me with that sense of pride in my work that comes from identifying oneself as an artist, and participating in a deeply emotional and human endeavor. Is it any wonder that programming language camps stage religious wars?

10:01 PM, January 17, 2007  
Blogger Martin DeMello said...

However, it has stopped growing, and programmers can sense momentum like a shark senses blood.

Disappointing article on the whole, but it was worth it for that one line. "If you would not perish, then grow" indeed - I'd love to see that extended into a post of its own.

Along similar lines, this is well worth a read.

10:30 PM, January 17, 2007  
Blogger kathy said...

The funny thing about biological "software" is that it's all running the same OS with every piece of hardware running it with different patches and different parameters.
All of these are faulty in some way, all of them have their own system-maintenance and debugging systems and need periods of regeneration. They use aware and unaware monitoring systems and they even monitor each other. And most importantly defective systems do not always stop functioning, they can sometimes perform all the tasks they did before, just not as efficiently.
Of course all systems become obsolete when an upgrade is available, so why whine if humans are going to be replaced one day, it's not necessary a bad thing, we just like to see ourselves as superior and irreplaceable, but one day we will be replaced, either by evolution or by revolution.

2:06 AM, January 18, 2007  
Blogger Unknown said...

Ah, you young 'uns, don't know yer history.

As someone said, DWIM was an actual typo-correcting feature of Interlisp back in the 70's written by Teitelman.

Interlisp itself, until it died/was murdered by being too closely associated with the PDP-10 family of hardware that eventually died (sob), is a good example of one of the "living systems" you're talking about. It was a huge, rich implementation of Lisp with lots of loadable packages, etc., developed at BBN and Xerox PARC and a few other places.

3:59 AM, January 18, 2007  
Blogger keithb said...

So, it's a surprise that the examples of systems that have the desirable properties doesn't include any RDBMSs, which mostly exhibit those properties very strongly. But think about persistance lead me to follow another line of thought on liveness

4:04 AM, January 18, 2007  
Blogger Phil said...

Living software has a command shell, since you need a way to talk to it like a grown-up. It has an extension language, since you need a way to help it grow. It has an advice system, since you need a way to train and tailor it. It has a niche, since it needs users in order to thrive. It has a plug-in architecture, so you can dress it up for your party.

Yes, but...

The primary purpose of a plug-in architecture is so that you can share. If you've already got an extension language, you can dress up your software all you please; what the plug-in system does is allow other hackers to use your modifications. This is what's been driving me batty about Firefox; when someone shows me a cool Conkeror mod, I can't just drop it in my .emacs file and eval it.

A decent plug-in system is about sharing. Without sharing, you only have a single set of eyes, and your software is limited by how smart you are as a single programmer. When you can just toss code up on the emacswiki (for instance) then it can evolve and be shaped by the community.

Great article though.

4:32 PM, January 18, 2007  
Blogger Mike Dierken said...

Hey Steve,
Looks like another popular post. A couple factual controversies, but no bother.
On the Firefox extension mention, you can actually update an extension during development without a full uninstall/reinstall. You still need to restart Firefox though.
The .xpi is expanded into a folder - and one of the files is a .jar If you build your app into that .jar and copy it to the correct location, then when you restart Firefox picks up the changes.
I have an Ant build file that does that, but you probably can figure it out.

11:04 PM, January 18, 2007  
Blogger Avi said...

Great post!

I think the essential idea here was stated over 20 years ago by Brooks' No Silver Bullet essay:

"Let us turn nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and selfrenewing [sic]. The secret is that it is grown, not built."

He further states the benefits of growing a system: the morale benefits as well as the practical benefits to incremental improvement. He doesn't mention the "no rebooting" requirement, since back then it would have been inconceivable.

Besides, even living things reboot - humans (for example) aren't improved over the course of one lifetime, but rather over many lifetimes and many deaths and births. Birth is the chance nature gets to fix things. The cycle is much slower than with software, but it is a similar process.

12:38 AM, January 19, 2007  
Blogger Mitja said...

How do you justify murdering an operating system by shutting it down when going away (like a vacation or something) for a certain period of time?

Or is this merely an end user's point of view which is not so important?

I mean who's going to understand developer's reasoning behind live software, right?

3:02 AM, January 19, 2007  
Blogger Chris Ryland said...

Some excellent thoughts about language design and what makes a living system, but your final call to build "intelligent" software is rather naive. Isn't that what the AI folks have been trying for 40 years?

The basic problem is that you're a materialist, and what you don't realize is that true intelligence involves an immaterial principle (the soul). Aristotle got it mostly right about 2300 years ago, arguing entirely from visible evidence. You would do well to really read and understand his work. (Not that I do. ;-)

6:03 AM, January 19, 2007  
Blogger Matt C. Wilson said...

cpr,

What if we created something so irresistably "full of life" that a soul chose to inhabit it?

What would be the difference? What would prevent the possibility? Are souls very particular about human body plans and physiology?

Steve - truly awesome stuff. I feel like you've given a voice to a lot of my own random musings. Thanks.

8:07 AM, January 19, 2007  
Blogger Unknown said...

Isn't the problem, one based on the that of the hardware that we are programming for? The hardware cannot take instructions that model the real world in its entirety, because it relies upon a wholely ficticious construct known as the 'rational number'? Here's a secret: rational numbers don't actually exist (outside of maths). What I mean, is that none of the real values in the real world are rational numbers, unless they are measured relative to themselves (for instance, only a litre of water is exactly a kilogram: a mole of water is roughly 18 grames. Measure anything, in the real world, in units other than whole numbers of themselves and the maths breaks down into approximations, because the values all become irrational). Why is this a progblem for programming? Well, because a computer can never hold anything other than a rough approximation of real world systems, whereas brains - assemblages of neurons - clearly grasp them in whays computers never will.

An illustration of the difference between intelligence (as demonstrated by neurons, that work with irrational numbers all the time) and clever programming (as based on someone writing reams and reams of code, based on an attempt to approximate the real world using rational numbers), is to watch a spider build a web across a span of space.

Let's first consider how a computer programmers would solve this problem. A computer programmer would attempt to start from a ridgidly predefined set of criteria about the space (or set of idealised spaces) that the web must be designed to span: its dimensions, the number of spokes the web must use to fill the space, the exact coordinates of the anchor points, etc. Our newly programmed robot spider would then depart for the real world, only to end up wandering the earth, for all eternity, seeking an example that matched the spaces it had been programmed to fill, like some autistic surfer, forever seeking the 'perfect wave'.

In nature, however, a real spider comes upon a space which is not an exact match for any of the spaces that any of her predecessors - in all 250 million years of their existence - might have come across before. Her data about that space is not ridgid, either, like the computer programme, but rather drawn from such vaugue inputs, as the scent sensors on here forelegs that tell her that there is possibly a juniper bush within eight to twelve inches on the other side of that space - coupled with whatever vague idea her primitive eyes may give her of the world around her, and so on.

Bear in mind that all of the coordinates to be used in the construction of this new web are not only irrational numbers, relative to each other (physically unrepresentable in any numeric system) but also, that all of them are changing, constantly, as a strong breeze causes the entire space to deform in a chaotic whirl that is even further unrepresentable, mathematically. From this, we must surely begin to see that - whatever calculations, there may be, taking place within that tiny bundle of neurons within our spiders miniature head - they do not resemble the calculations that programmers must use when writing software.

The difference lies in the hardware: neurons just don't use the kinds of maths that transistors do.

That the spider then proceeds to build her web, across that space - based on 250 million generations of expertise, that is wise to the fact the juniper bushes make good achor points, for instance, and that flies tend to fly around where the breeze is strongest - is strong evidence that a such a computational machine certainly exists. Nor does it debar the possibility that we humans, through our ingeniuity, might not one day build such a machine - but it will not a be 'computer', any more than whatever dwells within that bundle of neurons in our spider's head, making the decisions, is not 'software', as we would understand it.

Once you have grasped how fundamentally un-software-like real systems are, you begin to realise that software is fated, always, to be crap.

After all, although our new web is a clear example of its kind, and as such, represents a clear descendant of that basic template of 'web' - first built all those 250 million generations ago - it is, itself, unique. No web has ever been built like it, before, and no exact match will ever be built, like it, again. And, although it is, structurally, an engineering miracle - more perfect than the roughly bodged-together effort our poor robot spider might have build - it is, itself, ephemeral. No effort is made to store it's data. The system lives on, but it always destroys its output.

My opinion, is that this is because software is written for transistors; and that whatever drives neurons, it isn't software, because neurons clearly aren't transistors.

8:18 AM, January 19, 2007  
Blogger Unknown said...

Matt--No, the soul and body aren't really separable in the normal state of being; they're different aspects of the whole person.

That's another problem: the modern world suffers terribly from the Cartesian split: the view that the soul and body are somehow two different principles, that the soul "inhabits" or "drives" a body. From this simple mistake springs pretty much all errors of our time.

No, the soul is the (immaterial) "form" (in the Aristotelian sense) of the (material) body, but they're not separable--they're a unit.

And, yes, man's immortal soul is quite distinct from an animal's soul, which is "educed" from the matter.

(And, yes, the big question is what happens to an immortal being (man) when the material portion dies. That's the question on which hinges a lot, but which is another discussion.)

10:26 AM, January 19, 2007  
Blogger Matt C. Wilson said...

Chris, if a soul is educed by the configuration of the matter containing it, and only by such a pattern, then why would a software-based, soul-having lifeform not be possible? Just recreate the pattern, right? 10 million monkeys on IDEs eventually produce AI?

(I can't tell from your posts if you do think such a thing impossible or not.)

Daniel - For the most part, I agree with you.

The action of neurons, in my opinion, (and thus the expression of conscious thought) come from the associations they have to each other. No "real world" thing at all lives in our heads, but mental models of them do.

No one knows what the square root of 2 is in the sense of having the entirety of the number in their head. What is known is how to calculate it, what it represents, what it means. The special association it has to the number 2, and the significance of that association to other things (such as the length of the hypotenuse of an isoceles triangle).

Where I disagree is that I think these association chains can be modeled in what would be considered software. A neuron is a thing, it can be described and simulated. Maps of them can be associated to external concepts, and the strength of the associations can be changed to reinforce or reduce their relative importance.

Check out The Age of Spiritual Machines. Ray Kurzweil has a lot of fascinating ideas about what it would take to produce consciousness in a machine.

11:13 AM, January 19, 2007  
Blogger Jake said...

After reading this, writing code today has felt very much like a static endeavor with no real flexibility for becoming better over time.

I'm realizing that the Zen like happiness I've had lately from using dynamic language tricks in my C# career has been the rumblings of the broader picture of building software. Not hardware.

12:32 PM, January 19, 2007  
Blogger Unknown said...

I would argue that mind is an emergent property of body.

Perhaps mind is the body-software's method of introspection?

2:00 PM, January 19, 2007  
Blogger Braidwood said...

mmm interesting. This is why it is good for me to read things that I know nothing about- you almost made me want to learn programming. For my interests, I most appreciate the idea of QWANness. The link in the comments to the QWANness guy might be useful too.

I'm interested in QWANness in design of many kinds, especially systems that let humans interact, like our physical spaces and how we structure our relationships.

ps: On the off chance that machines or software become concious, I like QWANness more than DWIM - it is more respectful and more likely.

2:19 PM, January 19, 2007  
Blogger Noam Mor said...

I immensely enjoy your insights, each time anew. There are obviously many statements you've mentioned here that need rethinking (which makes them even more interesting), but the notion of living as opposed to dead systems - of monolithic systems built without an interface to their internals and with no hooks for extensibility being hardware, rather than software - is truly eye-opening.

It's a shame most people won't read articles of that length.

2:29 PM, January 20, 2007  
Blogger Илья Казначеев said...

With all java's static typing and stuff, J2EE servers feel dangerously alive.

They're teeming with cryptic services, random exceptions, do a lot of self-monitoring, reload resources as they change.

9:45 AM, January 21, 2007  
Blogger Unknown said...

I would argue that Eclipse (and probably IntelliJ - though I don't use it) should be bumped up your list a bit. And here's why:

I happened to be doing some low-level debugging of some very quirky jdk-version-spanning UI code in Eclipse the day I read this post. And one feature of Eclipse, which I've been using for a few years now, came in really handy.

It's the hot-replace feature.

(I think the first Java IDE to have this was one of Eclipse's direct ancestors: IBM's VisualAge for Java. In fact I tried to move my work development over to it more than once just to get this feature, but it was never really flexible enough to adapt to my existing projects)

Hot-replace of classes is a feature that's been there since the start of Eclipse, but doesn’t work well in a lot of situations. It's not always clear to me, in fact, why it's working well when it is.

But this particular day it was just zinging along. I was stopping at breakpoints, examining variables, making fixes, and just … continuing. No stopping to compile, no re-running.

No reboot.

Sure, Eclipse doesn't apply your no-reboot policy to itself (in fact, kill an Eclipse session before it's had a chance to save its state and you're in for a full 10 minute downtime while it repairs itself on the next startup), but it does support it in my Java development workflow.

So nicely does it do this, that even after years of using Eclipse and seeing this feature in action (with many years before that of coding Java mainly with vim, sometimes with VJ++) I found myself thinking pausing to reflect on how wonderful it truly is. Surely it qualifies for something?

Also, though I seldom use it, the refactoring features qualify as DWIM:

I remember one day at work a decade ago, I started complaining to the guy next to me that software hadn't come far enough. I was doing a lot of porting between different Unix platforms and it was just way too repetitive. I picked up the mouse and spoke into it a la Scotty in that Star Trek movie about the whales. I said, "I shouldn't have to repeat all of these steps every day. And I shouldn’t have to spent 6 mths writing a script just to avoid repeating them. I should be able to pick up this mouse and say [insert Scottish accent here]
'Computer. Computer! You know all those files I copied from the Sparc machine yesterday to the HPUX machine, then all the changes I made? Do the same thing to on the AIX machine please'"
[If you think that imitation Scottish accent was bad, you should hear my Sean Connery]

Now, in a way, the refactoring tools in Eclipse have given me this. I can now say things, not exactly by picking up the mouse, not exactly in a not-exactly-Scottish accent, but with a nice UI that's the equivalent of saying this:
"Hey Eclipse. Can you remove this parameter from this method for me please. Oh, and change all the calls to it so they don't pass an argument, while you're at it? Oh, and you'd better show me a nice structured preview of each change, because I don’t totally trust you yet. What's this, you've removed mention of the parameter from the comments too? Well done!"

Now that's pretty DWIMmy, no?

Eclipse has a lot of issues of course, and I still fall back to vim on a regular basis for some things, but I think it deserves some more stevey-points based on:
- the no-reboot development cycle it affords me
- the DWIMmy refactoring
- the plug-in arhcitecture
- the non-obvious command-line-like feature of adding text filtering to simplify the complex options dialog
- and the strong lexical checking as you type (way beyond syntax colouring, just short of compiling) which also helps in the no-reboot development department

(Actually, none of these is even close to being a favourite Eclipse feature of mine - the candidates for that are what happens when you ctrl-click, ctrl-T or ctrl-alt-H on a method)

By the way: substitute IntelliJ IDEA for Eclipse, and emacs for vim and I'm sure the same arguments apply - these are just the two I happen to use.

So here's my vote for more stevey-points for Eclipse.

11:35 AM, January 22, 2007  
Blogger IanRae said...

I don't get the need for 'never reboot'. Sounds like a desire for perfection; that of course is impossible.

His complaints about reboot are about the "user experience" of reboot-after-update. If Firefox downloads an update, exits, and fires up a new process with the same user state, then who would care?

Similarly, if OS reboots took 3 seconds, they wouldn't be an issue.

Good article though -- keep it coming!

6:53 AM, January 23, 2007  
Blogger Cristache said...

I tend to agree with Ian that rebooting per se is not necessarily bad. As a matter of fact, because I am a lazy guy, I would never spend the time to design a full blown Pinocchio.

Rather, I would build something smaller and give it the capability to adapt and mutate. Once the system has accumulated enough changes it will die off. But not before it creates some offspring (i.e. reboots).

Nah, scratch that, its not that intelligent a software design.

5:11 PM, January 23, 2007  
Blogger David Clark said...

I wanted to write a longish response so I simply posted it at my blog to avoid making people wade through my babblings.

10:30 PM, January 23, 2007  
Blogger Steve Yegge said...

I left sleep out of the blog because I considered it an unsolved technical problem - i.e. I hadn't made up my mind as to whether it's a reboot or not. I'm inclined to say that it is.

However, I think it's a little silly that you and everyone else seem to be positing that the design of a human being is the best design possible, so if we reboot, software should as well.

The fact of the matter is: we suck, too. We die, which nobody needs to tell you is a serious design flaw. We sleep in a big block, which is equivalent to a mark-and-sweep garbage collector: not exactly state of the art. We have huge single points of failure, e.g. the base of our spine (at which we're too often paralyzed for life), our throats (we too easily choke to death), our hamstrings and achilles tendons (cut them once and you crawl for the rest of your life.) We can't regrow limbs even though we possess all the code to do so. We have poor powers of concentration, our vision goes bad and never self-corrects, I mean really the list goes on.

Just because "Nature" decided something doesn't mean it's a good design. Aim higher!

10:16 AM, January 24, 2007  
Blogger Alan said...

"If you don't like the way a person works, you don't kill them, fix their DNA, and then regrow them."

Eh ... that's exactly what you do if you don't like the way a person (animal) works: wait till it dies, and hope its offspring have some mutation which makes them better adapted to the environment.

"Why, then, do we almost always develop software that way?"

a) Because it is a proven (by evolution and other engineers) method
b) because we can improve on evolution in that we can apply better problem-solving skills than "and hope that a mutation ..."
c) Because variation on the code are very cheap to produce (hit F9, YKMV), whereas a living system is very expensive (one full solar system + billions of years is the most recent estimate)

"We die, which nobody needs to tell you is a serious design flaw."

It's not a design flaw - it is the reason we exist. Asexual creatures do not "die", as in "process termination" - they become their offspring. The advent of sex, and the concomitant death, were a major design improvement, allowing much more rapid searching of the genosphere for fitter programs.

And each oif those "huge single points of failure" will only be fixed by death of the individuals with them, re-writing the code, and birth of new individuals. Sounds like a reboot to me !

My death is a flaw iff I am the goal of evolution. but I kinda reckon "the blind watchmaker" has larger (or no) goals, especially since I had the tubes snipped !

12:49 PM, January 24, 2007  
Blogger David Clark said...

Steve,

I never claimed that "the design of a human being is the best design possible, so if we reboot, software should as well." Evolutionary theory very much denies that such a thing as the best design possible even exists, so humans certainly don't have it.

I was simply responding to one of the main points of your entry which your put thusly: "So my first argument against rebooting is that in nature it doesn't happen." I simply wanted to show that nature does reboot and does so for good reasons.

2:02 PM, January 24, 2007  
Blogger Jim Martin said...

People reboot sometimes - at least good portions of them. Like a bone marrow transplant, which is a reboot of the crashing immune system.

Anyhow, good post Steve. I always like it when RSS tells me you've done a new one...

4:02 PM, January 24, 2007  
Blogger Steve Yegge said...

Alan, the need for reboots in nature may be the reason we exist today, but it has -become- a design flaw, or at least it's rapidly becoming one as our genetic technology improves. Obviously nature has had to do lots of reboots to get us where are today. But I think the overall trend, in both nature and in software, is towards not having to reboot. Requiring reboots is part of a bootstrapping process that eventually gets you to long-lived systems that don't need to do it.

I guess I'm subscribed to Richard Dawkins' point in his _Selfish Gene_ that at some point it'll be in our genes' best interest to stop behaving the way they've been doing for billions of years, and turn things over to us to let us decide what's best. If my genes turned things over to me, I'd vote for fixing my code without rebooting me, regardless of how it's been done previously.

All: sorry for missing Smalltalk and Erlang and RDBMSes in my exposition. It was written late at night. Also, my apologies to Warren (now at Google) for attributing DWIM to Larry Wall.

I'd edit the post itself, but I'm afraid Blogger might suddenly change the URL on me, the way it did with Good Agile, Bad Agile (which now ends in _27.html for mysterious reasons.)

6:05 PM, January 25, 2007  
Blogger Steve Yegge said...

Rhubarb - Eclipse gets -zero- stevey-points for any of the features you've listed. They're all great, wonderful end-user features: they help you write Java code. That's nice, for what it's worth.

However, I would surmise that you've never written an Eclipse plugin. Few Eclipse users ever have. I have, and it's a frigging pain in the ass. I once did a side-by-side comparison of hello, world in Emacs and Eclipse (both of them installed a menu with a selection that brings up a dialog that says hello). The Emacs one was 11 lines and made perfect sense; it was also completely declarative. The Eclipse one was close to 100 lines of miserable boilerplate crapola, and every change required you to restart the target Java process, because (as you point out) Eclipse can't hot-swap its own classes. Emacs can hot-swap its own functions, of course, except for the small (and annoying) subset that exist in the C/hardware layer.

You need to get *inside* Eclipse, as a programmer, and be able to peer at its data structures and modify the thing yourself, before you can fully appreciate the points I'm making here. You'll find that it's nontrivial. It's nontrivial in Emacs, sure, but do a deep dive into both and you'll find that all of Java's static typing hasn't helped at all, because you still gain all your knowledge of the system from the human-written documentation, and from interactively trying stuff in the code.

Even attempting a shallow dive into Eclipse is painful, since there are few shells (at least as far as you know) that give you access. You have to wire up Jython or some other scripting language, at which point you're no longer really using the Eclipse model; you've bolted real software atop your hardware, and it's cheating.

Eclipse gets no love from me. It's a really spiffy mannequin for doing Java development, but it's pretty far down the QWAN scale for doing *Eclipse* development.

8:16 PM, January 25, 2007  
Blogger macbeach said...

"Linux has gone through some growing pains here, and it's finally getting better, but I would guess it still has many more reboot scenarios than Windows does today."

Where did you get that idea? Quite a few people have first tried Linux precisely to avoid the frequent reboots of Windows. Windows has improved of course, but Linux has always been ahead in this game.

As a previous comment mentioned you only have to reboot if you update the kernel. Even then, you can delay that for as long as you like. I've never had a case where Linux told me I had to reboot now and I couldn't do anything else until I rebooted. Windows and OS X are famous for this behavior.

I don't have a problem with occasional reboots, as long as I can pick the time. No OS is so perfect that it doesn't leave some garbage laying around in memory from time to time, even if the OS is only obeying the request of some faulty application.

I tend to leave my computers running all the time. I used systems that can idle into a low power state without generating heat, fan noise, etc. When I am done using my computer for the day I reboot it. I don't have to wait for the process to finish, and the next time I use the computer it is in a relatively pristine state.

I don't use Windows any more, but I've noticed while helping people that do that the newer versions also allow updates and reboots to be done at the end of a users day, mimicking pretty much what I do with my Linux machines. If I HAD to use Windows today I'd probably hate it a lot less than I used to. But I can't think of any reason to go back either.

1:18 PM, January 26, 2007  
Blogger Kevin Fairchild said...

"If we ever wake up a real AI, I think it should be named Pinocchio."

...and if it were to ever lie, would the End-User License Agreement grow?

8:41 AM, January 30, 2007  
Blogger Michiel Trimpe said...

Wow, wow and wow! I haven't finished the entry yet but it's already the best I've read ... EVER!

I just hope I'll be the one to build that system. I'm working hard on it just so you know! It'll be a great Djinn!

3:25 AM, January 31, 2007  
Blogger bugm3n07 said...

I have recurring nightmares about living in a world where every application is an "extensibility tarpit", and in order to perform simple tasks like sending email, you need to know the ins-and-outs of s-expressions, monads, category theory, and the gang.

5:01 AM, February 02, 2007  
Blogger Daniel Berger said...

So, Christopher Alexander reinvented "je ne sais quoi" as "Quality Without a Name" and now he's famous, eh?

Wow, I need to look around for a foreign phase that I can reinvent as a computer related acronym so I can become famous, too!

2:02 PM, February 06, 2007  
Blogger Radek Maciaszek said...

It was waste of my time to read that article because it's crap. I wouldn't say though that all articles in the world which are not so crap, however this one is. Well following author argumentation let's say it's crap because it will dissapear from my screen once I will close the browser. Let's even call it dying. It's the same with software, author doesn't care that software could have a memory and save to it but it reboots so it's crap. Same with this article even because I have a memory and will unfortunately remember parts of this crap article in overall it's still crap.

2:24 AM, February 14, 2007  
Blogger Ralfoide _ said...

"If you don't like the way a government works, you don't shut it down, figure out what's wrong, and start it back up again. "

Well actually yes, governments get rebooted all the time. It's called revolution or coup d'état. And the new programming isn't always any better than the old previous one. I guess they suffer from version 2.0 syndrome too.

2:52 PM, February 15, 2007  
Blogger jto said...

This comment has been removed by the author.

6:49 AM, February 21, 2007  
Blogger jto said...

(part 1)

The great thing about Unix scripting is that each piece (each tool, like "grep" or "find" or "xargs") is just a machine. It starts, does its bit, and dies.

The Perforce client (p4) is a plug-in. What does it plug into? Unix.

The key vi-versus-emacs insight is that vi is not a standalone program, the way emacs is. vi is a Unix plug-in. They're two different things. vi is grep. Emacs is Google.

I think the IDE (emacs's approach) has basically won out over the "editor as Unix tool" (vi's approach). For programmers anyway. But generally the decision of whether a particular piece of software should be "conscious" or merely mechanical--a framework or a plugin--is crucial and nonobvious, and "framework" is often the wrong choice.

I really *really* like p4 and svn as strictly mechanical tools. Although I'm a committed emacs user, my P4EDITOR is set to vi.

7:10 AM, February 21, 2007  
Blogger jto said...

(part 2)

Make everything a framework and you get Windows. Seriously--the difference between Windows and Unix is that on Windows, everything's an API. Whereas on Unix some things are APIs and some things are just text files, and some things are scripts, and some things are magical imaginary filesystems (like /proc). Absolutely everything on Windows is programmable... and somehow, as a result, it feels less hackable as a whole. And less stable.

A pluggable system that never reboots eventually gets crufty, slow, and insane, like a comic-book monster that evolves too fast. Unix scripting is super-pluggable; it works because the individual tools run and exit--they die quickly. Windows is super-pluggable, and the longer you use it, the more stuff you install, the worse it gets. Reboots may help, but eventually you just need to format the drive (or more likely, buy a new computer) and start fresh. Which is like the ultimate reboot.

Pluggable systems need *some* kind of discipline to keep them from going insane like this. Something new. I don't think it's static typing. I think it's more of an install/uninstall discipline, automatically tracking the changes plugins make to the system, and a better kind of introspection aimed at troubleshooting and placing blame, so you can figure out which software you hate and uninstall it, and it is actually really uninstalled.

7:16 AM, February 21, 2007  
Blogger jto said...

Lastly... I don't know how much you know about Haskell. Your article makes it sound like you really don't know anything about it, but hey, at least you have an opinion.

I think Haskell requires a different way of thinking. More so than, say, Scheme. In a way Haskell is more adaptive than Python, but in a 100% declarative way instead of the 100% imperative way of Python.

In Haskell you don't get adaptivity by modifying class objects at runtime. But good Python programmers don't do that either.

7:19 AM, February 21, 2007  
Blogger Juanych said...

The truth must be said – I've never rebooted myself. Not even tried.
Woof! It was hard to confess, but now I definitely feel better when everybody knows about my inexperience.

After reading your note I’ve realized that all present developers believe in reincarnation. That obvious!
Now I suppose most of them are somewhat Buddhists, with several Zen peoples among 'em.
And *sigh* I am no better.

And no Christians. That's for sure.

I like it. Not Buddhist, but situation itself. It’s a kind of primal, tribal. Caves and stone axes are included.

So about Christians. The question is - are there some Jewish devs working for 40 years under Moses leadership somewhere in Haifa labs?
If they are, then there’ll be some powerful crusades in nearest future and a lot of speculations ‘bout software life after power off!
And just imagine all written code going straight to hell for their sins! Breathtaking.

Or is there some lead guy named... well... let it be Darwin. Guiding his fellow programmers to develop the evolutionary software?
PM named Karl Marx is a useful option, but do not let him to the marketing department or you’ll see a lot of black PR aimed primarily at some Russian hackers but spoiling entire brand as well!

Are you the first atheist in this software India jungle? Progressive movement! I’m in :)


Well, 10x thanks for your articles! All of 'em!
Lot of fun and such strange questions they provoke :)

4:18 AM, February 22, 2007  
Blogger Matthias Benkard said...

jto,

You're right about Unix, of course. That's why Steve listed it as one of the best pieces of crappy software out there. Unix lives. You can tinker with it, you can replace parts of it without rebooting, you can do everything with it that you may want to. Except replacing the kernel, but that one is a relatively small piece of the system. It's just like Emacs.

You're wrong about Windows, though. I don't know about you, but I can't see how Windows encourages me to tinker with its internals. First of all, there is no usable, interactive interface to its internals at all, either GUI or command-line based. Second, and I may be wrong about that, I don't see a way to actually replace Windows internals in a supported way without resorting to ugly, ugly hacks.

In Windows, everything may be an API -- but a closed one! That is the fundamental difference between Unix and Windows.

Your remark about Haskell is a bit weird, too. How is Haskell code adaptable at runtime? As far as I can see, it isn't. I don't think you could implement something Emacs-like in Haskell without effectively disabling or circumventing all the features that make Haskell special. Take hs-plugins, for instance: It's nice and all, making extending programs in a static way easy, but does it help you make an interactively adaptable system that lets you swap and try out stuff on the fly? (Maybe it does, I really don't know. Can you edit Yi inside of itself?)

As Steve wrote, It's all about developer-friendliness. A Haskell-based system may be user-friendly, but it will not be easy to make it developer-friendly. I'm not saying it's impossible, mind you. It's just that Lisp and Smalltalk and all those other highly interactive environments are probably much better suited for the task.

(Personally, I'd one day like to see a system that is Haskell-like in the small and Lisp/Smalltalk-like in the large. Although I must confess I currently don't really understand what that would mean, it would certainly be very cool. :))

8:22 AM, February 26, 2007  
Blogger Nate Cull said...

Just discovered your blog... excellent post. It sums up a number of things I've been thinking myself.

I would phrase the essential quality of QWAN software slightly differently, though: not foremost that it not reboot, but that it be *self-aware*. The two things might approach, but I think self-awareness is the key characteristic which is missing from many discussions of software engineering today. In fact, I think if you asked most engineers or OS or language designers whether self-awareness was desirable, let alone whether it was achievable, they'd blanch and say definitely not. Like you say, they want marionettes.

By 'self-awareness' I don't mean anything require deep Turing-complete AI. I just mean simply that it must be possible both to describe a system fully in itself, and modify that system from within itself. From this all else ought to follow.

This means there must be no exceptions to the system: no syntax unparseable by the runtime, no layers of type systems that can't be reasoned about at runtime, no compiler that does deep black-box magic and generates inferences that the running program can't refer to. Everything that a compiler or debugger can do, the core language runtime must be able to do.

From this derives everything else. A command shell, or equivalent, is needed because that is how a self-aware system talks to itself. Plugins likewise are ways of modifying a system from within itself. Not rebooting follows because you don't *need* to escape the boundaries of the running system in order to change it. Introspection/reflection and process health monitoring is a given. Advice, also.

That's also why C syntax is bad: it is not a self-aware syntax. You can't easily describe C syntax in itself. You need something like BNF to do that, and there's no good way to represent BNF in C.

It also strikes me that operating systems often seem to offer more in the way of self-awareness (control over running processes, ability to read and write files) than languages do - often by accident, or out of pragmatism rather than intentional design. Yet both OSes and languages also seem to be ways of solving the same problem. It surprises me that they have not yet converged.

On the flip side: working in systems administration, I often get extremely frustrated with the challenge of how to cleanly capture, distribute and reproduce configuration information from quasi-living systems. The nice thing about dead systems is that all the data necessary to reproduce them identically is stored in one place. But as soon as you put a program, no matter how restricted its type system and unmodifiable its code, into production with a live database, it instantly gets some of the bad characteristics of a true living system: it accumulates a huge layer of data and configuration information without which it can't function correctly, but offers no way of importing or exporting this reliably. So you resort to cobbling together database dumps, tape backups, virtual machine images, and disk cloning tools and pray that if the system goes south you can somehow reproduce something a little like it that's not eaten a fatal contradiction in its little mind that will choke it months down the track.

If all software trends toward the living, how do we keep living software systems physically and psychologically healthy? Have we even really begun to research this kind of question?

10:02 PM, March 11, 2007  
Blogger Ted Mielczarek said...

Just for the record, you're pretty much completely wrong about Firefox extension development. First of all, extensions are developed in the same language as the rest of the app, and second, no sane extension developer does the whole restart cycle these days: http://kb.mozillazine.org/Getting_started_with_extension_development#Registering_your_extension_in_the_Extension_Manager

6:25 PM, March 13, 2007  
Blogger Steve Yegge said...

I'm pretty far from "completely wrong" about them. The article you link to is relatively new, in the grand scheme of things, and all it really demonstrates is that the situation is improving, which is great.

It doesn't, however, make me wrong.

You shouldn't have to reload the window when you make a change, so the reload-chrome feature should work everywhere (including Thunderbird). You shouldn't have to download and install an extension in order to get this sped up development cycle. You shouldn't need this crazy package structure for hello-world:

helloworld.xpi/
chrome.manifest
install.rdf
components/
defaults/
preferences/
mydefaults.js
chrome/
helloworld.jar
content/
overlay.js
overlay.xul
locale/
en-US/
overlay.dtd
skin/
overlay.css

(from the link you cited). And you've obviously missed my point about the language they're written in. How many languages do you see in the manifest list above? I see css, dtd, js, rdf, and of course xul, which includes both html and at least one xml minilanguage. Plus whatever that manifest file is written in.

That, friend, goes well beyond mere ridiculousness and reaches almost into "evil" territory.

I know Brendan's pushing hard to get this stuff addressed in future versions of Firefox, and I applaud him for it (heck, I applaud him for a great many things, not just that).

But the state of the union -today- for Firefox extension development is just plain awful.

To be fair: it's no picnic in many other systems either, including Eclipse and IntelliJ. I'm just picking on Firefox because I expect better from what I consider to be the world's leading app -and- platform contender.

11:03 PM, March 13, 2007  
Blogger Alan_Ford said...

Why is it all programmers always quote their time that they needed to type in the code? You do it even with your blog - that's why software is crap! - everyone wants to solve the "Pinocchio problem" - but to show off more important it is to do it within 4 hours (no matter how much thinking went into that altogether). Until we change some attitudes and play less Gods for the sake of DWIN/QWAN we are far away from AI ... Otherwise quite a good think piece

2:23 AM, April 11, 2007  
Blogger root said...

This comment has been removed by the author.

12:16 PM, May 18, 2007  
Blogger root said...

I agree with a lot of what you said, but I'm a bit disappointed that you realize that lisp would be the best system, but immediately throw out the idea of actually using it.

Why can't Scheme grow? Why can't Common Lisp grow? Arguably, if they do they will no longer be Scheme or Common Lisp anymore, but at least consider the possiblility of a new dialect of lisp which addresses the fallacies of Common Lisp as we know it and provides a movement foreward into the future. Can you imagine what it may be called? Arc, perhaps?

One other problem I have with your post is plainly this: NFAs (Non-deterministic Finite Automatas) are equal in power to DFAs (Deterministic Finite Automata), so why do you claim that they are, in fact, more powerful? In terms of programming languages, the only difference is brevity, which is important for quick growth, but they are still equal in power. I get the idea that you think otherwise in this article, but perhaps I'm reading it wrong.

12:20 PM, May 18, 2007  
Anonymous Anonymous said...

Good, the article was vwry nice. It was inspiring too. Internet marketing is growing very rapidly day by day. Intersted in online marketing, start your business today.
VORTEX.COM

1:39 AM, June 29, 2007  
Blogger rblaa said...

Interesting article. Regarding reboot is bad, I think the situation is not so black and white, as usual.

Consider console games, in particular. Relying on their reboot property actually simplifies their implementation. In a hostile consumer environment you can usually count on the fact that these games simply work, are never corrupted, etc.

The measure of their success? People play them, yourself included. That's all the matters, really.

4:30 PM, July 03, 2007  
Blogger Sriram P C said...


My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible.

- Go To Statement Considered Harmful, Edsger W. Dijkstra.
I was reading for the nth time, and I just remembered your "The Pinocchio Problem". Is this the source of all Pinocchios ?

1:04 AM, July 13, 2007  
Blogger SYSMGR said...

Never reboot? Expand on-the-fly?

It can be done for over 25 years: Take an OpenVMS cluster, have it properly set up and you're done. There are numerous sites that have their user applications up and running for years (I know a number of sites that is up and running for more than 10 years, have all software updates applied, moved and replaced hardware, without users being shut off the system. It may even be possible to continue processing in case a box fails entirely, within seconds. In a multi-site environment, you can fail-over in seconds, where the users on the non-failing node wouldn't even notice the other went down.

10:46 PM, July 30, 2007  
Blogger admin said...

the article is long and took quite some time to read, but i enjoyed reading it.very nice..

7:21 AM, August 05, 2007  
Blogger Unknown said...

The article was really interesting, but like a good film which occaisionally lets a glaring inconsistency of plot through, the statement that "Excel is a good piece of software" is troubling.

I use it daily on mac and pc, and as a time waster, as a productivity drainer its second to none. No need to manually reboot, because it will crash and reboot for you. If one tries to use more than 100mb of Excel files, it runs out of resources - regardless of system memory. If you need to use the "mod system" (anything more than calculations), then its a clear sign that you are using it beyond its powers. It regularly corrupts data and any 3rd party "extensions" only add to its fragility and superb ability to destroy data - such is the beauty of its "mod system".

Rant reply sorry, but a reply had to come to "Excel is good software", If karma existed the authors of Excel(yes i know they have written lots of books)would be burning in many versions of Hell.

12:48 AM, August 30, 2007  
Blogger Unknown said...

Blog Comments
Easy to use SEO

4:09 PM, November 24, 2007  
Anonymous Anonymous said...

Neopets Cheats

Neopets Avatars

4:14 PM, November 28, 2007  
Blogger Caio Tiago said...

Enforcing some comments made before...

the only thing on Linux that requires a reboot is upgrading the kernel and... you can even upgrade your version of Linux (including the window manager and core libraries) without requiring a reboot.

Upgrading the window manager itself requires you stoping and starting it, so you logon and login again (OK, but you do twice a year and are only 15 seconds).

Sometimes there are security issues that enforce you update the kernel, but rebooting the system for kernel upgrades is strictly necessary.
That's much different from rebooting due to a third party driver or any product install.


And I strongly agree with you that Firefox needs a better extension system.
The only thing I can tell you is that you just need to copy some files on the folder and restart - there is no need to use the UI for this.

The session restore feature, including session cookies, makes these restarts less painful.
Actually you can invalidate the cache of the XUL window and shouldn't be hard to make it re-read the extensions folders.
Installing new ones may be harder (due to the registration of new components), but updating existing extensions could be improved easily.

10:05 PM, December 24, 2007  

<< Home