Against method: routine, innovation and epistemological anarchy

June 22, 2006

Confused is picking up on CIO's distinction between routine and innovation…

“Organizing for routine work:  Drive out variance”

“Organizing for innovative work:  Enhance variance”

Back in the early 90s the tension between the 'impose a routine' imperative of structured methods like JSP and Constantine/Yourdon and the necessary creativity and innovation of software problem solving drove me mad with frustration. Object orientation was crossing over into the mainstream, and a new generation of methodologies based on structured methods, but tweaked for OO were propounded by their advocates.

The promise of those methodolgies was to make software development routine and predictable, hence their appeal to managers. Of course, we now know software development isn't routine and predictable, and trying to make it so is like trying to nail a jelly to the wall. Trying to make it so is to organize for routine and to attempt to execute innovation. No wonder it didn't work! More recently, agile approaches have acknowledged the essential nature of dev work, and offer a process that works with, not against, its intrinsic nature.

So back in the early 90s I'd become so maddened by the impose routine/execute innovation dichotomy that I spent a week in the Bodleian Library researching a paper that would decisively reject the claims of the software methodologists. It was to be based on twin foundations: the empirical study of programmers, and Paul Feyerabend's Against Method.

I first came across Feyerabend 20 years ago as an undergrad, browsing the Philosophy shelves in Heffers. I came across a copy of Farewell to Reason, and intrigued by the title, bought it. I'd been studying the Philosophy of Science, and covered the usual ground with Popper, Kuhn, Lakatos, Papineau. But no one had mentioned Feyerabend. Once I read Farewell I realised why. It's packed full of exciting, radical & contrarian ideas – for instance he defends the Church against Galileo !

Much of the Philosophy of Science is about science's purported special status in human intellectual activity as a generator of truth and certainty. As such it attempts to give an account of how and why science can generate knowledge, usually in terms of the scientific method. The logical positivists characterised the scientific method in terms of the principle of verification. When that was shown to be self contradictory, Popper's falsificationist ideas came to the fore. They're all epistemology, being theories about knowledge.

Feyerabend is an epistemological anarchist. Against Method is a sustained polemic against the very idea of a scientific method. It argues that any conceivable scientific method is refuted by examples of actual science. He examines cases from medicine, chemistry and physics and shows that no progress could have been made if scientists had stuck by rigid rules as per Popper.

Against Method is full of delightfully contrarian insights such as "semantic sloppiness is the prerequisite of progress" and "the only principle that does not inhibit progress is anything goes". When I returned to the book in the early 90s, it seemed obviously applicable to software development. Now that Agile is widely accepted, that argument doesn't need to be made.

So when I see people asserting that we need more than innovations, we need a process for innovation, I think "the only principle that does not inhibit progress is anything goes". It's a lesson that's been learnt at great length and expense in software development. And it looks like there are many others who could still benefit from Feyerabend's insights.


3 Responses to “Against method: routine, innovation and epistemological anarchy”

  1. Paul Gladen Says:

    Certainly overly restrictive processes will kill creativity and innovation. But in the absence of ANY process how do you avoid chaos and actually create a profitable business?

    An innovation process is not about a step by step formula for developing the central concept, product or piece of software. An innovation process is about enabling that concept or product to reach the market and prove itself as a true innovation (i.e profitable). Without it we are usually left with ideas or perhaps art and in business a bunch of frustrated employees and customers.

    You have noted that Agile Approaches “offer a PROCESS that works with, not against,…[the] intrinsic nature [of dev work]”. Surely innovation processes can achieve this too? It’s about balancing different levels of creativity and discipline at each stage of moving an idea through to market reality.

  2. etrading Says:

    Guess I’m arguing that one should never formulate process without a sympathetic examination of practive. And that any process should allow exceptions, because of the ‘anything goes’ injunction.

  3. Paul Gladen Says:

    Now that I can definitely agree with!

Leave a Reply to etrading Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s