Against method: routine, innovation and epistemological anarchy
June 22, 2006
“Organizing for routine work: Drive out variance”
“Organizing for innovative work: Enhance variance”
Back in the early 90s the tension between the 'impose a routine' imperative of structured methods like JSP and Constantine/Yourdon and the necessary creativity and innovation of software problem solving drove me mad with frustration. Object orientation was crossing over into the mainstream, and a new generation of methodologies based on structured methods, but tweaked for OO were propounded by their advocates.
The promise of those methodolgies was to make software development routine and predictable, hence their appeal to managers. Of course, we now know software development isn't routine and predictable, and trying to make it so is like trying to nail a jelly to the wall. Trying to make it so is to organize for routine and to attempt to execute innovation. No wonder it didn't work! More recently, agile approaches have acknowledged the essential nature of dev work, and offer a process that works with, not against, its intrinsic nature.
So back in the early 90s I'd become so maddened by the impose routine/execute innovation dichotomy that I spent a week in the Bodleian Library researching a paper that would decisively reject the claims of the software methodologists. It was to be based on twin foundations: the empirical study of programmers, and Paul Feyerabend's Against Method.
I first came across Feyerabend 20 years ago as an undergrad, browsing the Philosophy shelves in Heffers. I came across a copy of Farewell to Reason, and intrigued by the title, bought it. I'd been studying the Philosophy of Science, and covered the usual ground with Popper, Kuhn, Lakatos, Papineau. But no one had mentioned Feyerabend. Once I read Farewell I realised why. It's packed full of exciting, radical & contrarian ideas – for instance he defends the Church against Galileo !
Much of the Philosophy of Science is about science's purported special status in human intellectual activity as a generator of truth and certainty. As such it attempts to give an account of how and why science can generate knowledge, usually in terms of the scientific method. The logical positivists characterised the scientific method in terms of the principle of verification. When that was shown to be self contradictory, Popper's falsificationist ideas came to the fore. They're all epistemology, being theories about knowledge.
Feyerabend is an epistemological anarchist. Against Method is a sustained polemic against the very idea of a scientific method. It argues that any conceivable scientific method is refuted by examples of actual science. He examines cases from medicine, chemistry and physics and shows that no progress could have been made if scientists had stuck by rigid rules as per Popper.
Against Method is full of delightfully contrarian insights such as "semantic sloppiness is the prerequisite of progress" and "the only principle that does not inhibit progress is anything goes". When I returned to the book in the early 90s, it seemed obviously applicable to software development. Now that Agile is widely accepted, that argument doesn't need to be made.
So when I see people asserting that we need more than innovations, we need a process for innovation, I think "the only principle that does not inhibit progress is anything goes". It's a lesson that's been learnt at great length and expense in software development. And it looks like there are many others who could still benefit from Feyerabend's insights.