One of the defining interests in the last 20 years or so of my life has been scientific skepticism. I try to live my life in such a way that this type of thinking colors pretty much everything I say and do. While at first blush this might sound like a miserable, confusing existence, in practical terms, it just means that I try to not believe anything without what I consider to be adequate evidence. The key words there are "belief" and "adequate" - it doesn't mean that I question every claim equally: if you tell me you had a ham sandwich for lunch, I'm not going to challenge you even though you offered me fairly weak evidence. The stakes are low, so my belief doesn't matter much. But as Carl Sagan famously said:
So the amount of evidence needed to hold what I would call an "actionable belief" should be directly proportional to how important or consequential that action is.
One important area, though, that I've never been able to square my skeptical proclivities with my own life is in my profession: software development.
There are other disciplines that have a sort of skeptical movement either built into them, or at least have a significant skeptical subculture. Most notably, one obvious industry that ought to be filled to the brim with skeptics has only relatively recently begun to formalize its way of skeptical thinking is medicine. Of course, when lives and fortunes are at stake, the quacks come out of the woodwork, which explains why medicine (and by extension, fitness, which will be a topic for another post or two) has probably had such a hard time gaining even as much ground as it has as a scientific discipline.
But back to software. I've always wanted to explore how to marry my love of science with what I do for a living. Yes, I have a degree in computer science, but let's not kid ourselves: not only do most software developers not do computer science, computer science isn't actually science. Science involves making predictions about the natural world and testing them either by observation or experiment. It's tough to argue that many programmers are getting paid to do anything resembling this.
In any event, I've been thinking more about this lately. Surely there's got to be a way to apply scientific thinking to improving what I do for a living. I've long been a practitioner - some would say "evangelist" - of Agile software development practices in my career. However, it's always nagged at me that while I really do believe in the benefit of TDD for instance, I don't see a tremendous amount of convincing evidence that it's actually better than a test-last approach, and considering the cost, that leaves me feeling like I'm running dangerously close to violating my own skeptical principles!
So I was recently listening to one of the few software development podcasts that I have on regular rotation, Software Engineering Radio and one of the guests (sorry, can't remember which one, it might have been this one on DevOps which in any case I recommend) mentioned Hypothesis Driven Driven Development which sounded at first like it might be what I was looking for.
But after a bit of poking around the web and reading a few good articles like this one I realized a couple of things:
- For better or for worse, there doesn't appear to be much traction around this movement. That doesn't mean it doesn't have any merit - heck, this fact alone could mean that it's worth more of my time exploring. But then there's the fact that...
- This is less a development discipline than it is a philosophy on how to approach what to build, rather than how to build it, like all the other "X-based-development" or "Y-driven-design" methodologies. Again, there's nothing wrong with this, but it doesn't really tell me much about the scientific (ie, evidence-based) validity of how I'm building my software, informing me on whether certain practices yield better results, which is what I am interested in. Hypothesis based development is more useful to a business to know if they're paying guys like me to build the right thing. Definitely valuable, but not necessarily of primary concern to a developer.
So even though it didn't seem like this was exactly what I was looking for, it at least finally spurred me into a bit action to do some more research. And then I came across this talk by Greg Wilson:
I thoroughly enjoyed this talk, and it is definitely honing in on what I was thinking about. I completely agree with Greg in that the pursuit of evidence is critical to everything we do, and if software is the first thing that gets you thinking about it, all the better. In any case, I have now gotten myself a copy of his book, Making Software: What Works, and Why We Believe It and will report back with an in-depth review ASAP.
It's unfortunate that most of the thinking on this topic seems to still be relegated to academia. I don't mean that as an insult - I have tremendous respect for academia - but unfortunately it moves much slower than industry, and in an industry that moves as quickly as software, it's always going to be hopelessly out of date. So in my opinion it's up to folks like us in the field to move this forward.
That's all for now - I'll come back when I have something more actionable to recommend. Until then, stay skeptical, my friends!