Cleanroom Software Engineering
The vast majority of software today is handcrafted by artisans using craft-based techniques that cannot produce consistent results.
— Cleanroom Software Engineering, Prowell et al., 1999
I've been interested in the idea of
Why is it that a system of standardising upfront planning seems to have evaporated? It helps avoid costly rewrites and debugging later on, helps align disparate teams and companies on project realities, and provides management with higher-confidence estimates (and thus timelines). This lack of adoption confuses me, and makes all the sense in the world.
Most every team I've worked on has been run under some semblance of scrum. I will not feign neutrality. I despise scrum. More accurately: I despise how scrum is often used and managed. I think scrum is too benign to itself be problematic. It's only problematic in its usage as a tool to measure people, not team and product; in ignorance of the idea of complexity, and its inherent and necessary separation from "time spent on task" (which every manager I've ever had has fallen victim to, whether feigned or real).
The baseline "common-sense" tenets that Cleanroom codifies are:
- Usage-based, statistically-valid testing
- Tests should be driven by real-world input and usage. What gets tested should be driven by some specific metric; that metric can be how frequently used a feature is, how critical the feature is (or how high the cost of its failure), or any other metric. Additionally, tests should be randomly generated (in terms of the steps they perform and the inputs to each of those steps inputs) so that they can prove a certain confidence (e.g. the ol' 95%) in the ability for the software to perform correctly. Broadly, this speaks to establishing metrics for measuring the actual reliability and performance of each piece of software over time.
- Incremental development
- Create a small, working subset of each new product, feature, or change first — the "hello world". Get something working. Make a ball of clay that can continue to be molded, so long as it is a subset of the end requirements. For example, this can involve creating a top-level thing, like a starter app, and stubbing out as many of the known sub-functionalities of the app as possible. This also gets the product to a point that multiple people can start to work on it ASAP without stepping on each others' toes.
- Planning and specification
- The description of functionality (of a function, library, or product) should progress from a black box to state box to clear box. A black box describes the range of possible inputs and outputs, as well as specific test cases (a list of inputs with their expected output). A state box describes the minimum state that must be tracked in to transform the input to the output. A clear box is the complete implementation (or a design thereof).
Mainly I wonder about validation, ownership, and liability. This seems closely related to the concept of intellectual control which is repeatedly used in Cleanroom Software Engineering. Intellectual control is a fancy phrase for the idea that some group of people should understand how a given piece of software works — and they should have some agreed-upon source of truth against which to validate any part of that piece of software.
This leaves me with a few questions:
- How much intellectual control do we have today, as a rough percentage?
- Put another way: How much of any given piece of software (proprietary or open-source) is understood by n+1 people, let alone one?
- What happens if we give up intellectual control, in the short term, and in the long term?
- What does giving up 20% of our current intellectual control look like? 50%? 90%? When do we start to feel it?
I've seen a lot of chatter about AI slop, AI ethics, AI vs human jobs, etc. They feel off to me. There is either flailing about in the legal aspect (which we need desperately) or the economic impact (which we do need to address). That all being said, the experience I have had with AI (and various friends in software echo) is that it is fast when given very specific and detailed instructions, but it's not necessarily writing better code. It doesn't necessarily have attention to detail. It's sold around doing things faster, cheaper.
I'd love to see us break this cycle. AI does not seem to experiment, nor learn from mistakes. It mimics. There is no feedback loop for it to truly make art, because art requires repeatedly making mistakes and learning from them. Art requires refinement. I think (and maybe I'm just jaded) that most people don't consider software engineering (or "coding", or any engineering discipline for that matter) to constitute an "Art". I have met and learned from far too many artists in this work to agree. It has always been the job of artists to critique, of engineers to constrain the blind march of progress into a realm where it continues to benefit the souls and lives of everyone. I think we engineers and artists have been slacking, or wilfully ignored, but we've always needed to fight harder for what is right.
I hope that we can find a way to break this cycle as a species and a global community; as friends; as artists; as engineers; as a species; as a global community; as a branching arm of the tree of life. I hope that we create a more sustainable, humane, and equitable way to live and work. I hope we don't clobber our living world in the process, a world of which we are all a part.