doing the thing right vs doing the right thing

Getting back to the concepts of unit testing etc - there was this intro article on TDD. (Not that we’re fully embracing that path at my current employer - though obviously some principles apply) I always hate how cherry picked TDD examples are - always functional programming-ish-- some subroutine function that does a simple data transformation, so the input is (often extremely) straightforward, and the output is similarly self contained, and easy to check, and there are no side-effects or external dependencies to think about.

Pondering on the coding I do (and preparing to drink the kool-aid of TDD and heavy unit testing-- or at least more deeply understand what its fans dig so much about it, and why they find it worth the self-evident cost of writing everything twice, and maintaining tons of mocks as systems grow) - I think most of the UI code I’ve done here falls into 3 rough categories:

1. ‘pure functional’ transformations (rare-ish)
2. functions fiddling with the UI, with fairly tightly coupled side-effects (angular does better than some at reducing the coupling)
3. general plumbing functions, really just infrastructure getting data from one place to another, maybe translating along the way. 

Only the first category is really friendly with unit tests. I know some of the drive of TDD is to learn to factor so that category 2 and 3 look more like 1. For example, refactoring a form validation so everything is happening at one place, a decoupling which helps both in testing and understanding. I see the value of that. (2020 Update: some frameworks like React are more amenable to slotting in with tests than old Angular was.)

But I feel like the majority of bugs I actually run into take place “out-of-band” of this kind of testing - like with that form validation, it’ll turn out the view doesn’t update the DOM like I expected with the scope validation variables. Or out-of-band in a “falls between the cracks” sense- say my new component made assumptions about the format of its input, and its tests are all green, but the old component its getting the data from had different assumptions about its output, but of course ITS tests are coded around THOSE assumptions.... so both pass their tests fine, but those 2 trees aren’t making a good forest.  Yeah, that’s what the functional or integration tests mean to catch, but then I can’t get away from the thought that most good “units” for unit testing are so trivial that when they fail, it’s some environmental glitch - and the testing environment is often a bit different than the real running environment anyway.

(I kind of scoff when I read, for the umpty-umpth time, 'see, you'll sleep better, because you KNOW YOUR CODE WORKS because it's unit tested'. People get paged at 2am because disks get filled up, or data in the format from 3 sprints ago got pulled up and mis-displayed (but all the tests were updated to assume the new data format), or connectivity to the backend server temporarily went down. It's almost always big stuff outside of the scope of tests, test that tend to be run in their own microcosm runtimes, and it's hardly-ever 'oh that error would have been caught as an edge case in your oh-so-clever unit test' - dang it, if it was an edge case I could have thought of when I was writing the unit, I would have made the code work right, so that then the unit test would just be bragging! (Admittedly, unit tests are pretty good at documenting the edge case to make sure a refactored version of the code supported the same weirdness. But I also hate, hate, hate the idea that unit tests can substitute proper, human readable documentation. You still want a map of the holistic forest, not just a listing of the reductionist trees.)

I've found SOME parallels in the way I write code relative to the Proper Unit Tester (PUTter). I'm a ridiculously incremental coder; I hate being "wrong", and kind of scared of making a block of code so big that when I'm finally ready to run it I don't what part might be going wrong, and so I've always relied on tight code/run/eval loops... lots of manual inspection and testing of assumptions at each little baby step. Often I'd even write little homebrew test functions for the various edge cases. The difference between old me and a true PUTter, then, is I would then throw away those little functions and logging statements in the same way a building site takes down the scaffolding once the building is standing. But PUTing says - look, that's how you know this code still works in the future, nurture that scaffolding, keep it around... make it survive any refactoring, fix all the mocks when the endpont input or output gets added to, etc etc.  That's a huge cost, and I'm still trying to convince my self it's worth it.

In short, unit tests are decent at A. saying you did what you said you'd do, but terrible at B. saying what you're doing is the right thing. And that's a problem for me, because they seem about as expensive as the core code to do up, and being benefit A alone make the expense harder to justify. (I mean really, shouldn't most proper "units" be too small to fail anyway?)

(See also this slide show I made, more gently and shallow-ly griping about this kind of status quo.)