Friday, June 9, 2023

confirmation bias and schadenfreude in testing

I was catching up on Slack on a team I'm not longer on, and saw the story of a serious production bug. Reviewing their post-mortem I saw the error description was 

jobs were not viewable on COMPANYNAME sites and COMPANYNAME jobs were not  indexed or displayed on Google for Jobs (G4J)."

 and the "blameless root cause" was:

In _document.tsx where json-ld is injected was refactored in order to get unit test scores above 80%.  The refactor was then the cause of the bug and prevented the json-ld types from being added into the code.

That definitely feeds in to my prejudices that focusing on "tree level" unit tests can lead to a culture that doesn't pay enough attention the forest.

On a different slack I participate in, someone put it:

as it turns out $DAYJOB’s business objective of reaching specific code coverage percentages has not made any impact on our DORA metrics — surprised pikachu face

No comments:

Post a Comment