Crafting the Worst DevEx
one decision at a time
I have never met a person that wakes up in the morning thinking: How can I worsen developer’s experience today? 🤔
But I often hear, how did we end up here?
In this post I’ll walk through how a bunch of good intentions end up with soul crushing development workflows
System of Empowerment (or not)
I don't know if is my explorer personality or the circumstances of my early software development days, somewhere It became extremely easy for me to recognize when I was in flow or no, it's like you are playing a video game and you are not having fun, you don't know why, it just feels boring
I was lucky enough to experience flow more often than not, I was in a position where I could identify what changed? Why was I having such a good time and now I don't?
Not surprising, many of the reasons were rooted by lack of empowerment
After many years and several companies, I realized that the lack of empowerment goes all the way up the career ladder
I concluded, When leaders are under pressure:
Do they continue to provide trust and support?
Take control and dictate orders?
I have experienced a few exceptional leaders that when under a lot of pressure and uncertain, they stayed vulnerable, taking responsibility from the team and providing aircover to those willing to try out-of-the-box ideas
The reality is that we are more likely to have leaders that take control and tell others what to do, so don't waste years looking for those exceptions, it doesn't matter what level in the ladder you are, you can forge 🔨 an empowered microculture on your team, but that's a story for another day
How did we end up here?
Really smart technical leaders are likely to believe they know more than the rest, and to be honest, they usually are 🤷 they are the only ones that know all the technical intricates of the whole mess the ended up building, this is like a self fulfilling prophecy
Another aspect is power, do these leaders follow those strict access control policies?
Of course not, these leaders often require root access to get their work done, this creates an unfair advantage, everyone else but me has to follow rules
They tend to manage complexity by:
Dumping their know-how into bland docs
Treat complex situations with simple absolute rules that become rigid policies
Expend their entire time on keep the lights on tasks, usually as human gates
Over time, these leaders are often promoted and given the opportunity to designing a new platform, starting the cycle again
The visible outcome, a good Intentional process
Decaying DevEx processes will cause incidents and actions from retrospectives will add even more steps to that soul crushing 80 actions deep process
One of the major influences of bad developer experience comes from the complexity of our platforms, implementing change needs
Navigating complex dependencies
Ask permission due to lack of required access to get things done
Take risks based on unreliable data
We are all trying to do our best with the skills and capabilities we have
How to deal with complexity
I often hear technical leaders referring to abstractions as: the action of hiding complexity away
Creating abstractions is a really hard work and it requires creating a new mental model of the problem, these split apart from the underline implementation and by definition, they aren't the optimal solution but one that it's simpler to reason about
One popular example of an abstraction is a file system, even the most non-tech savvy person can reason about files and folders, they represent known entities and interactions.
A file system pretends that a range of binaries stored in disk are folders and files, and files store text documents
It's a lot easier for me to reason and design solutions thinking about files, this abstraction is also the base for more advanced functionality like role based access control on those documents
What an abstraction is NOT, a parameterized bash scripts 🤦
I think we can learn from the state of the art infrastructure management, the core principles of GitOps can be used to help us deal with complexity:
Declarative: Desired state focus on the what and not on the how
Versioned and Immutable: Desired state is stored in a way that enforces immutability, versioning and retains a complete audit history
Pulled: Desired state is decoupled from underline dependencies, this means it doesn’t have context of what’s consuming it, one or thousand environments? It doesn’t care
Continuously Reconciled: Eventual consistent systems are easier to reason and manage
I’d add one more:
Functional Core, Imperative Shell: How the core behaves in specific conditions is known and backed by tests, connecting the core to side effects is a light and dump layer
Real life example
It takes time to think declarative, so let's do another example:
When you make a code change and create a PR, wait for tests ✅ and then merge to main.
This is imperative, you are doing the how
A more declarative approach would be to
When you make a code change create a PR with merge label
Something will reconcile the state of the PR with tests ✅ condition and eventually merge it
It's a lot easier to use and extend this solution, for example: we need to add extra conditions, let's say migrations will have a different set of checks than application changes
Something will add more label to describe the PR characteristics and the merge conditions will define more advanced merging rules
Developers declare the desire state as PR labels
PR maintains the history of what has happened
Tests can be executed on any CI provider, what matters is the status check ✅ for tests or migration ✅
The merge queue provider monitors when the merge conditions are met and merges or notifies the author the dequeue reason
Every GitHub Action is independently versioned, tested and linked together by a dump YAML workflow