-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
formatting tweaks for theh new posts of my blog
- Loading branch information
1 parent
b2d80b5
commit 2e55b62
Showing
8 changed files
with
722 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
497 changes: 497 additions & 0 deletions
497
src/blog/home-baked-abstractions-store-bought-implementations.md
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
--- | ||
title: "I Just Want The Simple Thing" | ||
date: 2024-06-10 | ||
description: >- | ||
*snorts line of code* HAVE YOU EVER WANTED TO DO ONE SIMPLE THING BUT INSTEAD YOU ENDED UP NEEDING TO RUN AN ENTIRE COMPUTE CLUSTER? | ||
--- | ||
|
||
_I am normally nuanced and full of empathy for differing perspectives; I try very hard to present things in a balanced way and to steer away from inflamatory nonsense that doesn't actually result in making things better._ | ||
_However, this is a rant, because I am having feelings, and those feelings are going to be feel'd._ | ||
|
||
--- | ||
|
||
\*snorts line of code\* **_HAVE YOU EVER WANTED TO DO ONE SIMPLE THING THAT COULD BE ACCOMPLISHED BY A WELL DEFINED LIBRARY + 200 LINES OF CODE BUT INSTEAD YOU ENDED UP NEEDING TO RUN AN ENTIRE KUBERNETES CLUSTER?_** | ||
|
||
Yeah, sometimes shits just like that. | ||
|
||
And by "sometimes" I mean all the time. | ||
And by all the time I mean this is why developers get burnt out and suffer. | ||
|
||
You know what I want? | ||
I want a library. | ||
Just a library. | ||
A simple elegant beautiful NIGH USELESS library. | ||
|
||
This library should be known as the ✨ `implementation-spec-lib` ✨ | ||
|
||
It must do absolutely nothing but the exact rules-lawyering minimum of the spec. | ||
It must BE the executable version of the spec. | ||
Change the spec? | ||
Spec change doesn't happen until library and test suite is changed. | ||
|
||
Does it handle the real world? NO. | ||
Does it gracefully error? NO. | ||
That's what the `acme-turbo-special-snowflake-extensions-lib` library does. | ||
_This_ library does one thing and one thing only: be pedantic, verifiably. | ||
|
||
It's the nerd that goes "well acktually"; it's the whisper in the night of "psst, someone's being wrong on the internet, you should yell at them"; it's the watcher of your sanity. | ||
|
||
It's also probably useless. | ||
|
||
But that's _fine_, actually. | ||
|
||
![The pikachu meme face of pikachu making an "oh" face.](./src/images/pikachu.webp "Mfw I just want a very simple library for so many things that isn't performant, isn't robust, doesn't handle errors, and does absolutely nothing except IMPLEMENT THE SPEC EXACTLY."){.w-full} | ||
|
||
Btw, implement this in a way that it can be easily formally verified, please. | ||
Performance hacks, "real world usage" and other nonsense can happen in a downstream library that can then be ran constantly against the **SPEC** to ensure _IT FUCKIN WORKS_. | ||
|
||
> "We have built the tower of the heavens, gaze upon its glory" | ||
Ok bud, what's your tower made of? | ||
|
||
> "Simple! We can parse html, email, yaml, and we run 273 different unbounded reconciliation control loops designed to take unspecified distributed systems and converge them into an unknown maybe-good" | ||
... Uhhh, ok, I see I asked too big of a question. | ||
I see you mentioned email??? | ||
How do you know you're parsing that correctly? | ||
|
||
> "Oh we don't, nobody does, we just give up and use BOOST::good_luck()" | ||
**_WHY. ARE. WE. LIKE. THIS._** | ||
|
||
> "You need to violate the spec because 13 out of 18 implementations of this follow the spec perfectly, but the one implementation that completely fucks up the spec is the one that has 98.76% of all traffic, usage, and mindshare from developers" | ||
Statements uttered by the mad and the deranged, and people who Ship Real Business Value™. | ||
|
||
> "I just want a simple thing that does what it promises and I'm willing to trade pretty much anything for simplicity, stability, and correctness" | ||
No sorry you need to suffer. | ||
Turbo McTurbo lib that was rewritten from C into raw assembly in the `YEAR OF OUR BULLSHIT $CURRENT_YR` for extra performance contains 763 extra features, three million extra lines of code, and 832 transitive dependencies. | ||
|
||
You have no other options. | ||
|
||
Fuck you for wanting nice things. | ||
All my homies hate people who want nice things. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
--- | ||
title: "Overengineering Isn't Real" | ||
date: 2024-01-20 | ||
--- | ||
|
||
I find it fascinating that we talk about overengineering as if it's a thing in software engineering. | ||
It's not, cut it out, stop talking about that. | ||
|
||
If I'm a construction contractor and I overengineer something, guess what? | ||
It works for that use case and all the future ones. | ||
The only thing I wasted was money and time. | ||
Did you make a 10 ton capacity deck when you only need 2 tons? | ||
Cool... It's still a deck. | ||
|
||
Software engineering? We don't have overengineering. | ||
We have "building to solve the wrong problem." | ||
|
||
In software we don't build a 10 ton deck when a 2 ton deck would do, we build a Deck Builder Factory that prints out decks that are fully extensible up and down but we build it into a concrete foundation where the only extension options are sideways. | ||
Then we bolt on safety join points to the house that prevent you from actually extending the deck at all. | ||
Then we only use the printing system once and throw it away. | ||
Then we complain that we overengineered the deck. | ||
But the deck? Rated for one ton. | ||
|
||
It sounds ridiculous when I phrase it that way, but I see things that are that obviously incorrect all the time. | ||
A huge problem is that software engineering is so incredibly abstract, ill defined, and nebulous, that it's non trivial to even find out that you're building a deck at all, much less what the load capacity should be. | ||
Software engineering is hard because there's no rules and no answers and it requires mutual understanding among groups to an extent that rivals the Tower of Babel. | ||
|
||
But overengineering? | ||
In software? | ||
It's not a thing. | ||
"YAGNI" might be a thing, but it's also still addressing the wrong part of the problem. | ||
|
||
If the hardest part of software engineering is building mutual understanding among a group of humans with different contexts, backgrounds, and approaches, then it stands to reason that trying to anticipate future problems will result in _solving the wrong problem_. | ||
We don't overengineer, we just build the wrong thing. | ||
|
||
One of the secrets to software engineering is to focus all of your energy as a team (and a company) on learning how to share information between each other better. | ||
Build that understanding. | ||
Build that ability to uplift and teach each other. | ||
Once you build that, you'll be amazed at how often it feels like you almost accidentally end up building the correct thing. | ||
It just starts to materialize out of nowhere, and becomes an emergent property of your shared understanding and collaboration. | ||
|
||
For fuck's sake, stop worrying about overengineering and start worrying about under-understanding the problem. | ||
The "overengineering" magically goes away the second you start putting humans first and start prioritizing understanding over an artificial roadmap built without context. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
--- | ||
title: "Pick Your Distributed Poison" | ||
date: 2024-06-20 | ||
--- | ||
|
||
One of the hardest things for people to understand with distributed systems is that eventual consistency is the same thing as eventual inconsistency. | ||
The very same pattern that lets you non atomically deal with things also ensures that eventually you'll have a system that doesn't match your understanding. | ||
Resources will go stale, things will go missing, stuff will exist without ever having been created, and data will be destroyed that never got manifested. | ||
|
||
> "How do you prevent this?" | ||
You don't. | ||
You figure out what flavor of wrong you want and what type of inconsistency is tolerable to you and you embrace the suffering and learn to mitigate the particularly painful outliers that bite you. | ||
|
||
Is bootstrapping your worst enemy? | ||
Regularly destroy and recreate the system to ensure no cycles exist in it. | ||
Of course, that means it will inevitably incur emergent instability and resource leaks. | ||
What's your preference? | ||
|
||
> "I know! I'll keep a fresh system around and recreate it to ensure no cycles, and I'll keep an old one around to ensure no long term leaks exist" | ||
Okay, suit yourself. | ||
I see you enjoy the wonders of non-deterministic metastability that comes from adaptive concurrency controls. | ||
Oh, you don't? | ||
So you have hard isolation between the two systems? | ||
I see. | ||
That gets you non-deterministic metastability but without needing adaptive concurrency controls. | ||
Fascinating innit? | ||
|
||
Dangling, stale, metastable, zombie. | ||
That only touches the very surface. | ||
|
||
- "This system only restarts with warmed caches" | ||
- "This system can't be rebooted and scaled up at the same time" | ||
- "This system can do anything except be highly available during updates" | ||
- "This system can only be restarted in topo-sort order" | ||
- "This system has a deadlock if you drain it geographically from east to west during daylight savings time" | ||
|
||
Pick your choice of madness, but don't pretend you won't be drinking it dry. | ||
|
||
My poison? | ||
I prefer reproducible and bootstrappable systems. | ||
That's my thing. | ||
I want cold caches, constant work, and young state. | ||
It minimizes, for me, the amount of things I need to keep in working memory. | ||
|
||
Of course, I pay the price: I lose the ability to detect leaks, stale references, clean shutdowns, and long lived properties. | ||
I also lose out on emergent performance, large amount of adaptability, and entire methodologies of systems safety. | ||
Living in ground zero means I never touch the sky | ||
|
||
Reproducible and bootstrappable systems get a lot of love among neurodivergent people. | ||
For good reason: they're very friendly to those with little working memory but vast amounts of working context They're harder to reason about, though, funnily enough. | ||
The path to running is never the same as the running loop. | ||
|
||
For all my love of liveness and safety properties when it comes to reasoning about systems, I ironically build ones that rely as little on them as possible. | ||
|
||
But, I'll take my poison. | ||
Neat, if you please. | ||
I prefer to sip it slowly and savor the madness within. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
title: "Scale Down to Scale Up" | ||
date: 2024-02-19 | ||
description: >- | ||
"If your service can't be run on a laptop, your service doesn't deserve to be ran in a cluster" - Marilyn Monroe | ||
--- | ||
|
||
_I am normally nuanced and full of empathy for differing perspectives; I try very hard to present things in a balanced way and to steer away from inflamatory nonsense that doesn't actually result in making things better._ | ||
_However, this is a rant, because I am having feelings, and those feelings are going to be feel'd._ | ||
|
||
--- | ||
|
||
> "If your service can't be run on a laptop, your service doesn't deserve to be ran in a cluster" - Marilyn Monroe | ||
I would give my kingdom for an engineering culture in which people figured out how to build good _primitives_ and _protocols_. | ||
Things that can scale down as well as scale up. | ||
Instead we have this weird middle ground. | ||
One thing that's wild to me is that we've gotten fairly solid at building distributed systems that are resilient, workable, and fairly decently designed... | ||
As soon as they hit a certain amount of scale, and only then. | ||
|
||
So much shit out there just gets slapped together with every single cloud scale mega-cluster service and tool like its a limited edition box of candies that's going out of stock | ||
Consequently... Things work on a 40 node stateful cluster just fine. | ||
But you're absolutely fucked if you ever want to run something ephemerally, "restart" anything, or even just sensibly provide something as a service for other people to run. | ||
Why in the absolute fuck did we write ourselves into a corner as an industry where "oh sorry minimum specs are 187 vCPU, 4TB of ram, 20TB of disks" seen as something sensible instead of absolutely ludicrous beyond belief? | ||
|
||
Ok, ok, yeah yeah, high availability and whatever. | ||
But _also_... Have you seen how highly available a single fucking binary running on a single fucking computer can be? | ||
Have you seen the performance specs of sqlite? | ||
I'm reminded of that [one paper from like 2010 or so](http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html) when a researcher showed that a single threaded binary could outperform many 100 core clusters because they were so poorly designed to scale _down_ at all. | ||
Did we learn nothing from that? Anything? Anything at all?? | ||
|
||
I'm all for highly distributed systems, I love them; I'm all for correctness, and resilience, and so many other fun things. | ||
That's my bread and butter. | ||
|
||
But **SERIOUSLY**. | ||
How the fuck is it that we ended up with this absolutely bimodal nonsense of "one single static binary of goodiness" vs "500,000 node compute cluster of goodiness" and this absolutely hellish quagmire of disgusting bullshit in the middle? | ||
You know what kubernetes is? Do you know what it is? It's 20 while true loops in a trench coat pretending to be a control plane. | ||
And that's fine! | ||
|
||
But somehow, somehow, the only fucking way you can run that thing is to shell out to a CSP and have them manage your control plane because absolutely nobody has ever figured out how to make it run reliably in bootstrap or less-than-5000,000-node-scale as a commodity. | ||
We've spent 800x more energy on service meshes than _actually getting good at running it_. | ||
The fuck? | ||
|
||
What have you seen lately that's good at scaling down _and_ scaling up? | ||
Have you seen anything that's a delight to operate at every scale? | ||
I'm not convinced we even know what that looks like let alone how to build it |
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters