I/O Wait States - The Ultimate A.I. Constraint
I/O is what data centers do - optimize I/O and you own the world
There are two great things about getting old - you know you aren’t going to die young and you have pretty much seen everything before.
At least in the technology space, history not only repeats itself, it rhymes, it uses the same words in the same order sung to the same tune.
Why do we have data centers?
There could be lots of responses but if we boil them down to first principles, the answer is to process I/O.
What is I/O?
Every transaction - when you hit that enter key goes into the data center and grabs info to tell you which BBQ is the best in Texas.
That answer isn’t sitting in a handy digital envelope - the computer did not know you were going to ask that question - so it didn’t prepare the answer.
The computer sends little messages all over the place, grabs pieces of data from what can be hundreds of places in the “database” and returns the answer.
Those little messages are called I/O - meaning “input/output” and they consume about 95% of what that data center does.
Not 2%, not 11% - they consume the data center.
Impact I/O and you change the world.
Yes, I said it.
In the world of A/I., if you tried Gemini for instance, you get back much more comprehensive answers than the best BBQ.
That means I/O explodes - more I/O means more data centers and you get to read the stories of an entire town in Georgia losing power, having its water polluted, for I/O.
If you thought I/O was a big deal before, A.I. makes I/O something you will be learning about very soon. Let’s begin teaching it here.
When we at Fractal started explaining I/O optimization to CTOs, and its ability to make their applications run 1,000 times faster - thus using 1/1000th the energy and equipment, - at 50% lower cost - they did not believe us.
One day, months ago, we were on a call with a well known tech company.
It was one of those calls - if you are in the software business you know that call - 10 people each of whom wants to prove his is bigger than yours.
It’s software sales call 001.
The first 20 minutes were hostile - with a number of SQL (the current tech database) guys claiming there was no way to make an application run 1,000 times faster - not pipelining, not in-memory processing - nothing.
They knew because they tried.
Hostility.
Then came the epiphany - from them, not us.
One of the older guys on the call - in his 60s for sure, asked us if we were building “compiled applications?”
Were we doing I/O optimization?
That’s when the water broke - our engineers responding that yes, that is precisely what we did.
Nobody ever used that term before.
He then asked how we did it - and the engineers took the entire team down that rabbit hole - about layer 2 cache, memory management, storing and partitioning the data, using A.I. to store the data in the way it is most likely to be processed.
Now there were only three people interacting - the old guy from the prospect - and two of our guys on I/O.
Then silence.
Nobody on our side said anything - it was a Zoom call so we knew they were still there.
Then their old guy said it.
“If you can optimize I/O “wait states” you would definitely get an application to run 1,000 times faster, and you would eliminate the need for more data centers.”
That was the epiphany moment - the moment separating the past from the future.
He then went on to say his previous company - one of the hardware behemoths, tried for 40 years to optimize I/O and they couldn’t get it done.
To hardware guys, I/O was the dragon to slay and anyone who did it was never going to miss a meal again.
From that point onward, we started talking about optimizing I/O wait states.
The next hurdle we encountered is nobody under 50 is likely to have written a full stack application from scratch - thus bypassing all the building blocks out there like databases, middleware and virtualization.
Nobody under 50 seemed to understand what impacting I/O wait states delivered.
Since few current techies ever had to worry about I/O, because chips were constantly made smaller - they did not know it was a thing.
Kind of like spending someone else’s money - if you do not feel constraints, they must not be there.
That reality is coming to a screeching halt and we are in the vanguard of the realization that everyone wants to deliver A.I. but not everyone has the processing computing power to do it.
Unless, they can constrain I/O.
Constrain I/O and you can deliver A.I. from your desktop or from the Apple devices in your firm today - read our post on how we enable that fun experiment.
Deliver A.I. without a data center - at 1/10th the cost, and you are on to something. Ya think?
We at Fractal are now experiencing the Overton Window phenomenon and we are reporting it to you - so we can claim credit later.
The Overton Window is something we heard about from politicos when we helped some Wisconsin officials doing deep voter roll analysis.
The Overton Window is definitely a thing - it means ideas so preposterous nobody would believe them start at the edge - with, of course everyone thinking they are crazy.
Then more people investigate the idea, talk about it, play around with outcomes, and the window - the Overton one, opens and the idea becomes mainstream.
You should Google it because it’s a real phenomenon.
Well, there is an Overton Window in technology and it’s on the wall of the Fractal house.
Two years ago when we did live demos of massive databases - the kind that consumed a full data center - and we ran them on a computer you can hold in your hand - nobody believed us.
We did videos and nobody believed us.
We did proofs of concept and in one case a CTO told us we could save his company $1.2 billion a year - but it was too big a saving to take to his board - they would fire him.
These things happen, here at the disruptive edge.
We write them down because they will make a great story in a year or two.
The Overton Window is beginning to open because old guys - who wrote full stack type code 40 years ago when I/O was king recognize optimizing I/O is the key to breaking A.I.’s biggest constraint - energy.
There is a growing realization in the A.I. journey, the constraint of energy - not enough for all the data centers people want to build - isn’t going to be there for years, if ever.
That is the constraint nobody can overcome in the time at hand.
All the money in the world, or Elon’s which is half of it, cannot build energy plants in a year.
I/O optimization can happen - for a massive Fractal system, in less than a quarter.
So we think we are onto something and we invite you to join us on this journey.
As some of you who engage with us know, we go to market through partners - so you the partner can deliver applications at a fraction of the cost, in a quarter or less, with widely differentiated offerings - and make great margins doing it.
Our partner pipeline is growing and we are always looking for those who understand that if you optimize I/O you change the world - and might want to do just that.
If you have some old guys around who want to optimize I/O, or young ones who want to see how - let’s chat sometime.
FractalComputing Substack is a newsletter about the journey of taking a massively disruptive technology to market. We envision a book about our journey so each post is a way to capture some fun events.
Subscribe at FractalComputing.Substack.com
Fractal Website: Fractal-Computing.com
Fractal Utility Site: TheFractalUtility.com
Fractal Government Site: TheFractalGovernment.com
Fractal Sustainable Computing: TheSustainableComputinginitiative.com
@FractalCompute
Portions of our revenue are given to animal rescue charities.
When I left a big financial company I calculated $27M of world transactions went through their computers every minute. I was tasked with making an 11 hour job run faster. Using pipelining and in memory processing I got the job to complete in under 60 minutes. If I knew about fractal computing I would now own the company. I was born too late!
It’s so encouraging to read this, especially right after reading about a data center in Abilene, Texas, that just consumed half a billion gallons of water while the residents are being told to cut back on showers.