Elon Cannot Merge Government Databases - Fractal Can!
Elon Musk this week demonstrated BIG time why relational technology is obsolete
This week The Elon Show was all about how his guys found incompatible databases all over the government - and the databases do not communicate with each other.
Wow, like who figured that might happen?
Just about anyone who ever visited a raised-floor computer center, using PostgreSQL, Oracle, Mongo or any of the current databases knows full well they do not easily communicate.
Different databases, built over 50 years is not a government thing - it is something that plagues every company in the world. ISAM, VSAM, embedded pointers, network database from Cullinet - they were built for different purposes, by different teams, over half a century.
Now you know why Larry Ellison, the head of Oracle, stood next to President Trump saying he wanted to raise half a trillion bucks to build huge data centers - because that obsolete Oracle crap needs massive data centers.
This is a Fractal piece and I am not going to save the punch line for the end - Fractal can ingest any database the government has and make it immediately “talk to” or “communicate with” any other database - in about 30 days or less.
A bunch of databases could be done in a single quarter - like 50 or 100 or more - and we have already done it with government data and you can see our videos on Rumble - type in Omega4America and watch away.
When the geek types start talking about communicating among databases, it goes into the weeds fast. There are levels of talking - like yelling over the counter, whispering, or 100% sharing every cell in every database with every cell in every other DB.
We are talking about the last one. That’s what Fractal does - in days or a couple of weeks, tops.
The geek guys will use terms like API - application program interface.
Your API is the plug in your coffee grinder cord which has 3 pins and you “make it talk to” your electrical system by plugging it into the wall - with its API - the prongs.
APIs are not useful in merging government databases.
Then there is “interfacing” rather than “integrating.”
Interfacing means you take some fields from database A, maybe the DMV driver license system and share the name and addresses with another system - the auto tax system. They are sort of interfaced.
You make a change to an address, both systems are updated.
They roughly share data.
This is the state of the art today - before Fractal - and it’s primitive.
Remember our little chat last month on I/O - input/output?
You will recall the reason we have data centers at all is because of I/O. We have several thousand new readers since that barn burner “The Black Swan of A.I.” so let me review.
I/O is the wait state when the CPU (brain) is waiting for the next item to process - and in relational it is the equivalent of forever.
Thus, massive data centers do the same amount of useful work in 125,000 square foot data centers as a Fractal system does in a 4 -inch cube. That is not hyperbole - we can prove it - watch our videos on Rumble - those babies are all proof.
There is this set of software called ETL (extract, transform and load) which is the family of programs taking rows and columns from database A and prepping them for database B.
In the relational world, ETL makes it much easier - so with current tools for the government to merge a couple of hundred related databases - big ones, think Medicaid, Social Security, the BIG ones, will take several years.
It would cost tens to hundreds of millions of dollars. It would need a data center larger than any data center in existence.
Fractal can do it in around 30 days but we would bid 90 so we don’t have lots of internal pressure.
Fractal would run the entire thing on a few hundred grand in hardware - lots of those little cubes, but nothing like what it would take with relational.
Fractal would eliminate the need for a data center consuming 500,000 gallons of water a day to cool the I/O - bound equipment - with equipment running on wall outlets.
And our security is sovereign level - far better than what much of the government uses today.
BS!
Say you, cubicle dweller in the SQL world - 21 year-old-Python developer - it cannot be done.
I am waiting for those emails!
OK, let’s do it, here, and show it to you.
You will recall we are the Fractal guys but also the Omega guys.
Our Omega project was the election work merging the voter rolls of 24 states with their property tax rolls, NCOA change of address rolls, etc.
When we started with Wisconsin, a state with 7 million voters on its database, we figured when we added Florida, Texas, Michigan we would just add the data - the field names would be pretty much the same.
A voter roll is a voter roll - how hard can it be?
Well, we can tell you we have done somewhere between 20 and 25 states - their entire voter roll. Every single one is different from the others. Every one was built with different technology - all relational but variants - by teams who never knew each other, and every state tracked different attributes.
Some have emails, some have Male, Female and Whatever, some have phone numbers, some have multiple residences like a home address and a mailing address - they are wildly different.
We track voters who split from Wisconsin and flee to Florida - so we do not interface, we integrate completely.
We built scores of queries: show me every voter over 115 years old, show me every voter who lives in a residence with 6 or more voters, show me every voter who is registered at a hotel.
Guess what?
Every query, every analysis, works across every one of the 20 something states as if it were written for that state. We fully integrated the election rolls of 2 dozen states - 2.7 billion records (more on that in a bit) and they run together flawlessly.
We could integrate the Social Security System database, the Medicaid database and Medicare database in about 30 days.
We do not run this data, twice the size of the Social Security System, in a raised floor data center, we run all of these states on a couple of 4 - inch cubes using less power than a kitchen microwave oven making popcorn.
So you SQL- heads, we prove every day we can merge massive government databases, all in unlike form, and we can add any state, California, in a weekend. Fully integrated.
We love Elon, you know that. Our engineers love the rocket stuff - but Elon cannot do anything like this and we just wish he would let us do it for him.
We aren’t done.
You will recall if you follow the Omega stuff, we cross search voter rolls with property tax rolls. Now, if you want to see apples and green bananas data - with little in common, it’s property tax rolls.
We had a reason.
Voter rolls tell us 139 Elmont Terrace has 12 registered voters. It does not tell us anything about the address. People assumed if 12 voters lived there it was cool.
Then we brought in property tax rolls.
Those do not tell us who our 12 voters are. Property tax data tells us the address is a closed JC Penny store, in an abandoned strip mall and now we know we found 12 bogus voters.
If you think voter rolls are different from state to state, we find 2 counties - in Kansas Wyandotte and Johnson - contiguous counties, next to each other too, and their property tax records are completely unalike. Is that a word?
There are 3,200 counties in the country. We ran the data for about 20 or so in the swing states and every one was totally different.
So we built a canonical model - which is fancy speak for a schema (row and column names) that worked for other counties. Now we ingest - and integrate - any property tax record set - from any county, in a day or two.
For Georgia, we brought in 11 counties in a weekend - and they not only all play well together as in every query runs against every such record, they are fully integrated with the voter rolls as in one big happy database.
Total time to do all of that, 24 -25 states would be about 2 to 3 weeks, one person, 8 hours a day.
That project is bigger than the Social Security System. By a lot.
Remember our voter database has 2.7 billion records. Yet there are only 165 million voters, including the phantoms. Why the disparity?
In Fractal, the Social Security database is not a big database. We have customers, in a single building, who create 10 times more data that the entire Social Security System in a day. We demo that sometimes.
So we do snapshotting of data.
We do not just take a copy of the Florida or Texas voter rolls, or Wisconsin - we take different copies over time. Guess what?
People move about - they move to better or worse neighborhoods - modify preferences - and we pick that up.
So the reason we have 2.7 billion real time records for 165 million voters is because for Pennsylvania we have 78 copies of their voter roll on different dates - and we can run them at quantum speed from your phone.
So when you watch our pal Elon on TV telling you how bad the government databases are and they do not communicate - remember, he is a relational database guy who does rockets better than he does data.
We suck at rockets but we can integrate the major government databases - as we have now completed for 20-something states - in a quarter.
Fractal is doing all we can to contact all the key players to show them a live demo of what we have already done.
We hope they listen because doing this integration with relational database will fail, and fail spectacularly.
FractalComputing Substack is a newsletter about the journey of taking a massively disruptive technology to market. We envision a book about our journey so each post is a way to capture some fun events.
Subscribe at FractalComputing.Substack.com
Fractal Website: Fractal-Computing.com
Fractal Utility Site: TheFractalUtility.com
Fractal Government Site: TheFractalGovernment.com
Portions of our revenue are given to animal rescue charities.
Because Elon's narrative is he is the smartest guy on the planet - he can see things nobody else can see - he has wizards who can do things nobody else can do. So if we come in and in 90 days solve this problem - which we can do - his narrative is screwed.
omg... I think I have heard about these cubes. If they are as you say they are, I hope you will not continue to pursue the idea of connecting all the government data bases for efficiency or whatever!
Technology is more valuable for the people to secure their own data and create networks that are private for human benefit. I would like to secure my data, privately transact, and choose who can look at my data for my benefit, not someone else's. For example can I secure my personal info and my health records? I can share them temporarily to a doctor for advice and then walk away witn my data and recommendations. No one else has consent to access my data. Can you do that for humanity, apart from the political parasites? I hope this is clear. Innovators need to be protected from governments too!
IF our government connects all the data, it will be much more efficient in controlling, regulating, and restricting human lives. That is tyranny at its highest and always leads to disaster for fellow humans.