Deloitte Data System Is Wonky Garbage
Absolutely no one surprised, except for the racism.
The U.S. Federal government contracted with Deloitte to make a “vaccine data system.” Sounds fancy.
Except the vaccine data system is really just a ghetto version of EventBrite. Yes, it’s a scheduling app that you could probably MVP with a few hours on github.
Obviously the government paid Deloitte tens of millions of dollars for it because it’s the government and Deloitte. You seriously think the government was going to get good software at a good price? Or did you think Deloitte was going to make good software at a good price? No on all fronts; it was over-priced wonky garbage.
The obvious question is: why do people keep paying Deloitte or Accenture or IBM or HCL for software? (I know, there are other suspects. But only so many can fit under the bus at once.)
First, let’s appreciate these companies’ business model: arbitrage the mean-variance between the price for which a man with a $300 haircut can sell code to gullible bureaucrats and the cost for which third-world people can produce that code.
That’s literally their entire business model. Third-world coders + suckers = profit. (In HCL’s defense, their business model is third-rate work for third-rate prices.)
The arbitrage between first-world prices and third-world costs works because of an assumption of capitalism: you get what you pay for. Of course, most people buying these services don’t even know what they’re paying for… how else to explain spending $44M for a scheduling app?
Unlike most, I’ve actually looked at the code you get for a $44M ghetto version of Eventbrite. It’s a patchwork Jenga tenuously held together with duct tape and prayer. (“But we use Big Data A.I. Mongo Python Agile Hybrid Edge Graph database Crypto Agile Agile Agile,” says the salesman’s PowerPoint.)
And that’s being generous. Once at IBM, I was speaking to the head of product delivery — an older guy who’d been doing this since the late 1970s — and he was complaining how his delivery team was also the de facto QA team solely because his job was to make the software work, and, as I’ve said before, no one really does QA. And so, he was annoyed that he was responsible for delivering software that never really worked.
So I spent some quality time reviewing the code by a 100+ member Bangalore team. It looked like a tornado ripped through a Lego factory. The general architecture of monolithic, which means that if anything breaks, there’s a chance that everything breaks. And it was wobbly, so feature updates, data updates, even UX changes — pretty much any change — could cause it to wobble over. That head of product delivery told me that he knows that he’s eventually going to get an emergency 3am call because the entire product stopped working.
I told him that the code looked pretty solid … for 1995. He chuckled and said he’d seen better when he started at IBM in 1978. Of course, by the 1980s, IBM assessed and paid coders by the number of lines of code they wrote (seriously), which shocked Microsoft on a late-80s co-dev project. Around that time, IBM almost went bankrupt. Probably not a coincidence.
But why does anyone keep buying bad code?
To understand why those big consulting companies produce garbage, consider that you’re buying code made at <$10/hour. Yes, that’s what those companies pay in India.
Yes, but $10/hour in India is…like…real money? Right?
No, it’s not. Good coders in India make much more. Very good coders can make six figures. Great Indian coders … usually aren’t in India. The people you’re hiring are at the bottomest of rungs.[1] And those coders are treated like unskilled factory cogs who take work orders like a Subway sandwich maker. If you ask for mayonnaise on a meatball sub, they will make it without question. Then the lawyers will point out that the SoW didn’t specify no mayonnaise on meatball subs.
Your Mission Critical Chalupa
At the top of a list of revenue per employee for U.S. companies are healthcare and energy companies and a few tech companies. At the bottom — usually around $50,000 of revenue per employee — would be a lot of low-end service companies: fast food and quick service restaurants, mid/low-tier retailers, etc. Those low-revenue-per-employee companies may only generate $50,000 per employee, and they have operating costs and c-suite salaries and probably corporate jets to pay for … which is why most of their employees make significantly less than $50,000.
Exactly as you’d expect. Taco Bell. McDonalds. Hotel maids and fajita makers.
Except that those consulting companies are also at the bottom of this list.
Of course, you could also look at tech company salaries. Amazon typically comes in last because, of course, Amazon isn’t a tech company; it’s a giant gift shop, so most of its employees are restocking that 275-gallons of lube you “accidentally” ordered or shipping out that giant chicken leg stuffed pillow for next-day delivery.
But right below Amazon is IBM with an average salary of $55k. When you consider that (almost) every IBM employee in the U.S. and Europe is making much more than that, you begin to realize what the Bangalore team is getting paid. Worth noting that nearly all of those tech companies have large operations in India (and elsewhere); they just aren’t paying all their Indian coders $18k/year.
IBM, like Accenture and Deloitte, hires cheap, then profits off the variance between those cheap coders and expensive sales pitches. (Fun fact: IBM execs have teams in Manila that make PPTs/pitches/proposals for you while you sleep. Works about as well as you’d imagine. They’re prone to putting a 30-page “history of IBM” at the beginning of every technical proposal because they think it looks cool; I never had the heart to tell them that no one cares what IBM invented in the 1920s. Curiously, the Manila-produced history didn’t mention the census tabulating machines that IBM sold the Nazis in the 1930s.)
So why do people keep buying cheap code, particularly in mission-critical situations? Cheap code may be okay for your light-blub firmware (that’s a thing), but it also drops Boeing 737s from the sky. Do CIOs not know that there’s nowhere on earth where $10/hour code will pass QA for a mission-critical app? (I know … don’t do QA. Problem solved.)
Why #1.
You tell someone that their great business innovation app is being developed by a $10/hour team in Bangalore, and they give you a shrug because, as it turns out, they’re a benevolent racist who thinks all Indians are smart. If you told them that their $10/hour coders are in Honduras or Pakistan or Myanmar or Nigeria, they’d be concerned. But no, it’s Indians. Surely even the dumb ones are smart, right? Racism has never been so profitable.
Why #2.
If you permit accountants and “procurement” to make decisions and treat coding as a cost-center and a “procurement” instead of an asset — a source of value — then you’re going to be blinded by metrics that shouldn’t be priorities and you’re going to have decision-makers who shouldn’t be making decisions. Deloitte and Accenture aren’t consultants or developers; they’re lobbyists whose sole client is themselves. They are experts at manipulating tech-handicapped bureaucrats with the right buzzwords and the right procurement procedures while there are dozens of dev shops in Austin, Texas (for example) that could under-price and over-deliver against Deloitte’s best team. A lot of those Austin dev shops even employ Indians, just not that $10/hour ones.
What’s amazing is the number of companies (IBM, Facebook) moving to places like Manila in pursuit of even cheaper labor. The $44M ghetto Eventbrite will still cost $44M, but Accenture/Deloitte/etc. will have more cash to defend themselves in the inevitable lawsuits.
In the final analysis, I think IBM/Deloitte/Accenture gets hired because “no one gets fired for hiring IBM.” In other words: cowardice. Bureaucrats aren’t incentivized to make the right decision; they’re incentivized to not make the wrong decision, to avoid blame, to wave around the brand name — Deloitte! — as some kind of explanation in the same way the salesman leveraged the brand a year before in the pitch meeting. And when you take delivery of some wonky garbage, you act surprised. That brand isn’t a indication of quality; it’s a risk-mitigation strategy.
Sources
Median salary
https://www.wired.com/story/what-tech-companies-pay-employees-2019/
Revenue per Employee
https://craft.co/reports/where-do-the-most-productive-employees-work
Profit per Employee
https://www.fool.com/investing/2020/11/12/these-companies-make-the-most-and-least-money-per/
Tech Revenue per Employee
https://www.businessinsider.com/tech-companies-revenue-employee-2017-8?op=1
S&P Rev per Employee
https://craft.co/reports/s-p-500-revenue-per-employee-perspective
[1] The average IT salary in India is about $5/hr. In the U.S., it’s about $50/hr. Not an apples-to-apples comparison though, because most low-end IT jobs (i.e. support) don’t exist in the U.S. anymore (they’re in India). You can’t compare IT salaries in India to overall salaries because overall Indian salaries are heavily weighted to agriculture, which isn’t true in the U.S. But if you compare average city-specific salaries, you’d find that Indian IT salaries are average for overall white-collar salaries, whereas in the U.S. they’re about 20% higher. That’s the effect of low-end IT jobs flowing from the U.S. to India. Of course, there are many highly paid Indian programmers, it’s just that they tend to be in Singapore/Australia/Canada/U.S./U.K. That again effects the average IT salaries in India vs. those countries. Finally, top U.S. tech firms (Facebook, Google) actually pay rather well in India, with top salaries typically in the $70k-$90k range. That’s not much of a discount to average U.S. IT salaries, but it is a huge discount to average tech salaries at those firms (typically over $200k).
