I’ll never forget what my manager said to me.
“Do you need me to break down some doors? Do you need me to flip over some tables?”
He was talking as if we were going to war.
We’ll call him Taylor.
He was my manager at one of my first data science jobs, at a Big American Bank, working in the marketing analytics department. (We were literally doing data science before they even called it data science.)
The reason that the conversation was unforgettable was because in contrast to his tough words, Taylor was a fastidiously polite and honorable guy.
About 6 feet tall, with a shock of salt-and-pepper hair, he looked wholesome in an all-American way.
He was the sort of guy who consistently wore khaki Dockers and non-iron shirts from LL Bean.
And in terms of personality, he was almost always courteous and polite.
I’m half-inclined to make a comparison to Ned Flanders, but that might come across as an insult, which would be unfair.
He was a good guy, a good manager, and well liked by almost everyone.
Yet he was talking as if we were going to war.
I guess, in a sense, we were.
You see, we were constantly battling with our business partners about one of the biggest problems for any data scientist.
The work backlog.
Too Much Work, and It Goes To the Work Backlog
The work backlog is one of the biggest problems for any data scientist or any data science team.
At every job I’ve ever worked at – at Bank of America, Apple, and several other places – every data scientist worked with multiple business partners.
For example, at Apple, I had 4 primary business partners that I was supposed to serve. And there were also 4 or 5 executives above me (or above my business partners) for whom I was supposed to answer questions.
So easily, I had 8 or 9 people who could be putting in work requests.
Some of those individuals would rarely ask questions or put in requests.
But some of them would give me multiple requests per week.
They wanted a report. Or a one-off analysis. Or they had a followup question. Or a followup to a previous followup.
And so on.
As a data scientist, there was always too much work.
And we’d keep the work in a backlog.
Battle of the Backlog
To call it a backlog makes it sound pretty innocuous.
The name almost disguises the amount of animosity that sometimes existed over the thing.
Business partners were often frustrated that they couldn’t get questions answered.
They might have had a brilliant idea and they needed some information from me to help them execute on that idea.
And my response: “Hey, there are 17 other requests right now, so I’ll put it in the backlog and prioritize it against everything else in our team’s next work management meeting.”
They hated hearing this.
And as a data scientist, I was often frustrated by the shear volume of work.
Most of the time, I was working with great people, and we almost all could do large volumes of work, but no matter how much we accomplished, there was always a flood of more work.
That’s why my mild-mannered manager Taylor (who I told you about at the beginning of this post), was talking like we were going to battle.
One of my business partners at the time was very angry about our limited bandwidth, and was really trying to push some boundaries. Taylor was doing his job as a manager and trying to hold the line.
It was one memorable example of an overall pattern in my data science career: there’s a constant struggle between the people who always want more work, and the limited bandwidth of the people doing that work.
The battle of the backlog.
AI Will Break the Backlog
Ok, so what does this have to do with AI?
AI will break the backlog.
Or rather, it will help data scientists clear the backlog.
Let me explain my thinking.
In the last few months, everyone has been discussing the impact that AI will have on a variety of careers.
And personally, I’ve been thinking about the impact of AI on the data science field.
Will it destroy jobs?
Will we even need data scientists?
I currently have a variety of thoughts on these questions which I’ll continue to expand upon over coming weeks and months.
But a critical pivot in my thinking happened a couple of weeks ago when I was watching an episode of The All In Podcast on Youtube.
Something they said really changed my thinking.
Will we still need data scientists?
My answer to this is a strong “yes.”
Let me tell you why.
David Sacks and the All In Podcast on How AI will Impact Knowledge Workers
For those of you who are not familiar with it, The All In Podcast is a weekly podcast that features a roundtable discussion with four successful Silicon Valley personalities: David Sacks, David Friedburg, Jason Calacanis, and Chamath Palihapitiya.
And when I say “successful,” I mean that all of them are Silicon Valley veterans with over 20 years experience in Tech, and who have net worths probably between $100 million and over $1 billion.
These are successful guys who know the Tech industry.
On a Episode 122 (released on March 31, 2023), the four of them discussed “AI’s impact on job destruction.”
The whole segment is great. They each made several good points.
But one thing struck me in particular.
Near the end of the segment, David Sacks made some comments about how AI systems will enable tech workers to meet demand for their work.
He points out that in most startups, engineering bandwidth is the limiting factor. It’s the thing that you almost always “need more of.”
And what I’ve seen in basically every startup I’ve ever been a part of is that the limiting factor on progress is always engineering bandwidth. That is always the thing you wish you had more of.
– David Sacks
Another way of saying this is that in most startups, there’s always a backlog of work.
His comments about the engineering backlog synch perfectly with my own experience with the data science backlog.
The important point here is that most organizations suffer from a lack of talent, and in turn, there’s almost always need to prioritize and limit work.
Sacks and Chamath go on to say that AI programming systems will make developers more effective.
“The engineers that you have … all of them will become 10X engineers.
… you’ll be able to do as much or more as you could have before.
– Chamath
Now some people will say that the AI-empowered, 10X tech worker will mean that companies will need fewer workers.
I think that’s incorrect.
As I already said: there was always more work than we could do.
In a technology organization, there’s always too much work.
So yes: AI will enable tech workers to do more work.
But that doesn’t mean that companies will need fewer workers.
Rather, it will enable them to finally meet demand for their services.
Superhuman Amounts of Work Output
Ultimately, AI will help data scientists do large volumes of work.
Superhuman amounts of work.
Quantities of work that heretofore could only be imagined in a business partner’s dollar-sign-filled dreams.
With the help of AI, my dear data scientist, you will be able to “do all the things.”
You’ll be able to answer all the questions.
Fulfill all of the requests.
You’ll finally be able to clear that damn data science backlog.
Leave your comments below
Do you agree?
Do you think that AI systems will empower data scientists to meet the extreme demands of most data science business partners?
Or do you think something else will happen?
I want to hear from you.
Leave your thoughts in the comments section below.