If you’re interested in data science and want to become a data professional, you’re probably going to need to make a choice.
I think that the industry is going to split – and in fact, it probably already has.
This split that I’m referring to is the divide between the top performers – the top few percent – and everyone else.
The the truth is that this divide already exists to some extent. If I had to guess, out of everyone who has ever started studying data science, only about 20% will ever get paid to use those data skills.
There are a lot of reasons for this.
Some people decide that data science is too hard and they give up.
Other people lose interest and focus their efforts elsewhere.
But one of the big reasons is that many people simply never develop enough skill.
It’s like many other skill areas, like sports or music. For example, many people learn basketball and play a basketball as sort of a hobby, but the vast majority of people will never be skilled enough to be paid for it.
Granted, basketball is an extreme example. And the market for data scientists is larger than the market for basketball players. Still, the principle is the same. There’s a relatively small percent of would-be data scientists who succeed, and a large percent who struggle, fail, or drop out.
You need to decide which you’ll be.
The divide between winners and losers will get bigger
The divide between the top performers and everyone else is already big. As I said, I suspect that only about 20% of people who study data science will ever get jobs in the field.
But I think that divide will get larger.
At least two things are driving this: layoffs and new technology.
The worst people are the first to go
Right now in August of 2020, we’re all dealing with the covid crisis.
The economic effects of covid haven’t hit tech companies quite as hard, but it has impacted some other industries who employ tech workers (e.g., retail).
Ultimately, many companies have experienced a drop in revenue, and many will use this as a reason (or excuse) to lay some people off.
I hate to break this to you, but the lowest skilled people are probably going to be hit by this the most.
In times like these, companies keep the top performers and get rid of the lowest performers. That’s just the way that it is.
AI systems will empower winners and remove low performers
Another big trend that’s just getting started is the use of AI systems to replace white-collar workers.
Most recently the GPT-3 AI system from OpenAI has shown the ability to generate working code based on English language input from a user. The user can describe what they want in English, and GPT-3 can create an application and working code based on the English language description.
The obvious question is whether or not these AI systems will replace programmers and data scientists.
I think that the answer is a little complicated.
AI-based systems will replace low skill programmers.
If you’re a low-skill, cut-and-paste coder, GPT-3 and similar systems may replace you. These systems will be able to write code faster and more accurately than lower-tier programmers by empowering people with zero experience. 23 year old marketing managers or will be able to generate their own code with the help of AI. They won’t need low-skill coders … they’ll just do it themselves.
Advanced Tools Will Augment The Best People
But the high skill coders will be a different story.
For the highest skilled people, AI systems will be a productivity tool.
For a person who already has high levels of skill and understanding, they will simply get more done.
AI systems will empower the top 10 to 20% and enable them to write more code, faster.
For highly skilled people, AI systems will enhance their skill. These AI systems and other tools will be productivity tools that will make the best people even better.
With emerging tools and technologies, the best data scientists will become even more valuable.
If you want to win, decide to be a top performer
Ultimately, what I want you to understand is that the market for data scientists will possibly split in two over the next few years.
New tools and technology will likely empower the best people to create massive amounts of value … and they’ll be rewarded with either high salary or high business valuations.
The lowest skilled people though are going to face ever more fierce competition, which will drive down wages.
You need to choose which you’ll become: a top-tier data scientist that succeeds wildly, or one of the bottom 80% that struggle endlessly.