The new fear arising in the domain of Data Science is whether data scientists will eventually automate themselves out of their positions.
The new fear arising in the domain of Data Science is whether data scientists will eventually automate themselves out of their positions.
Gartner as of late stated that 40% of data science tasks will be automated by 2020 and we are in 2022 now so imagine the percentage of data science tasks that has been automated. In the previous years, we have seen how companies that leverage data(whether internal data or from external sources) have thrive to uplift their return on investment. Companies that remained adamant and doubting the potential of data while relying on their traditional ways of delivering solutions have seen a nose dive in their returns.
Having realised the potential of data, many companies have jumped in building data science teams. But business managers have also started realising the huge cost they incur to build the best data science teams. In fact building a robust data science team is costlier than any other team in majority of the companies out there.
As a result, many organisations try to search for alternative arrangements. AutoML seems to be their godly saviour.
What is AutoML?
Automated machine learning (AutoML) is the process of applying Machine Learning (ML) models to real-world problems by giving just the initial commands and the rest of the pipelines are taken care of by the program itself. More specifically, it automates the selection, composition, and parameterization of machine learning models. Internally, it creates different pipelines by choosing the different hyperparameters and choose that pipeline that gives maximum accuracy for our dataset.
Automation in data science will certainly sweep some data science roles out of their feet but it will be absolutely premature to say that AutoML will totally kick data scientists out of their positions. Low-level capacities can be productively dealt with by Artificial Intelligence systems of course.
There are two areas that AI systems will find it very challenging to automate completely:
- Getting structured data
- Domain knowledge
Getting Structure Data
For all these past few years that Data Science has become a buzzword, it is because of its ability to deal with messy data by combining human intelligence with systems. Data has never been clean. There will definitely be one or two things to take out or add to a data at hand. That requires human intelligence to reason from several “relevant circumstances” to make such choices. Even models that are built using pre-trained models are tweaked for relevancy.
Data that we are going to receive in the next few years are even going to be messily messier(if such word exist). But what am trying to say is that, with the advance of AI technologies and the abusive use of it everywhere, data is going to overwhelm humans and systems are going to be choked with what to do next. It will take only the data scientist with some sort of astute data ingenuity to consume it and digest it successfully. Machines will offer help but at a limit.
Domain Knowledge
The Automated pipelines that AutoML boast of requires human judgment to transform raw data into something that can be fed to these pipelines and also transform outcomes into insights that bode well for a company, and consider all of a company’s complexities.
Feeding data to AI systems is easy but making it work for a “particular” organisation is the big deal. That is when expert data scientist are needed. Companies that do not have data science teams or “good” data science teams, to be liberal, are at a risk or at best blindly using AutoML tools. Such organisations will end up spending 1000s of dollars for no or minimal results.
In order for AI systems to work for your organisation, you need more than just models. You need systems: Human systems as well as AI systems. One cannot perform well without the highest contribution of the other.
Conclusion
AutoML will continue to surge. Certain tasks will surely be automated and certain roles will be removed for sure. However for decades to come, AutoML will act as a supplementary tool that will boost data science tasks and make them more efficient.
— — — -
If you like this article, kindly give it a like. Thanks in advance.
If you wish to write in this Newsletter, kindly reach out to us via Whatsapp: +919467891831
You may want to check: Full Stack Data Scientist A-Z™ BootCamp
