Overcoming surprises faced by Programmers migrating to Machine learning projects

Traditional programmers hit a stone-wall when they start working on Machine Learning (ML) development projects. They were used to coding based on design derived from requirements, logic/rules and when they move into a new paradigm in which rules are automatically generated from data, they are surprised. In ML paradigm, we do not find any output code and we see an output in the form of a binary object called model that goes into production. You may be wondering without measuring lines of code, how you get rated and assessed for productivity and quality of deliverables. It takes some time for traditonal programmers to overcome these surpises and get acquinted with the new world of ML development.

These days IT companies advertise machine learning powered software products and solutions. This means that existing software that is already been used in the past is now suddenly getting re-branded with the new machine learning buzz words. For survival, traditional programmers also should go through a re-banding process. Machine learning is data driven and data is the fuel for ML. Programmers were exposed to the schema definitions of the data used as input to ther programs. It was not necessary to see large instances of real data. But ML development starts with collection of huge volume of data. The quality of the deliverables of the ML project depends on the volume of data used for building the models. The more the volume of good quality data, the more the accuracy of the outputs generated by the model. Machines learn from data and as a programmer, what they can contribute ? Where do they apply their knowledge about optimized data structures and coding skills ?. In this blog, let us examine the surprises faced by traditional programmers when they migrate to the ML world.

ML is a new way to program computers and to succeed as a ML developer, you do not need a Masters or PhD in computer science or another complex mathematical or technological subject. In ML paradigm, programs are generated based on some known desired outputs created in the past. The expertize of traditional Java programmers is on creating instances of classes provided by Frameworks available in Core JAVA and making use of them to implement the logic envisaged in the design. A typical example is the Java collections framework, a combination of classes and interfaces. When a Java programmer makes use of the classes of the collections framework, he need not understand the details of the implementation of these classes. The expertize needed is in the identification of the most appropriate class and the interface to store data. He might have studied the implementation of these classes in a course on data structures during his college days. But to use them, it is not required to understand how they are being implemented in the collections framework.

In the ML paradigm of development, situation is almost identical. The expertize in ML development depends on identification of a suitable algorithm, selection of a set of APIs, functions and arguments. These APIs are for Optimization, Estimation of loss, Activation, Regularization etc. The selection of ML algorithms is identical to the logic applied when you choose between Linked list and Array list for storing data. Another essential expertize is in Hyper-parameter tuning. The developer need not have a clear insight into the implementation of the logic used in the ML APIs, but he should be an expert in making use of them in development. For a traditional programmer, it is hard to understand the theory of the machine learning algorithms implemented in these APIs. But an overall idea about these algorithms can be gained by listening to training videos and getting certified on few online machine learning courses. With this knowledge, he will be able to become productive in ML development projects. This expertize is almost similar to his expertize in making use of the classes in the Java collections framework. But the algorithms implemented in the Machine learning APIs are highly mathematical and statistical in nature and it will be a challenge for a traditional Java programmer to understand them completely. With a sense of dedication and commitment, traditional programmers can become productive in ML deveopment projects within a duration of 3 to 6 months.

The world is changing fast by adopting ML into products and solutions. Get ready to migrate from traditional programming to machine learning development.

See you next time…………