This work exposes a comparison between the different contributions to Machine Learning of the programming language Python and the parallel computing framework Apache Spark.
Particularly between the libraries Pandas and Scikit-Learn of Python and the library MLlib and the Dataframes module of Apache Spark. The goal of this work is to explain to the regular users of Pandas and Scikit-Learn the different functioning of several procedures and algorithms of Machine Learning between these libraries and MLlib, a library based on the Big Data paradigm.
For that purpose, comparisons of the basic procedures of a Machine Learning work (missings, outliers, …) and all the algorithms in the MLlib library will be made between both libraries. Besides the Machine Learning algorithms present in MLlib, we will describe the functioning of the library Elephas that extends the library Keras of Python for neuronal networks to parallel computing.
Moreover, explanations regarding the reasons between the differences in the implementations and options will be given. For some cases in which Apache Spark lacks options, we will develop our own one´s. Also, we will show code examples of a great number of the different procedures and algorithms and a complete MachineLearning flow.