MLOps: Continuous delivery and automation pipelines in machine learning

  • Large datasets
  • Inexpensive on-demand compute resources
  • Specialized accelerators for ML on various cloud platforms
  • Rapid advances in different ML research fields (such as computer vision, natural language understanding, and recommendations AI systems).

But Why Machine Learning Models Crash And Burn In Production

Why Do ML Models Fail?

MLOps

MLOps : CI/CD pipeline automation

Prerequisites to understand the work done in this project :

  • Git
  • Github
  • Docker
  • Jenkins
  • Linux OS
  • Very basic knowledge of Shell/Unix commands
  • ML
  1. CNN
  2. Deep learning basics
  3. Keras

Steps to be followed :

Steps / Approach to be followed :

  • accuracy.txt : This file reads and writes the accuracy obtained from the base model that we run in the previous step.
  • retrain.py : This is an intermediate file that reads the input file (main file) into a variable and reads the code line by line as it starts building an output file correspondingly with the added code. The added code in my case is the layers ( Conv2D and MaxPooling ) you can also add the Hidden/ Dense layers using this approach.
    Finally all these lines are written to a new output file ( new.py in my case ).
    You can rewrite over the same input file if you want.
  • new.py : This file is just the inout file plus the appended code in the retrain.py file.
  • You can tweak any hyperparameter this way such as neurons in the Dense layer, Activation function etc )
  • Syntax used here is too critical and sensitive. I had a real struggle while creating this job ( Required almost 52 builds )
  • sed -i : It is a UNIX command used to insert into an existing file
  • /^ : this is used to search for the line/keyword after which you would like to insert
  • */ : everything in the current directory that ends on a /. Usually files don’t end on /, but ending a path with a slash means you specified a directory, so this only means directories.
  • /ml_task/new.py : new.py is the file in which the changes are to be made

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store