SAI #03: Machine Learning Deployment Types, Spark - Architecture and more...

Machine Learning Deployment Types, ML Experiment/Model Tracking, Spark - Architecture, Kafka - Reading Data (Basics)

👋 This is Aurimas. I write the weekly SAI Newsletter where my goal is to present complicated Data related concepts in a simple and easy to digest way. The goal is to help You UpSkill in Data Engineering, MLOps, Machine Learning and Data Science areas.

In this episode we cover:

• Machine Learning Deployment Types.

• Experiment/Model Tracking.

• Spark - Architecture.

• Kafka -Reading Data (Basics).

---

MLOps Fundamentals or What Every Machine Learning Engineer Should Know

---

𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗧𝘆𝗽𝗲𝘀. 

There are many ways you could deploy a 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗠𝗼𝗱𝗲𝗹 to serve production use cases. Even if you will not be working with them day to day,  the following are the four ways you should know and understand as a 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿.
 
➡️ 𝗕𝗮𝘁𝗰𝗵: 
 
👉 You apply your trained models as a part of 𝗘𝗧𝗟/𝗘𝗟𝗧 𝗣𝗿𝗼𝗰𝗲𝘀𝘀 on a given schedule.
👉 You load the required Features from a batch storage, apply inference and save inference results to a batch storage.
👉 It is sometimes falsely thought that you can’t use this method for 𝗥𝗲𝗮𝗹 𝗧𝗶𝗺𝗲 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻𝘀.
👉 Inference results 𝗰𝗮𝗻 𝗯𝗲 𝗹𝗼𝗮𝗱𝗲𝗱 𝗶𝗻𝘁𝗼 𝗮 𝗿𝗲𝗮𝗹 𝘁𝗶𝗺𝗲 𝘀𝘁𝗼𝗿𝗮𝗴𝗲 and used for real time applications.
 
➡️ 𝗘𝗺𝗯𝗲𝗱𝗱𝗲𝗱 𝗶𝗻 𝗮 𝗦𝘁𝗿𝗲𝗮𝗺 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻: 
 
👉 You apply your trained models as a part of 𝗦𝘁𝗿𝗲𝗮𝗺 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲.
👉 While Data is continuously piped through your 𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗗𝗮𝘁𝗮 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀, an application with a loaded model continuously applies inference on the data and returns it to the system - most likely another Streaming Storage.
👉 This deployment type will most likely involve a real time 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗦𝘁𝗼𝗿𝗲 𝗔𝗣𝗜 to retrieve additional 𝗦𝘁𝗮𝘁𝗶𝗰 𝗙𝗲𝗮𝘁𝘂𝗿𝗲𝘀 for inference purposes.
👉 Predictions can be consumed by multiple applications subscribing to the 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 𝗦𝘁𝗿𝗲𝗮𝗺.
 
➡️ 𝗥𝗲𝗾𝘂𝗲𝘀𝘁 - 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲:
 
👉 You expose your model as a backend Service.
👉 It will most likely be a 𝗥𝗘𝗦𝗧 𝗼𝗿 𝗴𝗥𝗣𝗖 𝗦𝗲𝗿𝘃𝗶𝗰𝗲.
👉 The API service retrieves Features needed for inference from a 𝗥𝗲𝗮𝗹 𝗧𝗶𝗺𝗲 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗦𝘁𝗼𝗿𝗲 𝗔𝗣𝗜.
👉 Inference can be requested by any application in real time as long as it is able to form a correct request that conforms 𝗔𝗣𝗜 𝗖𝗼𝗻𝘁𝗿𝗮𝗰𝘁.
 
➡️ 𝗘𝗱𝗴𝗲: 
 
👉 You embed your trained model directly into the application that runs on a user device.
👉 This method provides the lowest latency and improves privacy.
👉 Data most likely has to be generated and live inside of device.
 
𝗧𝘂𝗻𝗲 𝗶𝗻 𝗳𝗼𝗿 𝗺𝗼𝗿𝗲 𝗮𝗯𝗼𝘂𝘁 𝗲𝗮𝗰𝗵 𝘁𝘆𝗽𝗲 𝗼𝗳 𝗱𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗶𝗻 𝗳𝘂𝘁𝘂𝗿𝗲 𝗲𝗽𝗶𝘀𝗼𝗱𝗲𝘀!

---

𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁/𝗠𝗼𝗱𝗲𝗹 𝗧𝗿𝗮𝗰𝗸𝗶𝗻𝗴.

A good 𝗠𝗼𝗱𝗲𝗹 𝗧𝗿𝗮𝗰𝗸𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺 should be composed of two integrated parts: 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁 𝗧𝗿𝗮𝗰𝗸𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺 and a 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗴𝗶𝘀𝘁𝗿𝘆.

From where you track 𝗠𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 metadata will depend on 𝗠𝗟𝗢𝗽𝘀 maturity in your company. 

If you are at the beginning of the ML journey you might be:

1️⃣ Training and Serving your Models from experimentation environment - you run 𝗠𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀 inside of your 𝗡𝗼𝘁𝗲𝗯𝗼𝗼𝗸 and do that manually at each retraining.

If you are beyond Notebooks you will be running 𝗠𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀 from 𝗖𝗜/𝗖𝗗 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀 and on 𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗼𝗿 𝗧𝗿𝗶𝗴𝗴𝗲𝗿𝘀.

In any case, the 𝗠𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 will not be too different and a well designed System should track at least:

2️⃣ 𝗗𝗮𝘁𝗮𝘀𝗲𝘁𝘀 used for 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗠𝗼𝗱𝗲𝗹𝘀 in 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗼𝗿 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗠𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀. Here you should also track your 𝗧𝗿𝗮𝗶𝗻/𝗧𝗲𝘀𝘁 𝗦𝗽𝗹𝗶𝘁𝘀. At this stage you should also save all important metrics that relate to 𝗗𝗮𝘁𝗮𝘀𝗲𝘁𝘀 - 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 etc.
3️⃣ 𝗠𝗼𝗱𝗲𝗹 𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿𝘀 (e.g. model type, hyperparameters) together with 𝗠𝗼𝗱𝗲𝗹 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗠𝗲𝘁𝗿𝗶𝗰𝘀.
4️⃣ 𝗠𝗼𝗱𝗲𝗹 𝗔𝗿𝘁𝗶𝗳𝗮𝗰𝘁 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻.
5️⃣ 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 is an 𝗔𝗿𝘁𝗶𝗳𝗮𝗰𝘁 itself - track information about who and when triggered it. Pipeline ID etc.
✅ 𝗖𝗼𝗱𝗲: Everything is code - you should version and track it.

When a 𝗧𝗿𝗮𝗶𝗻𝗲𝗱 𝗠𝗼𝗱𝗲𝗹 𝗔𝗿𝘁𝗶𝗳𝗮𝗰𝘁 is saved to a 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗴𝗶𝘀𝘁𝗿𝘆 there should always be a 1 to 1 mapping of previously saved 𝗠𝗼𝗱𝗲𝗹 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝘁𝗼 𝗧𝗵𝗲 𝗔𝗿𝘁𝗶𝗳𝗮𝗰𝘁 which was outputted to 𝗧𝗵𝗲 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗴𝗶𝘀𝘁𝗿𝘆: