That someone was my senior, Akash Singh. In a beginner friendly session addressed by him, he went on covering DevOps topics and introduced us to MLOps. Here is the knowledge transfer.

A developer builds an app, tests it, everything's smooth. They ship it to a colleague, or push it to a server, and suddenly... it breaks. Why? Because that other machine has a different OS version, a missing library, a conflicting dependency, or some obscure config setting that nobody remembers setting up.

No two setups were exactly alike, and that inconsistency cost teams countless hours in debugging, deployment failures, and "works here, not there" chaos. This is the problem Docker was built to solve.

1. Docker

Think of Docker as a way to pack your entire application code, libraries, dependencies, runtime, settings, everything into one neat, portable box. That box is called a container. A container is lightweight and isolated, which basically means wherever you run it , your laptop, a teammate's machine, a test server, or the cloud, it behaves exactly the same without any "lets do something different....suprisee!" situations.

Docker isn't one single thing, it's a small ecosystem of parts working together:

• Dockerfile : A plain text file with step-by-step instructions telling Docker exactly how to build your app's environment.

• Docker Image : This is the packaged version created from the Dockerfile, think of it as a ready to run template of your app.

• Docker Container : As discussed earlier container is the actual running instance of that image where the application executes.

• Docker Engine :The behind the scenes machinery that takes your Dockerfile, builds the image, and spins up containers from it.

• Docker Hub : A shared online library where people store and distribute Docker images, kind of like GitHub, but for pre-packaged environments instead of code.

Docker ensured that if your application fails, it can now fail consistently everywhere.

Alright, now your app is neatly packaged in a container, ready to run identically anywhere. Great!! but that raises the next big question: where does it actually run?

2. Managed Hosting vs Self-Hosting

This is where you hit a fork in the road. Do you rent out infrastructure that someone else builds, secures, and maintains for you? Or do you fancy a sleepless night, get your own servers, and manage everything yourself?

Managed Hosting

With managed hosting, you basically hand off the messy stuff like server maintenance, security patches, uptime to a third-party provider. They handle the infrastructure and operational headaches, so you can focus on what actually matters: building and shipping your application.

But "managed hosting" isn't one single bite, it comes in three levels, depending on how much control you want to give up or how much convenience you want in (or more precisely, how good is your bank account doin?): SaaS, PaaS, and IaaS.

SaaS (Software as a Service)

This is the most hands-off option. The software is already built, already hosted, and ready to use, you just log in and use it. No servers, no deployment, no updates to worry about. Someone else has already done all the work.

Examples include Google Docs, Gmail.

PaaS (Platform as a Service)

Here, you're a bit more involved. You bring your own application code, and the platform takes care of everything underneath it like servers, runtime environment, scaling, all of it. You deploy, they handle the plumbing.

Examples include Heroku, Render, Railway.

IaaS (Infrastructure as a Service)

This is where things get more hands-on. Instead of a ready-made platform, you're renting raw infrastructures like virtual servers, storage, networking. From there, it's on you to set up the operating system, install what you need, deploy your app, and manage most of the stack yourself. You get a lot more control, but with that control comes a lot more responsibility.

Examples include AWS EC2, Google Compute Engine, and Azure Virtual Machines.

Self-Hosting

Self-hosting flips the script entirely, now you're the provider (or may I say survivor). You manage the entire hosting environment yourself, whether that's on physical servers sitting in a rack somewhere or virtual machines you fully control.

And when we say "everything," we mean it. Server setup, OS configuration, security patches, updates, deployment, monitoring, scaling, it's all on you. There's no support ticket to raise when something breaks at 2 AM. You are the support ticket.

The upside? Total control and flexibility. You can tweak, configure, and optimize exactly how you want, with no platform limitations holding you back. The tradeoff? It demands serious technical knowledge and constant hands-on maintenance. Oh, and don't forget to make sure nobody accidentally unplugs the server in the rack, tends to affect uptime.

So let's say you've made your choices, you're running containers, maybe a bunch of them, maybe across multiple servers. Things are going fine... until they're not. One container crashes. Traffic spikes and your app needs three more instances, right now. Suddenly, manually managing containers one by one feels less like "control" and more like playing a Shakespearian character doomed to destruction. This is exactly the mess Kubernetes was built to clean up. If Docker's job was to package your app so it runs the same everywhere, Kubernetes' job is to make sure all those containers actually behave

3. Kubernetes

Once you're running more than a handful of containers, managing them by hand stops making sense. You need something that can keep track of every container, restart the ones that crash, spread traffic across them, and adjust how many are running based on load automatically, without a human clicking buttons.

That's what Kubernetes does. It's a container orchestration platform, a system that manages the deployment, scaling, and networking of containers across a group of machines. Here's the basic mental model:

Cluster : A Kubernetes setup is called a cluster. It's made up of multiple machines (physical or virtual) working together as one system.
Node : Each individual machine in the cluster is called a node. Some nodes run your actual application containers; one or more nodes act as the control plane, making decisions about the cluster.
Pod : Kubernetes doesn't run containers directly rather it wraps them in a Pod, the smallest deployable unit in Kubernetes. Most of the time, a Pod runs a single container, but it can hold more than one if they need to work closely together.
Deployment : Instead of manually creating Pods, you describe the desired state like "I want 3 replicas of this app running" in a Deployment. Kubernetes then continuously works to make reality match that desired state. Pod crashes? It replaces it automatically.
Service : Pods are constantly being created and destroyed, which means their IP addresses keep changing. A Service gives them a stable address and a way to be discovered and reached, acting as a consistent front door to a set of Pods.

What is Kubernetes Ingress?

So here's the thing, by default, everything running inside your Kubernetes cluster is basically hidden from the outside world. Your apps live in Pods, grouped behind Services, but those Services only exist within the cluster's private network. As far as the internet is concerned, they don't exist. Which is fine, until you actually want people to use your app.

This is where Ingress comes in. It acts as a smart gateway sitting right at the edge of your cluster. It watches for incoming HTTP and HTTPS traffic from the outside world, and based on a set of rules you define, it figures out exactly which internal Service that traffic should be handed off to. Besides, a more significant reason for using Ingress is the expenses it saves. If you have 10 services and expose each one with its own load balancer, you'll end up paying for 10 load balancers since each one has a cost. Instead, we use a single Load Balancer with Ingress, which routes requests to different backend services based on hostname and path. This way all the services are exposed through one Load Balancer and you pay only for that single one.

How Traffic Routing Works

To understand how Ingress really works, it helps to split it into two separate pieces: what you want to happen, and who actually makes it happen.

The first piece is the Ingress Resource ,its a YAML file where you write down your routing rules. Something like "requests coming in for api.myapp.com should go to the API Service, and everything under /blog should go to the blog Service." It's purely a set of instructions. On its own, it doesn't do anything.

The second piece is the Ingress Controller , this is the actual engine that reads those rules and enforces them. Popular options include NGINX and Traefik. The Controller constantly watches for Ingress Resources in the cluster, and the moment one is created or updated, it configures itself to start routing traffic accordingly. Once traffic hits the Controller, it decides where to send it using one of two main strategies:

1. Domain-Based Routing

This one's straightforward , as in the traffic gets routed based on the hostname itself. Different subdomains point to completely different applications.

Examples

blog.pointblank.club
careers.pointblank.club

Best for separating distinct applications that need their own web addresses.

2. Path-Based Routing

Here, instead of splitting by domain, you keep everything under one single domain and split traffic based on the URL path instead.

Examples

pointblank.club/events
pointblank.club/oss

This is the go to approach when you're breaking down a larger, monolithic application into smaller microservices, but still want everything to feel unified under one address.

Rolling Deployments and Rollouts in Kubernetes

Here's a real problem: your app is live, people are using it, and now you need to ship an update. Do you just... turn everything off, deploy the new version, and turn it back on? That would mean downtime and downtime means angry users , well.. it actually means the period during which a system, server, website, or service is unavailable or not working.

Kubernetes has a better way to handle this: rolling deployments. Instead of killing everything at once, it gradually swaps out old Pods for new ones, a few at a time, so the application stays up and running the entire time.

Say you've got 10 Pods running version 1.0. Kubernetes doesn't nuke all 10 and start fresh, it slowly spins up Pods running version 2.0, and as each new one comes online healthy, it retires an old one. Bit by bit, version 1.0 fades out and version 2.0 takes over, without your users ever noticing a gap.

So What Exactly Is a Rollout?

A rollout is simply the process Kubernetes follows to carry out this update it's how Kubernetes gets your app from "current state" to "desired new state" safely.

During a rollout, Kubernetes is quietly juggling a few things at once:

Spinning up new Pods with the updated version
Gradually shifting traffic over to them
Removing old Pods, but only once the new ones prove they're healthy
Keeping a close eye on the whole process in case anything goes wrong

Why Bother With All This?

Because the alternative is scary. Rolling deployments give you:

Near-zero downtime : users barely feel the update happening
Safer upgrades : you're not betting everything on one big risky switch
Easy rollbacks : if version 2.0 turns out to be broken, Kubernetes can back out to the last stable version
Continuous availability : the app just... keeps working, the whole time

The Flow, Simply Put

It basically looks like this:

Current Version Running → New Pods Created → Health Checks Pass → Old Pods Removed → Update Complete

And here's the safety net: if the new version fails its health checks along the way, Kubernetes doesn't just barrel forward. It can pause the rollout right there, or roll back to the previous stable version entirely keeping the damage to users as close to zero as possible.

So far, everything we've talked about Docker, hosting, Kubernetes, Ingress, rollouts has been about one core problem: how do you reliably ship and run code. And honestly, the industry has gotten pretty good at that. Once your app is packaged and deployed, it either works or it doesn't, and you'll know pretty quickly which one it is.
But what happens when the thing you're shipping isn't just code it's a machine learning model? Turns out, that changes everything.

4. Intro to MLOps

Shipping a machine learning model isn't like shipping code. Code doesn't quietly get worse over time but models do. As real-world data shifts and evolves, a model's accuracy can silently erode. Retraining isn't a quick fix either; it means rerunning entire pipelines from scratch. A drop in performance usually doesn't throw an error or crash anything. It just... degrades. Nobody notices until predictions start quietly going wrong, and by then, the damage might already be done.
This is exactly the gap MLOps exists to fill.

MLOps (Machine Learning Operations) is essentially DevOps principles like automation, versioning, CI/CD, monitoring, reproducibility ,but applied to the entire machine learning lifecycle. It's not just about deploying a model once, it's about managing everything from data preparation and training, all the way through deployment and long-term maintenance.

How an ML Pipeline Is Actually Built

Before a model ever makes a prediction, it goes through a fairly structured pipeline think of it as an assembly line, where each stage feeds into the next:

Data processing: Data scientists assemble and prepare the data that will be used to train the ML model. Phases in this stage include data collection, preprocessing, cleaning and exploration.
Model development: Data practitioners choose or create a ML algorithm that fits the needs of the project. The algorithm is trained on the data from the previous step, and the resulting model is tested and validated until it is ready for use.
Model deployment: Developers and software engineers deploy the model for real-world use, integrating it into a production environment and monitoring its performance.

The key idea here is that this isn't a one-time process it's a loop. Data keeps changing, so the pipeline needs to keep running.

Training vs Inference ,What's the Difference?

training and inference are the two core phases of an ML model's lifecycle

Training is the learning phase. You feed the model a large dataset, and it adjusts itself internally, learning patterns, weights, and relationships. This is usually slow, resource-heavy, and only needs to happen occasionally (or whenever you want the model to improve).
Inference is the using phase. Once the model is trained, inference is what happens every time it's asked to make an actual prediction on new input, like when a user types a prompt and expects a response back. This needs to be fast, since real users are waiting on the other end.

Training teaches the model, inference is the model doing its job.

What Is vLLM, and Why Does It Matter?

A real bottleneck teams run into: even after a large language model is fully trained, actually running it efficiently, serving thousands of user requests quickly and cheaply is a massive engineering challenge on its own. LLMs are huge, and naively running inference on them can be painfully slow and expensive.

This is where vLLM comes in. It's an open-source library specifically built to make LLM inference faster and more memory-efficient. Instead of naively processing requests one at a time and wasting GPU memory, vLLM uses smarter memory management techniques (like something called PagedAttention) to serve way more requests simultaneously, without needing proportionally more hardware.

vLLM helps you serve a large language model to real users quickly and cheaply, instead of it choking under load or burning a hole in your GPU budget.

Conclusion

If you're currently learning DevOps and feeling overwhelmed, don't worry. The roadmap only gets longer. But at least now we know why Docker exists, why Kubernetes exists, and why machine learning engineers have started borrowing all of DevOps' homework.

Thanks to Akash Singh for this session and sharing knowledge shaped by real-world industry experience.

For anyone wanting to learn more about docker and Kubernetes I recommend reading the official documentation at https://kubernetes.io/docs/home/ and https://docs.docker.com/get-started/ to get started with.