Chapter 2: Kubeflow

Kubeflow Components

Component	Purpose
Pipelines	ML workflow orchestration
Notebooks	Jupyter notebooks on K8s
Katib	Hyperparameter tuning
KFServing	Model serving
Training Operators	Distributed training (TFJob, PyTorchJob)

Kubeflow Pipeline Example

from kfp import dsl, compiler

@dsl.component
def preprocess_data(input_path: str, output_path: str):
    import pandas as pd
    df = pd.read_csv(input_path)
    # Preprocessing logic
    df.to_csv(output_path, index=False)

@dsl.component
def train_model(data_path: str, model_path: str):
    import joblib
    from sklearn.ensemble import RandomForestClassifier
    # Training logic
    model = RandomForestClassifier()
    joblib.dump(model, model_path)

@dsl.pipeline(name="ml-training-pipeline")
def ml_pipeline(input_data: str = "gs://bucket/data.csv"):
    preprocess_task = preprocess_data(input_path=input_data, output_path="/tmp/processed.csv")
    train_task = train_model(data_path=preprocess_task.output, model_path="/tmp/model.pkl")

compiler.Compiler().compile(ml_pipeline, "pipeline.yaml")

---

Next: Chapter 3: Scaling ML Workloads