Resources
A curated collection of tools, platforms, and learning resources for MLOps and LLMOps.
🛠️ MLOps Tools
Experiment Tracking
| Tool | Description | Link |
|---|---|---|
| MLflow | Open-source ML lifecycle platform | mlflow.org |
| Weights & Biases | Experiment tracking and collaboration | wandb.ai |
| Neptune.ai | ML metadata store | neptune.ai |
| Comet | ML experiment platform | comet.com |
Data & Model Versioning
| Tool | Description | Link |
|---|---|---|
| DVC | Data version control | dvc.org |
| Git LFS | Large file storage | git-lfs.com |
| LakeFS | Data lake versioning | lakefs.io |
Feature Stores
| Tool | Description | Link |
|---|---|---|
| Feast | Open-source feature store | feast.dev |
| Tecton | Enterprise feature platform | tecton.ai |
| Hopsworks | ML data platform | hopsworks.ai |
Model Serving
| Tool | Description | Link |
|---|---|---|
| TorchServe | PyTorch serving | pytorch.org/serve |
| TF Serving | TensorFlow serving | tensorflow.org/tfx/serving |
| Triton | NVIDIA inference server | nvidia.com/triton |
| BentoML | ML model packaging | bentoml.com |
| Seldon | ML deployment platform | seldon.io |
Pipeline Orchestration
| Tool | Description | Link |
|---|---|---|
| Apache Airflow | Workflow orchestration | airflow.apache.org |
| Prefect | Modern workflow engine | prefect.io |
| Dagster | Data orchestration | dagster.io |
| Kubeflow Pipelines | ML pipelines on K8s | kubeflow.org |
Data Quality
| Tool | Description | Link |
|---|---|---|
| Great Expectations | Data validation | greatexpectations.io |
| Soda | Data quality platform | soda.io |
| dbt | Data transformation | getdbt.com |
🤖 LLMOps Tools
LLM Serving
| Tool | Description | Link |
|---|---|---|
| vLLM | High-throughput serving | vllm.ai |
| TGI | HuggingFace inference | github.com/huggingface/tgi |
| Ollama | Local LLM serving | ollama.ai |
| LMDeploy | LLM deployment toolkit | github.com/InternLM/lmdeploy |
LLM Frameworks
| Tool | Description | Link |
|---|---|---|
| LangChain | LLM application framework | langchain.com |
| LlamaIndex | Data framework for LLMs | llamaindex.ai |
| Semantic Kernel | Microsoft's LLM framework | github.com/microsoft/semantic-kernel |
LLM Evaluation
| Tool | Description | Link |
|---|---|---|
| RAGAS | RAG evaluation | ragas.io |
| DeepEval | LLM evaluation framework | github.com/confident-ai/deepeval |
| LangSmith | LangChain's evaluation platform | smith.langchain.com |
| Phoenix | Arize's observability | phoenix.arize.com |
Prompt Management
| Tool | Description | Link |
|---|---|---|
| PromptLayer | Prompt management | promptlayer.com |
| Humanloop | LLM evaluation & prompts | humanloop.com |
| DSPy | Programmatic prompting | github.com/stanfordnlp/dspy |
☁️ Cloud Platforms
| Platform | ML Service | Link |
|---|---|---|
| AWS | SageMaker | aws.amazon.com/sagemaker |
| GCP | Vertex AI | cloud.google.com/vertex-ai |
| Azure | Azure ML | azure.microsoft.com/ml |
📚 Learning Resources
Books
- "Designing Machine Learning Systems" by Chip Huyen
- "Machine Learning Engineering" by Andriy Burkov
- "Building Machine Learning Pipelines" by Hannes Hapke & Catherine Nelson
Courses
Blogs
Newsletters
🏢 Communities
Keep Learning
The MLOps and LLMOps landscape evolves rapidly. Follow the communities and newsletters to stay updated with the latest tools and best practices.