LoginSign Up
← Back to all sparks
M

MLflow

AI-ML-PLATFORMS
Velocity6.3

Open source platform for managing the machine learning lifecycle

mlflow.org ↗

MLflow 3.18 starts watching agents in production, not just grading them before release.

llm-observabilityevaluationagent-monitoringissue-detectionmlops
◆Current state
MLflow's 3.x line has been building GenAI tracing and evaluation, and 3.18.0 extends it past the pre-release check. Automatic Issue Detection groups recurring agent failures into an Issues tab linked to trace evidence and can run on a schedule; online evaluation scores new traces and conversations as they arrive. A redesigned comparison view traces score changes back to the inputs and steps behind them, and an experimental Skill Registry catalogs reusable agent skills.
◆Where it's heading
The arc runs from logging traces (3.16) to typed judges and lineage (3.17) to continuous monitoring and triage (3.18). MLflow is turning into an operations surface for agents, with each release adding a step after deployment rather than before it. The cost is a steady stream of schema migrations and breaking changes for self-hosters.
◆Prediction
Expect 3.18.x patches to harden issue detection and online evaluation and to drop experimental labels on the Skill Registry. Whether code-based online evaluation reaches servers without job containers is not yet clear from the entries.

◆Recent moves

  1. 10d ago

    MLflow 3.18.0

    ⚡ SPARK

    The release that moves MLflow's GenAI work from pre-ship evaluation into production: scheduled issue detection, online evaluation of live traces, and comparisons that explain score changes. It builds directly on the 3.16 trace UI and 3.17 lineage work.

    View source ↗
  2. 24d ago

    3.17.2 records dataset, scorer and agent lineage on evaluate runs

    A patch that adds one meaningful feature: evaluation runs now record the dataset, scorers and agent they used, so results can be traced back to inputs. It continues the 3.17 push toward structured, reviewable evaluation; the Optuna fixes are routine.

    View source ↗
  3. 1mo ago

    MLflow 3.17.1

    A patch that extends 3.17's typed-feedback work: make_judge() now supports the Decision API, and review queues on Databricks become manageable from the SDK. The Claude Code OTel rendering fix shows coding-agent traces are now a use case the UI has to handle.

    View source ↗
  4. 1mo ago

    Weekly model catalog refresh (CI)

    The CI job that refreshes the per-provider model catalog files ran its weekly update. Useful plumbing for the AI Gateway's model lists, but not a product release.

    View source ↗
  5. 1mo ago

    MLflow 3.17.0

    3.17 adds typed Boolean and categorical judge feedback, resource-level permissions with wildcard grants and explicit DENY, and opt-in daily rollups for trace analytics. It continues the shared-server hardening from 3.16, but it brings a mandatory schema upgrade with every writer stopped.

    View source ↗
  6. 1mo ago

    MLflow 2.11.5

    A 2.x maintenance patch adding opt-in Databricks SDK Files API routing for Unity Catalog model artifacts. It shows the legacy line is still serviced for Databricks users while new work lands in 3.x.

    View source ↗