PyCaret
GitHub
← Back to blog
2026-05-15

Session 55: End-to-end UI test pass + csv_upload path fix

Engineering log for session 55.

Live exploration session walking the platform as a first-time user: workspace → project → CSV upload → EDA → (planned) experiment → promote → deploy → downstream features (monitoring, drift, schedules, webhooks, approvals, lineage, LLM). Bugs found while clicking through are tagged below.

FIXED — CSV upload stored a relative path, breaking refresh + future loaders#

  • FIXED — data_sources.py:175 (upload_csv) now stores str(target.resolve()) instead of str(target). The CSV upload writes to settings.artifact_dir / "data-sources" / "<uuid>.csv". With the default artifact_dir = Path("./artifacts"), the unresolved string was the relative "artifacts\data-sources\<uuid>.csv". On the refresh path (POST /data-sources/{id}/refresh in connections.py:394), csv_driver._read_bytes calls LocalFsObjectStore.get_bytes(path). The store's _path_for_uri doesn't recognise the relative form as a file:// URI or absolute path, so it falls through to _path_for(key) which joins against artifact_root → <artifact_root>/artifacts/data-sources/<uuid>.csv → file not found → driver raises → handler returns 400. Symptom in the UI: "Refresh failed: Request failed with status code 400" on the dataset Versions tab. Fix is one-line; behaviour for new uploads is to store the resolved absolute path, which _path_for_uri accepts via its uri[1] == ":" Windows-absolute branch.
  • FIXED — One-shot migration for existing rows: any data_sources.config.path for kind='csv_upload' that is a relative string was rewritten to its absolute form by resolving against artifact_dir.parent (the BE CWD). Done locally on the dev DB; for prod we'll need a proper Alembic data migration if any wheels with the buggy upload code shipped — none have, this only affects dev DBs.

REMOVED — Pipelines navbar entry + page (folded into Model Registry)#

  • REMOVED — apps/web/src/pages/Pipelines.tsx + apps/web/src/pages/PipelineDetail.tsx deleted. The "Pipelines" entry is gone from the navbar; the /workspaces/:wsId/pipelines + /workspaces/:wsId/pipelines/:pipelineId routes are removed from App.tsx. The Pipeline DB table stays — it's still the artifact pointer the predict path uses — but the UI for browsing it has folded into the Model Registry surface (since session-56's unified promote, every Pipeline has a matching RegisteredModelVersion).
  • ADDED — apps/web/src/pages/RegisteredModelDetail.tsx now carries the Deploy action: a per-version Deploy button opens a Dialog that calls registryApi.deploy(modelId, versionId, {endpoint_slug, auth_mode}). The version row shows existing endpoint slugs inline as /slug chips that link to the Deployment detail page. The model-detail page is now the canonical "look at an artifact, deploy it, see what's serving it" surface.
  • CHANGED — services/api/pycaret_server/api/deployments.py::_serialise_pipeline now back-fills registered_model_id + registered_model_version_id by walking Pipeline → Trial → RegisteredModelVersion. The Pipeline TS type gains both fields. Pre-session-56 Pipelines (no matching registry row) return null — which is the correct historical fact and gates the "Open in registry →" link in the UI accordingly.
  • CHANGED — Sweep of UI references to the deleted pages:
    • TrialDetail.tsx — "Open pipeline →" became "Open in registry →" (gated on registered_model_id).
    • TrialsCard.tsx — promoted-trial "view →" link now points at /workspaces/:wsId/models (registry list).
    • RunDetail.tsx::PromotedPipelinesSection — section renamed to "Promoted versions"; rows link to /workspaces/:wsId/models/:registered_model_id (or the workspace list as fallback). Empty-state copy reframed around the Model registry.
    • DeploymentDetail.tsx — "· pipeline {name}" → "· model {name}" with link into the registry.
    • Deployments.tsx empty state — link rerouted from /pipelines to /models.
    • WorkspaceDetail.tsx KPI tile relabeled "Pipelines" → "Model versions" with link to /models.
    • WorkspaceHome.tsx KPI tile + quick-action card same.
    • CommandPalette.tsx — collapsed two entries ("Pipelines registry" + "Model registry") into one "Model registry" command with the pipeline keyword preserved for discoverability.
    • TrialCompare.tsx section title "Pipelines" → "Pipeline structure" (it shows the sklearn pipeline DAG per trial, not our Pipeline table — disambiguation only).

CHANGED — Single unified promote (Pipeline + Model Registry version, atomic)#

  • CHANGED — services/api/pycaret_server/api/runs.py::promote_trial is now the single source of truth for promotion. One atomic transaction writes the Pipeline row (legacy artifact pointer used by predict / rollback) AND a RegisteredModelVersion off the same Trial bytes. The RegisteredModel is found-or-created by (workspace_id, name), so the first promote of a name creates the registry entry; subsequent promotes bump both Pipeline.version and RegisteredModelVersion.version in lock-step. The response now includes registered_model_id + registered_model_version_id so the UI can deep-link without a follow-up call.
  • CHANGED — unpromote_trial symmetrically deletes both the Pipeline AND the matching RegisteredModelVersion. Returns 409 if either has an active Deployment. The parent RegisteredModel is preserved (other versions may still exist; the user can delete the empty container explicitly).
  • CHANGED — services/api/pycaret_server/api/deployments.py::create_deployment (pipeline-based) now back-fills registered_model_id + registered_model_version_id on the new Deployment by walking Pipeline → Trial → RegisteredModelVersion. Deployments created from pre-unified-promote Pipelines stay registry-less (no governance row existed to point at) — which is the correct historical fact.
  • REMOVED — POST /api/v1/registered-models/{model_id}/versions (registry.py::create_version). It was always-broken (read Run.stored_path / sha256 / params / metrics, which live on Trial instead — would AttributeError on the first non-empty trial) and had zero test coverage. Removal is forward-only; no migration needed.
  • REMOVED — apps/web/src/components/RegistryPromoteDialog.tsx file deleted. The "Promote to registry" button on the Trial Detail page is gone; the legacy "Promote (legacy)" button is renamed to plain "Promote". Single button, single mental model.
  • REMOVED — registryApi.promote call in apps/web/src/api/endpoints.ts (the FE wrapper around the deleted backend endpoint).
  • TESTS — test_session25.py::test_trial_promote_creates_pipeline extended to assert the unified write: response carries both registered_model_id + registered_model_version_id, the workspace Registry list includes the new model, and a second promote under the same name produces a v2 attached to the SAME RegisteredModel. All 13 promote-related tests across test_session25.py / test_session32_revert_phase0.py / test_phase9_finish.py pass green.
  • DOCS — docs/revamp/DECISIONS.md ADR added (see entry below). The maintainer (Moez, 2026-05-15) explicitly approved the consolidation; the Phase-7 split was leaky from inception (broken handler, no tests, parallel-table UX).

ADDED — Interactive dataset-explore modal (click the row body)#

  • ADDED — apps/web/src/components/DatasetExploreModal.tsx — full-width Dialog (size="full") opened when a user clicks the body of a dataset row in DataSourcesSection. Single GET /data-sources/{id}/profile?sample_rows=50 call drives the whole experience — no per-column round trips, no loading flicker between selections.
    • Layout (viewport-bounded, both panes scroll internally — fixed h-[78vh] body so the modal never overflows the laptop screen): top region holds the dataset-health pill strip (rows · cols · memory · duplicates · missing% with red/amber tones at thresholds) + a heuristic-chip strip (clickable → jumps to the column). 7/5 grid below: sample-rows table on the left, tabbed pane on the right.
    • Right-pane tabs: Columns (filterable picker + selected column detail), Overview (dtype-distribution bars, top-N most-missing columns ranked, full warning list with severity), Correlations (top-10 strongest pairs + diverging-colour SVG mini-heatmap with cell tooltips). Tab counts: column count on Columns, warning count badge on Overview, disabled state on Correlations when no numeric columns exist.
    • Column picker rows carry a dtype-kind badge, the column name, an inline 14-bar sparkline for numeric columns (a peek at distribution shape without a click), and a missing-% bar.
    • Per-column detail renders a stats grid (unique / cardinality / missing / dtype + numeric extras: min/max/mean/median/std/Q1/Q3/skew/kurt/zeros%/IQR-outliers), a pure-CSS histogram (numeric) or weighted top-values bars (categorical/text/boolean), and a "Top correlated" strip with diverging negative-on-the-left bars for numeric columns. Selecting a column highlights the matching cells column-wise in the sample table.
    • "Looks-like" hints — informational badges on the column header inferring role: target? (low-cardinality numeric/boolean), id (id-like), drop? (constant), leak? (high-cardinality categorical close to row count), sparse (>50% missing). Not consumed by the engine — surfaced for the human before they configure an experiment.
  • ADDED — apps/web/src/components/Dialog.tsx — new size="xl" (max-w-5xl) and size="full" (max-w-7xl) options, used by the explore modal.
  • CHANGED — apps/web/src/components/DataSourcesSection.tsx — the row body (icon + name + size summary) is now a button that opens the explore modal. The existing Explore (page) / Versions / AI / Delete action buttons are unaffected.

CHANGED — LLM provider settings page: clear stored-state UI#

  • CHANGED — apps/web/src/pages/LLMSettings.tsx rewritten to make stored-vs-not-stored state obvious. When has_api_key === true: emerald "✓ API key on file" status card at the top with provider, model, relative saved-at timestamp + Test connection / Rotate key / Clear buttons. The password input is hidden behind a lock icon ("API key hidden — use Rotate key above to replace") so users no longer face an empty password field with a "keep existing (leave blank)" placeholder they have to mentally parse. When has_api_key === false: amber "No API key on file — LLM features will fail" banner above the form; password input is required. Clear button calls the new DELETE /workspaces/{id}/llm/settings endpoint after a confirm prompt.
  • CHANGED — Default Anthropic model in LLMSettings.tsx PROVIDERS array bumped from claude-sonnet-4-5 (stale) to claude-sonnet-4-6. Same bump applied to the model-name input placeholder and the initial modelName state.

ADDED — DELETE endpoint for LLM provider settings#

  • ADDED — DELETE /workspaces/{workspace_id}/llm/settings in services/api/pycaret_server/api/llm.py. Admin-gated. Idempotent (204 even if nothing was stored). Deletes every LLMProviderSetting row for the workspace (handles the rare case where a previous provider's row was disabled but not removed). Wired into the frontend as llmApi.deleteSettings in apps/web/src/api/endpoints.ts.

ADDED — Workspace-level Datasets navbar entry + page#

  • ADDED — apps/web/src/pages/AllDatasets.tsx + new route /workspaces/:wsId/datasets (apps/web/src/App.tsx). Workspace-level index of all DataSources, mirrors the data model (DataSources are workspace-scoped, not project-scoped — see DOCS entry below). Thin wrapper over the existing DataSourcesSection component so behaviour stays in lockstep with the project-page surface.
  • ADDED — Datasets nav link in apps/web/src/components/Layout.tsx, placed between Projects and Pipelines with a database-cylinder icon. The Project-page "Data sources" section still works exactly as before — the new top-level entry is an additional, more-prominent surface.

FIXED — Lineage writes now log at WARNING instead of silently swallowing#

  • FIXED — services/api/pycaret_server/api/connections.py:record_lineage now logs the failed edge (workspace_id, kind/id pair, relation, error type+message) at WARNING when an insert raises, instead of except Exception: db.rollback() with no breadcrumb. Behaviour is unchanged (still best-effort, never propagates) — but the previous "lineage graph silently goes stale" failure mode flagged in the session-55 audit is now observable in services/api/logs.log. Added logging import + _log module logger.

ADDED — Display-only banner on Model Library#

  • ADDED — apps/web/src/components/ModelLibrarySection.tsx now renders a prominent amber banner above the controls: v1 — display only. Toggles below are saved to the catalog but the engine does not consult them yet. The prior treatment was a single muted line inside the description text that users had no reason to read. Engine-side enforcement is now tracked as an explicit ROADMAP V2 bullet.

DOCS — ROADMAP V2: worker heartbeats + engine-side Model Library enforcement#

  • DOCS — docs/revamp/ROADMAP.md V2 section gained two explicit bullets, both surfaced by the session-55 build-status audit:
    • Worker heartbeats — replace the "derive workers from Job.locked_by" hack with a real worker_heartbeats table. Currently idle workers vanish from the queue admin and a dead worker is indistinguishable from a healthy idle one.
    • Engine-side Model Library enforcement — wire compare_models to skip enabled=False rows from the workspace's ModelLibrary. Today the UI toggle does nothing; the new banner above calls this out, but the fix is V2.

DOCS — Clarified workspace-vs-project ownership of DataSources#

  • DOCS — DataSource belongs to the workspace, not the project. Confirmed against db/models.py:161 ("Registered CSV/S3/Postgres source a Project can point at") and data_sources.py route surface (all upload/register routes are workspace-scoped). The "upload dataset" modal on project pages is a UX convenience — it still calls POST /workspaces/{id}/data-sources/upload. Projects/experiments reference DataSources; they don't own them. No code change needed; flagged for the eventual user docs so the modal placement doesn't mislead.

Sessions 20–54 — 2026-04-24 → 2026-05-08 — V2 platform + UI surfacing + design refresh#

Consolidated catch-up entry covering the arc from the failure-debugger / API-key landing in session 19 through the engine MVP-1 finish (sessions 22-46), the V2 platform features (sessions 22-24, server-side; tagged on top of MVP-1 numbering), and the front-end surface that exposes them (UI sessions through s54). Written retroactively from the shipped code; individual sessions weren't entered as their own headers during the build, but each block below maps to a concrete roadmap milestone.

Baseline: session 19 left the platform with 4-of-6 LLM copilots, API keys + workspace members + audit logs not yet wired, and the engine god-class still partially undrained. Engine 4.0.0a2 was the last published wheel.

Theme: graduate every "stub" to "real implementation" across all three layers. Engine: revive every NotImplementedError verb. Server: deliver V2 control-plane features (drift monitor, scheduled retraining, deployment versioning, webhooks, AutoML pipeline search, encrypted secrets, backup/restore, expanded roles). Web: surface every new endpoint + a full design-system unification pass.

ADDED — Engine: revive the six legacy-only verbs (sessions 53–54)#

  • ADDED — Experiment.get_leaderboard() returns a defensive copy of the most recent compare_models leaderboard. Snapshotted into _fit_state["last_leaderboard"] after every supervised + time-series compare_models call. Raises RuntimeError if no compare has run.
  • ADDED — Experiment.automl() — convenience wrapper for compare_models(n_select=1) → tune_model. Returns a fitted sklearn.pipeline.Pipeline. Pass-throughs for optimize / n_iter / turbo / include / exclude / fold / fit_kwargs / round / verbose.
  • ADDED — Experiment.interpret_model() — wraps shap.Explainer over a pipeline's preprocessor + final estimator. Returns a shap.Explanation. plot= argument renders 'summary' / 'beeswarm' / 'bar' / 'waterfall' inline. SHAP is an opt-in extra: pip install pycaret[interpret].
  • ADDED — TimeSeriesExperiment.check_stats() — runs Summary / Ljung-Box (white noise) / ADF + KPSS (stationarity) / Shapiro-Wilk (normality) over the experiment's series. Returns a 6-column DataFrame. test= filter and split='all'|'train'|'test'.
  • ADDED — Experiment.plot_model(estimator, plot=None, save=False, **kw) — looks up plot in a per-task registry and returns a plotly.graph_objects.Figure. save=True writes f"{plot}.png" (requires pycaret[export] for kaleido); save="my.png" writes to that path.
  • ADDED — Experiment.evaluate_model(estimator) — returns a dict[str, Figure] of the curated diagnostic bundle for the task. Plots that fail (e.g. shap missing, estimator lacking feature_importances_) are silently dropped rather than raising.
  • ADDED — Per-task plot registries on every leaf class. Classification (16 kinds), Regression (12), Clustering (6), Anomaly (4), Time-series (8). Default plots: auc / residuals / cluster / score / forecast.
  • ADDED — [interpret] = ["shap>=0.46"] optional extra. Folded into [full].
  • ADDED — 22 new tests in test_session53_revive_legacy_verbs.py + test_session54_plot_model_dispatcher.py.

REMOVED — Engine: drop transitional core deps (sessions 47-50 area)#

  • REMOVED, BREAKING — imbalanced-learn from core deps. The legacy Pipeline class that subclassed imblearn.pipeline.Pipeline is also gone — every native verb has been operating on sklearn.pipeline.Pipeline since session 24. fix_imbalance users now install imbalanced-learn themselves.
  • REMOVED, BREAKING — category-encoders from core deps. The 4.0 native preprocessor uses sklearn-native encoders (OneHotEncoder / OrdinalEncoder / TargetEncoder).
  • REMOVED — Deleted packages/engine/pycaret/internal/preprocess/ (the entire legacy preprocessor: preprocessor.py, transformers.py, iterative_imputer.py, target/, time_series/forecasting/preprocessor.py).
  • REMOVED — Deleted packages/engine/pycaret/internal/pipeline.py (the legacy Pipeline and TimeSeriesPipeline classes).
  • REMOVED — Deleted packages/engine/pycaret/internal/patches/sklearn.py (the only file that imported internal.pipeline).
  • CHANGED — packages/engine/pycaret/utils/_show_versions.py — pruned the dep list from a 35-name 3.x relic down to a clean 10-required + 9-optional view that matches the actual ship surface.

ADDED — Engine: published 4.0.0a3 → a8 to PyPI#

  • BUILD — 4.0.0a3 (first PyPI release of the 4.0 line). 4.0.0a1 and 4.0.0a2 had only ever been distributed via GitHub release downloads.
  • BUILD — 4.0.0a4 — relaxed scikit-learn>=1.7 → >=1.5 and imbalanced-learn>=0.13 → >=0.12 after the a3 install triggered Colab kernel restart-loops by force-upgrading the preinstalled scientific stack.
  • BUILD — 4.0.0a5 — capped ipython>=8.18,<9 after a4 pulled IPython 9.x which removed IPython.utils.coloransi.TermColors.Green that google.colab._shell_customizations reads at kernel startup.
  • BUILD — 4.0.0a6 — revived get_leaderboard / automl / interpret_model / check_stats. Added [interpret] extra. Fixed __version__ hardcoded to 4.0.0a2 in pycaret/__init__.py.
  • BUILD — 4.0.0a7 — revived plot_model / evaluate_model (Plotly dispatchers wired into per-task plot registries).
  • BUILD — 4.0.0a8 — dropped imbalanced-learn + category-encoders from core deps. Core surface lands at 10 deps (numpy, pandas, scipy, scikit-learn, joblib, plotly, tqdm, requests, jinja2, ipython).

ADDED — Server: control-plane V2 features#

  • ADDED — Secrets encryption (services/api/pycaret_server/crypto.py). Fernet-backed at-rest encryption for LLM API keys + future cloud creds. Stored ciphertext is prefixed with ENC:v1: so reads can transparently fall back to plaintext for rows written before this module existed. Key rides in env var PYCARET_SECRETS_KEY; missing key in dev synthesises an ephemeral per-process key with a loud warning. Added cryptography>=43 to backend core deps.
  • ADDED — PredictionLog table (db/models.py). Append-only log of every served prediction. Drift detection, latency forensics, audit. The /deployments/{slug}/predict handler writes one row per call (ok or error path). Sample of input/output rows capped at 50 per log entry. New GET /api/v1/deployments/{id}/prediction-logs endpoint paginates with status_filter.
  • ADDED — Trial table + auto-persistence. Promotes the JSON Run.leaderboard rows into queryable entities so the UI can sort, filter, link to fitted pipelines. Written by RunOrchestrator._transition whenever leaderboard JSON lands. New GET /api/v1/runs/{id}/trials endpoint.
  • ADDED — ModelLibrary table + lazy-seed CRUD. Workspace-scoped, editable mirror of the engine's model registry. Lazy-seeded from pycaret.api.list_models on first read of a (workspace, task) pair. v1 enforcement is informational; engine-side filtering ships in V2.
  • ADDED — Admin user routes (/api/v1/admin/users GET + PATCH). Superuser-only. Toggles is_superuser / is_active with last-superuser + self-deactivation guards.
  • ADDED — ScheduledJob table + services/api/pycaret_server/scheduler.py (APScheduler in-process). Two job kinds: drift_monitor (snapshots prediction-log distributions vs. baseline; writes a DriftReport row) and retrain (re-runs a configured experiment via dispatch_run). New /workspaces/{id}/schedules CRUD + POST /schedules/{id}/run-now for immediate execution.
  • ADDED — drift_monitor.py — PSI for numeric features, chi² (Cramer's V) for categorical, mean as aggregate. Auto-writes DriftReport row + fires drift.alert webhook on moderate/severe scores.
  • ADDED — WebhookSubscription table + delivery infrastructure (webhooks.py). HMAC-SHA256-signed POSTs fired on platform events: run.{succeeded,failed,cancelled}, deployment.{created,deleted,rollback}, drift.alert, schedule.failed. Secrets stored encrypted via the new crypto module. Best-effort fire-and-forget on a daemon thread so a slow webhook target can't pin a user request. New /workspaces/{id}/webhooks CRUD + POST /webhooks/{id}/test for synthetic ping.
  • ADDED — ExperimentTemplate table + CRUD. Saved (task, setup_params, plan_params) bundles users can pre-fill the New Experiment screen from. New /workspaces/{id}/experiment-templates CRUD.
  • ADDED — Pipeline versioning (deployments.py). Pipelines that share (workspace_id, name) are revisions of the same logical model and share a family_id. Promote bumps version. New GET /pipelines/{id}/versions. New POST /deployments/{id}/rollback repoints the deployment at any earlier pipeline in the same family; in-memory registry is evicted so the next /predict reloads.
  • ADDED — plan="search" (runs/plans.py). AutoML pipeline search: orchestrator iterates a list of preprocessing variants (each a setup_params override merged on top of the experiment defaults), runs compare_models per variant, concatenates leaderboards into a single Variant-tagged DataFrame, returns the globally-best fitted pipeline.
  • ADDED — Backup/restore (api/backup.py). Superuser-only. GET /admin/backup streams a tarball containing database.json (every row from every table) + raw artifacts under the artifact dir. POST /admin/restore (multipart) wipes + reloads from the tarball with a confirm=true guard.
  • ADDED — Expanded user-role lattice (Spec § 17.2). WorkspaceMember.role widened from {admin, member} to 7 roles: owner / admin / project_admin / ml_engineer / data_scientist / viewer / service_account plus the legacy member. ADMIN_ROLES set governs admin gating + the last-admin guard.
  • ADDED — pycaret-client SDK (packages/sdk-python/). Python HTTP client for the Control Plane. Hand-written namespaced façades (cp.workspaces.list(), cp.runs.submit(...), cp.deployments.predict(...), etc.). To be published as pycaret-client on PyPI.
  • ADDED — runs/dispatch.py:dispatch_run extracted from the route handler so the scheduler (and any future programmatic enqueue path) can submit a Run without going through HTTP.
  • ADDED — Single Alembic migration b2c3d4e5f6a7 covering scheduled_jobs + webhook_subscriptions + experiment_templates + pipelines.family_id + pipelines.version. Plus prior a1b2c3d4e5f6 for prediction_logs + trials + model_library.

FIXED — Server bootstrap auto-migrate#

  • FIXED — db/bootstrap.py — pre-Alembic legacy DBs were being stamped at head even when their actual schema was older, so newer migrations never landed and OperationalError: no such table: trials would appear at first prediction. Fix: detect the schema state by walking distinguishing tables (newest fingerprint first), stamp at the matched revision, then upgrade head if behind. When already at head, stamp head (no upgrade) — this also avoids polluting Alembic's process-global ScriptDirectory cache that breaks tests bootstrapping many DBs.
  • CHANGED — db/bootstrap.py — when alembic_version is already present in dev, also run upgrade head so newer migrations land between server boots without operator action.

ADDED — Web: surface every new endpoint#

  • ADDED — Schedules screen (/workspaces/:wsId/schedules) + <NewScheduleForm>. Lists active drift-monitor + retrain schedules with last-run state. Toggle / run-now / delete inline.
  • ADDED — Experiment templates screen (/workspaces/:wsId/templates). CRUD with a JSON setup_params editor. Templates surface as a "Start from template" picker on the New Experiment screen.
  • ADDED — Webhooks screen (/workspaces/:wsId/webhooks) + <NewWebhookForm>. Event-type checkboxes, secret input, test-fire button.
  • ADDED — <TrialsCard> on RunDetail. Resolves model_id → full model name via describeApi.models(task) so rows show "Logistic Regression / Random Forest" instead of lr / rf. Two view modes: Table (sortable columns + inline progress bar on the primary metric) and Chart (Plotly horizontal bar chart with metric picker; "higher is better" / "lower is better" caption auto-flips per metric; best-row tinted green).
  • ADDED — <PredictionLogsCard> on DeploymentDetail. Status filter, auto-refresh, latency in tabular-nums, truncated request-id.
  • ADDED — <DeploymentVersionsCard> on DeploymentDetail. Lists every Pipeline in the family with rollback button per row; current version pilled green.
  • ADDED — <RunRunningCard> — animated "in flight" card that replaces the empty leaderboard while a run is queued / running. Pulsing accent dot, current stage from the latest event, live elapsed-time counter, skeleton bars where the leaderboard will land. Polls /runs/{id}/events every 1.5 s.
  • ADDED — Admin screens — /admin/users (superuser-only platform user mgmt with is_superuser + is_active toggles), /workspaces/:wsId/admin (workspace admin hub: members + LLM + audit cards + ModelLibrarySection inline), /workspaces/:wsId/admin/integrations (LLM / webhooks / object-storage / SSO cards).
  • ADDED — <BackupRestoreCard> on AdminUsers. Download tarball / upload + confirm restore.
  • ADDED — <AIAdvisorWidget> floating button + side panel (mounted in Layout). Shows the workspace's recent LLM consultations + provider status + quick-jump links. "Connect a provider" prompt when none configured.
  • ADDED — API client methods in endpoints.ts: pipelinesApi.versions, deploymentsApi.rollback / predictionLogs, full schedulesApi, templatesApi, webhooksApi, modelLibraryApi, adminApi, runsApi.trials.
  • ADDED — AutoML search plan option on NewExperiment (4th plan kind). Submits with default preprocessing variants [{}, {normalize:true}, {normalize:true, transformation:true}].
  • ADDED — Sidebar navigation entries — Schedules / Templates / Webhooks (each with a custom 16px lucide-style SVG icon — Clock, Template, Hook). Workspace-scoped Admin link below LLM. Superuser-only "Platform users" entry under Account.
  • ADDED — workspace_id + project_id on the Run response so /runs/:id deep-links can resolve the active workspace for the sidebar. Layout.tsx reads runForCtx.data?.workspace_id as a fallback when the URL doesn't carry a /workspaces/:wsId/... segment.
  • ADDED — Run-context fields propagated into api/types.ts + Pipeline.family_id + Pipeline.version + RunPlan widened to include 'search' + WorkspaceRole widened to the 7-role lattice + ADMIN_WORKSPACE_ROLES exported set.

CHANGED — Web: design-system unification#

  • CHANGED — WorkspaceHome dashboard rebuild — replaced inline style={{...}} everywhere with design tokens. KPI strip uses card-tight + tabular-nums. Recent-runs row uses pill-success / pill-accent / pill-warn / pill-danger / pill-neutral (no more bespoke colored badges). Shortcuts panel rebuilt to the modern Linear/Vercel/GitHub pattern: icon + label + small description + chevron-on-hover, hover:bg-ink-50 rows, text-ink-400 → text-ink-700 icon transition. 10 lucide-style 16px SVG icons inline.
  • CHANGED — ProjectDetail full rewrite — clean breadcrumb, h-page header, tags as kbd, primary "New experiment" CTA. Drops <DataSourcesSection> inline so users upload CSVs in project context. Experiments list rendered as a tight rows-with-divider list (matches the data-sources list visually) instead of a stack of cards. Empty-state with beaker icon + first-experiment CTA.
  • CHANGED, BREAKING — ExperimentDetail full rewrite — replaced the permanent right-sidebar "New run" form with a header CTA button that opens a <Dialog>. New-run defaulting fixed: defaults to whatever data source the experiment's most recent run used (was hard-coded to sklearn:iris). Sklearn samples now only appear in the dropdown when zero CSVs are registered (true demo case) or when a prior run actually used one. Runs table resolves data_source_id UUIDs to friendly CSV names + uses pill primitives for status. Plan dropdown gains the search option.
  • CHANGED — RunDetail polish — full run UUID in breadcrumb (mono, no truncation). Status badge in header is now a pill-* class. Removed duplicate UUID under the title. Request snapshot redesigned as a 3-col dl with divide-y rows. Dropped the standalone <Leaderboard> section — TrialsCard is the single canonical view. "Event stream" → "Event log" with a one-line description; status pill instead of bullet-prefix text; empty-state copy is context-aware ("No events were emitted by this run." when terminal vs. "Waiting for events…" while connecting).
  • CHANGED — Schedules / Templates / Webhooks / Admin pages design unification. All use h-page / h-section, space-y-8, canonical table pattern (bg-white text-ink-500 thead, px-4 py-2 text-left font-medium, border-t hover:bg-ink-50 rows), pill-* for status, rounded-xl border-dashed border-ink-300 p-8 text-center empty states.
  • CHANGED — <AIAdvisorWidget> — was a circular candy-teal floating bubble; now a small rounded-full pill with the spark icon + "AI" label that flips to bg-ink-900 text-white when open.
  • CHANGED — <Leaderboard> — canonical white-thead pattern, model column bolded, sort chevron in accent.
  • CHANGED — <EventStream> — header uses h-section, status pill instead of ● open text, context-aware empty state.

TESTS#

  • TESTS — 265 → 330 engine tests passing (1 skip on [interpret] extra). New: test_session53_revive_legacy_verbs.py (11 tests) + test_session54_plot_model_dispatcher.py (19 tests). Sessions 47-52 plot tests now run as part of the engine suite (previously excluded).
  • TESTS — 115 backend tests passing. New: test_session22.py (9 tests — encryption + prediction logs + trials), test_session23.py (8 tests — model library + admin), test_session24.py (9 tests — schedules + templates + webhooks + pipeline versioning + search plan).
  • TESTS — 59 UI tests passing. Updated EventStream.test.tsx to match the new pill-based status indicator (replaced the ● closed regex match).

DEPS#

  • DEPS, BREAKING — Engine core deps: 12 → 10. Dropped imbalanced-learn + category-encoders.
  • DEPS — Engine core: capped ipython>=8.18,<9 to coexist with Colab's preinstalled google.colab extension which depends on the IPython 8.x coloransi.TermColors API.
  • DEPS — Engine core: relaxed scikit-learn>=1.7 → >=1.5 + imbalanced-learn>=0.13 → >=0.12 (before the dep was dropped) to avoid forcing upgrades on Colab/Kaggle preinstalled images.
  • DEPS — New engine optional extra [interpret] = ["shap>=0.46"] — folded into [full].
  • DEPS — Server core: added cryptography>=43 (was previously only in [mysql] extra) for Fernet at-rest secret encryption.
  • DEPS — Server core: added apscheduler>=3.10 for the in-process scheduler.