Rebuilding Visdom AI Visualizations in Five Layers

Visdom is FOSSASIA's open source dashboard for watching machine learning experiments live, with over 10,000 GitHub stars. Over five months we rebuilt a large part of it in five stages, with 92 pull requests merged upstream

1. Storage abstraction layer: a DataStore interface so every environment save, load, delete and undo goes through one backend instead of raw JSON files.

2. Search layer: an experiment metadata store, an injection-safe query parser, search and field-by-field comparison.

3. Hyper-parameter layer: a dashboard with a run table, scatter matrix, parallel coordinates, a compare view and live updates.

4. Pytest: a suite of nearly 1500+ tests that uncovered real crashes and security bugs.

5. asyncio: disk I/O moved off the event loop and a new AsyncVisdom client, tackling an issue open since 2020 and reaching 2.1x the old client's throughput.

I'll show how each stage made the next one possible, the bugs we found along the way, and why small, reviewable PRs that never break the existing API are what got a mature project to accept this much change.

Speakers: Manik Khajuria

Track: [CLS] - OTS