Projects

InferenceFlow

An end-to-end Go systems project. A reliable, observable, multi-tenant platform for orchestrating document processing and ML inference, built one working increment at a time.

PlannedStarted Nov 2026

What it is

Customers submit document references. InferenceFlow validates submissions, schedules work, calls an inference service, evaluates versioned policy rules, and exposes job status and results. The interesting part isn’t the happy path but keeping that workflow correct when requests repeat, dependencies slow down, workers crash and tenants compete for capacity.

The build plan lives in the repository: the README describes the system, and Roadmap.md sets out phases P0 to P15, milestones, exercises and a pattern atlas. The Go roadmap on this site is the companion: each stage teaches the patterns and practices one of those phases needs.

Status

Documentation only so far. The next increment is M0: a standard-library HTTP executable with a health check, timeouts and bounded shutdown.

Milestones

Milestone Deliverable
M0 Go module, cmd/api, /healthz, signal handling
M1 Job state machine, rules, application service, in-memory repository
M2 Submit, poll, cancel and result routes, with OpenAPI
M3 PostgreSQL, migrations, typed queries, transactions
M4 Bounded workers, inference stub and adapter
M5 Leases, fencing, attempt history, scheduled retries
M6 Transactional outbox, relay, JetStream, a separate worker
M7 Telemetry, runbook, authenticated tenancy, quotas
M8 Compatibility checks and a measured performance improvement
M9 Compose, CI, deployment and a fault campaign
M10 Optional focused technology comparisons

What done looks like

These are learning targets, not demonstrated results:

  • A batch of 1,000 small jobs runs without unbounded queue or goroutine growth.
  • Replaying the same tenant, key and request returns one logical job, and conflicting reuse is rejected.
  • A killed worker leaves its work recoverable through lease expiry and retries.
  • A resumed stale worker can’t overwrite a newer attempt’s result.
  • Transient inference failures recover, or reach a documented terminal state within retry budgets.
  • One tenant can’t read, cancel or fetch another tenant’s jobs.
  • Someone else can start the system, inject a fault, diagnose it and recover using only the repository docs.