A live brush, not a prompt queue
potocolom turns an image model into a drawing partner. You sketch on a canvas and a diffusion model re-renders your strokes as a finished image while you draw, several times per second, not once after a button press. The prompt sets the scene; your strokes decide the composition.
Around that loop sits a full studio: classic prompt-to-image generation with per-model controls, one-click upscaling, a history of everything you make, laid out as a canvas of how each image was derived, and a star for the keepers. This page explains the machinery underneath, in the order a stroke meets it.
The map below follows one stroke through the system. Your canvas sends a frame to the platform, the platform hands it to a GPU slot running a model described by its manifest, and the generated frame comes back the same way. Finished images take a separate road through storage. Each stop carries the number of the section that explains it: the anatomy of the platform, the realtime loop, GPU scheduling, models, images and privacy, credits, and what happens when something fails.
Not every part is running yet. The self-hosted platform works today on your own GPU. Parts of the hosted cloud, such as an admission queue shared across API servers, the billing service and relaying across several API servers, are designed and specified but not yet built. Where a section describes one of those, it says so.
The anatomy of the platform
Three moving parts. The studio runs in your browser. The platform API is a stateless relay and scheduler: it owns no GPU; it routes realtime frames, dispatches work and keeps the authoritative record of every job and asset in PostgreSQL. GPU workers hold the models and do the actual rendering.
Workers always dial out to the platform over one persistent WebSocket. Because the connection starts on the worker's side, a GPU machine needs no inbound ports, no public address and no firewall exceptions. Any machine that can open a connection can serve, from a data-center card to the desktop under your desk.
On connect, a worker announces its models in a manifest: capabilities, parameter schema, VRAM needs. The platform validates every request against that schema, picks a worker that serves the model, and streams progress back live. Finished images stay off the realtime channel: workers upload them to a one-time upload target, and your browser loads them through a link that expires in minutes when storage is S3, or through the API's owner-checked file route on a local disk. With S3 storage the API reads each finished image once to verify it, but never serves the bytes itself. If the platform restarts, it rebuilds its queue from the database and carries on.
Two WebSockets carry everything live. The browser's carries canvas frames up and generated frames down as binary WebP, each behind a small header: a one-byte type and the 16-byte session id. The platform checks that every frame belongs to the session it names before relaying it, because the browser is untrusted. The worker's socket opens with its hello and manifest, then carries session control, canvas and generated frames, jobs and progress in both directions.
PostgreSQL holds the record of every job and asset, and nothing else is trusted to. After a restart, a job that was running is retried once or marked failed, and queued jobs are pushed back from the database. A worker that falls silent for 90 seconds is closed: its jobs go back to the queue, and its drawing sessions move to another worker with room.
The realtime loop
While you draw, the canvas is snapshotted and sent as a compact binary frame (a small header plus WebP pixels) on the same WebSocket that brings results back. A distilled diffusion model re-renders it in a handful of denoising steps at 512 px, and generated frames stream back at 2 to 4 per second.
The loop never queues stale input: if a new stroke arrives while the GPU is busy, it replaces the pending frame rather than lining up behind it. Latest input wins; superseded frames are simply dropped. You always see the newest state of your drawing, never a replay of where it was.
Sessions are admitted only while a worker's measured frame budget has room, and are never oversubscribed. If the machine under your session vanishes and another has room, the platform reassigns you, the canvas is re-sent, and drawing continues. Idle release runs today: after about a minute without input your slot is freed, the canvas stays in the browser, and the next stroke takes a slot again; if none is free, the session waits in the admission queue and resumes when one frees. One account holds at most two sessions at a time.
Opening a session is one decision. The platform checks your prompt and parameters against the model's schema, then looks for room on a worker that holds a realtime model fully in memory. If there is one, the worker readies the session, the platform waits at most ten seconds for its answer, and the loop starts. If no worker has room, the session waits in the admission queue and sees its place in line. The hosted service is designed to queue it instead, with a live position, and to treat that queue as the signal to bring another GPU online.
A session is live while you draw. If its worker disappears it is interrupted, then resumed on another worker with room, and the studio re-sends the whole current canvas, so nothing is lost; with no other worker free, as on a one-GPU install, it waits in the queue until one is. A session is idle once its slot is released after about a minute without input, and queued while it waits for a slot, with its place in the queue shown; both run today. In the hosted service, credits are designed to meter only while a session is active, reserved about 60 GPU-seconds at a time, so waiting and idle time cost nothing.
GPU scheduling
GPU seconds are the scarce resource, so scheduling is specified rather than improvised. Realtime sessions come first and are never oversubscribed: the platform gives a worker a new session only while its measured frame times still fit the 2 fps bar, and a full pool puts a new session in a queue, showing its place in line. In the specified design, queued generations fill whatever capacity is idle and pause between denoising steps when a drawing session needs the slot; nothing ever preempts an active session. That preemption is designed, not built yet: today jobs and sessions simply share the worker.
How many sessions one GPU can hold is measured, not guessed: at warmup a worker benchmarks its models and advertises the concurrency it can sustain while keeping frame pacing above the bar.
Model placement is designed to follow demand. A hot set of fast models would stay pinned in VRAM for instant sessions, and everything else would load on demand, with a visible loading state of about a minute the first time. When the pool is genuinely full, new sessions wait in an admission queue with a live position, which runs today; in the hosted design that queue is also the signal that brings more GPUs online. This fleet-level placement is not built yet; what runs today is the memory management inside each worker, described below.
The specified scheduler is one loop, run by a single leader among the API replicas, in a fixed order. First it reconciles: a worker whose heartbeat has lapsed is forgotten, its jobs requeue once and its sessions move to another machine. Then it releases sessions idle for 60 seconds, so metering stops. Then it admits waiting sessions into free slots. Last, it fills the remaining capacity with queued jobs. Today one in-process loop dispatches queued jobs, a sweep releases idle sessions, and an in-process admission queue lets waiting sessions into slots as they free; the leader lease is still designed.
Inside one worker, memory management is real today. The worker reads free VRAM live and keeps several models warm; before loading another it evicts the least recently used one rather than push the new model onto a slower offload rung. Warm models still take turns on one GPU lock, so keeping them resident saves switching time, not compute. Only a fully resident model may serve drawing sessions, and at warmup the worker times its frames and advertises the most sessions whose p95 frame time still meets the 2 fps floor.
Models as manifests
A model, to the platform, is a JSON manifest: what it can do, how much VRAM it needs, and a parameters block that is standard JSON Schema. That schema is the single source of truth: the platform validates every request against it and the studio sizes its controls from it, so a model that uses the standard parameters becomes usable everywhere, with the right ranges, without touching a line of frontend code.
Manifests also carry the recipe: distilled few-step models (the Turbo and LCM class) for the drawing loop, Lightning-distilled models for fast queued stills, heavier samplers for quality renders, optional scheduler overrides and fused acceleration LoRAs. On a single consumer 16 GB card, once warm, the fast class renders a 512 px image in under three tenths of a second, the Lightning class takes a few seconds at 1024 px, and the default quality model, SDXL, about twelve seconds. The numbers are not marketing: they come from a benchmark harness anyone can run, and the results are published on the live benchmark page.
Models ship only after their licenses are vetted. Each manifest can record its license, its attribution text and a commercial revenue cap, and the studio shows a required attribution next to the model.
The manifest is split by who needs it. The recipe (weight source, VAE, scheduler and LoRA) is read only by the worker, which builds the pipeline from it and strips it before announcing the manifest, so weight locations never cross the wire. The platform receives the rest: identity, capabilities as the routing key, the VRAM requirement, license fields and the parameter schema. The API checks every request against that schema and rejects an invalid one before any GPU time is spent; the studio reads the ranges, defaults and size options of its fixed controls from the same schema.
The VRAM figure in a manifest is the model's own need for full residency, not a promise about any card. The worker compares it with the memory it actually has free and picks a rung: fully resident, offloaded to system memory in whole components, or streamed layer by layer from system memory or disk. Offloading lets a small card run a large model slowly, but it drops the model from drawing, because only full residency holds the frame bar.
Images, privacy and safety
Transport and storage have different jobs, so they use different formats: canvas and generated frames travel as WebP, chosen for quality per byte at interactive speeds, while the stored master of every generation is lossless PNG, the archival copy of your work, with WebP thumbnails for fast browsing. Every generation stores its parameters next to the image, including the seed when you set one, so a seeded result can be reproduced, and provenance is recorded in the schema so an image can tell where it came from.
Assets are private by default; sharing is an explicit act. In the cloud they are delivered through signed URLs that expire after five minutes, and a self-hosted install serves them through the API's own file route. Signed-in state is an opaque server-side token in an HttpOnly cookie, invisible to page scripts, listable and instantly revocable. Nothing you create is used to train models.
Follow one generation through the figure. The prompt and your canvas go to a GPU worker, and while you draw, frames come back as WebP. The finished result is written once as a lossless PNG master with a WebP thumbnail, next to a record of its parameters (with the seed, when you set one) and parent image, under a storage key built from your account and an unguessable job id. It reaches your browser only through the delivery route above, and anyone else only if you create a share link.
Content safety for the hosted service is designed, not built yet. The design screens every prompt through layers (normalization, then rules, then a classifier) on every path a prompt can take, including live drawing, and enforces on strikes without retaining prompt text. No screening code runs today, in either deployment.
The platform measures itself without watching you: no tracking cookies (the cookies it sets serve sign-in and remember the studio sidebar), no trackers, not a line of client-side analytics. Usage statistics are rows the server already knows (which model ran, for how long, and a category label meant to come from a CLIP pass over the output image), never the prompt, the image, or your address. The label field is wired, but the CLIP pass is not built yet, so every row reads other today. Self-hosted installs send one anonymous daily aggregate that can be previewed locally and disabled with a single switch.
Credits that cannot double-charge
The hosted service is designed to meter usage in credits, and none of this runs yet. A credit corresponds to GPU time actually spent. Before any GPU second is spent, the platform would reserve an estimate; when the work finishes, it would commit the actual amount, and failed work would refund automatically. Drawing sessions would meter in small chunks only while active, so an idle canvas costs nothing.
Every money path is designed to tolerate replay. Payment webhooks deduplicate on event ids, the credit ledger is append-only with unique source keys, and reservations are idempotent with a timeout that returns stranded credits on its own. Retries and redeliveries are no-ops by construction.
Every charge is specified to follow one contract: a reservation with three possible endings. The API picks the reservation id and sends it with an estimate and a time limit, so a call that times out can be retried with the same id without booking twice. The reservation then ends exactly once: committed with the actual GPU time, refunded when the work fails, or expired when the time limit passes with neither, which returns credits a crash would otherwise strand. Repeating a step does nothing; a conflicting step is refused.
A drawing session is designed to use the same contract in chunks. About 60 GPU-seconds are reserved at admission and extended chunk by chunk while you draw, and the actual use is committed when the session goes idle or closes. If an extension fails, the session ends gracefully with an out-of-credits message, so the most a session can overdraw is one chunk.
Payment pages are to be hosted by the payment provider, so card data never touches platform servers. If billing is ever unreachable, reserving is designed to fail closed, so no GPU time is granted on credit, while finished work settles through a retry queue until it lands exactly once.
When billing cannot be reached, the priorities are fixed: never hand out free GPU time, then never lose a charge. Reserving fails with a billing-unavailable error. Commits and refunds wait in an outbox table in the platform's own database and retry until billing acknowledges them, and the ledger's unique keys make a repeated delivery harmless. In the open code this contract is specified but not wired yet: there is no quota client or QUOTA_SERVICE_URL setting today, and GPU time is recorded for analytics only.
Built to fail safely
GPU machines vanish; the design treats that as normal. A worker dying mid-generation gets the job requeued exactly once before it fails visibly, so an input that crashes workers cannot burn GPU time in a loop. A worker dying mid-session triggers a transparent resume when another machine has room: it takes over and your canvas is re-sent.
The platform restarting rebuilds its queue from the database. The database being down degrades the service instead of stopping it: drawing sessions already open keep running, and on an install without accounts new ones still open, while history waits. Retries are bounded, and the hosted service is designed so failed work refunds and no failure mode produces a surprise bill.
Compute failures, a worker lost mid-job or mid-session, are handled in the open code today: the lost job is requeued exactly once, and a second failure leaves the job marked failed in your history. The retry button and the refund that goes with it are designed for the hosted service. Money failures, a renewal that fails or a billing service that cannot be reached, are designed never to claw back credit already granted and never to grant GPU time on credit. Infrastructure failures hit a limit, not a bill.
Those limits are recorded decisions for the hosted service, not yet running. The fleet autoscaler has a hard machine ceiling and a monthly budget, so demand past them waits in the admission queue behind a high-demand banner instead of renting more. The database runs in one availability zone with point-in-time recovery, accepting up to five minutes of lost writes and about an hour of manual recovery if that zone fails. The money rows depend on the billing service, which is also designed and not built yet.
Developed in the open
The product is AGPL 3.0 and complete on its own: the code that serves you, including this page, is the code in the repository, and you can run the entire platform on your own GPU.
The design is public too: the architecture documents, the wire protocol, the benchmark methodology, and a decision ledger that records not only every choice but the alternatives that were rejected and why.
Running it yourself is the same software, not a lite edition. A release produces one set of container images and one static studio build. Self-hosters install a release with one script that pulls the images from GHCR, and the cloud is designed to mirror the same images by digest. Application code never asks which deployment it is in. It reads specific settings, and the studio adapts to what GET /api/v1/config returns.
Where the two deployments must differ, a seam owns the difference, with a self-hosted and a cloud implementation. Storage is the local disk or S3, and auth is none or accounts; both switch with one setting today. Job queues live inside one process, frames pass between sockets in one API process, and there is no quota check. Their cloud versions, Redis queues, a Redis pub/sub relay between API replicas and a billing service, are designed and not built yet.
Billing, the GPU fleet autoscaler and the cloud infrastructure code belong to a separate private repository, and the billing service there is not built yet. That code must never import a line of the open code: it is written against a documented HTTP contract, versioned under /v1 like the worker protocol. The boundary is a process boundary, so the open code stays AGPL and complete on its own, and anyone can read the contract the private side must honor.
