The first generation of AI API demos treated inference like a fast database query: send a prompt, wait for a response, and display the result. Modern media systems do not behave that way. A video may wait for capacity, run for minutes, pass through several models, fail during delivery, or finish after the caller has already timed out.

That makes queue design part of the product, not infrastructure housekeeping. One careless retry can create two paid renders. One global concurrency setting can let thumbnails block customer videos. One preview endpoint change can turn a healthy queue into a backlog.

The reliable pattern is to accept uncertainty explicitly. Record intent before submission, give every operation a durable identity, model its states, and make retry policy depend on what is actually known.

Five limits that teams often confuse

A rate limit controls requests over time, such as calls per minute. A concurrency limit controls how many jobs can run at once. A quota limits total use over a longer period. Queue delay is the time before work begins. Generation latency is the time after it begins.

Those numbers answer different questions. Sending fewer requests per second may satisfy a rate limit but still exceed a two-job concurrency cap. Buying more monthly quota does not guarantee shorter waits. A fast model can still feel slow when a shared queue is saturated.

Track each dimension independently. At minimum, record acceptance time, provider start time when available, terminal time, model, output duration or resolution, attempt count, and status code. Client-observed latency should be split into local queue time, provider queue time, generation, validation, and delivery.

Do not turn a vendor marketing latency into a service-level objective. Runway says Runway Dev brings its models, recipes, workflows, and characters into one developer platform and advertises managed controls and 99.9 percent uptime. That is useful platform information, but its announcement does not promise one fixed generation latency or document one universal rate limit for every model. Capacity still has to be tested against the exact account and workload.

Luma makes the commercial distinction explicit: pay-as-you-go access has rate limits and no latency service-level agreement, while scale capacity is positioned around dedicated throughput and latency. The architectural lesson is broader than either vendor. Product promises must be based on the purchased tier and observed workload, not the existence of an API.

Put a durable queue before the provider

An application should write a local job record before calling a media API. A useful record includes:

The local queue is the source of orchestration truth. The provider task ID is evidence about one external attempt, not a replacement for the application record.

Use a state machine rather than a vague processing flag. Practical states include created, validated, queued, submitted, running, succeeded, failed, canceled, and needs_review. Keep delivery separate from generation: a render can succeed while its download, virus scan, upload, or final validation fails.

State transitions should be monotonic unless an explicit recovery action creates a new attempt. A delayed webhook for running must not overwrite a later succeeded state. Store provider events with timestamps and deduplicate them before applying transitions.

Idempotency begins in the client

Some APIs accept an idempotency key and some do not. A production client needs its own duplicate protection either way.

Create a stable request fingerprint from the authorized input asset versions, normalized parameters, model, and intended operation. Before submission, atomically claim that fingerprint for the job. If the caller repeats the same request, return the existing job instead of generating again—unless it explicitly asks for a new creative candidate.

This does not mean two prompts with the same text always share an output. Random seeds, model versions, ownership, retention policy, and creative intent matter. The duplicate boundary should be defined by the product, not by string equality.

The most dangerous moment is an uncertain timeout. The provider may have accepted the request even though the client never received its task ID. Mark the attempt unknown, reconcile through provider history or supportable lookup mechanisms, and require review when it cannot be resolved. Blind resubmission converts uncertainty into likely duplicate spend.

Polling is a protocol, not a loop

For an asynchronous API, poll only after preserving the external task ID. Use exponential backoff with jitter, respect Retry-After when supplied, and cap both the interval and total wait. A thousand workers polling on the same fixed schedule can create a synchronized traffic spike.

Poll frequency should reflect the job class. A short image edit can justify a tighter first interval than a long video generation. Slow down once the job is known to be running. Stop immediately on terminal states and on authorization errors that will not improve with time.

Webhooks reduce needless polling but do not remove reconciliation. Events may arrive twice, out of order, or not at all. Verify signatures where supported, acknowledge quickly, enqueue event processing, and run a periodic sweeper for jobs that have exceeded their expected update window.

Google’s Managed Agents preview illustrates why resumable state matters beyond media generation. Managed Agents can execute in isolated, ephemeral remote Linux environments, preserve files and state across follow-up interactions, and run as background work. The preview model and behavior can change, so an application should preserve its own interaction ID, request intent, and output checks rather than assuming that remote state is permanent.

Separate technical retries from creative retries

A network reset, a temporary capacity response, and an image with the wrong product color are not the same failure.

Technical retries repeat an operation because delivery or execution failed. They should be bounded, use backoff, and target only the failed stage. If generation succeeded but the output download failed, retry the download rather than buying another render.

Creative retries request a new candidate because the completed output did not meet acceptance criteria. They may change a prompt, seed, reference, model, or edit. Count them as new paid work. Give each job a candidate budget and require approval before exceeding it.

Policy failures are usually not retryable at all. If a user lacks rights to an input, a moderation rule blocks it, or a required consent record is absent, waiting 30 seconds changes nothing. Route the job to correction or human review.

A retry matrix should map error class to action: retry same stage, retry with reduced load, switch a permitted model, pause for capacity, request user correction, or fail permanently. Never use a model fallback that silently changes quality, licensing, safety behavior, or data geography.

Queue fairness and cost controls belong together

One queue for every workload invites starvation. Divide capacity by product class or use weighted scheduling. Interactive previews need short deadlines; batch localization can tolerate delay; high-resolution video may need a separate concurrency pool.

Reserve capacity for retries so a provider incident does not fill the entire queue with fresh submissions. Apply per-user and per-project limits before provider limits are reached. A circuit breaker should pause a failing route, but queued work must retain a clear status and estimated next action.

Attach cost to admission control. Estimate the maximum generation, retry, storage, and processing spend before accepting a batch. Stop a workflow when the budget is exhausted, even if more retries might eventually improve it. A dead-letter queue should preserve diagnostics and references without automatically replaying expensive work.

Useful operating metrics include queue age by percentile, accepted-to-start time, success rate by model, duplicate suppression, retry amplification, cost per approved asset, webhook delay, and jobs stuck in nonterminal states. High success with rising queue age is still an incident.

Multi-stage workflows need compensation, not wishful transactions

Runway Dev emphasizes recipes and workflows because media creation is increasingly composed: generate a shot, remove a background, add audio, create captions, and package several formats. No distributed transaction can roll all providers back.

Make each stage independently observable and restartable. Store an immutable input and output manifest. If stage four fails, resume from the validated stage-three asset. If a user cancels, attempt provider cancellation when supported, but label the outcome honestly—already running work may still complete and incur cost.

Delete temporary assets according to policy after the workflow is terminal. Compensation might revoke a delivery URL or remove a draft; it cannot unspend a completed render. That is why approval gates should precede expensive stages.

Applying the pattern to Medux tasks

Medux media operations are separate external tasks that Codex or Claude can call after MCP configuration. They do not share queues, credentials, or billing with Runway, Google, or Luma.

The Codex video-editing guide and Codex lip-sync guide both demonstrate task-oriented media work. In production, save the Medux task ID on the local job, then poll that task to a terminal result. Do not submit the transformation again merely because an agent turn ended or a polling request timed out.

If Medux does not expose a provider-side idempotency parameter for a particular operation, the orchestrator’s local job ID, request fingerprint, and atomic submission lock still prevent most duplicates. Describe this as client-side idempotent behavior, not as a native Medux guarantee.

Keep generation and finishing states distinct. A source video may come from another model, while Medux performs a separately authorized edit or lip-sync task. Validate URLs, duration, format, ownership, and consent before submission. After completion, verify the returned asset before marking the overall workflow successful.

Reliable AI media is not the absence of failures. It is the ability to know which work was accepted, what it cost, where it stopped, and whether trying again would create a duplicate.