Many inputs¶
A handful: call in parallel¶
Runs are independent, so the simplest way to run a model over a few dozen inputs is to make the calls at the same time.
const photos = await Promise.all(paths.map((p) => upload(p)));
const answers = await Promise.all(photos.map((image) => run("marigold-depth", { image })));
answers = f.run_batch(model="marigold-depth", inputs=[{"image": p} for p in paths])
One call, run in parallel on our side, answers in order. An item that fails comes back as the error saying why; the others are unaffected. Up to 256 inputs.
Add max_cost_usd to cap what a run may spend, and price one input first with
POST /v1/quote - the total is that times the number of inputs.
Thousands: run a recipe over a dataset¶
For a folder of images or every frame of a long video, save the pipeline as a recipe and run it over the dataset. It runs a small trial first and shows the results with a projected total, and goes on only when you confirm. It keeps going if your process stops, retries the items that failed, and gives you the results as JSON, JSONL or CSV.
Language models¶
Many chat requests as one job - the OpenAI batch shape - go to
POST /v1/batches.