Testing only on a developer machine, which hides how the same model behaves on a mid-range phone with a cold cache.
AI and on-device processing
On-device inference
Also called local inference, client-side ai or edge inference. Here is what it means, when it changes what you export, and what people get wrong.
AI and on-device processingRuns on your deviceUpdated 10 August 2026
The short answer
On-device inference runs a machine learning model on the hardware in front of you, your laptop, phone or browser tab, rather than sending data to a server. The image never leaves the device and there is no upload wait, but the model itself has to be downloaded first.
Updated
The short version
On-device inference at a glance
| Quick fact | Detail |
|---|---|
| Runs on | The user's own CPU or GPU |
| Browser GPU path | WebGPU |
| Browser CPU path | WebAssembly |
| Main cost | A one-time model download |
| Main benefit | Images are never uploaded |
| Weakest point | Low-memory and older devices |
Slide the table sideways to see every column.
What actually runs the model in a browser tab?
Three pieces have to line up. First the weights, the trained parameters, usually shipped in an interchange format such as ONNX. Then a runtime that reads the model graph and executes its operators. Then an execution backend that does the arithmetic.
On the web there are two realistic backends. WebGPU hands the work to the graphics hardware, where the large parallel matrix operations a neural network is made of run many times faster. WebAssembly runs the same operators on the CPU at close to native speed, which is slower but works essentially everywhere. A well-built tool detects WebGPU, uses it when present, and falls back to WebAssembly when it is not.
What do you genuinely gain?
- The photo stays put. Nothing is uploaded, so there is no copy of your image on someone else's disk and no retention policy to read.
- No round trip. Once the model is loaded, processing starts immediately instead of queueing behind an upload, a server queue and a download.
- No per-image cost to the operator. Server GPU time is the reason many hosted tools meter usage tightly; local execution moves that cost to hardware you already own.
- It keeps working offline. After the first load, a cached model does not need the network.
Photo tools built this way, Photomop's on-device studio among them, trade a one-time model download for images that are processed where they already are.
What do you give up?
The honest costs are real. The first run is slow, because a model measured in tens of megabytes has to arrive before anything happens, and on a poor connection that is the whole experience. Performance then varies enormously: the same page can be near-instant on a recent laptop and sluggish on an older phone, and you cannot fix that from your side.
Memory is the hard ceiling. A browser tab has far less headroom than a server, and a large photo expands considerably once decoded and converted to model input, so very large images and very large models are the two things most likely to fail. Sustained processing also drains battery and can throttle a phone thermally.
How do you keep a local model small enough to ship?
The engineering is mostly about budget. Quantisation cuts precision from 32-bit floats to 16-bit, and sometimes to 8-bit integers, which shrinks the download and the memory footprint proportionally. Architecture choice matters more than compression tricks: a compact network designed for edge devices beats a large one squeezed down.
Beyond that, models are cached after first load so the cost is paid once, large images are processed in tiles rather than in one allocation, and inference is moved to a worker so the interface stays responsive. The aim is a first-use download you only notice once and never again.
What people get wrong about on-device inference
Each one is a real failure mode, not a style preference.
Loading the full-precision model when a 16-bit version would halve the download with no visible quality change.
Running inference on the main thread, which freezes the interface and makes a two-second job feel broken.
Assuming local processing means nothing is sent anywhere, without checking what analytics and error reporting still transmit.
Questions people ask
On-device inference, answered
The follow-up questions people search for once they have the definition.
Is on-device processing actually more private?
For the image itself, yes: if the pixels are never transmitted, there is no server copy to be stored, logged or used for training. It says nothing about the rest of the page, so telemetry, crash reports and analytics can still leave the device unless they are also disabled.
Why is the first run slow and the second one fast?
The first run downloads the model weights and compiles the compute pipelines. Both results are cached, so later runs skip straight to processing. If the tab is reloaded after the cache is cleared, or the model is updated, you pay that startup cost once more.
Does it work on a phone?
Usually, for cut-out sized models. Modern phones have capable GPUs and browser support for local inference, but they have far less memory headroom than a laptop, so very large photos are the usual failure point. Reducing input resolution is the standard fix.
Is local always better than a server?
No. Local wins on privacy, latency and cost for small and medium models. Servers still win when a model is too large to download, when you need identical results across every device, or when a job needs sustained compute that would flatten a laptop battery.
Same cluster
More from the glossary
Neighbouring entries, so every term in the set is one hop from every other.
Nearest-neighbour interpolation
Nearest-neighbour interpolation sets each new pixel to the value of the closest pixel in the original, with no blending at all.
Orientation tag
The orientation tag is an EXIF value from 1 to 8 that tells software how to rotate or flip a photo for display.
Native resolution
Native resolution is the pixel dimensions an image was actually captured or rendered at, before anything resampled it.
From the studio
Tools that do this
Each one runs on your own device, so the photo stays where it is.
Sources
Where the facts on this page come from. Every link opens the specification, standard or documentation the claim was read at.
Keep reading
Elsewhere on Photomop
Platform image rules
Image size, file type and background rules for 22 selling platforms, each one sourced.
Read itConvert image
JPG, PNG, WebP and AVIF, with an honest answer about HEIC and alpha.
Read itAll the tools
The whole toolbox in three families, with what each one needs before it can run.
Read itOn-device privacy
The pipeline in five steps, what leaves your device, and how to check it yourself.
Read it
Now do it to a photo
Photomop is a photo studio that runs on your own device. Resize, crop, compress, convert, batch edit, remove a background and change one all work in the browser tab you are reading this in, at full resolution and with no watermark. The on-device page shows you how to check that the photo stays put.