Treating the raw mask as the finished cut-out, which leaves jagged edges anywhere the subject has hair, fur or fine detail.
AI and on-device processing
Image segmentation
Also called semantic segmentation, instance segmentation or object segmentation. Here is what it means, when it changes what you export, and what people get wrong.
AI and on-device processingRuns on your deviceUpdated 10 August 2026
The short answer
Image segmentation is the task of labelling every pixel in a photo with what it belongs to, rather than drawing a box around it. Models output a mask: one value per pixel saying subject or background, or which of several objects that pixel is part of.
Updated
The short version
Image segmentation at a glance
| Quick fact | Detail |
|---|---|
| Output | A per-pixel mask, same size as the image |
| Main variants | Semantic, instance, panoptic, salient object |
| Raw model output | Confidence scores, then thresholded |
| Edge behaviour | Hard boundaries, no partial transparency |
| Usually paired with | Matting, for soft edges |
Slide the table sideways to see every column.
How does a segmentation model decide what each pixel is?
A segmentation model takes the whole image and returns a second image of the same size, where each pixel carries a score instead of a colour. Trained on enough labelled examples, it learns which textures, shapes and surrounding context belong to a person, a shoe or a sky.
That score map is then thresholded into a binary mask: above the cut is foreground, below it is background. The mask is what the rest of your pipeline actually consumes, whether that means cutting the subject out, restricting a filter to one region, or telling an object-removal tool which pixels to rebuild.
What is the difference between semantic, instance and panoptic?
The three common flavours answer different questions about the same photo.
| Type | What it labels | Two overlapping chairs |
|---|---|---|
| Semantic | Every pixel gets a class name | One merged chair region |
| Instance | Each countable object separately | Chair 1 and chair 2, background ignored |
| Panoptic | Classes and instances together | Chair 1, chair 2, plus floor and wall |
Cut-out work usually wants something simpler again: salient object segmentation, which only asks which pixels are the subject of the photograph, with no class names involved at all.
Why do segmentation edges look crunchy on hair?
A mask is a decision, and a decision has to be yes or no. Real edges are not. A single pixel along a strand of hair is part hair and part whatever sits behind it, so forcing it to one side gives you either a chopped-off strand or a rim of the old background welded to the subject.
This is why segmentation is step one rather than the whole job. Locating the subject is segmentation. Working out how transparent each boundary pixel is, and what colour it would be without the background bleeding through, is matting. Cut-outs that survive close inspection run both stages.
What people get wrong about image segmentation
Each one is a real failure mode, not a style preference.
Thresholding the confidence map at the halfway point out of habit when a slightly different cut gives a cleaner boundary.
Feeding a heavily compressed JPEG to the model and blaming the mask for edges the compression had already destroyed.
Assuming a class-based model knows your product, when a salient object model handles unlabelled objects far better.
Questions people ask
Image segmentation, answered
The follow-up questions people search for once they have the definition.
Is segmentation the same as object detection?
No. Object detection draws a rectangle around an object and names it. Segmentation labels the individual pixels, so the shape you get back follows the object outline rather than a box. Detection is often run first to find candidates, then segmentation refines each one down to pixel level.
Why does the mask look fine on screen but bad when I zoom in?
Masks are usually predicted at a lower resolution than your photo and then scaled up. At normal viewing size the softening hides it, but at full zoom you see stair-stepping and lost detail. Running the model at a larger input size, or refining the boundary separately, removes most of it.
Can one model segment several objects at once?
Yes. Instance and panoptic models return a separate mask per object in a single pass, each with a label and a confidence score. Whether that helps depends on the job: for a straightforward subject cut-out, one foreground mask is faster and much easier to work with.
Does segmentation need a server?
Not necessarily. Cut-out sized segmentation models are small enough to run in a browser tab or on a phone, using the GPU through WebGPU or the CPU through WebAssembly. Server inference still makes sense for very large models, but it is no longer a requirement for this kind of task.
Same cluster
More from the glossary
Neighbouring entries, so every term in the set is one hop from every other.
Inpainting
Inpainting fills a selected region of a photo with content that matches its surroundings, so whatever was there looks as though it never existed.
Image noise
Image noise is the random speckle in a photo that does not correspond to anything in the scene.
Interpolation
Interpolation is how software works out pixel values at positions that did not exist before, which is what happens whenever you resize, rotate or warp an image.
From the studio
Tools that do this
Each one runs on your own device, so the photo stays where it is.
Keep reading
Elsewhere on Photomop
Platform image rules
Image size, file type and background rules for 22 selling platforms, each one sourced.
Read itConvert image
JPG, PNG, WebP and AVIF, with an honest answer about HEIC and alpha.
Read itAll the tools
The whole toolbox in three families, with what each one needs before it can run.
Read itOn-device privacy
The pipeline in five steps, what leaves your device, and how to check it yourself.
Read it
Now do it to a photo
Photomop is a photo studio that runs on your own device. Resize, crop, compress, convert, batch edit, remove a background and change one all work in the browser tab you are reading this in, at full resolution and with no watermark. The on-device page shows you how to check that the photo stays put.