RotoScope and Magic Mask

RotoScope cuts a moving subject out of a video layer. You point at the subject, the editor traces its outline and tracks it through the shot, and the result becomes a layer mask — so the person stands in front of a new background, a title slides behind them, or a grade touches only them. On an image layer the same tools are called Magic Mask and trace something in the picture in one go.

Nothing is written to the layer while you work. The traced outline plays over the preview as a mint preview, and Copy to mask is the only action that turns it into a mask — so a bad trace costs a click, not an undo.

Where it lives

Select a video or image layer, open the Mask tab on the inspector’s rail and find the LAYER MASK section. Below the mask list is a block titled RotoScope on a video layer, or Magic Mask on an image layer, with its own On/Off switch. Switch it on to show the tools.

  • The switch is remembered per layer: turning it on for one clip leaves every other clip off.
  • You do not need to switch the Layer Mask section on first. The block is there with the section off, and copying a result switches the mask on for you.
  • Switching the block off hides the tools and drops a preview you have not copied yet. A mask you have already copied stays.

The work happens in three numbered steps: Subject, Region, then Trace.

Step 1 — Subject

Pick how the subject is found. The cog on the Subject row holds the engine’s settings (see Roto options below).

SubjectEngineBest forNeeds a region
PersonSelfie AIA person filling much of the frame, and long tracks — small and very fast.No
ForegroundBiRefNetWhatever the main subject is, person or object, with much finer edges than Person. Slower.No
ObjectMagic TouchPicking one subject out of several — the region says which.Yes
ColourColour keyA subject that differs from its surroundings by colour. No AI model: the colour is sampled from inside the region.Yes
Hand rotoManual keysRotoscoping by hand: the region you draw is the mask, keyed frame by frame. Video only.Yes

Person and Foreground can run with no region at all. Object and Colour use the region to seed the model and to fence it in, so background between a subject’s limbs stays out.

Step 2 — Region

Click Draw and outline the subject on the canvas with a few points — the region is drawn like a pen path:

  1. Click to place points around the subject; the outline curves smoothly through them.
  2. Drag out of a click to pull a tangent, or Alt-click for a corner. Placed points stay live — drag one to move it, click one to pull its tangent knobs.
  3. Backspace steps back a point and ⌘Z / ⌘⇧Z step the drawing back and forward. Escape drops the drawing; a second Escape leaves the tool.
  4. Finish by clicking the first point, pressing Enter, or clicking Finish in the panel. It needs three points or more.

A single point instead of an outline — place one and press Enter or Finish — means “the subject is here” and becomes a circle around that spot. That is often all Object needs.

As soon as the region closes, the current frame traces, so you see the contour straight away. The panel then reads Region set, with two buttons:

  • Edit region — drag the region’s anchors, click one for its tangents. Each release traces the frame again, so a region that missed an arm is fixed with one drag. Click Done when it is right.
  • Redraw — start the region again.

While you drag a region point, the round magnifier from the mask editor shows the spot at 2.5×; on a tablet the point rides above your fingertip.

Step 3 — Trace

Choose the Mask output first — it decides what Copy to mask writes (see Detail or Outline below). Then:

  • Trace frame traces only the frame under the playhead.
  • Track forward traces from the playhead to the end of the clip, following the subject frame by frame.

While a track runs, the panel shows the progress and a strip coloured by frame: tracked, bridged (the tracker filled a short gap) and occluded (something covered the subject). Stop & keep ends the run and keeps what was traced so far.

Analysis reads the original footage rather than the playback proxy, so a fast-moving edge is traced from the sharp frames, not from a re-encode’s smear.

Review the preview

The traced outline plays over the preview in mint while you scrub — check it at the frames that matter. The review card shows the same frame strip with a tick for each key, and the number of frames, anchors (points on the outline) and keys. If the run ended early the card says why, and keeps the keys up to the last good frame.

  • Copy to mask (Copy matte to mask for a Detail result) writes the result onto the layer as one undo step, as a mask named Roto. Running it again replaces the Roto mask rather than adding a second one.
  • Re-track (or Re-trace for a single frame) runs the analysis again with the current settings.
  • Discard throws the preview away.

Once copied, Roto is an ordinary layer mask in the list above: set its Mode, Feather or Opacity, or switch Invert on to keep everything except the subject — to remove a person rather than isolate them.

Detail or Outline

Mask outputWhat Copy to mask writesKeepsGives up
Detail (default)The model’s own matte for every frame, baked into a small video file saved to a mattes folder on your board.Hair, holes, the gap between an arm and a body, soft and motion-blurred edges.Editing with the Pen; Expand and Grow.
OutlineAn editable outline, animated with a path key at every key of the track.Everything a mask can do: the Pen, Smooth, Expand, Grow.Fine detail the outline cannot follow.

A Detail mask still has Mode, Invert, Feather and Opacity, and it carries the traced outline underneath as a fallback — if the matte file ever goes missing, the mask falls back to that outline instead of vanishing. If the matte cannot be saved, Copy writes the outline instead and says so.

Continuing or correcting a track always produces an outline, and a copied Detail mask cannot be continued or corrected — choose Outline when you expect to repair a track after copying it. Hand roto always writes an outline.

Continue after an occlusion

A track stops when the subject is lost — behind a passer-by, out of frame, split in two — or when you press Stop & keep. It does not have to start over:

  • Before copying, the review card offers Continue from the playhead. Scrub to where the subject is clear again (redraw the region there if it has moved a long way) and continue: the new stretch is added to the held preview, and one Copy to mask writes the whole track.
  • After copying an outline, the panel offers Continue tracking from the last key, which resumes the Roto mask from its last key. Move the playhead past the gap first and the button reads Continue here from the last key.

Repair a bad stretch

When a stretch of the track goes wrong, fix one frame and let the tracker redo everything after it:

  1. Scrub to the first bad frame and click Correct frame — on the review card before copying, or on the Roto mask row once an outline is copied.
  2. The outline turns amber on the canvas. Drag it, or drag its points, onto the subject. Nothing is written yet, and the mask section’s editing button reads Cancel correction while you work.
  3. Click Re-track from this frame. The frames before the correction stay exactly as they were.
  4. Review the result:
    • on a copied mask, Apply correction replaces the correction frame and every key after it in one undo step, Adjust correction goes back to the amber outline, and Discard leaves the mask as it was;
    • on a preview, Copy repaired track writes the whole repaired track, and Discard returns to the original preview.

Hand roto

Hand roto is manual rotoscoping with no AI model — for a subject the models cannot see, or a shape you want to control exactly. The region you draw is the mask:

  1. At the first frame, draw the region. It previews as a mask key straight away (Preview mask key runs it again); click Set mask key to write it.
  2. Move the playhead to where the subject has moved or changed shape, click Redraw, outline it again, preview and set another key.
  3. Add keys wherever the subject changes. The mask moves by morphing between them.

To fix a hand-rotoscoped frame, use Correct frame, reshape the amber outline and click Set correction key.

Magic Mask on image layers

On an image layer the block is called Magic Mask: the one-frame half of RotoScope. The subjects are Person, Foreground, Object and Colour — Hand roto is left out, because on a still the Pen already does that job.

  1. Switch Magic Mask on.
  2. Click Draw and outline what you want masked with a few loose points, or place a single point on it and press Finish. It traces straight away. With Person or Foreground you can skip the region and click Magic trace.
  3. Check the traced outline and click Copy to mask. It becomes a mask named Magic Mask spanning the whole image, in one undo step.

Pick Outline output to fine-tune the result afterwards with the Pen or Select tool, or Detail to keep soft edges and fine hair. Draw a new region and trace again to replace it.

Roto options

The cog on the Subject row holds the engine’s dials. They apply to the next trace or track, persist as your preference, and show a dot when changed; Reset to defaults puts them back.

OptionWhat it doesDefault
Analysis detailThe size frames are analysed at — 384 px (fastest), 512 px or 768 px (finest).512 px
Key reductionHow closely the keys must reproduce the traced outline — looser means fewer keys: Precise (0.5 px), Balanced (1.0 px), Light (2.0 px), or Off to key every frame.Balanced
Max keysThe most keys one track may write (20–600).200
AnchorsPoints on the traced outline — Auto (by shape), or a fixed 32, 48, 64, 96 or 128.Auto
Region marginHow far the subject may move from one frame to the next and still count as the subject (0–80 px).24 px
Matte clean reachHow faint a part of the matte may be and still count, as long as it touches the solid core — lower keeps more of a soft edge (0.1–1).0.55
Matte smoothSoftening of the matte before it is traced (0–4 px).1.2 px
Colour toleranceIn Colour mode, how far from the sampled colour still counts as subject.0.26
Smoothing windowFrames averaged to remove jitter: Auto (by frame rate), Off, 5, 9, 13 or 21.Auto

Region margin and Matte clean reach are the first two to try when a matte struggles. The remaining dials — Anchor spacing, Min area, Template pull, Template blend, Occlusion δ, Phase search, Sub-anchor refine, Median-3, Pose decompose and, for Person, Person mask idx — tune the tracker itself and rarely need changing.

Models, downloads and PRO

Every mode runs on your own device — no footage is sent to an AI service and no credits are used. It works in the desktop app and in the web version.

SubjectWhat it loads
PersonA small selfie-segmentation model (about 250 KB) shipped with the app.
ObjectThe Magic Touch model, shipped with the app.
ForegroundBiRefNet, downloaded once on first use — about 94 MB, or 183 MB on a machine where it has to run on the CPU instead of the GPU. It is cached, so app updates do not download it again.
Colour, Hand rotoNo model.

Tracing a single frame is free: Trace frame, Magic trace on images and Hand roto keys. Tracking across frames — Track forward, Continue, Re-track and Re-track from this frame — is PRO; without a PRO licence the desktop app opens the licence window when you start one.

Where to go next

  • Layer Masks — everything you can do with the Roto mask once it is copied
  • Chroma Key — the faster route when the subject is in front of a coloured screen
  • Track Matte — use one layer’s shape to mask another
  • Motion Tracking — make a title or a blur follow a detail instead of cutting it out

Was this helpful?

0

Updated

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *