AI Sound Design writes sound for a shot: foley, ambience or a music score, generated from the picture itself. You pick a model, audition what comes back playing over the shot, keep every attempt as a take to compare, and only then place the one you like on the timeline as an audio layer. It works on silent clips too — a shot with no sound of its own is exactly what it is for.
Open it
Select a video layer and open Smart AI ▸ AI Sound Design… in the timeline toolbar. The entry is greyed out until a video layer is selected.
The wizard works on the layer’s trimmed range. Its header card names the clip and shows the range it will score — for example “Scoring 00:04 → 00:12 · 8s · has original audio”, or “silent clip” when the shot has no sound.
The models come from the AI providers you have keys for (see Configuring API Keys). If none of them offers a video-to-audio model, the wizard says so.
Choose a model
Every video-to-audio and video-to-music model is in one list, cheapest first for this range. Each row shows:
- the model name and its provider;
- capability tags — SFX (sound effects and foley), Music, Speech (can keep the dialogue of the shot) and Remux (the model hands back the whole video with a new soundtrack);
- the price — per run, per second (/s) or per minute (/min), or “Provider priced” when the provider sets it.
Use All, Effects or Music above the list to narrow it; the count beside them reads “3 of 9”, for example. Below the list, the selected model’s description shows the price for this range (“≈ $0.05 for this range”), what it returns (an audio format, or “video → track extracted”) and whether it can keep the dialogue. The wizard remembers the model you used last.
A model whose provider published no inputs has nowhere to send the clip, so it is listed under the models as hidden, by name, rather than silently left out.
Describe the sound
What else the Setup step shows depends on the model — each one’s own inputs appear only on that model:
| Option | Shown for | What it does |
|---|---|---|
| Sound direction / Music direction (or Sound effects prompt) | Models that take a prompt | Describe the foley, ambience and on-screen effects, or the music. Marked “— required by this model” or “— optional”; when it is required, Generate sound stays disabled until you write something. |
| Background music prompt | Models that can also score a bed | Mood, instruments and pacing for the music. Optional — leave it empty for effects only. |
| Mute original audio | Every model (on by default) | Silences the clip’s own sound when the generated track lands. |
| Dialogue | Models that handle speech | Score only, or Keep & duck to isolate the speech in the shot and keep it over the score. |
| ASMR mode | Models that offer it | Close-mic, high-detail texture instead of room sound. |
| Output format | Models that offer a choice | The file format of the returned audio. |
A Remux model returns the shot with its soundtrack already replaced; the wizard keeps the new track and places that — your video layer is never overwritten.
Only the range is cut on your machine and uploaded to the model’s provider; the returned audio is cached in the board’s folder. The footer sums up the run — length, model, price per run, and whether the original will be muted. Click Generate sound.
While it generates
A progress screen shows the phases — Cutting the range, Generating with the model, Extracting the track (Remux models only) and Caching and decoding — with the elapsed time and a log. Cancel run goes back to Setup. Closing the window abandons the run, but the provider still charges for it.
Audition over the picture
When the take is ready, the shot plays with the generated track over it, so you judge the sound against the picture:
- Preview / Stop, or the transport’s play button and scrub bar. You can also drag along the waveform to scrub.
- Two lanes show the real waveforms: Generated (sound design or music score) and Original clip audio — marked Muted while the original is muted, or Silent when the range has no sound of its own.
The Mix settings on the right shape what you hear and what gets placed:
| Setting | What it does | Range / Default |
|---|---|---|
| Generated | Level of the generated track; it becomes the placed layer’s Volume. | 0–100%, default 100% |
| Original | How loud the clip’s own sound plays under the preview when it isn’t muted. | 0–100%, default 0% |
| Fades | Fade in and fade out applied to the placed layer. | None, 0.3s, 1.0s; default 0.3s |
| Mute original | Mutes the video layer when you place the sound. | on by default |
Reset mix puts the levels and mute back to their defaults. Nothing is written to the timeline until you place a take.
Takes and A/B
Every run keeps its take. The Takes list shows each one — “Take 2 · model name”, with its tags and prompt — and clicking a take auditions it. Generate again returns to Setup with your settings kept, and the next run lands as another take, so trying a second direction is an A/B comparison rather than a replacement. Each extra take costs one provider call.
The takes are saved on the layer. Reopen AI Sound Design on the same layer and a note offers them back (“This layer has a sound run from … — 2 takes”); click Open takes to go straight to the preview. Generating again adds to them.
Place it on the timeline
Under Place as, choose where the selected take goes:
- New Audio channel (default) — a new channel called “AI Sound” right above the video, with the sound aligned to the start of the clip. Trim and fade it like any audio layer.
- An existing Audio channel — the card is named after the channel. It is offered only when an Audio channel is free across the whole range; otherwise it reads “No audio channel has room here.”
- Media library only — keep the take in the board’s media without touching the timeline.
Click Place on timeline (or Save to library). The placed layer carries the Generated level and the fades from the mix, and the video layer is muted if Mute original was on. Take files are kept in the board’s aisound media folder.
Where to go next
- Audio Tracks — volume, fades and editing the placed sound
- Audio, Speech and Music Generation — the same video-to-audio models in the generation panel
- AI Beats — cut the video to the music
- Sound, Music and Assets Libraries — ready-made sound effects and music
Leave a Reply