Give Claude the ability to watch any video.

Claude Code (recommended — auto-updates via marketplace):

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

Codex, Cursor, Copilot, Gemini CLI, or any of 50+ Agent Skills hosts:

npx skills add bradautomates/claude-video -g

(-g installs globally for your user, available across all projects. Drop it to scope per-project.)

Zero config to start — yt-dlp and ffmpeg install on first run via brew on macOS (Linux/Windows print exact commands). Captions cover most public videos for free. Whisper API key is only needed when a video has no captions.

Claude can read a webpage, run a script, browse a repo. What it can't do, out of the box, is watch a video. You paste a YouTube link and it has to either guess from the title or pull a transcript that's missing 90% of what's on screen.

With Claude Video /watch you can paste a URL or a local path, ask a question, and Claude fetches captions first, downloads only what it needs, extracts frames (scene-aware, or fast keyframes at efficient detail), pulls a timestamped transcript (free captions when available, Whisper API as fallback), and Reads every frame as an image. By the time it answers, it has seen the video and heard the audio.

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?

What people actually use it for

Analyze someone else's content. /watch https://youtu.be/<viral-video> what hook did they open with? Claude looks at the first frames, reads the opening transcript, breaks down the structure. Same for ad creative, competitor launches, podcast intros, anything where the how matters as much as the what.

Diagnose a bug from a video. Someone sends you a screen recording of something broken. /watch bug-repro.mov what's going wrong? Claude watches the recording, finds the frame where the issue appears, describes what's on screen, often catches the cause without you ever opening the file.

Summarize a video. /watch https://youtu.be/<long-thing> summarize this does the obvious thing — pulls the structure, the key moments, what was actually said and shown. Faster than watching at 2x.

Cut the hype out of an update video. /watch https://youtu.be/<launch-video> what's actually new — skip the hype Strip a "game-changer" feature drop down to the few things that matter, so you get the substance without ten minutes of intro and overselling.

Turn a playlist into notes. /watch https://youtu.be/<video> summarize this to a note Run it across a series and file a per-video summary, so a channel or course becomes a searchable set of notes instead of hours you have to sit through.

How it works

1. You paste a video and a question. URL (anything yt-dlp supports — YouTube, Loom, TikTok, X, Instagram, plus a few hundred more) or a local path (.mp4, .mov, .mkv, .webm).
2. yt-dlp checks captions first. At transcript detail, captioned URLs return without downloading video. Otherwise, or when Whisper needs audio, it downloads only what the run needs.
3. ffmpeg extracts frames at the chosen detail. efficient decodes keyframes only (near-instant); balanced/token-burner prefer scene-change frames and fall back to the duration-aware uniform sampler when they under-produce. JPEGs are 512px wide by default and clamped to 1998px tall for Claude Read compatibility.
4. The transcript comes from one of two places. First try: yt-dlp pulls native captions (manual or auto-generated) from the source. Free, instant, accurate-ish. Fallback: extract a mono 16 kHz 64 kbps mp3 audio clip (~480 kB/min) and ship it to Whisper — Groq's whisper-large-v3 (preferred — cheaper and faster) or OpenAI's whisper-1.
5. Frames + transcript are handed to Claude. The script prints frame paths with t=MM:SS markers and the transcript with timestamps. Claude Reads each frame in parallel — JPEGs render directly as images in its context.
6. Claude answers grounded in what's actually on screen and in the audio. Not "based on the description" or "according to the title." It saw the frames. It heard the transcript. It answers the way someone who watched the video would.
7. Cleanup. The script prints a working directory at the end. If you're not asking follow-ups, Claude removes it.

Frame budget — why it matters

Token cost is dominated by frames. Every frame is an image; image tokens add up fast. The script's auto-fps logic exists so you don't blow your context budget on a sparse scan of a 30-minute video that would have been better answered by a focused 30-second window.

When the user names a moment ("around 2:30", "the last 30 seconds", "from 0:45 to 1:00"), pass --start / --end. Focused mode gets denser per-second budgets, capped at 2 fps. Far more useful than a sparse pass over the whole thing.

Frame deduplication

Frame selection — keyframes (efficient), scene-change detection (balanced/token-burner), or the uniform sampler it falls back to — can still surface near-identical frames: a screen recording that holds one slide for 90 seconds produces a dozen, each billed as a separate image. A dedup pass drops them before frames reach Claude. It runs by default on every frame mode (--no-dedup turns it off):

1. One ffmpeg call scales each extracted JPEG to a 16×16 grayscale thumbnail. Everything after is pure-stdlib Python — no image libraries.
2. For each frame, compute the mean absolute difference against the last frame that was kept (average per-pixel brightness change, 0–255 scale).
3. If that difference is at or below the threshold (2.0), the frame is a near-duplicate and is dropped. Otherwise it's kept and becomes the new reference.
4. The frame-budget cap applies after dedup, so the budget is spent on distinct frames.

Comparing against the last kept frame (not the previous one) catches slow fades that never trip a frame-to-frame threshold. The threshold is deliberately low and measures absolute brightness rather than structure, so a one-line code diff, a terminal scrolling a row, or two differently-colored flat slides all survive.

The Frames line reports what was collapsed, e.g. 6 selected from 14 candidates (… 8 near-duplicates dropped …). On always-moving footage nothing is dropped and you pay what you would have anyway.

Detail modes — measured

The --detail dial trades speed and token cost for visual fidelity.

Install

Claude Code

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

Update later with /plugin update watch@claude-video.

Codex, Cursor, Copilot, Gemini CLI, and 50+ other hosts

The Agent Skills CLI installs the skill into whatever agents it detects:

npx skills add bradautomates/claude-video -g

claude.ai (web)

1. Download watch.skill from the latest release.
2. Go to Settings → Capabilities → Skills.
3. Click + and drop the file in.

Enable "Code execution and file creation" under Capabilities first — the skill shells out to ffmpeg and yt-dlp, so it won't run without it.

Usage

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?
/watch https://www.tiktok.com/@user/video/123 summarize this
/watch ~/Movies/screen-recording.mp4 when does the UI break?
/watch https://vimeo.com/123 what tools does she mention?

Focused on a specific section — denser frame budget, lower token cost:

/watch https://youtu.be/abc --start 2:15 --end 2:45
/watch video.mp4 --start 50 --end 60

Develop

python3 -m pytest -q
bash skills/watch/scripts/build-skill.sh      # → dist/watch.skill

Open source

MIT license.

Built on yt-dlp, ffmpeg, and Claude's multimodal Read tool. Whisper transcription via Groq or OpenAI.

Built by Brad Bonanno.