Media formats

There is no user-facing extension whitelist for video or stills. Playback is decided by who can actually decode a real first frame.

How the decoder is chosen

Every file goes through one rule set, in order:

1. If ImageIO recognises it as an animated image, the animated-image decoder takes it.
2. Otherwise, if the extension is one of ten — mkv, webm, avi, wmv, asf, mxf, ts, m2ts, mpg, mpeg — the bundled FFmpeg decoder takes it.
3. Otherwise AVFoundation gets first refusal, and it only wins if opening and decoding a real frame at position 0 both succeed.
4. Otherwise it falls back to the bundled LGPL FFmpeg 9.0.2 decoder.

The governing comment says it plainly: "Public AVFoundation remains preferred for readable native video. Fallback is accepted only after a real frame is decoded, never just a container probe."

You can see which backend won, per source, reported as AVFoundation, FFmpeg · <codec>, ImageIO · GIF or ImageIO · APNG.

What the FFmpeg fallback will open

Only eleven demuxers are whitelisted, and only the file protocol — so no network or streaming sources:

Inputs must be absolute paths to regular files. Probe settings are bounded: 8 MiB probesize, 2 s analyze duration, 128 probe packets, 128 streams maximum.

Codecs compiled into that fallback

Video: dxv, hap, prores, h264, hevc, vp8, vp9, mpeg1video, mpeg2video, mpeg4, mjpeg, qtrle, dn×hd, ffv1, utvideo, huffyuv, rawvideo, wmv1, wmv2, wmv3, vc1, libdav1d (AV1).

Audio: aac, mp3, mp3float, vorbis, flac, opus and the PCM family.

Everything else is compiled out: no encoders, no muxers, no filters, no network, no hardware decoding, no VideoToolbox. The license is asserted to be LGPL 2.1 or later.

.mp4 and .m4v are not on the forced-FFmpeg extension list, so they attempt AVFoundation first and only reach FFmpeg if that fails. Which one handles your file depends on runtime probe outcomes.

Stills

Anything ImageIO can open. There is no extension, name or UTI whitelist — acceptance is delegated entirely to ImageIO.

• Orientation 5–8 swaps width and height, as you would expect.
• A still used in a clip gets a 8-second timebase by default.
• "Full size" still decoding caps at 8192 pixels per side.

Because there is no list in the code, the exhaustive set of still formats is not a fact of this codebase — it is a fact of your macOS build's ImageIO.

Animated images

GIF and APNG only. Anything else animated — animated WebP, animated HEIF — falls through to AVFoundation (which will not find a video track) and then to the FFmpeg whitelist, whose containers do not match. It will not play.

Timing details worth knowing:

• APNG timings are read from the raw fcTL chunk rationals, deliberately avoiding ImageIO's float32 rounding, accumulated in an integer nanosecond clock so exact centisecond and millisecond boundaries do not drift.
• GIF uses the unclamped delay first, falling back to ImageIO's adjusted value only when it is absent or zero.
• A separate static poster before the first animation frame is detected and stripped, and the timed APNG is rebuilt from scratch with each chunk's CRC verified.
• A separate-poster APNG whose first frame is partial is explicitly rejected with "A separate-poster APNG with a partial first frame needs canvas reconstruction and is not yet supported."

Audio-only files

Audio-only clips exist and draw a measured waveform instead of video frames — in tiles and in thumbnails.

AVAudioFile handles most formats; FLAC, and everything in mkv, webm, flac, ogg, opus, are routed to the bundled FFmpeg audio decoder instead. Audio-only clips get a nominal 1280 × 720 record.

Note that a video with a soundtrack is not an audio-only file: it shows video frames, not a waveform.

Colour handling — and the honest refusals

The decoder refuses to guess colour when it has no grounds to guess. This means some files that "play anywhere else" are rejected here, by design, with a message saying exactly why.

Declared interpretation profiles

• Declared metadata — everything comes from encoded tags.
• Windows Media profile — for WMV1/WMV2/WMV3/VC-1 streams whose tags are absent: the absent fields only receive BT.601 coefficients, MPEG (limited) range and MPEG-2 chroma placement. Tags that are present are honoured, and unsupported explicit declarations are rejected rather than treated as absent. This mirrors the documented behaviour of decoding unrecognised inputs through an FFmpeg fallback and showing the result rather than refusing them outright.

The chosen profile is reported per source rather than applied silently.

Chroma siting substitutions

Where a codec's spec defines a placement, the decoder substitutes it — each of these is a declared reconstruction policy, not a fidelity proof:

Anything else with unspecified chroma location is rejected.

Complete list of explicit rejections

Pixel formats and alpha

The pipeline is: 8-bit BGRA with premultiplied alpha from the decoders → rgba16Float inside the GPU frame.

• The FFmpeg path always emits tightly packed 8-bit BGRA with premultiplied alpha.
• Hardware-accelerated pixel formats are rejected outright.
• Premultiplication is applied whenever a source has alpha and is not already premultiplied.
• DXV and HAP are the only codecs allowed an unspecified alpha association.
• AVFoundation output requests BGRA only, with no explicit alpha configuration.
• Stills are drawn through a premultiplied sRGB context and retained as premultiplied RGBA8.
• Animated images use whatever ImageIO produced, with no premultiplication step in this codebase.

The colour scaler is pinned to the stable swscale backend, never the unstable one, with the strict flag set at open time. The dependency builder records why: enabling the unstable backend fails to link, and the project explicitly does not claim HQ colour-conversion acceptance.

One reviewed local patch is applied to the signed FFmpeg source, with pre- and post-image hashes recorded and a stated reason: decoding the complete final chroma block for half-resolution DXV HQ planes. A hash mismatch aborts the build.

Not verified

• Exactly which still-image formats succeed — delegated entirely to ImageIO.
• Which specific HAP or DXV product variants play. The code knows only the decoder names hap and dxv, plus the DXV HQ YCoCg branch. Claims about "Hap1/Hap5/HapY" or "DXV3" are not source facts.
• Whether a given .mp4 lands on AVFoundation or FFmpeg — depends on runtime probes.
• Sustained playback performance for any format. No throughput or endurance guarantee exists in code.