* Export videos for named platforms and check each file against its limits Exports only knew aspect ratios, so a cut could still be rejected or degraded on upload: too long for the platform, over its size limit, or far off its loudness. export_video now takes platforms (TikTok, Instagram Reels and Feed, YouTube Shorts and YouTube, Facebook Reels, LinkedIn, X, Pinterest, Snapchat Spotlight), up to six targets with the presets. Each platform is rendered to its frame with the export's fit, its frame rate cap, a bitrate cap that also keeps the file under the size limit, and its loudness target in one loudnorm pass. Every render is probed and measured afterwards. A file over the size limit is encoded once more at a bitrate scaled from what it averaged, and audio more than 2 LU off the target gets one linear loudness pass. The result is stored on the output as a pass, warn or fail check with language-neutral issue codes. A platform the video is too long or too short for is not rendered: the card lists it with the reason, the other targets still render, and the bot is told to trim first. The limits come from the earlier profile research, with its sources, review date and unverified flag; content, disclosure and caption rules are left out because they need inference or a review flow. Preset definitions move to their own module so the platform schemas can use their aspect ratios without an import cycle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * Add remove_silence step to edit_video Talking-head clips are mostly cut by hand to drop pauses, and the bot had no way to find them: it could only keep segments it guessed. The new remove_silence step finds every pause quieter than threshold_db (default -35 dB) and at least min_silence long (default 0.6 s), keeps padding (default 0.12 s) on each side so word onsets survive, and cuts the rest through the existing keep-segments graph, so picture and sound stay in sync. Steps run in order, so silence is detected on the timeline the steps before it leave: the planner gets an analysis mode that sends that audio through silencedetect into a null output, and ametadata prints the marks to a file in the job directory. The detected speech is handed back to the planner per step. Nothing silent makes the step a no-op; nothing left fails with a clear error. At most 200 pieces are kept, bridging the shortest pauses first, so the filter graph stays a sensible size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * Offer platforms in the media card's export menu and show their checks The card's Export menu now lists the platforms first, by brand name and untranslated, then the aspect ratios after a separator; the platform an output was made for is left out, like its own shape. A platform export shows the platform's name under the player, or, when its check did not pass, one line for the first issue instead ("Too long for TikTok (max 10:00)"), in the warning or destructive color. Skipped platforms get the same kind of line, and they stand in for the error of a job that skipped every platform, so the card does not say the same thing twice in two languages. The E2E spec seeds a platform export with a failed check and a skipped platform, and captures the card and the open menu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * Add analyze_video to find cut points for the bot To cut a social video well the bot has to know where the cuts fit, and it could not look at a video at all. analyze_video is a read-only media job: one ffmpeg pass prints scene changes (scene score above 0.3, one per cut), pauses (silencedetect with the remove_silence defaults) and short-term loudness (ebur128) to files in the job directory, and a transcript made from the video by transcribe (an SRT whose parent is the video) adds when each line is spoken. The job stores nothing; its card just says done. The result reaches the bot through the existing media job path: the job always wakes the bot that started it (the tool has no continue_after) and the summary carries a short legend plus compact JSON. The runtime's summary limit grows to 12,000 characters and the summary is fitted into it: at most 200 entries per list, then every list shrinks evenly, keeping the strongest cuts, the longest pauses and the start of the transcript, with omitted counts. The wake prompt no longer claims the card shows the result when a job made no files. Jobs read derived artifacts through a new fetchDerivedArtifact, scoped to the job's user like fetchArtifact. A migration admits the operation in the media_jobs check and adds the tool to existing Studio specialists that still edit video. The web card gets an icon; no new copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * fix(media): keep one skipped entry per platform on the card A job that recovers after a crash runs again and reports the same skipped platforms; the newest reason per platform now replaces the earlier one instead of listing it twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * chore(db): add the widened media job operation check as NOT VALID and validate it separately Follows the repository's migration convention: the constraint commits without scanning media_jobs, and a second migration validates it with a lock that lets reads and writes continue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u * Brand videos with a logo overlay and an intro before the cut edit_video gains an overlay step that lays a PNG, JPEG or WebP image over the video at a corner or the center, sized to a share of the frame width, with optional opacity and time range; tall frames keep the 9:16 safe areas clear. concat takes at: "start" so an intro can go before the video. The image is authorized like other inputs, read with its own demuxer and refused above 8192 pixels on a side. New Studio specialists are told to brand with the user's logo and intro. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u --------- Co-authored-by: Claude <noreply@anthropic.com>
123 lines
4.6 KiB
TypeScript
123 lines
4.6 KiB
TypeScript
import * as z from "zod";
|
|
import { Id } from "./ids.js";
|
|
import { RemoveSilenceStepSchema } from "./media-analysis.js";
|
|
import { MediaPlatformIdSchema } from "./media-platforms.js";
|
|
import { MediaFitSchema, MediaPresetIdSchema } from "./media-presets.js";
|
|
|
|
/**
|
|
* Inputs of the `edit_video` and `export_video` media operations, which are also the arguments
|
|
* of the tools with those names. Times are seconds on the timeline left by the previous steps.
|
|
*/
|
|
|
|
const Seconds = z.number().min(0).max(86_400);
|
|
|
|
const TimeRangeSchema = z
|
|
.object({ start: Seconds, end: Seconds })
|
|
.refine((range) => range.end > range.start, "end must be after start");
|
|
|
|
/** Written by the bot in the user's language; shown on the media card and used as file name. */
|
|
export const MediaTitleSchema = z.string().trim().min(1).max(120);
|
|
|
|
/** Lays an image, a logo or watermark say, over the video: the whole time or from start to end. */
|
|
const OverlayStepSchema = z
|
|
.object({
|
|
op: z.literal("overlay"),
|
|
artifact_id: Id,
|
|
position: z
|
|
.enum(["top_left", "top_right", "bottom_left", "bottom_right", "center"])
|
|
.default("top_right"),
|
|
/** Share of the frame width. */
|
|
width: z.number().min(0.02).max(1).default(0.15),
|
|
opacity: z.number().min(0.05).max(1).default(1),
|
|
start: Seconds.optional(),
|
|
end: Seconds.optional(),
|
|
})
|
|
.refine(
|
|
(step) => step.start === undefined || step.end === undefined || step.end > step.start,
|
|
"end must be after start",
|
|
);
|
|
|
|
export const EditVideoStepSchema = z.discriminatedUnion("op", [
|
|
z
|
|
.object({ op: z.literal("trim"), start: Seconds, end: Seconds })
|
|
.refine((step) => step.end > step.start, "end must be after start"),
|
|
/** Keeps these parts and cuts everything between them. */
|
|
z.object({ op: z.literal("keep"), segments: z.array(TimeRangeSchema).min(1).max(50) }),
|
|
/** Adds these videos at the end, or at the start for an intro, fitted to the current frame. */
|
|
z.object({
|
|
op: z.literal("concat"),
|
|
artifact_ids: z.array(Id).min(1).max(20),
|
|
at: z.enum(["end", "start"]).default("end"),
|
|
}),
|
|
z.object({
|
|
op: z.literal("reframe"),
|
|
preset: MediaPresetIdSchema,
|
|
fit: MediaFitSchema.default("crop"),
|
|
}),
|
|
z.object({ op: z.literal("speed"), factor: z.number().min(0.25).max(4) }),
|
|
/** Linear gain: 0.5 halves the loudness, 2 doubles it. */
|
|
z.object({ op: z.literal("volume"), gain: z.number().min(0).max(8) }),
|
|
z.object({ op: z.literal("mute") }),
|
|
/** Burns in an SRT or VTT artifact. */
|
|
z.object({ op: z.literal("subtitles"), artifact_id: Id }),
|
|
/** Last step only: the result is the audio as M4A. */
|
|
z.object({ op: z.literal("extract_audio") }),
|
|
/** Cover frame at this second of the result; as the only kind of step, the result is that image. */
|
|
z.object({ op: z.literal("poster"), at: Seconds }),
|
|
RemoveSilenceStepSchema,
|
|
OverlayStepSchema,
|
|
]);
|
|
export type EditVideoStep = z.infer<typeof EditVideoStepSchema>;
|
|
|
|
export const EditVideoInputSchema = z
|
|
.object({
|
|
artifact_id: Id,
|
|
steps: z.array(EditVideoStepSchema).min(1).max(20),
|
|
title: MediaTitleSchema,
|
|
})
|
|
.superRefine((input, context) => {
|
|
const ops = input.steps.map((step) => step.op);
|
|
const extract = ops.indexOf("extract_audio");
|
|
if (extract >= 0 && extract !== ops.length - 1) {
|
|
context.addIssue({
|
|
code: "custom",
|
|
path: ["steps", extract],
|
|
message: "extract_audio must be the last step",
|
|
});
|
|
}
|
|
if (extract >= 0 && ops.includes("poster")) {
|
|
context.addIssue({
|
|
code: "custom",
|
|
path: ["steps"],
|
|
message: "poster cannot be combined with extract_audio",
|
|
});
|
|
}
|
|
if (ops.filter((op) => op === "poster").length > 1) {
|
|
context.addIssue({ code: "custom", path: ["steps"], message: "use at most one poster step" });
|
|
}
|
|
});
|
|
export type EditVideoInput = z.infer<typeof EditVideoInputSchema>;
|
|
|
|
export const MAX_EXPORT_TARGETS = 6;
|
|
|
|
/** One output per platform and per preset: at least one, at most MAX_EXPORT_TARGETS in all. */
|
|
export const ExportVideoInputSchema = z
|
|
.object({
|
|
artifact_id: Id,
|
|
platforms: z.array(MediaPlatformIdSchema).max(MAX_EXPORT_TARGETS).optional(),
|
|
presets: z.array(MediaPresetIdSchema).max(MAX_EXPORT_TARGETS).optional(),
|
|
fit: MediaFitSchema.default("crop"),
|
|
title: MediaTitleSchema.optional(),
|
|
})
|
|
.refine(
|
|
(input) => {
|
|
const targets = (input.platforms?.length ?? 0) + (input.presets?.length ?? 0);
|
|
return targets >= 1 && targets <= MAX_EXPORT_TARGETS;
|
|
},
|
|
{
|
|
path: ["platforms"],
|
|
message: `name 1 to ${MAX_EXPORT_TARGETS} platforms and presets in all`,
|
|
},
|
|
);
|
|
export type ExportVideoInput = z.infer<typeof ExportVideoInputSchema>;
|