Modernising the Compressed-Domain Video Captioner: A Controlled Study of SigLIP2 and GPT-2 Substitutions

arXiv:2609.31700v1 Announce Type: new Abstract: Compressed-domain video captioning avoids full video decoding by operating directly on I-frames, motion vectors and residuals, trading a small amount of accuracy for a large gain in inference speed. CoCap established this pipeline using a CLIP vision encoder and a shallow BERT-style multimodal decoder. Both components predate substantially stronger alternatives. We ask a narrow, controlled question: how much of CoCap's accuracy is limited by these two components, and which of the two is the binding constraint? We replace the CLIP I-frame encoder with SigLIP2 and the BERT-style decoder with GPT-2, and evaluate three configurations (the original pairing, the encoder substitution alone, and both substitutions together) under identical data, sampling budget and optimisation schedule. All comparisons are made against our own reproduction of CoCap rather than its published numbers, because we train on a 4,999-clip subset of VATEX at a reduced sampling budget; absolute values are therefore not comparable with the original work. Our reproduction tracks the published result closely: CIDEr and METEOR run slightly above it (54.9 against 52.7; 23.4 against 23.2), BLEU-4 and ROUGE-L slightly below (29.7 against 31.4; 48.9 against 49.4). We attribute the differences to our evaluation subset rather than to any improvement in either direction. We find the two substitutions pull in opposite directions: SigLIP2 alone improves every metric (+4.5 CIDEr), while adding GPT-2 on top erodes that gain, because a pretrained decoder overfits 4,999 clips within two epochs. We additionally report inference latency for each configuration, since speed is the property that motivates compressed-domain captioning in the first place, and an accuracy gain purchased at a latency cost should be reported as such.