Tencent WeVisOne-VL Unifies Detection, OCR, Segmentation and Matting

Tencent’s WeChat Vision team has released WeVisOne-VL, an open vision language model in 2B and 4B sizes. The WeVisOne-VL vision language model handles detection, referring, counting, pointing, OCR, segmentation and alpha matting in one model. Pixel outputs come from a 9.6M-parameter decoder, so no separate SAM is needed. It also answers “None” when a queried object is absent. For developers, that can replace a detector, an OCR engine and a segmenter with one model.

TL;DR

• Size: 2B (2.44B params) and 4B (4.83B params) on Hugging Face, and a 9.6M-parameter pixel decoder. Trained with up to 8,192 tokens of context; deployment context length is not disclosed.
• Runs on: A single GPU through Transformers or vLLM in BF16. The package also lists NPU and CPU devices. VRAM needs are not disclosed. A community GGUF ran object-level tasks on a 6 GB RTX 2060.
• Performance: Beats the same-scale VLM baselines in Tencent’s tables on most of 30+ benchmarks (self-reported).
• Best: 71.3 F1 Mean on Dense200 dense detection, versus 58.7 for LocateAnything-3B.
• Worst: 52.6% rejection rate on PR-Bench, so many absent-target queries still get a box.

What is WeVisOne-VL?

WeVisOne-VL (“WeChat Vision All-in-One”) is a perception-focused model from Tencent built on Qwen3-VL. It targets fine-grained localization rather than open-ended chat. Users choose the output granularity: points, boxes, binary masks or continuous alpha mattes.

How does WeVisOne-VL work?

Every task is cast as structured next-token prediction. The model adds coordinate tokens <0> to <1000> for normalized boxes and points. An absent target produces None instead of a guessed box.

For pixel tasks, the model emits an <OBJ> token after each predicted box. Its hidden state becomes a query for the pixel decoder, according to the model card. The decoder has 3 parts:

• Visual Feature Adapter: reuses multi-level Qwen3-VL ViT features instead of a second image encoder.
• Box-Guided Query Decoder: 6 deformable cross-attention blocks seeded by the <OBJ> state and the box.
• Dense Prediction Head: refines output at S/4, S/2 and S/1 into a full-resolution mask or matte.

How was it trained?

A knowledge-graph-driven data engine annotated over 100M images with about 3.6B boxes. SAM3, Rex-Omni and an in-house detector proposed regions, and Qwen3-VL verified them.

Training used 3 SFT stages. Stage 1 aligned output formats on 90M examples. Stage 2 raised input to 1,960 visual tokens and up to 500 boxes per response. Stage 3 trained the pixel decoder, then specialized it for matting on 130M samples. A final GRPO stage used 178K uncertainty-filtered examples, as described in the research paper.

How does WeVisOne-VL perform on benchmarks?

All numbers below are self-reported, using greedy decoding and at most 1,960 visual tokens per image. Detection, referring and OCR use F1 averaged over IoU 0.5 to 0.95.

• Detection: 55.3 F1 on LVIS and 71.3 on Dense200 for the 4B model.
• Referring: 81.8 F1 on RefCOCOg test, versus 77.6 for LocateAnything-3B.
• Rejection: 96.4% on Tencent’s in-house WeRef-Bench (2,880 queries), versus 73.0%.
• Counting: 89.6% exact-match on PixMo-Count test, versus 82.8%.
• OCR: 52.0 F1 on TotalText, ahead of the PP-OCRv5 specialist at 25.7.
• Segmentation: 71.2 gIoU on ReasonSeg val, versus 65.0 for LISA-13B.
• Matting: 7.3 MAD on P3M-500-NP box prompts, versus 15.4 for SAM2-Matting (SAM3). Lower is better.

The 2B model trails the 4B model on most tasks. It still scores better box-prompted matting on P3M-500-NP, at 6.6 MAD.

WeVisOne-VL vs Qwen3-VL, Rex-Omni and LocateAnything

How do you run WeVisOne-VL?

Weights for the 2B and 4B models are on Hugging Face. The wevisone_vl package wraps Transformers and vLLM backends. Tasks include detection, referring, point, count_box, ocr, segmentation and matting.

from wevisone_vl import WeVisOneVL
model = WeVisOneVL("tencent/WeVisOne-VL-4B", backend="transformers")  # or "vllm"
result = model.predict(image, task="matting", query="person")

Hosted demos are live for 2B and 4B. Tencent also open-sourced Mask2Alpha, the matting annotator from its data engine. Fine-tuning code is still on the TODO list. GGUF builds cannot produce masks, since the decoder needs native visual features.

Key Takeaways

• 1 model covers detection, OCR, counting, pointing, segmentation and matting.
• A 9.6M-parameter decoder replaces SAM for masks and alpha mattes.
• 4B scores 71.3 F1 on Dense200, 12.6 points above LocateAnything-3B.
• Apache 2.0 weights, unlike non-commercial LocateAnything-3B.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us