Tag
SimLoss uses embedding-space contrastive supervision to enable single-pass fine-grained image captioning that matches multi-stage quality at much lower latency, with variants like SimLoss FFT achieving high precision.
A new vision-encoder-decoder model is introduced that can process both images and text to generate human-like responses, suitable for tasks like image captioning and visual question answering.
Photoroom details their data strategy for training PRX, including assembling diverse datasets, re-captioning with a VLM, and using Mosaic Data Shards for efficient training.
Introduces ClaimDiff-RL, a reinforcement learning framework for long-form image captioning that uses typed, verifiable claim differences as reward units to separately measure and balance hallucination and missing facts, improving faithfulness and coverage.
The paper introduces BalCapRL, a balanced reinforcement learning framework for multimodal large language models that jointly optimizes correctness, coverage, and linguistic quality in image captioning. It demonstrates improved performance over existing methods by addressing trade-offs between utility and fluency through reward decoupling and length-conditional masking.
Florence-2 is an advanced vision foundation model from Microsoft that uses a prompt-based approach to handle various vision and vision-language tasks like captioning and object detection, trained on a large annotated dataset.