screen-understanding

Marv is an AI cursor companion that lets users hold a hotkey and ask about anything on screen, answering out loud while drawing arrows, circling buttons, and annotating the display to guide tasks like debugging code, writing spreadsheet formulas, or navigating apps.

The author questions traditional mobile agent approaches relying on API access and proposes using screen understanding via hardware like aiden-firmware to enable more general-purpose agents that interact like humans.

LiquidAI released LFM2.5-VL-3B, a 3.1B local vision-language model that outperforms Gemma-4 E4B and demonstrates a significant jump in screen understanding, making on-device AI more practical.

LiquidAI announces LFM2.5-VL-3B, an efficient vision-language model for edge hardware with improved screen understanding, grounding, multi-image input, and function calling, trained with 4x more vision data and post-training via SFT and RL.