A user reports that DeepSeek-V4-Flash-0731 is unreliable for non-coding office tasks like summarization and meeting notes, failing at concept extraction and speaker understanding despite strong benchmark scores, while Gemma-4-31B performs better.
(I am not a native speaker, written by myself, so please bear with me) I really want to like DeepSeek-V4-Flash-0731. But it has serious flaws that don't align with the high score on intelligence benchmarks. And those flaws render it useless unfortunetely for anything else except, maybe, coding. It fails in subtleties that seem small but are crucial, and fails in more obvious tasks that should be easy to solve. Those errors make it unreliable enough for me to not even trust it the simplest tasks in office work like summarizing text or writing letters. Maybe I am doing something wrong here. Some of the issues below don't seem to be normal for an LLM of that size. To be clear: I want this model to work. It's faster than Gemma-4-31B and its total parameter count is 8 times higher. It is good at thinking things through, excellent at doing research if given web search access. But for language it not only fails on "beautiful wording" but on extracting the relevant concept from a context. Those areas seem not to be tested in benchmarks, but they are essential when doing office work. They are easier to explain with examples. Below I'll show you three. Ability 1: Including the revelant yet being concise Given a text to create meeting notes from. DeepSeek-V4-Flash-0731: Spreading irregular income over the year to make sure the essentials are available every month. Gemma-4-31B: Concept: The financial investments are designed to cover only part of the needs. The remaining gap will be filled by averaging the irregular income from self-employment throughout the year. DeepSeek-V4-Flash-0731's version is missing that there are two income sources. So while it points out the essence (the issue), that doesn't become clear enough because it is only part of the story. Gemma somehow has an ability to understand the essence and put it into sentences that are concise yet precise in beautiful wording. Look at "by averaging the irregular income", that is a very elegant way to say what is happening with just the two words "by averaging". DeepSeek is not able to do that, and worse, it is missing the context of the financial investments being one part of the cost coverage. This is not a "beautiful" language issue (we know Gemma is good at language), but also a "concept understanding" issue or a "figuring out what is relevant" issue. Ability 2: Understanding who is the speaker Given is a text transcript of a voice message and the question. "What would be her best option? How should she handle the situation? What are her possibilities? Please find the best way forward." DeepSeek-V4-Flash-0731: Writes its whole answer like if I am the person who spoke the voice message and to be addressed. Given that the question contained "her", and that the two voice messages had headlines "Voice message 1 of the person" and "Voice message 2 of the person", this is a mistake I can't accept. Being pressured on it, it tries to explain the reason for writing "you" in the answer instead of "she" is that the voice transcript talks in the person "I", and the voice message takes a big portion of the context, so it had just weight on "I" being the person asking and assumed it's me asking. But I clearly wrote "What could be her best option", and it was really clear by the headlines those messages were of another person. DeepSeek failed here, and this failure is unacceptable to me. An AI needs to understand the context, not just get confused by the amount of text written as "I". Gemma-4-31B: No issue here. It understood clearly that who made the request is not the same person as who spoke the voice message. Ability 3: Not getting confused by minor phrases DeepSeek-V4-Flash-0731 got confused by the start of the message being "Hi, hi. So, Jon, his message is kind of funny. They’re currently up north, ..." Only because "So, John, ..." could be a greeting, it assumed the whole text of 5 paragraphs was addressed to John, even though the rest of the text was saying "he". Gemma-4-31B: Understood from the whole context that "So, John" was context, not a greeting. It understood "John" is not the person being written to, but the people being talked about. The Verdict DeepSeek-V4-Flash has 284 billion parameters. I thought it to be at least on the same level as Gemma-4-31B in those areas. Language doesn't need to be as beautiful as Gemma-4, but I had the expectation that DeepSeek knows how to include all relevant information or getting the context right, and to my surprise, it fails.
User benchmarks DeepSeek v4 Flash against Qwen3.6-27B, Qwen3.5-122B, and Gemma 4 31B on a local coding benchmark, finding Flash wins overall but Qwen 122B performs surprisingly well with better first-try success and lower token usage.