research-methodology

Tag

Cards List
#research-methodology

OpenAI revealed how its ~10,000-agent Navier–Stokes swarm actually worked: separate research teams explored different ideas, communicated across groups, and combined what worked

Reddit r/singularity ↗ · 2026-09-30

OpenAI revealed how its swarm of approximately 10,000 AI agents solved Navier-Stokes equations by having separate research teams explore ideas, communicate across groups, and combine successful approaches.

0 favorites 0 likes
#research-methodology

Beyond the Stitching Assumption: A Unified Framework for Multimodal Synthetic Data Evaluation via Semantic Quantization

arXiv cs.CL ↗ · 2026-09-22 Cached

This paper presents a unified framework for evaluating multimodal synthetic data using semantic quantization and cross-modal metrics, emphasizing the need for explicit evaluation with permutation baselines and coverage reporting.

0 favorites 0 likes
#research-methodology

Evolution or Illusion? Rethinking Evaluation in LLM Evolutionary Search

arXiv cs.CL ↗ · 2026-09-18 Cached

This paper critiques single-point evaluations for LLM evolutionary search, showing that optimal performance depends on varying seeds and iterations, and proposes a budget-grid evaluation protocol.

0 favorites 0 likes
#research-methodology

Metacognitive Steering: Learning the Structure of Scientific Judgment

arXiv cs.AI ↗ · 2026-09-16 Cached

The paper introduces Metacognitive Steering, an inference-time controller that dynamically adjusts language model reasoning for scientific discovery using process-level data, and demonstrates its application in the Columbus-1 autonomous research system.

0 favorites 0 likes
#research-methodology

@Xudong07452910: Over the past few years, the act of doing ML/NLP research has changed at a somewhat exaggerated pace. CMU PhD student Emmy Liu's piece 'When We Did Research By Hand' reflects on her PhD journey, roughly dividing these years into three phases starting with 2021: reading papers, writing code…

X AI KOLs Timeline ↗ · 2026-08-30 Cached

CMU PhD student Emmy Liu reflects on the evolution of ML/NLP research, from manual work to AI-assisted to the Agent era, discussing the value of researchers in the new age.

0 favorites 0 likes
#research-methodology

Using Human-LLM Disagreement to Improve Checklist-Based Quality Appraisal

arXiv cs.CL ↗ · 2026-08-24 Cached

This paper explores using human–LLM disagreement to enhance checklist-based quality appraisal in systematic reviews, showing that analyzing disagreements can identify ambiguous items and improve checklist design for better agreement and study ranking preservation.

0 favorites 0 likes
#research-methodology

AQuA's "self-improvement" updates research state, not the agent LM. What should a local port freeze?

Reddit r/LocalLLaMA ↗ · 2026-08-20

The article analyzes AQuA's preprint on recursive self-improvement in AI agents, clarifying that the agent LM remains fixed while research state updates, and advocates for detailed ablation studies and artifact sharing to enable credible local model ports.

0 favorites 0 likes
#research-methodology

🤡 How to make any Sparse Attention / KV Compression look good? 🤡

Reddit r/ArtificialInteligence ↗ · 2026-08-17

The article critiques common practices in AI research to make sparse attention and KV cache compression techniques appear more effective than they might be, by exploiting benchmark flaws and implementation details.

0 favorites 0 likes
#research-methodology

@KirkDBorne: "Designing Experiments and Analyzing Data: A Model Comparison Perspective" [Third Edition]: http://amzn.to/3QM1TE2 Amaz…

X AI KOLs Timeline ↗ · 2026-08-10 Cached

Tweet promoting the third edition of the statistics textbook 'Designing Experiments and Analyzing Data: A Model Comparison Perspective' by Maxwell, Delaney, and Kelley, highlighting its pedagogical features and the authors' academic credentials.

0 favorites 0 likes
#research-methodology

Partnership with AI Guide updated to v7

Reddit r/artificial ↗ · 2026-07-21

The AI Guide partnership updates to v7 with more honest reflections on unresolved contradictions and methodology, including re-evaluating a key term's valence and a metaphor's effectiveness, plus an outside review clarifying model sentiment vs. training data bias.

0 favorites 0 likes
#research-methodology

The Agentic Garden of Forking Paths

arXiv cs.AI ↗ · 2026-07-03 Cached

This paper introduces AI agents that reproduce human analytical variation across datasets, finding that different personas lead to divergent conclusions. It proposes the m-value and Agentic Bootstrap to assess the credibility of reported analyses.

0 favorites 0 likes
#research-methodology

@iamgrigorev: https://x.com/iamgrigorev/status/2071688181628678468

X AI KOLs Timeline ↗ · 2026-06-29 Cached

A detailed guide on designing effective ML experiments, emphasizing starting with a clear research question, developing research taste, and scaling results. Based on the author's experience running ~100 experiments weekly at Poolside.

0 favorites 0 likes
#research-methodology

Mitigating LLM-based p-Hacking by Preregistering for the Next LLM

arXiv cs.CL ↗ · 2026-06-29 Cached

Proposes a protocol to mitigate p-hacking in LLM-based research by preregistering experiments and running them on the first eligible model released after preregistration, demonstrating substantial mitigation across multiple models.

0 favorites 0 likes
#research-methodology

@VincentLogic: https://x.com/VincentLogic/status/2070704146605564230

X AI KOLs Timeline ↗ · 2026-06-27 Cached

This article shares a piece written by Anthropic researcher Vivek Nair on how to conduct good research, emphasizing that choosing the right problem is more important than solving it, cultivating research taste, upgrading information sources, and using writing as a thinking tool. These insights are not only applicable to academic research but also provide inspiration for career development and investment decisions.

0 favorites 0 likes
#research-methodology

Thinking Like a Scientist? A Structural Study of LLM-Generated Research Methods

arXiv cs.CL ↗ · 2026-06-26 Cached

This study examines how LLMs suggest research methods (datasets, models, metrics) when prompted only with a research question, finding that LLMs exhibit a strong provider bias and propose a much narrower range of methods compared to actual papers, potentially narrowing researchers' methodological search space.

0 favorites 0 likes
#research-methodology

Medical students are using popular research tool to pump out misleading studies

Hacker News Top ↗ · 2026-06-25

Medical students are reportedly using a popular research tool to produce misleading studies, raising concerns about academic integrity and research quality.

0 favorites 0 likes
#research-methodology

The Return of Rigorous Full-System Timing Simulation

Hacker News Top ↗ · 2026-06-16 Cached

This blog post argues for a return to rigorous full-system timing simulation in computer architecture to overcome the 'timing simulation wall' and accurately capture modern system behaviors, advocating for measuring the right execution intervals with statistically sound methods rather than simulating everything in detail.

0 favorites 0 likes
#research-methodology

@MinLiBuilds: I wish I had read such an excellent article during my undergraduate and graduate studies; my career would have turned out completely differently. This is her research methodology, very smart and solid, with compounding returns. Translation: vivek @itsreallyvivek how to be good at r…

X AI KOLs Timeline ↗ · 2026-06-14 Cached

A methodology article on how to excel at AI research, emphasizing problem selection, literature reading, writing notes, and other skills, suitable for researchers.

0 favorites 0 likes
#research-methodology

Thinking Through Signs: PEEL as a Semiotic Scaffolding for Epistemically Accountable AI-Enabled Research

arXiv cs.AI ↗ · 2026-06-04 Cached

This paper introduces PEEL (Protocols for Epistemically Engaged Literacy in AI), a framework combining deterministic text analysis via Voyant Tools with LLM interpretation via Claude, grounded in Peircean semiotics, to expose systematic distortions in AI-generated research summaries and promote epistemic accountability.

0 favorites 0 likes
#research-methodology

@yibie: 3 new additions this round: 1. Auto-Quant: Apply Karpathy autoresearch to FreqTrade cryptocurrency strategy backtesting, evolving multi-strategy combinations across 5 trading pairs through a keep/discard loop. 2. Universal…

X AI KOLs Timeline ↗ · 2026-06-02 Cached

yibie shared three new entries from the awesome-autoresearch list, covering automated quantitative trading, universal skill optimization, and a Claude Code plugin.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback