methodology

After facing methodology criticisms, the author conducted extensive experiments on an LLM resume-screening study, revealing that initial bias measurements were largely due to noise and that factors like prompt design and wrapper choice significantly affect scores.

An introduction to Extreme Programming, an Agile software development methodology that emphasizes customer satisfaction, teamwork, and iterative development through simple rules and values.

Ethan Mollick suggests that AI-generated analyses should be accompanied by multiverse-style reporting and full disclosure of prompts to enhance reproducibility in science.

This perspective paper from arXiv articulates a four-stage inferential framework for evaluating foundation models as cognitive and developmental models, emphasizing that behavioral alignment alone is insufficient and must be embedded within theoretical commitments and contrastive evaluation.

Samuel Ting shared his four methodologies at the conference on AI accelerating scientific research, emphasizing redoing what everyone thinks has already been done.

This paper presents a methodology for integrating knowledge graphs into ROS 2-based robotic systems to improve autonomy and decision-making in missions like search and rescue, demonstrated with the Aerostack2 framework.

This paper introduces intra-paper claim verification, a framework that uses LLMs to evaluate whether novelty claims in a paper are supported by its methodological evidence, addressing a gap in existing automated peer review systems. Human evaluation shows significant alignment with human reviewer concerns, especially for novelty-related issues.

This paper presents a systematic geographical clustering method for identifying language areas and macroareas in linguistic typology, addressing the lack of a formal approach to defining regions of potential language contact.

Skill-from-masters is an open-source tool that searches local databases and the web to surface expert best practices before generating AI agent skills, helping users incorporate proven methodologies from domain experts.

Anthropic outlines a six-step process for conducting large-scale code migrations using Claude Code, focusing on creating rulebooks, dependency maps, and iterative validation with a strong judge.

Discussion of two approaches to software design, as documented on the C2 wiki.

This paper analyzes validation practices for using LLMs as measurement instruments in social science, identifying epistemic threats and proposing emerging norms for robust validation.

SAAG (Simplify, Automate, Agentify, Guard) is a practical methodology for deciding where AI fits in a business, emphasizing simplification before automation and agentification, with guardrails for risk mitigation.

Cognition releases FrontierCode 1.1, an updated benchmark for evaluating code quality with refined guidelines for fair internet use and grading criteria, along with new model scores for Sonnet 5 and Fable 5.

This paper presents a stepwise methodology for developing NLP systems in the clinical domain, applying the Systems Development Life Cycle approach, and discusses the challenges of using large language models for information extraction from electronic medical records.

Recommend Jin Guantao's book 'Cybernetics and Scientific Methodology', which is believed to fundamentally reshape thinking and raise the ceiling of personal work and learning.

This tweet argues that Spec-Driven Development is ineffective for AI agents, drawing parallels to the failure of waterfall methodology in software development. It advocates for 'intent engineering'—communicating context, strategy, and constraints to agents to handle unknown unknowns.

This article critically examines the accuracy of AI visibility tools that claim to measure brand presence in generative AI responses, arguing that they provide false precision due to nondeterminism, personalization, and scraping biases. It calls for transparency in methodology and warns against treating opaque dashboards as stable truth.

This paper argues that NLP research on culture is a material-discursive practice where language models participate in constituting cultural reality rather than passively recording it, drawing on Barad's concept of agential cut.

Recommends two amazing Skills, focusing mainly on obra/superpowers, a complete software development methodology designed for coding agents, supporting automated sub-agent-driven development and TDD workflow.