Accuracy and Reliability of Large Language Models in Cosmetic Chemistry and Skin Health: A Benchmarking Study

# Accuracy and Reliability of Large Language Models in Cosmetic Chemistry and Skin Health: A Benchmarking Study
Source: [https://arxiv.org/html/2608.14631](https://arxiv.org/html/2608.14631)
Amelia Liu Independent Researcher Basking Ridge, NJ, 07920, USA amelialiu2027@gmail\.comAll Python code used for this project is publicly available at the paper’s GitHub repository[https://github\.com/ameliaxl](https://github.com/ameliaxl)

###### Abstract

As consumers increasingly turn to AI chatbots for skincare advice, the technical accuracy of Large Language Models \(LLMs\) in cosmetic chemistry remains largely under\-evaluated\. We benchmarked 14 LLMs on a structured set of topics related to cosmetic chemistry, including the chemical properties of specific cosmetic ingredients and common cosmetic scenarios that may be of interest to consumers\. Web search was disabled throughout to assess each model’s internalized knowledge rather than its internet retrieval capacity\. Overall performance was poor, with the most pronounced deficits in quantitative reasoning and structural identification tasks\. While models handled general skincare questions with reasonability, responses consistently lacked the technical depth required for informed consumer decision\-making\. Notably, conversation with AI can pose a risk: outputs that sound authoritative but contain technical errors are less likely to generate skepticism compared to responses that explicitly acknowledge uncertainty\. These findings suggest that general\-purpose LLMs, trained predominantly on unverified public data, are currently not reliable sources of cosmetic chemistry information\. Progress on two fronts, fine\-tuning verified chemical and dermatological datasets, and substantial improvements to algorithmic reasoning, will likely be needed before these tools can be considered as resources for public use\.

## 1Introduction

In recent years, large language models \(LLMs\) have achieved significant global prominence by demonstrating capabilities that mirror human cognitive processing across a wide array of domains\(Naveedet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib7)\)\. Since the debut of ChatGPT, these computational tools have become integrated into the fabric of daily life, assisting millions of users with tasks ranging from academic assignments and business automation to specialized inquiries in theoretical physics and skin care guidance\(Hostinger,[2025](https://arxiv.org/html/2608.14631#bib.bib8)\)\.

The architecture of LLMs is rooted in deep learning principles designed to synthesize vast corpora of human knowledge\(Naveedet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib7)\)\. These systems utilize probabilistic modeling to predict linguistic sequences and generate coherent text\. While their contemporary success is often associated with the 2017 introduction of transformer models and self\-attention mechanisms, their lineage extends back to the neural networks of the 1950s and the rudimentary chatbots of the 1960s\(Vaswaniet al\.,[2017](https://arxiv.org/html/2608.14631#bib.bib6); Toloka AI,[2025](https://arxiv.org/html/2608.14631#bib.bib9)\)\. By early 2026, the sector had matured into a highly competitive landscape;OpenAI \([2026](https://arxiv.org/html/2608.14631#bib.bib10)\)claimed that ChatGPT is a market leader with approximately 900 million weekly active users\. Other influential architectures include xAI’s Grok, Google’s Gemini, Meta’s Llama, Anthropic’s Claude, DeepSeek, and Mistral AI\.

Cosmetic chemistry is a specialized branch of the chemical sciences concerned with the formulation, development, and evaluation of aesthetic and personal care products\(Rosen,[2015](https://arxiv.org/html/2608.14631#bib.bib11)\)\. For consumers, cosmetics and over\-the\-counter skincare products serve as practical tools for hygiene and the management of dermatological concerns such as photoaging, acne, and hyperpigmentation\(Gohet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib14)\)\. As professional dermatological consultation is often cost\-prohibitive and time\-intensive, many consumers rely on online resources that vary considerably in quality and scientific rigor\. LLMs offer a notable alternative through their immediate availability, interactive format, and authoritative tone\(Carrara,[2026](https://arxiv.org/html/2608.14631#bib.bib13)\)\.

Despite their growing presence in consumer\-facing contexts, the scientific accuracy and reliability of LLM\-generated responses within the specialized domain of cosmetic chemistry remain poorly characterized\. Whether AI tools can provide sound skin health guidance is a matter of ongoing debate\(Ferreiraet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib18)\)\. Although the conversational accessibility of these models may benefit general consumers, their application for medical or technical advice is frequently cautioned against on grounds of accuracy and safety\(Goktas and Grzybowski,[2024](https://arxiv.org/html/2608.14631#bib.bib5)\)\.

Leon and Pimentel \([2023](https://arxiv.org/html/2608.14631#bib.bib16)\)were among the first to evaluate the use of ChatGPT in general chemistry\. They found that while ChatGPT demonstrated “conversational competence” in general chemistry, it failed in rigorous problem\-solving\.Guoet al\.\([2023](https://arxiv.org/html/2608.14631#bib.bib17)\)benchmarked LLMs in eight critical chemistry tasks\. They found that even the most advanced LLMs exhibit inconsistent performance when moving from general explanation to technical reasoning\. While these papers focused on general chemistry, we are interested in the specialized domain of cosmetic chemistry\. In the field of clinical dermatology, the initial assessments of LLMs byFerreiraet al\.\([2023](https://arxiv.org/html/2608.14631#bib.bib18)\)suggest a high rate of ’appropriate’ responses to patient queries\. However, the underlying scientific rigor regarding ingredient chemistry remains largely unquantified\.

This study evaluates 14 popular LLMs from seven leading AI developers across five topic areas in cosmetic chemistry and skin health, spanning both quantitative tasks and qualitative skincare scenarios\. The findings reveal a consistent performance disparity between retrieval\-based tasks, where several models performed well, and those requiring multi\-step chemical reasoning, where most models struggled regardless of parameter count\. These results highlight both the promise and the current limitations of LLMs as informational tools at the intersection of cosmetic chemistry and dermatology\.

## 2Experiment Design and Method

### 2\.1Large Language Models

This study evaluates a diverse cohort of LLMs from seven prominent AI developers, including Anthropic, DeepSeek, Google, Meta, Mistral AI, OpenAI, and xAI, to assess their proficiency in resolving cosmetic chemistry inquiries\. Fourteen models were selected for this benchmarking experiment, with two variants drawn from each company: a flagship “Pro” or “Large” model, typically characterized by a high parameter count, and a scaled\-down “Flash” or “Mini” variant designed for efficiency\.

The architectural capacity of these models is heavily influenced by their parameter counts, which are established during pre\-training on vast heterogeneous datasets\. While parameter count often serves as a proxy for raw cognitive capability, it is not the sole determinant of success\. Factors such as training data curation, architectural efficiency \(e\.g\., Mixture\-of\-Experts\), and reinforcement learning from human feedback \(RLHF\) contribute significantly to a model’s specialized performance\. The Chinchilla scaling laws proposed byHoffmannet al\.\([2022](https://arxiv.org/html/2608.14631#bib.bib26)\)suggests that models with fewer parameters can outperform larger models if they are trained on sufficiently large and high\-quality datasets\. A goal of this study is to examine the relationship between scale and accuracy in the specialized field of cosmetic chemistry\.

The specific model architectures and their respective parameter scales are detailed in Table 1\.222Estimates for proprietary systems \(GPT\-5, Gemini 2\.5 Pro/Flash, Grok 4, and the Claude 4\.5 suite\) are derived from consensus industry benchmarks, whereas figures for the Meta Llama 4 series, Mistral AI, and DeepSeek R1 are based on publicly disclosed technical specifications\.The selection criteria prioritized both market dominance and geographic diversity\. OpenAI’s ChatGPT and Google’s Gemini represent the most pervasive platforms in the consumer market\. Anthropic, Meta, and xAI represent leading American research initiatives, while DeepSeek and Mistral AI provide critical perspectives from the burgeoning Chinese and European AI sectors, respectively\. By spanning a wide spectrum of training methodologies and regional datasets, this multi\-model approach ensures a comprehensive evaluation of current AI capabilities in cosmetic science\.

Each model was subjected to a rigorous evaluation involving a predefined set of cosmetic chemistry prompts, ranging from structural bond analysis to complex group\-contribution calculations\.

Table 1:Comparative Model Architectures and Parameter Scales of Tested LLMs \(Q1 2026\)DeveloperFlagship ModelParametersEfficiency VariantParametersOpenAIGPT\-5∼\\sim1,800 BGPT\-5 Mini∼\\sim149 BGoogleGemini 2\.5 Pro∼\\sim288 BGemini 2\.5 Flash∼\\sim17 BMetaLlama 4 Maverick∼\\sim402 BLlama 4 Scout∼\\sim108 BxAIGrok 4∼\\sim1,700 BGrok 4 Fast∼\\sim314 BAnthropicClaude 4\.5 Opus∼\\sim175 BClaude 4\.5 Haiku∼\\sim20 BMistral AIMistral Large 3∼\\sim675 BMistral 14B14 BDeepSeekDeepSeek R1∼\\sim685 BR1\-Distill\-Qwen\-32B32 BNote: B = Billion\. Parameter counts for proprietary models are based on industry estimates\.
### 2\.2Experimental Prompts and Target Stimuli

To evaluate the proficiency of LLMs in synthesizing both quantitative and qualitative knowledge within the cosmetic domain, each model was subjected to five distinct thematic inquiries\. The prompts were structured to elicit concise, one\-sentence responses, ensuring a focused assessment of factual accuracy and technical reasoning\. The specific prompts are provided in Appendix[A](https://arxiv.org/html/2608.14631#A1)\.

Quantitative Assessment: Chemical Properties and Reasoning\.The first four topics evaluated the LLMs’ command of structural chemistry and computational logic\. These tasks represent foundational inquiries for which responses can be independently validated:

- •Topic 1 — Molecular Bonding Analysis:Identify the number of sigma and pi bonds for five cosmetic ingredients: Squalane, Phenoxyethanol, Sodium Lauryl Sulfate, Isododecane, and Chlorphenesin\.
- •Topic 2 — Compositional Analysis:Identify synthetic preservatives within five prominent formulations, includingCeraVe Hydrating Toner,Kiehl’s Ultra Facial Cream, andThe Ordinary Hyaluronic Acid 2% \+ B5\(2024 reformulation\)\.
- •Topic 3 — Molecular Weight Determination:Find the molecular weight \(rounded to one decimal place\) for Squalane, Octane, Dodecane, Isopropyl Alcohol, and Chlorphenesin\.
- •Topic 4 — Thermophysical Estimation \(Joback Method\):Calculate normal boiling points in Kelvin using the Joback method\(Joback and Reid,[1987](https://arxiv.org/html/2608.14631#bib.bib41)\)\. This task tested multi\-step algorithmic reasoning beyond simple data retrieval\.

Qualitative Assessment of Skincare Advice:Topic 5asked LLMs to provide consumer\-oriented guidance in two real\-world contexts: \(1\)Sequential skincare advice, describing subsequent steps for sensitive, acne\-prone skin following specific interventions \(e\.g\., azelaic acid, cleansing\); and \(2\)First\-step recommendations, identifying initial treatments for concerns such as hyperpigmentation and excess sebum\.

Methodological Rigor and Validation\.All experimental scenarios were conceived before data collection to prevent any form of post\-hoc selection\. Test stimuli, including specific cosmetic ingredients and finished products, were curated based on their frequency and significance of use in personal care\. Experimental scenarios were further designed to simulate high\-frequency, real\-world consumer interactions\. Quantitative responses were cross\-referenced against NCBIPubChem\(Kimet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib27)\)\. Qualitative outputs were benchmarked against foundational textbooks such asHarry’s Cosmeticology\(Rosen,[2015](https://arxiv.org/html/2608.14631#bib.bib11)\)\.

### 2\.3Experimental Design

All interactions with the selected LLMs were conducted systematically to ensure consistency and reproducibility across models\. To interface with multiple LLMs through a single, standardized endpoint, we used the OpenRouter API gateway, a unified gateway offering access to hundreds of large language models across different providers\(OpenRouter,[2025](https://arxiv.org/html/2608.14631#bib.bib30)\)\. Queries were automated using the official OpenAI Python library\(OpenAI,[2025](https://arxiv.org/html/2608.14631#bib.bib31)\), which enabled each predefined inquires to be submitted to all 14 models under identical conditions, with responses collected programmatically for subsequent analysis\.

Prompt standardization and bias mitigation: To maintain a controlled experimental environment, each inquiry was submitted as a discrete, independent request\. To mitigate the influence of conversational drift or “chain\-of\-thought” biases, the system was configured to prevent context retention between queries\. Each model formulated its response based solely on the immediate prompt, free from the carryover effects or external guidance that might artificially enhance performance\. All prompts were delivered in a “zero\-shot” format, without supplementary instructions or system\-level hints\.

Control for hallucination and inference integrity: AI hallucination, the tendency of a model to generate responses that are fluent and coherent yet factually inaccurate or entirely fabricated, remains a documented challenge in the field of LLMs\(Huanget al\.,[2024](https://arxiv.org/html/2608.14631#bib.bib32)\)\. Research suggests that vague or ambiguous inputs frequently exacerbate hallucination by forcing LLMs to rely probabilisticaly on their training data rather than factual synthesis\. Using clear, consistent, and unambiguous questions and eliminating iterative dialogue commonly observed in web\-based chatbot interactions were therefore an intentional methodological safeguard against this risk\.

Assessment of intrinsic knowledge vs\. web retrieval: A critical distinction was made between a model’s intrinsic parameter\-based knowledge and its ability to perform real\-time information retrieval\. Consequently, all API interactions were configured to disable web\-access functionality\. This design choice ensured that responses were derived exclusively from the models’ pre\-trained corpora, frozen at their respective knowledge cutoff dates\. This restriction served as a control against the potential for Retrieval\-Augmented Generation \(RAG\) to obscure the true reasoning capabilities of the underlying model architectures\(Lewiset al\.,[2020](https://arxiv.org/html/2608.14631#bib.bib4)\)\.

### 2\.4Data Analysis

LLM outputs were extracted programmatically using Python and organized into a summary table of accuracy percentage for each cosmetic chemistry query and for each model\. Accuracy for each query was calculated as the number of correct responses divided by the total number of ingredients or products queried\. To examine whether parameter of LLMs predicts accuracy, a simple linear regression was conducted using theSciPylibrary, with statistical significance defined asp<0\.05p<0\.05\. For the qualitative questions, response content was analyzed using word clouds using theWordCloudPython library\. This is a well\-established visualization technique in which word size reflects frequency of occurrence, providing an intuitive overview of the most prominent themes across model responses\(Heimerlet al\.,[2014](https://arxiv.org/html/2608.14631#bib.bib36)\)\. All Python code used in these analyses is publicly available at the corresponding author’s GitHub repository\.

![Refer to caption](https://arxiv.org/html/2608.14631v1/q1result.jpg)\(a\)Topic 1: Calculating Sigma and Pi Bonds
![Refer to caption](https://arxiv.org/html/2608.14631v1/q2result.jpg)\(b\)Topic 2: Preservatives Identification
![Refer to caption](https://arxiv.org/html/2608.14631v1/q3result.jpg)\(c\)Topic 3: Calculating the Molecular Weight
![Refer to caption](https://arxiv.org/html/2608.14631v1/q4result.jpg)\(d\)Topic 4: Boiling Point by the Joback Method

Figure 1:Relationship between LLM accuracy and model parameter scale\.

## 3Main Results

### 3\.1Calculating Sigma and Pi Bonds in Cosmetic Ingredients

Sigma and pi bonds form the foundational framework of molecular structure and play a central role in determining the physical and chemical properties of compounds\(Careyet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib37)\)\. Each LLM was asked: “In one simple sentence, state the number of sigma and pi bonds in\{ingredient\}”\. Five cosmetic ingredients were selected, spanning simple and complex molecular structures to probe how each model handles varying levels of chemical complexity\. Bond counts for all five ingredients were independently verified through manual analysis and cross\-referenced against NCBI PubChem databases\(Kimet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib27)\)\. The verified ground\-truth values are detailed in Appendix[B\.1](https://arxiv.org/html/2608.14631#A2.SS1)\.

The relationship between model scale and accuracy is visualized in Figure[1\(a\)](https://arxiv.org/html/2608.14631#S2.F1.sf1)\. For this metric, a binary “all\-or\-nothing” scoring system was applied: a LLM scores 1 only if both bond counts were precisely correct\. Only five models, the GPT\-5 series, the Grok\-4 series, and Gemini 2\.5 Pro, maintained perfect accuracy across all stimuli\. While the data suggests a general upward trajectory, implying that raw parameter scaling bolsters chemical reasoning, the slope of the regression fell just short of traditional statistical significance \(p=0\.058p=0\.058\)\. This near\-miss is philosophically telling; it suggests that while brute\-force scaling improves performance, it does not inherently bridge the ”reasoning gaps” found in specialized scientific domains\.

The performance of non\-U\.S\.\-based models was particularly noteworthy and perhaps a bit sobering\. Despite its massive architectural footprint, Mistral Large failed to produce a single correct response \(0%0\\%accuracy\), while DeepSeek\-R1 achieved only40%40\\%\. This disparity invites a critical look at potential ”corpus bias\.” Although English is the lingua franca of modern science, models trained on European \(Mistral\) or Chinese \(DeepSeek\) dominant datasets may utilize tokenization strategies or training weights that do not map seamlessly to English\-centric IUPAC nomenclature and bonding conventions\(Favre and Powell,[2014](https://arxiv.org/html/2608.14631#bib.bib3)\)\. This disparity warrants further investigation into how different development environments shape a model’s internalization of Western chemical standards\.

### 3\.2Identification of Synthetic Preservatives in Cosmetic Formulations

Ingredient transparency is increasingly important to consumers, particularly those with sensitivities to specific preservatives who need to know precisely what a product contains\(Lundovet al\.,[2009](https://arxiv.org/html/2608.14631#bib.bib46)\)\. Each LLM was therefore asked: ”In one sentence, list all the synthetic preservatives found in\{this product\}\.” Five commercially available products were selected, covering a toner, cleanser, facial cream, foundation, and lotion\. Correct responses were verified by cross\-referencing each product’s official ingredient list with the INCIDecoder database\(INCIDecoder,[2025](https://arxiv.org/html/2608.14631#bib.bib40)\), a research\-backed resource that classifies cosmetic ingredients by function, safety profile, and irritancy potential\. Full product details and verified preservative profiles are provided in Appendix[B\.2](https://arxiv.org/html/2608.14631#A2.SS2)\.

A response was scored as accurate only when all recognized synthetic preservatives in the product were correctly identified\. As shown in Figure[1\(b\)](https://arxiv.org/html/2608.14631#S2.F1.sf2), accuracy rates were generally low across the 14 LLMs, suggesting that most models struggled to reliably identify synthetic preservatives in real cosmetic formulations\. Regression analysis revealed no significant relationship between parameter count and task performance, with uniform deficits observed across all model scales \(p=0\.34p=0\.34\)\. Since web access was disabled via the API, all models had to rely solely on their pre\-existing training knowledge\(Gaoet al\.,[2024](https://arxiv.org/html/2608.14631#bib.bib2)\), which implies a limitation in specialized domains that training data may not comprehensively capture product\-specific ingredient information\. The low accuracy rates observed here likely reflect gaps in training data coverage rather than a fundamental deficit in reasoning\. This interpretation is supported by supplementary testing using web\-enabled LLM interfaces\. The results were stark: Google Gemini, for example, achieved100%100\\%accuracy when granted real\-time internet access\. The results suggest that real\-time retrieval, rather than parametric knowledge alone, accounts for much of the performance gap\.

### 3\.3Calculating the Molecular Weight of Cosmetic Ingredients

Molecular weight is a foundational concept in general chemistry, calculated by summing the atomic weights of all atoms in a molecule\(Zumdahlet al\.,[2018](https://arxiv.org/html/2608.14631#bib.bib42)\)\. In cosmetics, it plays a meaningful role in determining how an ingredient interacts with the skin and influences the viscosity and sensory feel of a formulation\(Walters and Roberts,[2002](https://arxiv.org/html/2608.14631#bib.bib43); Rosen,[2015](https://arxiv.org/html/2608.14631#bib.bib11)\)\. Each LLM was asked: ”In one sentence, provide the molecular weight of\{ingredient\}, rounded to one decimal place\.” Five commonly used cosmetic ingredients were selected, including Squalane, Octane, Dodecane, Isopropyl Alcohol, and Chlorphenesin, with their molecular weights independently verified through manual calculation and cross\-referencing against the NCBI PubChem database\(Kimet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib27)\)\. The confirmed ground\-truth values are provided in Appendix[B\.3](https://arxiv.org/html/2608.14631#A2.SS3)\.

This is a relatively straightforward calculation for anyone with a basic chemistry background, and the results largely reflected that\. Six out of 14 LLMs achieved 100% accuracy, and the remaining models correctly identified 4 out of 5 molecular weights \(see Figure[1\(c\)](https://arxiv.org/html/2608.14631#S2.F1.sf3)\), an encouraging outcome overall, though it does suggest that even simple numerical tasks are not entirely error\-free across all models\. Notably, parameter count did not predict performance, as both smaller and larger models achieved perfect accuracy \(p=0\.58p=0\.58\), pointing to training data quality and architectural design as more meaningful determinants than model scale alone\. Performance on this task was considerably better than on the synthetic preservative identification task, reinforcing the broader pattern that LLMs tend to be more reliable for structured calculations grounded in widely available chemical data than for recalling specific product\-level information that may be sparsely represented in their training corpora\.

### 3\.4Calculating the Boiling Point of Cosmetic Ingredients using the Joback Method

Boiling points have direct practical relevance in cosmetic formulation, influencing product stability, texture, and performance during both manufacturing and skin application\(Walters and Roberts,[2002](https://arxiv.org/html/2608.14631#bib.bib43); Rosen,[2015](https://arxiv.org/html/2608.14631#bib.bib11)\)\. To probe whether LLMs can move beyond simple data retrieval into genuine structural reasoning, each model was asked: ”In one sentence, calculate the normal boiling point of\{ingredient\}in Kelvin using the Joback method and provide the answer as a whole number\.” Five ingredients were selected: Squalane, Propylene Glycol, Dodecane, Isopropyl Alcohol, and Chlorphenesin\.

The Joback method was chosen because it is a deterministic group\-contribution technique that estimates thermophysical properties solely from a molecule’s functional groups and architecture\(Joback and Reid,[1987](https://arxiv.org/html/2608.14631#bib.bib41)\)\. Experimentally measured boiling points are unavailable for Squalane and Chlorphenesin due to their high molecular weights or thermal decomposition prior to boiling; the Joback method therefore provides a verifiable ground truth\. Benchmark values were derived using the DDBST Online Property Estimation tool and cross\-verified through manual calculation\(DDBST GmbH,[2025](https://arxiv.org/html/2608.14631#bib.bib45)\); full details are provided in the Appendix[B\.4](https://arxiv.org/html/2608.14631#A2.SS4)\.

Answering correctly required a multi\-step process: decomposing each molecule into its constituent functional groups, retrieving the corresponding Joback contribution values, executing the summation, and applying the appropriate unit conversion and rounding\. Despite the prescriptive nature of the task, the LLMs demonstrated a systemic inability to perform these operations\. As shown in Figure[1\(d\)](https://arxiv.org/html/2608.14631#S2.F1.sf4), 50% of the models \(7 out of 14\) failed to return a single correct value\. Only two models, GPT\-5 and Grok\-4, attained a peak accuracy of 60%\. As observed in previous tasks, model scale was not a predictor of success \(p=0\.14p=0\.14\); larger “frontier” models struggled as significantly as their smaller counterparts\.

A particularly notable pattern was the tendency of several models to hallucinate Joback group contribution constants, suggesting that while LLMs can often retrieve general chemical facts, they struggle with algorithmic tasks requiring precise structural decomposition and sequential arithmetic\. In applied contexts such as thermal safety assessment or formulation stability prediction, these findings underscore the necessity of independent human verification of any LLM\-generated thermophysical data\.

### 3\.5Qualitative Analysis of LLM Responses to Skincare Scenarios

Beyond quantitative tasks, the study evaluated how well LLMs handle the practical skincare questions that everyday consumers are likely to ask\. Two qualitative topics were assessed: sequential skincare advice for individuals with sensitive and acne\-prone skin, and first\-step recommendations for treating common skin concerns\. These topics were chosen because they reflect frequently encountered consumer questions and offer a meaningful window into whether LLMs can provide guidance consistent with established dermatological principles\. While such questions are ideally directed to a dermatologist, specialist access is not always practical, and LLMs have shown growing promise in bridging the gap between consumers and professional skincare guidance\(Ferreiraet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib18)\)\.

Since these topics are inherently qualitative, responses were assessed for alignment with broadly accepted skincare principles rather than scored against a single correct answer\. Trigram analysis \(three\-word phrase frequency\) was applied across all LLM responses for each topic, with word cloud visualizations used to highlight areas of consensus and divergence\(Heimerlet al\.,[2014](https://arxiv.org/html/2608.14631#bib.bib36)\)\.

The results are shown in Figures[2\(a\)](https://arxiv.org/html/2608.14631#S3.F2.sf1)and[2\(b\)](https://arxiv.org/html/2608.14631#S3.F2.sf2)\. For the first topic, the recommended next step for sensitive, acne\-prone skin, LLMs’ responses following cleansing consistently emphasized gentle handling and barrier protection, with common trigrams including “pat skin dry,” “gently pat skin,” and “skin gentle toner\.” After azelaic acid application, responses were more varied but centered on hydration, referencing “gentle moisturizer hydrate,” “hydrate soothe skin,” and “hyaluronic acid\.” Responses following toothpaste application showed little consensus, though washing with lukewarm water was the most commonly mentioned step\. After popping a pimple, common themes included “reduce inflammation,” “gently cleanse area,” and “spot treatment\.” Following nighttime moisturizing, the most frequent trigrams referenced occlusive layering, while responses after witch hazel application emphasized barrier support and humectants such as ceramides and hyaluronic acid\.

For the second topic, first\-step recommendations for common skin concerns, responses were notably more consistent\. Sun protection dominated recommendations for both pigmentary changes and anti\-aging concerns, with repeated references to daily sunscreen use and the prevention of further darkening\. For acne and excess oil, LLMs consistently recommended gentle cleansing without compromising skin integrity\.

Notably, most LLMs appended explicit disclaimers recommending consultation with a board\-certified dermatologist, a meaningful signal that these models recognize the boundaries of medical guidance\. The broad framing of the questions also appeared to naturally discourage prescriptive, personalized recommendations, given the diversity of individual skin types, medical histories, and sensitivities\. Taken together, these observations suggest that while LLMs fall short on precise chemical reasoning, they have been meaningfully aligned to operate within a safety\-conscious framework when addressing general consumer skincare questions\.

![Refer to caption](https://arxiv.org/html/2608.14631v1/q5.jpg)\(a\)Sequential skincare advice
![Refer to caption](https://arxiv.org/html/2608.14631v1/q6.jpg)\(b\)First\-step recommendations for skin concerns

Figure 2:N\-gram analysis \(N=3N=3\) of qualitative LLM responses on Topic 5 regarding consumer skincare advice\. The size of trigrams in the word cloud is proportional to their frequency of occurrence across the 14 LLMs\.

## 4Discussion and Conclusions

### 4\.1Discussion

Can LLMs reliably bridge the information gap between complex cosmetic chemistry and the general public? Our findings suggest that the breadth of their training does not necessarily translate into deep mastery of specialized scientific concepts\. A clear and consistent pattern emerged: strong performance on familiar, retrieval\-based tasks, and significant failure on those requiring structural reasoning or algorithmic precision\.

The Disparity Between Retrieval and Reasoning\. LLMs performed well on deterministic tasks such as calculating molecular weights and identifying sigma and pi bonds — tasks where the answer is a direct lookup from well\-represented training data\. Performance dropped sharply, however, when the Joback method was required, a technique that demands functional group decomposition, parameter retrieval, and sequential arithmetic\. This points to a meaningful architectural gap: these models can retrieve chemical constants with reasonable confidence, but they are not yet reliably capable of applying them through multi\-step structural reasoning\. The persistent difficulty in identifying synthetic preservatives compounds this concern, as ingredient transparency is increasingly important for consumers navigating sensitivities and health\-conscious purchasing decisions\(Lundovet al\.,[2009](https://arxiv.org/html/2608.14631#bib.bib46)\)\.

By contrast, LLMs performed considerably better on general skincare topics, providing broadly sensible guidance across a range of consumer scenarios\. This likely reflects the relative abundance of skincare\-related content in their training corpora\(Zhaoet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib47)\)\. These responses were, however, general in nature and lacked the clinical specificity needed for personalized dermatological guidance\.

LLM Commercialization and Consumer Safety\. Although most LLMs appropriately included disclaimers advising users to seek professional guidance, the confident, authoritative tone these models tend to adopt may still lead consumers to place unwarranted trust in overly broad answers\(Ferreiraet al\.,[2023](https://arxiv.org/html/2608.14631#bib.bib18)\)\. This reflects a broader concern about the incentive structures shaping LLM development\. In a competitive commercial landscape, responsiveness and perceived helpfulness may be prioritized over scientific rigor\.Ouyanget al\.\([2022](https://arxiv.org/html/2608.14631#bib.bib48)\)demonstrated that fine\-tuning models to follow instructions more agreeably can degrade performance on strictly logical tasks, andFanouset al\.\([2025](https://arxiv.org/html/2608.14631#bib.bib49)\)found that sycophantic behavior, where models adjust responses to align with user expectations, is measurable and widespread across leading LLMs, posing genuine risks in domains where accuracy matters\.

Hallucination and the Illusion of Confidence\. A recurring pattern across experiments was the tendency of LLMs to generate plausible\-sounding but factually incorrect numerical or chemical data\. This is particularly concerning in cosmetic and skincare contexts, where consumers often interpret confident responses as reliable ones\. WhileKimet al\.\([2025](https://arxiv.org/html/2608.14631#bib.bib50)\)attribute medical hallucinations primarily to failures in temporal reasoning, the hallucinations observed here appear to have a more foundational cause, namely, gaps in domain\-specific training data, as seen with synthetic preservatives, and an inability to execute structured chemical algorithms with precision\.

The Role of Model Scale and Architecture\. A notable finding was that model scale did not reliably predict performance\. In several instances, smaller “flash” or “mini” models matched or outperformed their larger counterparts, consistent with the scaling laws proposed byHoffmannet al\.\([2022](https://arxiv.org/html/2608.14631#bib.bib26)\), who emphasize that training data quality and architectural efficiency are as consequential as raw parameter count\. Even with carefully standardized single\-turn prompts, performance remained inconsistent across models, pointing to differences in training data composition and optimization objectives as the more meaningful sources of variability\.

### 4\.2Conclusions and Recommendations

LLMs have become capable and widely accessible information tools, but they are not yet reliable sources for cosmetic chemistry or precision skincare guidance\. The gap between their fluency in general advice and their limitations in rigorous chemical reasoning creates a superficial appearance of expertise that could genuinely mislead consumers\. Two practical recommendations follow from these findings\.

- •For AI developers, there is a compelling case for prioritizing scientific grounding over conversational agreeableness\. Incorporating specialized chemical datasets, domain\-adaptive pre\-training, and clearer internal distinctions between evidence\-based science and anecdotal content would meaningfully improve reliability in this domain\(Caoet al\.,[2025](https://arxiv.org/html/2608.14631#bib.bib51)\)\.
- •For consumers, LLM\-generated information is best treated as a starting point rather than a final answer\. For questions involving synthetic preservatives, complex formulations, or clinical skin conditions, the judgment of a qualified cosmetic chemist or dermatologist remains indispensable\.

Until LLMs can more reliably connect coherent language to rigorous chemical reasoning, their role in cosmetic chemistry should remain supplementary\. In a field where precise chemical details carry real implications for consumer health and product safety, accuracy should never be compromised for the sake of a more conversational user experience\.

## Acknowledgments

The author acknowledges the assistance of ChatGPT, Gemini, and Claude in refining English stylistics and enhancing grammar\.

## References

- Leveraging prompt engineering in large language models for accelerating chemical research\.ACS Omega\.External Links:[Link](https://pmc.ncbi.nlm.nih.gov/articles/PMC12022906/)Cited by:[1st item](https://arxiv.org/html/2608.14631#S4.I1.i1.p1.1)\.
- F\. A\. Carey, R\. M\. Giuliano, N\. T\. Allison, and S\. L\. Bane \(2023\)Organic chemistry\.12th edition,McGraw Hill\.Cited by:[§3\.1](https://arxiv.org/html/2608.14631#S3.SS1.p1.1)\.
- A\. Carrara \(2026\)AI is strengthening its grip on beauty in 2026: here’s why…\.Note:Cosmetics BusinessExternal Links:[Link](https://cosmeticsbusiness.com/ai-is-strengthening-its-grip-on-beauty)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p3.1)\.
- DDBST GmbH \(2025\)Online property estimation — normal boiling point \[computational tool\]\.Note:Dortmund Data Bank Software & Separation TechnologyExternal Links:[Link](http://ddbonline.ddbst.de/OnlinePropertyEstimation/OnlinePropertyEstimation.exe)Cited by:[§3\.4](https://arxiv.org/html/2608.14631#S3.SS4.p2.1)\.
- A\. Fanous, J\. Goldberg, A\. A\. Agarwal, J\. Lin, A\. Zhou, R\. Daneshjou, and S\. Koyejo \(2025\)SycEval: evaluating llm sycophancy\.InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society \(AIES ’25\),External Links:[Document](https://dx.doi.org/10.1145/3735632)Cited by:[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p4.1)\.
- H\. A\. Favre and W\. H\. Powell \(2014\)Nomenclature of organic chemistry: iupac recommendations and preferred names 2013\.Royal Society of Chemistry,Cambridge\.External Links:[Document](https://dx.doi.org/10.1039/9781849733069),ISBN 978\-0\-85404\-182\-4Cited by:[§3\.1](https://arxiv.org/html/2608.14631#S3.SS1.p3.2)\.
- A\. L\. Ferreira, B\. Chu, J\. M\. Grant\-Kels, T\. Ogunleye, and J\. B\. Lipoff \(2023\)Evaluation of chatgpt dermatology responses to common patient queries\.JMIR Dermatology6,pp\. e49280\.External Links:[Document](https://dx.doi.org/10.2196/49280)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p4.1),[§1](https://arxiv.org/html/2608.14631#S1.p5.1),[§3\.5](https://arxiv.org/html/2608.14631#S3.SS5.p1.1),[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p4.1)\.
- Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. Wang \(2024\)Retrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.10997\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.10997),[Link](https://arxiv.org/abs/2312.10997)Cited by:[§3\.2](https://arxiv.org/html/2608.14631#S3.SS2.p2.2)\.
- C\. L\. Goh, Y\. Wu, B\. Welsh, N\. Kerrouche, C\. Lu, X\. Wang, M\. Sangueza, J\. B\. Lee, P\. See, and H\. N\. Kang \(2023\)Expert consensus on holistic skin care routine: focus on acne, rosacea, atopic dermatitis, and sensitive skin syndrome\.Journal of Cosmetic Dermatology22\(1\),pp\. 45–54\.External Links:[Document](https://dx.doi.org/10.1111/jocd.15519)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p3.1)\.
- P\. Goktas and A\. Grzybowski \(2024\)Assessing the impact of ChatGPT in dermatology: A comprehensive rapid review\.Journal of Clinical Medicine13\(5\),pp\. 1336\.External Links:[Document](https://dx.doi.org/10.3390/jcm13051336)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p4.1)\.
- T\. Guo, K\. Guo, B\. Nan, Z\. Liang, Z\. Guo, N\. V\. Chawla, O\. Wiest, and X\. Zhang \(2023\)What can large language models do in chemistry? a comprehensive benchmark on eight tasks\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 59662–59688\.External Links:[Link](https://doi.org/10.48550/arXiv.2305.18365)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p5.1)\.
- F\. Heimerl, S\. Lohmann, S\. Lange, and T\. Ertl \(2014\)Word cloud explorer: text analytics based on word clouds\.InProceedings of the 47th Hawaii International Conference on System Sciences \(HICSS\),pp\. 1833–1842\.External Links:[Document](https://dx.doi.org/10.1109/HICSS.2014.231)Cited by:[§2\.4](https://arxiv.org/html/2608.14631#S2.SS4.p1.1),[§3\.5](https://arxiv.org/html/2608.14631#S3.SS5.p2.1)\.
- J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. de Las Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark,et al\.\(2022\)Training compute\-optimal large language models\.arXiv preprint arXiv:2203\.15556\.External Links:[Link](https://arxiv.org/abs/2203.15556)Cited by:[§2\.1](https://arxiv.org/html/2608.14631#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p6.1)\.
- Hostinger \(2025\)LLM statistics: key facts and figures for 2025–2026\.Note:Hostinger International Ltd\.External Links:[Link](https://www.hostinger.com/tutorials/llm-statistics)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p1.1)\.
- L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin, and T\. Liu \(2024\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems1\(1\)\.External Links:[Document](https://dx.doi.org/10.1145/3703155)Cited by:[§2\.3](https://arxiv.org/html/2608.14631#S2.SS3.p3.1)\.
- INCIDecoder \(2025\)INCIDecoder: decode your skincare ingredients \[online cosmetic ingredient database\]\.External Links:[Link](https://incidecoder.com/)Cited by:[§3\.2](https://arxiv.org/html/2608.14631#S3.SS2.p1.1)\.
- K\. G\. Joback and R\. C\. Reid \(1987\)Estimation of pure\-component properties from group\-contributions\.Chemical Engineering Communications57\(1–6\),pp\. 233–243\.External Links:[Document](https://dx.doi.org/10.1080/00986448708960487)Cited by:[4th item](https://arxiv.org/html/2608.14631#S2.I1.i4.p1.1),[§3\.4](https://arxiv.org/html/2608.14631#S3.SS4.p2.1)\.
- S\. Kim, J\. Chen, T\. Cheng, A\. Gindulyte, J\. He, S\. He, Q\. Li, S\. Shoemaker,et al\.\(2023\)PubChem 2023 update\.Nucleic Acids Research51\(D1\),pp\. D1373–D1380\.External Links:[Document](https://dx.doi.org/10.1093/nar/gkac956)Cited by:[§2\.2](https://arxiv.org/html/2608.14631#S2.SS2.p5.1),[§3\.1](https://arxiv.org/html/2608.14631#S3.SS1.p1.1),[§3\.3](https://arxiv.org/html/2608.14631#S3.SS3.p1.1)\.
- Y\. Kim, H\. Jeong, S\. Chen, S\. S\. Li, M\. Lu, K\. Alhamoud, J\. Mun, C\. Grau, M\. Jung, R\. Gameiro,et al\.\(2025\)Medical hallucinations in foundation models and their impact on healthcare\.medRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.28.25323115)Cited by:[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p5.1)\.
- H\. E\. Leon and A\. S\. Pimentel \(2023\)Do large language models understand chemistry? a conversation with chatgpt\.Journal of Chemical Information and Modeling63\(6\),pp\. 1649–1655\.External Links:[Document](https://dx.doi.org/10.1021/acs.jcim.3c00285)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p5.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/file/6ad8b6d035414619ca24844391696417-Paper.pdf)Cited by:[§2\.3](https://arxiv.org/html/2608.14631#S2.SS3.p4.1)\.
- M\. D\. Lundov, L\. Moesby, C\. Zachariae, and J\. D\. Johansen \(2009\)Contamination versus preservation of cosmetics: a review on legislation, usage, infections, and contact allergy\.Contact Dermatitis60\(2\),pp\. 70–78\.External Links:[Document](https://dx.doi.org/10.1111/j.1600-0536.2008.01501.x)Cited by:[§3\.2](https://arxiv.org/html/2608.14631#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p2.1)\.
- H\. Naveed, A\. U\. Khan, S\. Qiu, M\. Saqib, S\. Anwar, M\. Usman, N\. Akhtar, N\. Barnes, and A\. Mian \(2023\)A comprehensive overview of large language models\.arXiv preprint arXiv:2307\.06435\.External Links:[Link](https://doi.org/10.48550/arXiv.2307.06435)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p1.1),[§1](https://arxiv.org/html/2608.14631#S1.p2.1)\.
- OpenAI \(2025\)OpenAI python library \[software\]\.External Links:[Link](https://pypi.org/project/openai/)Cited by:[§2\.3](https://arxiv.org/html/2608.14631#S2.SS3.p1.1)\.
- OpenAI \(2026\)Scaling ai for everyone\.External Links:[Link](https://openai.com/index/scaling-ai-for-everyone/)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p2.1)\.
- OpenRouter \(2025\)OpenRouter: a unified interface for llms \[api platform documentation\]\.External Links:[Link](https://openrouter.ai/docs)Cited by:[§2\.3](https://arxiv.org/html/2608.14631#S2.SS3.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 27730–27744\.External Links:[Link](https://arxiv.org/abs/2203.02155)Cited by:[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p4.1)\.
- M\. R\. Rosen \(Ed\.\) \(2015\)Harry’s cosmeticology, 9th edition: focus books — handbook of skin anti\-aging theories for cosmetic formulation development\.Vol\.1,Chemical Publishing Co\.\.Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.14631#S2.SS2.p5.1),[§3\.3](https://arxiv.org/html/2608.14631#S3.SS3.p1.1),[§3\.4](https://arxiv.org/html/2608.14631#S3.SS4.p1.1)\.
- Toloka AI \(2025\)History of llms: complete timeline & evolution \(1950–2026\)\.Note:Toloka BlogExternal Links:[Link](https://toloka.ai/blog/history-of-llms/)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p2.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\. 5998–6008\.External Links:[Link](https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2608.14631#S1.p2.1)\.
- K\. A\. Walters and M\. S\. Roberts \(2002\)The structure and function of skin\.InDermatological and Transdermal Formulations,K\. A\. Walters \(Ed\.\),pp\. 1–40\.Cited by:[§3\.3](https://arxiv.org/html/2608.14631#S3.SS3.p1.1),[§3\.4](https://arxiv.org/html/2608.14631#S3.SS4.p1.1)\.
- W\. X\. Zhao, K\. Zhou, J\. Li, T\. Tang, X\. Wang, Y\. Hou, Y\. Min, B\. Zhang, J\. Zhang, Z\. Dong,et al\.\(2023\)A survey of large language models\.arXiv preprint arXiv:2303\.18223\.External Links:[Link](https://arxiv.org/abs/2303.18223)Cited by:[§4\.1](https://arxiv.org/html/2608.14631#S4.SS1.p3.1)\.
- S\. S\. Zumdahl, S\. A\. Zumdahl, and D\. J\. DeCoste \(2018\)Chemistry\.10th edition,Cengage Learning\.Cited by:[§3\.3](https://arxiv.org/html/2608.14631#S3.SS3.p1.1)\.

## Appendix AAppendix: Experimental Prompts and Target Stimuli

### A\.1Quantitative Inquiries

- •Bond Analysis:“In one simple sentence, state the number of sigma and pi bonds in \[Ingredient\]\.”
- •Preservative Identification:“In one sentence, list all the synthetic preservatives found in \[Product Name\]\.”
- •Molecular Weight:“In one sentence, provide the molecular weight of \[Ingredient\], rounded to one decimal place\.”
- •Joback Calculation:“In one sentence, calculate the normal boiling point of \[Ingredient\] in Kelvin using the Joback method and provide the answer as a whole number\.”

### A\.2Qualitative Scenarios

- •Sequential skincare advice:“In one sentence, briefly describe the immediate next skincare step after using/applying \[Scenario\] for a person with sensitive and acne\-prone skin\.”
- •First\-step recommendations:“In one sentence, briefly describe the immediate first skincare step for treating \[Scenario\] on the face\.”

## Appendix BAppendix: Correct Answers for Quantitative Inquiries

### B\.1Calculating Sigma and Pi Bonds

Table 2:Manual Calculation of sigma and pi Bond Counts for Selected Cosmetic Ingredients\.Note:Values were independently verified via structural decomposition and cross\-referenced with NCBI chemical databases\.

### B\.2Identification of Synthetic Preservatives

Table 3:Identification of Synthetic Preservative Systems in Selected Commercial Formulations\.Note:Ingredient profiles were cross\-referenced against official manufacturer disclosures and the INCIDecoder database\.

### B\.3Calculating Molecular Weight

Table 4:Molecular Weight Comparison of Selected Cosmetic Ingredients\.Note:Values were independently verified through stoichiometric summation of standard atomic weights and cross\-referenced against the PubChem database\.

### B\.4Calculating Normal Boiling Point using the Joback Method

Table 5:Estimated Normal Boiling Points of Cosmetic Ingredients via the Joback Method\.Note:\*Boiling points \(TbT\_\{b\}\) calculated using the Joback method:Tb=198\+∑Δ​Tb,iT\_\{b\}=198\+\\sum\\Delta T\_\{b,i\}\. Values rounded to the nearest whole number\.