distilbert

This paper presents a hybrid model combining DistilBERT embeddings with Holographic Reduced Representation vectors encoding cognitive-linguistic features (first-person pronouns, absolutist words, negative emotion ratios) to detect depression in Reddit posts, achieving a macro F1 of 0.94 and demonstrating that theory-driven features complement contextual embeddings for explainable mental health NLP.

Trained a prompt injection classifier using ml-intern and DeepSeek V4 Flash, achieving 99% F1 with DistilBERT, optimized to ONNX int8 (~65MB) and deployable in the browser via Transformers.js v3.

This paper introduces Switchcraft, the first AI model router specifically optimized for agentic tool calling to reduce inference costs. By using a lightweight DistilBERT classifier, it achieves significant cost savings while maintaining high accuracy in tool-use tasks.

A comparative study evaluating three explainability techniques (Integrated Gradients, Attention Rollout, SHAP) on fine-tuned DistilBERT for sentiment classification, highlighting trade-offs between gradient-based, attention-based, and model-agnostic approaches for LLM interpretability.