data-preparation

DataFlow is an open-source tool with visual, low-code pipelines to generate, clean, and prepare high-quality LLM training datasets from raw data. It includes a technical report on arXiv.

DataFlow is an LLM-driven framework for automated data preparation and workflow engineering, featuring nearly 200 reusable operators and six domain-general pipelines that improve LLM performance across tasks like math, code, and Text-to-SQL.