Synthetic Data Generator

Part of the worlds-biggest-software-project initiative.

AI-generated realistic synthetic datasets for testing and training, with privacy guarantees and natural-language scenario specification.

An open-source synthetic data platform for ML engineers, QA and DevOps teams, and data scientists in regulated industries who cannot use production data for model development or test environments. It generates statistically faithful tabular, relational, and time-series datasets with built-in PII handling, privacy risk scoring, and audit-ready documentation.

Why Synthetic Data Generator?

The defining incumbent — Gretel — was acquired by NVIDIA in March 2025 for ~$320M and integrated into the NeMo Data Designer / GPU ecosystem, leaving non-GPU users with higher operational complexity and a roadmap focused on LLM training data.
Enterprise platforms (MOSTLY AI, Tonic.ai, K2view, Hazy) use opaque value-based or volume-based pricing, with regulated-industry contracts running $50k–$500k+/yr — inaccessible to SMEs and developer teams.
Existing open-source options are narrow: SDV lacks production-scale tooling, PII detection, differential privacy, and a UI; Faker/Mimesis do not preserve statistical distributions.
No current tool offers natural-language constraint specification ("10,000 transactions with a 3% fraud rate following Q4 seasonal patterns") without manual schema configuration.
CI/CD-native lightweight tooling that integrates with GitHub Actions / GitLab CI without enterprise procurement is an unmet need identified across the competitive landscape.

Key Features

Core Generation

Tabular synthetic data generation for single-table and multi-table relational datasets, preserving referential integrity
Time-series and sequential data support for transaction sequences and customer journeys
Constraint-based generation with logical business rules
Natural-language constraint input for specifying data scenarios without manual schema configuration
Export to CSV, JSON, and Parquet

Privacy and Compliance

PII detection and synthesis/redaction pipeline integrated into the generation workflow
Differential privacy with configurable epsilon/delta parameters and privacy budget tracking
Re-identification risk scoring with per-record risk flagging before dataset release
Privacy guarantee documentation aligned with GDPR Recital 26 and NIST SP 800-188
Automated audit-ready compliance report templates, including EU AI Act training data summaries

Quality and Reporting

Statistical fidelity metrics covering univariate and bivariate distribution comparisons
Automated data quality report comparing synthetic and source datasets
LLM-as-a-judge quality evaluation for generated outputs
Intelligent class imbalance detection with targeted minority-class augmentation

Developer Experience

Python SDK, REST API, and CLI
CI/CD pipeline integration for GitHub Actions and GitLab CI for automated test data provisioning
Multi-format output spanning structured data, nested JSON, and unstructured documents (PDF, DOCX)
Optional MCP server interface for AI agent-driven generation workflows

Extended Capabilities (Backlog)

LLM fine-tuning dataset generation (instruction-following, preference, RLHF formats)
Real-time synthetic data streaming output (Kafka-compatible)
Image and additional time-series modalities
On-premises / air-gapped deployment for regulated enterprise environments

AI-Native Advantage

AI is used to interpret natural-language scenario descriptions into schemas and constraints, removing the manual YAML/JSON configuration overhead that dominates incumbent workflows. Privacy risk is assessed automatically using techniques such as membership inference attack simulation, surfacing specific at-risk records before release. Domain-adaptive fine-tuning on small seed datasets produces statistically faithful synthetics without exposing full source data, and LLMs generate regulatory compliance narratives directly from statistical metrics.

Tech Stack & Deployment

The project is designed around a Python SDK with a REST API and CLI, suitable for embedding in ML pipelines and CI/CD workflows. Deployment modes target both local/self-hosted use and managed cloud, with on-premises / air-gapped options on the roadmap for regulated environments. Compliance alignment is grounded in GDPR Recital 26, NIST SP 800-188, ISO/IEC 27701, HIPAA Safe Harbour / Expert Determination, and EU AI Act technical documentation requirements.

Market Context

The synthetic data market is estimated at approximately $710–$791 million in 2026, up from $510 million in 2025, with projections ranging from $3.67B by 2031 (Mordor Intelligence, 38.96% CAGR) to $6.9B by 2034 (31% CAGR) [research.md]. Model training accounts for roughly 46% of current usage, and Gartner projects synthetic structured data will grow at least 3x as fast as real structured data for AI training through 2030. Primary buyers are ML engineers, QA/DevOps teams, and data scientists in healthcare, finance, and insurance; enterprise contracts typically run $50k–$500k+/yr [research.md].

Project Status

This project is in the research and specification phase.
Contributions, feedback, and domain expertise are welcome.

Contributing

We welcome contributions from developers, domain experts, and potential users. See CONTRIBUTING.md for guidelines.

Important: All contributions must be your own original work or clearly attributed open-source material with a compatible licence. Copyright infringement and licence violations will not be tolerated and will result in immediate removal of the offending contribution. If you are unsure whether a piece of code, text, or other material is safe to contribute, open an issue and ask before submitting.

Note: K2view holds a patent on its Micro-Database entity architecture; contributors proposing entity-centric data management designs should conduct independent freedom-to-operate analysis. Avoid copying from SDV components carrying a Business Source Licence (BSL) without a DataCebo licence.

Licence

Licence to be determined. See discussion for context.