Post

Categories of Synthetic Data Generation Models

Posted Updated
By
18 min read
Categories of Synthetic Data Generation Models

Guide

Table of Contents

Overview

This post provides a comprehensive overview of the main categories of synthetic data generation models, with a focus on their application to health and privacy-preserving data sharing. It covers statistical, machine learning, deep learning, diffusion, and LLM-based approaches, highlighting their strengths, limitations, and practical use cases.

Introduction

Synthetic data generation methods learn the statistical structure of the original dataset and then sample new records from the learned distribution. These records maintain key characteristics of the real data such as:

  • Marginal distributions of variables
  • Correlations between attributes
  • Complex interactions among features

However, synthetic records are not copies of real individuals, which reduces the risk of privacy disclosure.

Why Synthetic Data Is Useful in Healthcare

Direct sharing of patient data can expose sensitive information and lead to re-identification attacks, where individuals are identified using quasi-identifiers such as age, location, gender, or diagnosis codes.

Synthetic data addresses these challenges by enabling:

  • Privacy-Preserving Data Sharing: Synthetic datasets allow researchers to share information without exposing real patients.
  • Development and Testing of Machine Learning Models: AI models can be trained and validated using synthetic datasets when access to real data is restricted.
  • Reproducible Research: Synthetic data enables reproducibility without distributing sensitive data.
  • Simulation and Policy Analysis: Researchers can simulate scenarios using synthetic populations derived from healthcare datasets.
  • Evaluation of Privacy Risks: Synthetic data can also be used to simulate adversarial attacks and evaluate re-identification risk.

Categories of Synthetic Data Generation Models

Synthetic data generation models aim to learn (or approximate) the joint distribution of a dataset and then sample new records that preserve important statistical properties such as marginals, correlations, and conditional relationships. This section explains the main categories of models used in practice, especially for health tabular data, where we often deal with mixed feature types (continuous + categorical), missingness patterns, and complex dependencies.

1. Statistical Generative Models

Statistical models explicitly represent the data distribution (or parts of it). They are often fast, stable, and interpretable, which is why they’re widely used in healthcare settings for synthetic tabular data and synthetic population modeling.

1.1 Copulas (The “core” idea)

A copula is a function that “couples” marginal distributions into a joint distribution. It lets you model:

  • Marginals: $F_1(x_1), \ldots, F_d(x_d)$
  • Dependence structure separately: $C(\cdot)$

Key formula (Sklar’s Theorem): \(F(x_1, \ldots, x_d) = C(F_1(x_1), \ldots, F_d(x_d))\) Where:

  • $F$ is the joint CDF
  • $F_i$ are marginal CDFs
  • $C$ is the copula that captures dependence

Intuition (pipeline):

1
2
3
4
5
6
7
8
9

Real data X
   │
   ├─ Fit marginals F1..Fd  ───────────┐
   │                                  │
   └─ Transform each column: Ui = Fi(Xi)  (Uniform[0,1])
                                      │
                                      ├─ Fit copula C over U = (U1..Ud)
                                      │
                                      └─ Sample U~C  →  Xi_syn = Fi^{-1}(Ui_syn)

This is why copulas are powerful for healthcare: you can preserve realistic marginals (age distribution, lab value distributions, etc.) while capturing cross-feature dependencies.

1.2 Gaussian Copula Models

Gaussian copula models are the most common “classic” tabular synthesizers. They are popular in tools like SDV because they are fast and usually give strong baseline performance.

How Gaussian copula works (step-by-step):

  1. Transform each column into a uniform variable: $U_i = F_i(X_i)$
  2. Map uniform variables to a Gaussian space: $Z_i = \Phi^{-1}(U_i)$ ($\Phi^{-1}$ is the inverse CDF of standard normal)
  3. Estimate correlation matrix $\Sigma$ for $Z$
  4. Sample: $Z_{syn} \sim N(0, \Sigma)$
  5. Transform back: $U_{i,syn} = \Phi(Z_{i,syn}), X_{i,syn} = F_i^{-1}(U_{i,syn})$

SDV implementation (code):

1
2
3
4
5
6
7
8
9
10
11

import pandas as pd
from sdv.metadata import SingleTableMetadata
from sdv.single_table import GaussianCopulaSynthesizer

data = pd.read_csv("health_data.csv")
metadata = SingleTableMetadata()
metadata.detect_from_dataframe(data)

synth = GaussianCopulaSynthesizer(metadata)
synth.fit(data)
synthetic_data = synth.sample(num_rows=1000)

1.3 Vine Copula Models (e.g., D-vine, C-vine)

Vine copulas generalize copulas by modeling dependency as a composition of pairwise copulas organized in a tree structure (a “vine”). They are often better than Gaussian copulas when dependencies are nonlinear or high-dimensional.

Core idea: Instead of one global correlation matrix, vines build the joint distribution from many bivariate copulas:

  • Tree 1: captures strongest pairwise dependencies
  • Higher trees: capture conditional dependencies

Simple D-vine sketch:

1
2
3

Tree 1:  X1—X2—X3—X4
Tree 2:  (X1,X3|X2) — (X2,X4|X3)
Tree 3:  (X1,X4|X2,X3)

Python example (copulas library):

1
2
3
4
5
6
7

import pandas as pd
from copulas.multivariate import VineCopula

data = pd.read_csv("health_data.csv")
vine = VineCopula('center')  # common choice
vine.fit(data)
synthetic_data = vine.sample(1000)

1.4 Bayesian Networks

Bayesian networks represent variables as a directed acyclic graph (DAG) and generate data by sampling from learned conditional distributions. They are interpretable and useful when you want explicit conditional structure.

Factorization:

\[P(X_1, \ldots, X_d) = \prod_{i=1}^d P(X_i | Parents(X_i))\]

Python example (pgmpy):

1
2
3
4
5
6
7
8

import pandas as pd
from pgmpy.models import BayesianNetwork
from pgmpy.estimators import MaximumLikelihoodEstimator

data = pd.read_csv("health_data.csv")
bn = BayesianNetwork([('Age','Disease'), ('Disease','Treatment')])
bn.fit(data, estimator=MaximumLikelihoodEstimator)
synthetic = bn.simulate(n_samples=1000)

2. Machine Learning Models (Classical ML)

These methods use predictive models (trees, forests, boosting) to generate each column conditionally. They are common in statistical disclosure control tooling (e.g., sequential synthesis).

Sequential conditional synthesis: A common approach is: \(X_1 \sim P(X_1), X_2 \sim P(X_2|X_1), \ldots, X_d \sim P(X_d|X_{<d})\) Where each conditional is modeled by a classifier/regressor such as:

  • Decision Trees
  • Random Forests
  • Gradient Boosted Trees

Pros: often works well with mixed data + small samples
Cons: order-dependence; may struggle with global coherence at high dimension

3. Deep Learning Models

Deep generative models can capture complex nonlinear dependencies. For tabular data, the hardest parts are: mixed feature types, multimodal continuous columns, and imbalanced categorical columns.

3.1 GAN-based models

GAN structure (high-level):

  • $z \sim N(0, I) \rightarrow$ Generator $G(z, cond) \rightarrow$ synthetic row $\tilde{x}$
  • real row $x \rightarrow$ Discriminator $D(x) \rightarrow$ real/fake score
  • Train: $G$ tries to fool $D$; $D$ tries to detect fakes

CTGAN (Conditional Tabular GAN): CTGAN (Conditional Tabular GAN) is a generative model designed specifically to address the challenges of generating realistic tabular datasets with mixed data types and imbalanced categorical variables. It extends the standard Generative Adversarial Network (GAN) architecture by introducing conditional generation and specialized data transformations. In CTGAN, the generator learns to produce synthetic records while the discriminator attempts to distinguish between real and generated samples. The key innovation is the conditional vector mechanism, which forces the generator to produce samples conditioned on specific categorical values, enabling the model to better represent rare categories and imbalanced distributions. Additionally, CTGAN uses a mode-specific normalization technique for continuous variables, allowing the model to capture multimodal distributions often found in tabular datasets. During training, the model employs a training-by-sampling strategy that balances categories to prevent the model from ignoring minority classes. As a result, CTGAN typically produces high-fidelity synthetic data with realistic correlations between variables, making it one of the most widely used deep learning models for synthetic tabular data generation. CTGAN is also part of the SDV (Synthetic Data Vault) ecosystem and has been widely applied in domains such as healthcare, finance, and privacy-preserving data sharing.

SDV implementation (code):

1
2
3
4
5
6
7
8
9
10
11

import pandas as pd
from sdv.metadata import SingleTableMetadata
from sdv.single_table import CTGANSynthesizer

data = pd.read_csv("health_data.csv")
metadata = SingleTableMetadata()
metadata.detect_from_dataframe(data)

ctgan = CTGANSynthesizer(metadata)
ctgan.fit(data)
syn = ctgan.sample(1000)

TGAN (Tabular GAN): TGAN (Tabular Generative Adversarial Network) is one of the earliest deep learning architectures designed specifically for generating synthetic tabular datasets containing both continuous and categorical variables. Traditional GANs were originally developed for images, where spatial structure exists, but tabular data lacks such spatial correlations and often includes mixed data types. TGAN addresses this by introducing a special preprocessing pipeline and representation learning strategy that allows the generator to model complex relationships between columns. In TGAN, continuous variables are typically modeled using a Gaussian mixture representation, while categorical variables are generated using probability distributions learned by the model. The architecture still follows the classic GAN structure with a generator network that produces synthetic records and a discriminator that tries to distinguish real from fake samples, but it includes additional components that encode tabular features and preserve correlations between columns. Empirical studies show that TGAN can generate synthetic datasets that maintain statistical properties and feature relationships similar to the original data, making it useful for applications such as healthcare data sharing, model training, and data augmentation.

TableGAN: TableGAN is another GAN-based model designed specifically for tabular data generation, with a particular focus on preserving both data utility and privacy properties. Unlike early GAN approaches that treated tabular features independently, TableGAN incorporates an auxiliary classifier and additional loss functions to ensure that correlations between features and target labels are preserved in the generated data. The architecture includes three main neural components: a generator, a discriminator, and a classifier network. The generator produces synthetic rows, the discriminator evaluates whether the data is real or synthetic, and the classifier ensures that the generated data maintains meaningful relationships with the target variable in classification tasks. This structure helps ensure that machine learning models trained on synthetic data achieve performance similar to models trained on real datasets. TableGAN has been widely used in synthetic data research and is often evaluated alongside models such as CTGAN and CopulaGAN for tasks like intrusion detection data synthesis, healthcare data generation, and privacy-preserving data publishing.

CTAB-GAN: CTAB-GAN (Conditional Tabular GAN) is an advanced GAN-based model designed to address several major challenges in real-world tabular datasets, including mixed variable types, skewed distributions, and imbalanced categorical variables. Many real datasets—especially in domains such as healthcare, finance, and insurance—contain variables that have both continuous and categorical components or exhibit long-tailed distributions. CTAB-GAN improves upon earlier models by introducing conditional generation mechanisms and specialized encoding methods that allow the model to better represent these complex data patterns. The architecture includes a conditional generator, discriminator, and auxiliary classifier, which together help capture relationships between features and maintain realistic distributions of rare categories. The model also uses novel conditional vectors and sampling strategies to handle imbalanced data and prevent mode collapse during training. Experimental evaluations show that CTAB-GAN produces synthetic data with higher statistical similarity and machine-learning utility than earlier models such as TableGAN and CTGAN.

CopulaGAN (Hybrid: Copula + GAN): CopulaGAN is a hybrid synthetic data generation model that combines statistical copula transformations with GAN-based deep learning architectures. The motivation behind CopulaGAN is that traditional GANs struggle to learn tabular distributions when the marginal distributions of features vary widely or when strong correlations exist between variables. To address this, CopulaGAN first applies copula-based transformations that convert each variable into a uniform probability space, effectively separating marginal distributions from dependency structure. Once the variables are transformed, the GAN component learns the complex relationships between variables in this normalized space. During generation, the model samples synthetic points using the GAN and then applies inverse copula transformations to reconstruct realistic feature values in the original data domain. By combining probabilistic modeling with deep neural networks, CopulaGAN can capture both complex nonlinear dependencies and realistic marginal distributions, making it particularly effective for high-dimensional tabular datasets. This hybrid approach has become popular in synthetic data libraries such as SDV (Synthetic Data Vault) and is frequently compared against other GAN-based tabular models like CTGAN and TableGAN.

1
2
3
4
5
6
7
8
9
10
11

import pandas as pd
from sdv.metadata import SingleTableMetadata
from sdv.single_table import CopulaGANSynthesizer

data = pd.read_csv("health_data.csv")
metadata = SingleTableMetadata()
metadata.detect_from_dataframe(data)

cg = CopulaGANSynthesizer(metadata)
cg.fit(data)
syn = cg.sample(1000)

3.2 VAE-based models (TVAE)

VAEs are usually more stable than GANs.

VAE structure:

  • Encoder: $x \rightarrow (\mu, \sigma)$
  • Sample: $z = \mu + \sigma \odot \epsilon$
  • Decoder: $z \rightarrow \tilde{x}$
  • Loss: recon(x, x~) + KL(q(z|x) || p(z))

TVAE (Tabular Variational Autoencoder) For tabular data, a common baseline is TVAE. TVAE is a deep generative model specifically designed for tabular data synthesis based on the framework of Variational Autoencoder (VAE). In TVAE, the model learns a probabilistic latent representation of the tabular dataset by encoding input records into a lower-dimensional latent space and then decoding them back to reconstruct the original data distribution. The encoder network estimates the parameters of a latent distribution (typically Gaussian), while the decoder generates synthetic records by sampling from this latent space. During training, TVAE optimizes a loss function composed of reconstruction loss (ensuring the generated samples resemble the real data) and Kullback–Leibler divergence (regularizing the latent distribution). Unlike many GAN-based approaches, TVAE is relatively stable to train and handles mixed data types (continuous and categorical variables) by using specialized encoding schemes such as Gaussian mixture normalization and one-hot encoding. However, although TVAE can capture complex distributions and correlations between features, it may sometimes produce slightly blurred or less diverse samples compared to adversarial models. TVAE is implemented in the SDV (Synthetic Data Vault) framework and is commonly used in healthcare and financial datasets where stable training and probabilistic modeling are important.

SDV includes TVAE as a synthesizer option SDV TVAE synthesizer.

4. Diffusion Models for Tabular Data

Diffusion models have become very strong generative models. For tabular data, the challenge is mixed data types; modern approaches adapt diffusion to handle continuous + categorical variables. The core idea is to learn a denoising model that can reverse a gradual noise process. Although it often achieves high fidelity and effectively captures complex dependencies, it generally requires slower sampling and higher computational resources compared to GAN-based or copula-based models.

Diffusion process steps:

  • Forward process (adds noise):

    \[q(x_t|x_{t-1}) \approx \mathcal{N}((1-\beta_t)x_{t-1}, \beta_t I)\]

  • Reverse process (denoising):

    \[p_\theta(x_{t-1}|x_t)\]

  • Generation:

    Start from pure noise $x_T$ and iteratively denoise step-by-step down to $x_0$. Practical structure:

  • Preprocessing layer: continuous normalization, categorical encoding
  • Denoiser network: often an MLP adapted for tabular features
  • Diffusion schedule: noise schedule $\beta_t$
  • Sampling procedure: iterative denoising loop

TabDDPM (Diffusion for tabular): TabDDPM is a deep generative model designed for synthetic tabular data generation based on the framework of Denoising Diffusion Probabilistic Models (DDPM). Diffusion models generate data through a two-step process: a forward diffusion process, where noise is gradually added to the data over many time steps, and a reverse denoising process, where a neural network learns to progressively remove the noise to reconstruct realistic samples. In TabDDPM, this framework is adapted for mixed-type tabular datasets containing both continuous and categorical variables. Continuous features are typically modeled using Gaussian noise diffusion, while categorical variables are encoded using specialized embeddings or probabilistic transformations to allow diffusion-based learning. During training, the model learns the conditional probability of denoising the data at each step, enabling it to approximate the underlying data distribution more accurately than many GAN-based approaches. Compared to GAN models such as Generative Adversarial Network-based tabular generators, TabDDPM often provides more stable training and better coverage of complex feature dependencies, reducing issues like mode collapse. Because diffusion models generate samples through gradual refinement, they can capture intricate correlations between variables and produce high-quality synthetic datasets. As a result, TabDDPM has recently emerged as one of the state-of-the-art approaches for tabular synthetic data generation, particularly in domains such as healthcare and finance where accurate modeling of complex relationships is critical.

5. LLM-based Generators (Recent)

Large Language Models (LLMs) can be used in several different ways to generate synthetic tabular data, depending on how the model interacts with the dataset and whether it directly generates rows or assists other generative models. The main approaches can be grouped into the following categories.

5.1. Direct LLM-Based Tabular Generation (Prompt-Based Generation)

In this approach, an LLM directly generates synthetic rows of tabular data through prompting. The dataset schema, column descriptions, and example rows are provided in the prompt, and the LLM produces additional rows consistent with the structure. This method treats tabular data as structured text, allowing models such as GPT‑4 or Gemini to generate new records.

Characteristics:

  • No specialized generative model required
  • Works well for small datasets
  • Easy to control through prompt instructions

Limitations:

  • Difficult to maintain statistical distributions
  • Correlations between variables may be inaccurate

Typical workflow: Schema + Example Rows → Prompt → LLM → Generated Rows

5.2. LLM-Assisted Tabular Generation (LLM as a Controller)

In this approach, the LLM does not generate the data directly but instead guides another generative model (such as diffusion or GAN models). The LLM analyzes the dataset schema, dependencies, or constraints and helps configure the generative process.

Examples:

  • Inferring column relationships
  • Generating constraints
  • Designing generation rules

Advantages:

  • Maintains statistical properties better
  • Captures logical relationships

Example pipeline: Dataset Schema → LLM → Relationship Discovery → Generator (GAN/Diffusion) → Synthetic Data

5.3. LLM + Diffusion Hybrid Models

Some recent research combines LLM reasoning with diffusion-based tabular generators such as Denoising Diffusion Probabilistic Models.

The LLM helps identify:

  • dependencies between columns
  • domain constraints
  • logical rules

Then a diffusion model generates the rows.

Advantages:

  • Better modeling of complex dependencies
  • Improved logical consistency

Example architecture: Schema → LLM reasoning → diffusion generator → synthetic dataset

5.4. LLM-Based Data Augmentation

In this approach, LLMs are used to generate additional samples to augment existing datasets, especially for:

  • rare classes
  • missing data
  • imbalanced datasets

LLMs generate rows conditioned on specific attributes.

Example prompt: Generate 50 patient records where

  • Age > 70
  • Diagnosis = diabetes

Advantages:

  • Improves downstream model training
  • Useful in healthcare datasets with rare cases

5.5. Constraint-Guided LLM Generation

LLMs can generate synthetic data while respecting domain constraints or logical rules.

Examples:

  • Age ≥ 0
  • Admission date < discharge date
  • Pregnancy → Gender = Female

The LLM ensures generated rows satisfy these rules.

Advantages:

  • Ensures logical consistency
  • Reduces unrealistic records

5.6. Agent-Based Synthetic Data Generation

In this emerging approach, multiple LLM agents collaborate to generate and validate tabular data.

Typical agents:

  • Generator agent
  • Evaluator agent
  • Corrector agent

Pipeline: Generator → Evaluator → Feedback → Regeneration

Advantages:

  • Iterative improvement
  • Higher quality synthetic datasets

5.7. LLM-Based Schema and Distribution Learning

Another approach uses LLMs to understand dataset semantics before generation.

The LLM extracts:

  • column meanings
  • relationships
  • value ranges
  • data types

This information is then used to configure a traditional generator (GAN, copula, diffusion).

Conclusion

In summary, synthetic data generation models have evolved from simple statistical approaches to advanced deep learning and LLM-based methods. Each category offers unique strengths for different data types and privacy requirements. Understanding these models enables researchers and practitioners to select the most appropriate tools for privacy-preserving data sharing, simulation, and machine learning in healthcare and beyond. Ongoing research continues to improve the fidelity, utility, and safety of synthetic data.

Resources

Recent publications:

Truthworthy and Privacy-Preserving AI, Data Sharing
ML synthetic data generative models privacy health data copula GAN VAE diffusion LLM machine learning
This post is licensed under CC BY 4.0 by the author.