Gaussian Mixture Models and Characteristic Function Estimation

Fourier-based density estimation, empirical characteristic function methods, and mixture models for neuroimaging

2026-06-07 18:16 PDT

Overview

Gaussian mixture models (GMMs) and their estimation occupy a long thread in this research program, beginning with thesis work in 1982 on Fourier-based decomposition of two-component normal mixtures and continuing through a current manuscript on empirical characteristic function (ECF) methods targeting Statistics in Medicine, with applied extensions to Alzheimer’s disease neuroimaging data.

The unifying methodological challenge is parameter estimation for normal mixture distributions when the likelihood surface is pathological. Maximum likelihood estimation of normal mixture parameters is undermined by an infinite spike in the likelihood at each sample point – a well-known theoretical flaw that motivates alternative estimation strategies. The Fourier cosine series and the empirical characteristic function provide principled alternatives that avoid the degenerate likelihood surface while achieving competitive efficiency.

Background: the 1982 Fourier approach

The original thesis document, Parametric Estimation for Normal Mixtures Based on Fourier Density Estimation Methods, addressed the problem of decomposing a two-component normal mixture

f(x) = \frac{p}{\sqrt{2\pi\sigma_1^2}}\,e^{-\frac{(x-\mu_1)^2}{2\sigma_1^2}} + \frac{1-p}{\sqrt{2\pi\sigma_2^2}}\,e^{-\frac{(x-\mu_2)^2}{2\sigma_2^2}}

estimating the mixing proportion p and the four component parameters. The approach represents the density as a truncated Fourier cosine series, following Kronmal (1966), and matches the empirical cosine coefficients \hat{A}_k to the theoretical coefficients implied by the mixture model. This avoids the degenerate likelihood and outperforms both Pearson’s method of moments and the Quandt-Ramsey moment generating function method in simulation.

Current work

Empirical characteristic function methods (res/26-ecf-estimation). The working paper Empirical Characteristic Function Methods for Statistical Inference: From Classical Estimation to Modern Deep Learning Applications systematically develops ECF-based parametric estimation, including:

  • A new analytic method for selecting the evaluation points of the ECF – resolving the principal practical obstacle to ECF estimation that had limited prior work to ad hoc point selection.
  • Closed-form least-squares ECF estimators for the normal and exponential families, with Monte Carlo verification.
  • Application to stable distributions, where MLE is computationally intractable and the ECF provides the natural estimation vehicle.
  • Extension to modern deep learning: characteristic function distances as a principled alternative to the Jensen-Shannon divergence in generative adversarial networks (GANs), providing a theoretically grounded approach to distribution matching.

The manuscript targets Statistics in Medicine and continues the ECF program’s engagement with clinical and biostatistical applications.

Fourier methods for neuroimaging (res/25-fourier-gmm, res/29-mixnormalmri). Applied GMM work applies normal mixture models to MRI-derived distributions in Alzheimer’s disease, where the multimodal structure of brain-volume and cortical-thickness distributions across normal aging and disease stages motivates mixture decomposition. Whitepapers address both the general clinical use case and the AD-specific application.

Research opportunities in Fourier-based mixture methods. A companion white paper, Low-Hanging Fruit: Research Opportunities in Fourier-Based Mixture Methods for Modern ML and AI, catalogs accessible research projects for students and collaborators at undergraduate, master’s, and doctoral levels – connecting the 1982 theoretical framework to contemporary problems in transformers, time series, and generative modeling.

Methods

Fourier cosine series density estimation (Kronmal 1966); empirical characteristic function estimation with analytic evaluation-point selection; minimum-distance estimation for stable distributions; normal and exponential closed-form ECF estimators; Monte Carlo simulation for bias and efficiency assessment; Gaussian mixture models via mclust for neuroimaging data; characteristic function GAN loss functions.

Key documents

  • Thomas RG (1982). Parametric Estimation for Normal Mixtures Based on Fourier Density Estimation Methods. [Thesis/working paper, res/25-fourier-gmm]
  • Thomas RG. Empirical Characteristic Function Methods for Statistical Inference: From Classical Estimation to Modern Deep Learning Applications. [Working paper, res/26-ecf-estimation; targeting Statistics in Medicine]
  • Gaussian Mixture Models for Alzheimer’s Disease Neuroimaging. [Whitepaper, res/29-mixnormalmri]
  • Low-Hanging Fruit: Research Opportunities in Fourier-Based Mixture Methods for Modern ML and AI. [White paper, res/25-fourier-gmm]

Publications

Early work on characteristic function and density estimation methods is accessible through the full publications list by filtering on biostatistics or statistical-computing.