Distances, dissimilarities, divergences, diversities, discrepancies, discriminations, displacements, deviations, etc.

Fisher-Rao distance

• A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions
• Optimal transport vs. Fisher-Rao distance between copulas for clustering multivariate time series, IEEE statistical signal processing workshop (SSP), 2016.
• On clustering financial time series: a need for distances between dependent random variables , Computational Information Geometry pp 149–174. chapter

Jensen-Shannon divergence

• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius, Entropy 2021.
• On a Generalization of the Jensen-Shannon Divergence and the Jensen–Shannon Centroid, Entropy 2020
• On the Jensen-Shannon Symmetrization of Distances Relying on Abstract Means, Entropy 2019
• A family of statistical symmetric divergences based on Jensen's inequality, arXiv:1009.4004 2010.

f-divergences

• On f-divergences between Cauchy distributions, IEEE Transactions on Information Theory, 2022 arXiv:2101.12459
• On the f-divergences between hyperboloid and Poincaré distributions, submitted, 2023.
• On the Chi square and higher-order Chi distances for approximating f-divergences, IEEE Signal Processing Letters, 2013
• The analytic dually flat space of the mixture family of two prescribed distinct Cauchy distributions
• α-divergences:
  • Generalizing the Alpha-Divergences and the Oriented Kullback-Leibler Divergences with Quasi-Arithmetic Means. Algorithms 15(11): 435 (2022)
    A generalization of the α-divergences based on comparable and distinct weighted means, arXiv:2001.09660
  • Non-flat clustering with alpha-divergences, ICASSP 2011: 2100-2103

Bregman divergences and some generalizations

• Generalizing the Alpha-Divergences and the Oriented Kullback-Leibler Divergences with Quasi-Arithmetic Means. Algorithms 15(11): 435 (2022)
• Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences, Entropy 2022, 24(3)
  Duo Fenchel-Young divergence , arXiv:2202.10726.

  We define duo Bregman divergences, duo Fenchel-Young divergences, duo Jensen divergences. We show how those divergences occur naturally when calculating the Kullback-Leibler divergence and skew Bhattacharyya distances between densities belonging to nested exponential families. We report the KLD between truncated normal distributions as a duo Bregman divergence.
  Demo code:

  • KLDTruncatedNormalDistributions.java
  • KLDTruncatedExponentialDistributions.java
  • Computing Statistical Divergences with Sigma Points, GSI 2021
  • Quasiconvex Jensen Divergences and Quasiconvex Bregman Divergences, SPIGL 2021
  • Generalizing Skew Jensen Divergences and Bregman Divergences With Comparative Convexity, IEEE Signal Process. Lett. 24(8): 1123-1127 (2017)
  • Reranking with Contextual Dissimilarity Measures from Representational Bregman k-Means, VISAPP (1) 2010: 118-123
  • Bregman Divergences and Surrogates for Learning, IEEE Trans. Pattern Anal. Mach. Intell. 31(11): 2048-2059 (2009)
  • The Dual Voronoi Diagrams with Respect to Representational Bregman Divergences, ISVD 2009: 71-78

  Jensen divergences and some generalizations

  • The Chord Gap Divergence and a Generalization of the Bhattacharyya Distance, IEEE ICASSP 2018
  • Skew Jensen-Bregman Voronoi Diagrams, Trans. Comput. Sci. 14: 102-128 (2011)
  • Jensen-Bregman Voronoi Diagrams and Centroidal Tessellations, ISVD 2010: 56-65

  Conformal divergences

  • On conformal divergences and their population minimizers, IEEE Transactions Information Theory, 62.1 (2015): 527-538
  • Total Jensen divergences: Definition, Properties and k-Means++ Clustering, IEEE ICASSP 2015
  • Total Bregman Divergence and its Applications to Shape Retrieval, IEEE CVPR 2010
  • Total Bregman divergence and its applications to DTI analysis, IEEE Transactions on Medical Imaging 30(2):475-83, 2011
  • Shape Retrieval Using Hierarchical Total Bregman Soft Clustering, IEEE Trans. Pattern Anal. Mach. Intell. 34(12): 2407-2419 (2012)

  Projective divergences

  • On Hölder Projective Divergences, Entropy 2017, 19(3), 122
  • Patch Matching with Polynomial Exponential Families and Projective Divergences, SISAP 2016 (projective gamma divergence)

  Optimal transport/Wasserstein distances/Sinkhorn distances

  Earth mover distances (EMD), Wasserstein distances
  • Optimal Transport with Tempered Exponential Measures, arxiv:2309.04015
  • Sinkhorn AutoEncoders, UAI 2019
  • On The Chain Rule Optimal Transport Distance, Progress in Information Geometry, 2021
  • Tsallis Regularized Optimal Transport and Ecological Inference, AAAI 2017
  • Clustering Patterns Connecting COVID-19 Dynamics and Human Mobility Using Optimal Transport, March 2021, Sankhya B 83(S1)
  • Optimal copula transport for clustering multivariate time series, IEEE ICASSP 2016
  • Exploring and measuring non-linear correlations: Copulas, Lightspeed Transportation and Clustering, Time Series Workshop at NeurIPS 2016
  • Earth Mover Distance on superpixels, IEEE ICIP 2010

  Shannon, Rényi, Tsallis, Sharmal-Mittal entropies, cross-entropies and divergences

  • MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions, IEEE Signal Process. Lett. 24(4): 402-406 (2017)
  • A closed-form expression for the Sharma-Mittal entropy of exponential families, Journal of Physics A: Mathematical and Theoretical 45.3 (2011).
  • On Rényi and Tsallis entropies and divergences for exponential families, arXiv:1105.3259
  • Entropies and cross-entropies of exponential families, IEEE ICIP 2010
  • Texture Regimes for Entropy-Based Multiscale Image Analysis, ECCV (3) 2010: 692-705

  Chernoff information

  • Revisiting Chernoff Information with Likelihood Ratio Exponential Families. Entropy 24(10): 1400 (2022), arXiv:2207.03745
  • Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means , Pattern Recognition Letters 42(1), 2014
  • An Information-Geometric Characterization of Chernoff Information , IEEE Signal Processing Letters, 2013
  • Hypothesis Testing, Information Divergence and Computational Geometry , GSI 2013

  Other dissimilarities

  • A note on Onicescu's informational energy and correlation coefficient in exponential families , arXiv:2003.13199
  • Deep rank-based transposition-invariant distances on musical sequences , arXiv:1709.00740
  • Quantifying the Invariance and Robustness of Permutation-Based Indexing Schemes , SISAP 2016: 79-92

  Loss functions and proper scoring rules

  • Loss factorization, weakly supervised learning and label noise robustness, ICML 2016: 708-717
  • Gentle Nearest Neighbors Boosting over Proper Scoring Rules, IEEE Trans. Pattern Anal. Mach. Intell. 37(1): 80-93 (2015), TPAMI

  Divergences between statistical mixtures

  • Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions, Entropy 2021, 23(11), 1417
  • The Statistical Minkowski Distances: Closed-Form Formula for Gaussian Mixture Models, GSI 2019
  • Guaranteed Deterministic Bounds on the total variation Distance between univariate mixtures, IEEE MLSP 2018
  • Comix: Joint estimation and lightspeed comparison of mixture models , IEEE ICASSP 2016
  • Bag-of-Components: An Online Algorithm for Batch Learning of Mixture Models, GSI 2015
  • Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities , Entropy 2016, 18(12), 442
  • Combinatorial bounds on the α-divergence of univariate mixture models, ICASSP 2017
  • Closed-form information-theoretic divergences for statistical mixtures, ICPR 2012