Mutual 

Information

Least-dependent 

Component

Analysis            

  






                                                                                     
                    




MILCA and SNICA are Independent Component Analysis (ICA)-algorithms which use an accurate Mutual Information (MI) estimator to find the least dependent components under a linear transformation (SNICA uses non-negativity constraint). The MI estimator is data efficient, adaptive and has minimal bias [3].
The direct access to the dependency between the components can be used not only as a contrast function to find the most independent components but also in performance tests, reliability tests and for cluster analysis (when the signals are not independent). We provide here algorithms for performing ICA (or rather LCA) and these tests.
Furthermore, we extended all these approaches to use additional time structure information which is often contained in physical data. For an overview see [1].

Have a look at the MILCA demonstration page with audio and image examples.

MILCA and SNICA were recently applied to spectroscopic analysis of complex chemical mixtures [7]. This web page contains data used in this paper along with simple matlab codes to reproduce paper figures.

Along with the basic MILCA, the package contains also the extended version which use time structure (MILCAdelay), reliability tests (ICATests), a clustering algorithm (MIClustering), and mutual information estimators for testing independences of arbitrary signals (MIhigherdim and MIxnyn). Specifically:


MILCA (standard ICA ) - uses only the instantaneous information in the signal, optimal for signals with white spectra (no time structure)
            Input:     original (mixed) components
            Output:  least dependent components, de-mixing matrix

MILCAdelay - uses in addition any time structure in the signals. Can separate also two Gaussians with different spectra [1].
            Input:     original (mixed) components
            Output:  least dependent components, de-mixing matrix

ICATests (Reliability Tests)- can be used for any ICA-output, not necessarily from MILCA [2,1]
            Input:     any ICA-output
            Output:  dependency matrix, variability matrix, square array of 2-d plots showing MI vs. rotation angle from all two-channel combinations

MIClustering - hierarchical clustering algorithm based on the grouping property of MI [4,1]
            Input:     any ICA-output
            Output:  Dendrogram of the dependency

MIxnyn - calculates mutual information between two input channels of arbitrary dimensions [3,1]
            Input:     multidimensional input signal
            Output:  MI value

MIhigherdim - calculates mutual information (redundancy) between any number of one-dimensional input channels [3,1]
            Input:     multidimensional input signal
            Output:  MI value

SNICA - "Stochastic Non-negative Independent Component Analysis" algorithm. This is a new Monte  Carlo approach to blind separation of non-negative well-grounded sources from their linear mixtures based on a constrained search for statistically least-dependent components. SNICA is particularly suited for multivariate spectral curve resolution. You can download the zip file with the SNICA source codes (along with the test example) here.
If you have questions or comments, feel free to email to Sergey Astakhov.
To reduce computational time, all MATLAB functions call executables which have to be generated from the included C source code.
For those who are not so familiar with C, here is a short description how to generate the executables.
The C-executables are autonomous  (MATLAB is not required).  Run a program without arguments to get help.
DOWNLOAD -  ZIP-file with all C-Source-Codes, m-files (including tutorial) and data

DOWNLOAD windows version produced with the free DJGPP (gcc) Compiler
(freely available Borland C compiler can be used as well)



Main References

These are the main references if you want to publish results obtained by our codes:
  1. Least Dependent Component Analysis Based on Mutual Information (preprint)

    Harald Stögbauer, Alexander Kraskov, Sergey A. Astakhov, and Peter Grassberger , Phys. Rev. E 70 (6)  066123, 2004,

  2. Reliability of ICA estimates with mutual information  

    Harald Stögbauer, Ralph G. Andrzejak, Alexander Kraskov and Peter Grassberger, INDEPENDENT COMPONENT ANALYSIS AND BLIND SIGNAL SEPARATION LECTURE NOTES IN COMPUTER SCIENCE 3195: 209-216, 2004

  3. Estimating mutual information (preprint)

    A. Kraskov, H. Stögbauer, and P. Grassberger,  Phys. Rev. E 69 (6) 066138, 2004

  4. Hierarchical clustering using mutual information (preprint)

    A. Kraskov, H. Stögbauer, R. G. Andrzejak, and P. Grassberger, EUROPHYSICS LETTERS 70 (2): 278-284,  2005

  5. Spectral Mixture Decomposition by Least Dependent Component Analysis (preprint)

    Sergey A. Astakhov, Harald Stögbauer, Alexander Kraskov,  and Peter Grassberger ,

  6. Monte Carlo Algorithm for Least Dependent Non-Negative Mixture Decomposition (preprint)

    Sergey A. Astakhov, Alexander Kraskov, Harald Stögbauer, and P. Grassberger , ANALYTICAL CHEMISTRY 2006; 78(5); 1620-1627

  7. Independent components in spectroscopic analysis of complex mixture (preprint)

    Yulia B. Monakhova, Sergey A. Astakhov, Alexander Kraskov, and Svetlana P. Mushtakova (2010), Chemometrics and Intelligent Laboratory Systems, 103, 108-115


Notices

These programs and packages are free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

These programs are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You will receive a copy of the GNU General Public License enclosed in the downloaded archive. See also http://www.gnu.org/licences/.

                                                             
 ©  Sergey Astakhov
      Peter Grassberger
      Alexander Kraskov
      Harald Stögbauer