Mutual 

Information

Least-dependent 

Component

Analysis            

  






                                                                                     
                    




MILCA and SNICA are Independent Component Analysis (ICA)-algorithms which use an accurate Mutual Information (MI) estimator to find the least dependent components under a linear transformation (SNICA uses non-negativity constraint). The MI estimator is data efficient, adaptive and has minimal bias [3].
The direct access to the dependency between the components can be used not only as a contrast function to find the most independent components but also in performance tests, reliability tests and for cluster analysis (when the signals are not independent). We provide here algorithms for performing ICA (or rather LCA) and these tests.
Furthermore, we extended all these approaches to use additional time structure information which is often contained in physical data. For an overview see [1].

Have a look at theMILCA demonstration page with audio and image examples.

Along with the basic MILCA, the package contains also the extended version which use time structure (MILCAdelay), reliability tests (ICATests), a clustering algorithm (MIClustering), and mutual information estimators for testing independences of arbitrary signals (MIhigherdim and MIxnyn). Specifically:


MILCA (standard ICA ) - uses only the instantaneous information in the signal, optimal for signals with white spectra (no time structure)
            Input:     original (mixed) components
            Output:  least dependent components, de-mixing matrix

MILCAdelay - uses in addition any time structure in the signals. Can separate also two Gaussians with different spectra [1].
            Input:     original (mixed) components
            Output:  least dependent components, de-mixing matrix

ICATests (Reliability Tests)- can be used for any ICA-output, not necessarily from MILCA [2,1]
            Input:     any ICA-output
            Output:  dependency matrix, variability matrix, square array of 2-d plots showing MI vs. rotation angle from all two-channel combinations

MIClustering - hierarchical clustering algorithm based on the grouping property of MI [4,1]
            Input:     any ICA-output
            Output:  Dendrogram of the dependency

MIxnyn - calculates mutual information between two input channels of arbitrary dimensions [3,1]
            Input:     multidimensional input signal
            Output:  MI value

MIhigherdim - calculates mutual information (redundancy) between any number of one-dimensional input channels [3,1]
            Input:     multidimensional input signal
            Output:  MI value

SNICA - "Stochastic Non-negative Independent Component Analysis" algorithm. This is a new Monte  Carlo approach to blind separation of non-negative well-grounded sources from their linear mixtures based on a constrained search for statistically least-dependent components. SNICA is particularly suited for multivariate spectral curve resolution. You can download the zip file with the SNICA source codes (along with the test example) here.
If you have questions or comments, feel free to email to Sergey Astakhov.
To reduce computational time, all MATLAB functions call executables which have to be generated from the included C source code.
For those who are not so familiar with C, here is a short description how to generate the executables.
The C-executables are autonomous  (MATLAB is not required).  Run a program without arguments to get help.
DOWNLOAD -  ZIP-file with all C-Source-Codes, m-files (including tutorial) and data

DOWNLOAD windows version produced with the free DJGPP (gcc) Compiler
(freely available Borland C compiler can be used as well)



Main References

These are the main references if you want to publish results obtained by our codes:
  1. Least Dependent Component Analysis Based on Mutual Information Harald Stögbauer, Alexander Kraskov, Sergey A. Astakhov, and Peter Grassberger , Phys. Rev. E 70 (6)  066123, 2004, (preprint)
  2. Reliability of ICA estimates with mutual information  Harald Stögbauer, Ralph G. Andrzejak, Alexander Kraskov and Peter Grassberger, INDEPENDENT COMPONENT ANALYSIS AND BLIND SIGNAL SEPARATION LECTURE NOTES IN COMPUTER SCIENCE 3195: 209-216, 2004
  3. Estimating mutual information A. Kraskov, H. Stögbauer, and P. Grassberger,  Phys. Rev. E 69 (6) 066138, 2004 (preprint)
  4. Hierarchical clustering using mutual information A. Kraskov, H. Stögbauer, R. G. Andrzejak, and P. Grassberger , EUROPHYSICS LETTERS 70 (2): 278-284,  2005 (preprint)
  5. Spectral Mixture Decomposition by Least Dependent Component Analysis Sergey A. Astakhov, Harald Stögbauer, Alexander Kraskov,  and Peter Grassberger , (preprint)
  6. Monte Carlo Algorithm for Least Dependent Non-Negative Mixture Decomposition Sergey A. Astakhov, Alexander Kraskov, Harald Stögbauer, and P. Grassberger , ANALYTICAL CHEMISTRY 2006; 78(5); 1620-1627 (preprint)

Notices

FOR SCIENTIFIC USE ONLY

These codes are free of charge for research and education purposes only. Any commercial or military use of this software is prohibited.

NO WARRANTY

The software on this site is provided "as-is," without any expressed or implied warranty. In no event  we are liable for any damages arising from the use of the software.
                                                             
 ©  Sergey Astakhov
      Peter Grassberger
      Alexander Kraskov
      Harald Stögbauer