Hierarchical clustering with gene expression matrix in python

Question

how can I do hierarchical clustering (in this case, for gene expression data) in Python in such a way as to show the matrix of the values ​​of the gene expression along with the dendrogram? I mean, for example, an example:

http://www.mathworks.cn/access/helpdesk/help/toolbox/bioinfo/ug/a1060813239b1.html

shown after bullet point 6 (Figure 1), where the dendrogram is depicted to the left of the gene expression matrix, where the rows were reordered to reflect clustering.

How can I do this in Python using numpy / scipy or other tools? Furthermore, is it advisable to do this with a matrix of about 11,000 genes, using the Euclidean distance as a metric?

EDIT: Many have proposed clustering packages, but I still don't know how to build such an image that I linked to above in Python. How can I overlay a dendrogram with a heatmap matrix, for example using Matplotlib?

thank.

Answer 1

Many clustering methods, including scipy.cluster, start with sorting all the pair distances, ~ 60 million in your case, not too big.
How much time do they take for you?

import scipy.cluster.hierarchy as hier
import pylab as pl

def fcluster( pts, ncluster, method="average", criterion="maxclust" ):
    """ -> (pts, Y pdist, Z linkage, T fcluster, clusterlists)   
        ncluster = n1 + n2 + ... (including n1 singletons)
        av cluster size = len(pts) / ncluster
    """
    pts = np.asarray(pts)
    Y = scipy.spatial.distance.pdist( pts )  # ~ N^2 / 2
    Z = hier.linkage( Y, method )  # N-1                         
    T = hier.fcluster( Z, ncluster, criterion=criterion )
        # clusters = clusterlists(T)
    return (pts, Y, Z, T)

hier.dendrogram( Z )

How to set up the matrix and graph here in So in March, with a partial answer.

Answer 2

scipy cluster.hierarchy. . correlation corr pdist, cluster scipy- fcluster. , dendrogram scipy clustergram Matlab.

(, pdist). , 11 000 , 11000 * (11000-1)/2 = 60494500 (11000 2) , . , , , , .

Answer 3

scipy matplotlib:

, , scipy

, , . , . :

http://altanalyze.blogspot.com/2012/06/hierarchical-clustering-heatmaps-in.html