Fig. 3

Clustering analysis reveals differences in tumour heterogeneity between Palbociclib-treated and control samples. a Comprehensive scanning of different clustering resolutions from 0.1 to 1.2 (an interval of 0.05) for each of the four samples. The resolution parameter is used in the graph optimisation procedure to find the optimal number of cell communities (clusters)—a higher resolution leads to more clusters. Cluster partitioning is visualised hierarchically from low resolution (top) to high resolution (bottom), where each row and colour corresponds to the result from one resolution (visualised by Clustree). Branching and merging of cells and clusters are shown by horizontal and vertical arrows when moving from one resolution to the next. CI represents connectivity index, a measure of differences in heterogeneity due to separating or grouping spots that are similar (nearest neighbour). CI values of controls (2709 and 2029) are higher than Palbociclib (1171 and 1431), indicating the reduced heterogenity in samples under drug treatments. b Shannon entropy per sample estimated from each clustering resolution used in a. The x-axis is the same clustering resolution in panel a. The y-axis is the Shannon entropy per sample. Colours represent different individuals. The bar plot on the bottom-right corner shows the differences in Shannon entropy of Palbociclib and control (i.e. mean of entropy in Palbociclib − mean of entropy in control). A high Shannon entropy indicates high heterogeneity, with lower Shannon entropy and heterogeneity in Palbociclib-treated PDOX compared to controls (P < 0.001 for Wilcoxson signed-rank exact test and two-tailed paired t-test). c Similar to panel b but shows Simpson index. Simpson index represents the probability that two randomly selected spots belong to the same cluster. Thus, a low Simpson index indicates high heterogeneity, with higher Simpson index indicating lower heterogeneity in Palbociclib-treated PDOX compared to controls (P < 0.001 for Wilcoxon signed-rank exact test and P < 0.01 for two-tailed paired t-test). d Differences in Simpson entropy of Palbociclib and control (i.e. mean of entropy in Palbociclib − mean of entropy in control). e Density plot of per-spot Shannon entropy using a clustering resolution of 0.8. f Pair-wise similarity (i.e. Jensen–Shannon Divergence (JSD)) between distributions in panel e. The same colour bars on the right and bottom indicates the same sample in panel e. The value in the heatmap is the JSD between corresponded samples. The lower the JSD value, the more similar the corresponding distribution