Unsupervised machine learning-based stratification and immune deconvolution of liver hepatocellular carcinoma

Background: Hepatocellular carcinoma (HCC) is the most prevalent type of liver cancer and a leading cause of cancer-related deaths globally. The tumour microenvironment (TME) influences treatment response and prognosis, yet its heterogeneity remains unclear.

Methods: The unsupervised machine learning methods— agglomerative hierarchical clustering, Multi-Omics Factor Analysis with K-means++, and an autoencoder with K-means++ — stratified patients using microarray data from HCC samples. Immune deconvolution algorithms estimated the proportions of infiltrating immune cells across identified clusters.

Results: Thirteen genes were found to influence HCC subtyping in both primary and validation datasets, with three genes—TOP2A, DCN, and MT1E—showing significant associations with survival and recurrence. DCN, a known tumour suppressor, was significant across datasets and associated with improved survival, potentially by modulating the TME and promoting an anti-tumour immune response.

Conclusions: The discovery of the 13 conserved genes is an important step toward understanding HCC heterogeneity and the TME, potentially leading to the identification of more reliable biomarkers and therapeutic targets. We have stratified and validated the liver cancer populations. The findings suggest further research is needed to explore additional factors influencing the TME beyond gene expression, such as tumour microbiome and stromal cell interactions.