Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysis
Weiqin Zhao, Ziyu Guo, Yinshuang Fan, Yuming Jiang, Maximus Yeung, Lequan Yu.

Nature npj Digital Medicine, 2024   

Developed a knowledge concept-based framework for precise Whole Slide Imaging (WSI) analysis by integrating human expert knowledge with data-driven concepts. Employed Large Language Models (LLMs) to derive reliable human expert knowledge from medical literature and align it with histology images using a pathology vision-language model. Achieved enhanced automated diagnostic precision in various cancers.

MulGT: Multi-Task Graph-Transformer with Task-Aware Knowledge Injection and Domain Knowledge-Driven Pooling for Whole Slide Image Analysis
Weiqin Zhao, Shujun Wang, Maximus Yeung, Tianye Niu, Lequan Yu.

The AAAI Conference on Artificial Intelligence (AAAI), 2023   

This framework leverages Graph Neural Networks and Transformers to learn both task-agnostic local features and task-specific global representations. It incorporates a Task-aware Knowledge Injection module that adapts shared graph embeddings into task-specific feature spaces and a Domain Knowledge-driven Graph Pooling module that uses diagnostic patterns to enhance accuracy and robustness.

ConSlide: Asynchronous Hierarchical Interaction Transformer with Breakup-Reorganize Rehearsal for Continual Whole Slide Image Analysis
Yanyan Huang$^{*}$, Weiqin Zhao$^{*}$, Shujun Wang, Yu Fu, Yuming Jiang, Lequan Yu.

International Conference on Computer Vision (ICCV), 2023   

Propose the first continual learning framework for WSI analysis, named ConSlide, to tackle the challenges of enormous image size, utilization of hierarchical structure, and catastrophic forgetting by progressive model updating on multiple sequential datasets.

From Layers to States: A State Space Model Perspective to Deep Neural Network Layer Dynamics
Qinshuo Liu$^{*}$, Weiqin Zhao$^{*}$, Wei Huang, Yanwen Fang, Lequan Yu, Guodong Li.

International Conference on Learning Representations (ICLR), 2025   

This paper novelly treats the outputs from network layers as states of a continuous process and considers leveraging the state space model (SSM) to design the aggregation of layers in very deep neural networks. Designed a new module called Selective State Space Model Layer Aggregation (S6LA) to combine traditional CNN or ViT within a sequential framework, enhancing its representational capabilities.

HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image
Ziyu Guo$^{*}$, Weiqin Zhao$^{*}$, Shujun Wang, Lequan Yu.

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2023   

This paper introduces a novel Hierarchical Interaction GraphTransformer (HIGT) for Whole Slide Imaging (WSI) analysis. Built on Graph Neural Networks and Transformers, HIGT effectively captures both short-range local information and long-range global representations from WSI pyramids. Recognizing the complementary nature of information across different resolutions, we design a Bidirectional Interaction block to enable communication between different levels of the WSI pyramids, enhancing the learning process.

Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement
Yanyan Huang, Weiqin Zhao, Yihang Chen, Yu Fu, Lequan Yu.

Conference on Neural Information Processing Systems (NeurIPS), 2024   

Present Concept Anchor-guided Task-specific Feature Enhancement (CATE), an adaptable paradigm that can boost the expressivity and discriminativeness of pathology foundation models for specific downstream tasks.

Unleash the Power of State Space Model for Whole Slide Image with Local Aware Scanning and Importance Resampling
Yanyan Huang, Weiqin Zhao, Yu Fu, Lingting Zhu, and Lequan Yu.

IEEE Transactions on Medical Imaging, 2024   

This paper introduces a new Pathology Mamba (PAM) for more accurate and robust WSI analysis. It includes three carefully designed components to tackle the challenges of enormous image size, the utilization of local and hierarchical information, and the mismatch between the feature distributions of training and testing during WSI analysis.

Hist2Cell: Deciphering Fine-grained Cellular Architectures from Histology Images
Weiqin Zhao, Zhuo Liang, Xianjie Huang, Yuanhua Huang, Lequan Yu.

Cell Genomics Peer Review   

Developed a one-stage prediction model for identifying spatial transcription-related fine-grained cell types from histology images, enhancing cost-efficient analysis for cancer studies and clinical applications. Achieved accurate cell-type identification and colocalization on external datasets, validating existing biological findings using cost-effective histology images; Enabled large-scale fine-grained cellular analysis on public histology cohorts, producing significant consensus findings that were previously unattainable.

BRIDGE: A Cross-organ Foundation Model for Bridging Histology Imaging and Spatial Transcriptomics
Zhuo Liang$^{*}$, Weiqin Zhao$^{*}$, Fuying Wang, Yuanhua Huang, Lequan Yu.

Nature Journal Submission

Present a multi-organ foundation model pre-trained on over 600,000 image-gene paired profiles from 13 organs with various sequencing techniques, to connect cross-organ morphology and genomics for cost-effective histology analysis. Without fine-tuning, BRIDGE enables accurate gene inference for over 80 biomarker genes across 10 human organs, enhances survival analysis for six different types of cancer by using histopathology images from TCGA cohorts and shows potential as a cost-efficient alternative to bulk RNA sequencing