Zeyu Zhang

Zeyu Zhang is an undergraduate researcher advised by Prof. Richard Hartley and Prof. Ian Reid. He is also a visiting student researcher at MIT CSAIL, working with Assoc. Prof. Stefanie Mueller. He is an incoming research assistant at USC, working with Asst. Prof. Yue Wang. His research interests are rooted in computer vision, focusing on generative 3D modeling and AI for health. Specifically, he is dedicated to advancing efficient and high-quality motion and avatar generation, as well as 3D medical imaging segmentation and representation learning. With extensive experience across multiple research disciplines, Zeyu actively explores cutting-edge advancements in both the foundational and applied aspects of artificial intelligence. He also collaborates closely with Asst. Prof. Hao Tang (PKU), Prof. Bohan Zhuang (ZJU), Dr. Yang Zhao (La Trobe), Dr. Minh-Son To (FHMRI), and many others. Zeyu is actively seeking PhD, RA, and internship in the US.

News

(07/19/2024) 🎉 Our paper Motion Avatar has been accepted to BMVC 2024!
(07/02/2024) 🎉 Our paper Motion Mamba has been accepted to ECCV 2024!
(06/18/2024) 🎉 Our paper JointViT has been selected as oral presentation at MIUA 2024!
(05/14/2024) 🎉 Our paper JointViT has been accepted to MIUA 2024!
(03/13/2024) 🎉 Our paper Motion Mamba has been featured in Daily Papers!
(02/10/2024) 🎉 Our paper SegReg has been accepted to ISBI 2024!

Publications

Selected publications are highlighted.

	Motion Mamba: Efficient and Long Sequence Motion Generation Zeyu Zhang^∗, Akide Liu^∗, Ian Reid, Richard Hartley, Bohan Zhuang, Hao Tang^✉ *ECCV 2024* Human motion generation is a key goal in generative computer vision, and we propose Motion Mamba, a model using state space models (SSMs) with Hierarchical Temporal Mamba (HTM) and Bidirectional Spatial Mamba (BSM) blocks, achieving up to 50% FID improvement and 4x speedup on HumanML3D and KIT-ML datasets, showcasing efficient and high-quality long sequence motion modeling.
	MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection Zeyu Zhang^∗, Nengmin Yi^∗, Shengbo Tan^∗, Ying Cai^✉, Yi Yang, Lei Xu, Qingtai Li, Zhang Yi, Daji Ergu, Yang Zhao *Preprint* Cervical disc herniation (CDH) is a common disorder needing expert analysis. Current automated detection methods face challenges: high computational demands and MRI noise. We propose MedDet for efficient detection, leveraging knowledge distillation, generative adversarial training, and nmODE². Our model improves mAP by 5%, reduces parameters by 67.8%, and speeds inference fivefold.
	Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion Zeyu Zhang^∗, Yiran Wang^∗, Biao Wu^∗, Shuo Chen, Zhiyuan Zhang, Shiya Huang, Wenbo Zhang, Meng Fang, Ling Chen, Yang Zhao^✉ *BMVC 2024* Our paper introduces a novel agent-based approach called Motion Avatar for generating customizable human and animal 3D avatars with motions via text queries, coordinated by an LLM planner, and supported by the new Zoo-300K animal motion dataset.
	JointViT: Modeling Oxygen Saturation Levels with Joint Supervision on Long-Tailed OCTA Zeyu Zhang, Xuyin Qi, Mingxi Chen, Guangxi Li, Ryan Pham, Ayub Qassim, Ella Berry, Zhibin Liao, Owen Siggs, Robert Mclaughlin, Jamie Craig, Minh-Son To *MIUA 2024* *Oral* Our paper introduces JointViT, a Vision Transformer model with a novel joint loss function and balancing augmentation technique that significantly improves the accuracy of diagnosing sleep-related disorders using OCTA, achieving up to a 12.28% accuracy improvement.
	SegReg: Segmenting OARs by Registering MR Images and CT Annotations Zeyu Zhang, Xuyin Qi, Bowen Zhang, Biao Wu, Hien Le, Bora Jeong, Zhibin Liao, Yunxiang Liu, Johan Verjans, Minh-Son To, Richard Hartley^✉ *ISBI 2024* To improve the accuracy and efficiency of organ at risk (OAR) segmentation in radiotherapy, we propose SegReg, a method that combines CT and MRI using Elastic Symmetric Normalization, outperforming traditional CT-only methods by 16.78% in mDSC and 18.77% in mIoU.
	Thin-Thick Adapter: Segmenting Thin Scans Using Thick Annotations Zeyu Zhang, Bowen Zhang, Abhiram Hiwase, Feng Chen, Akide Liu, Christen Barras, Biao Wu, Adam Wells, Daniel Ellis, Benjamin Reddi, Andrew Burgan, Minh-Son To, Ian Reid, Richard Hartley, Yutong Xie^✉ *Preprint* Medical imaging segmentation is critical for medical analysis, predominantly using thicker CT slices due to the scarcity of annotated thin slices, so we propose segmenting thin scans with thicker slice annotations, introduce the CQ500-Thin dataset, and present the Thin-Thick Adapter to bridge domain gaps, significantly improving segmentation performance.
	A Deep Learning Approach to Diabetes Diagnosis Zeyu Zhang, Khandaker Asif Ahmed, Md Rakibul Hasan, Tom Gedeon, Md Zakir Hossain *ACIIDS 2024* We propose a non-invasive diabetes diagnosis method using a Back Propagation Neural Network with batch normalization, addressing class imbalance and improving performance over traditional methods, achieving high accuracy on multiple datasets.
	ESA: Annotation-Efficient Active Learning for Semantic Segmentation Jinchao Ge, Zeyu Zhang, Minh Hieu Phan, Bowen Zhang, Akide Liu, Yang Zhao^✉ *Preprint* Active learning improves annotation efficiency by selecting the most informative samples for labeling. We propose Entity-Superpixel Annotation (ESA), an efficient strategy using a mask proposal network and superpixel grouping. Our method reduces click cost by 98% and boosts performance by 1.71%, outperforming pixel-based methods with only 40 clicks per image.
	SegStitch: Multidimensional Transformer for Robust and Efficient Medical Imaging Segmentation Shengbo Tan, Zeyu Zhang, Ying Cai^✉, Daji Ergu, Lin Wu, Binbin Hu, Pengzhang Yu, Yang Zhao *Preprint* Medical imaging segmentation, crucial for lesion analysis, has seen advances with transformers in 3D segmentation. Despite their scalability, transformers struggle with local features and complexity. We propose SegStitch, combining transformers with denoising ODE blocks, improving mDSC by up to 11.48% and reducing parameters by 36.7%, promising real-world clinical adaptation.
	Sine Activated Low-Rank Matrices for Parameter Efficient Learning Yiping Ji^∗, Hemanth Saratchandran^∗, Cameron Gordon, Zeyu Zhang, Simon Lucey^✉ *Preprint* We propose a novel theoretical framework integrating a sinusoidal function into low-rank decomposition, enhancing parameter efficiency and model accuracy across diverse neural network applications such as Vision Transformers, Large Language Models, Neural Radiance Fields, and 3D shape modeling.
	XLIP: Cross-modal Attention Masked Modelling for Medical Language-Image Pre-Training Biao Wu^∗, Yutong Xie^∗, Zeyu Zhang, Minh Hieu Phan, Qi Chen, Ling Chen, Qi Wu^✉ *Preprint* Vision-and-language pretraining (VLP) in the medical field faces challenges with reconstructing pathological features due to data scarcity and limited use of paired/unpaired data. This paper proposes XLIP, using AttMIM and EntMLM modules, to enhance feature learning from unpaired data, achieving state-of-the-art results in medical classification tasks.
	A Landmark-Based Approach for Instability Prediction in Distal Radius Fractures Yang Zhao, Zhibin Liao, Yunxiang Liu, Koen Oude Nijhuis, Britt Barvelink, Jasper Prijs, Joost Colaris, Mathieu Wijffels, Max Reijman, Zeyu Zhang, Minh-Son To, Ruurd Jaarsma, Job Doornberg, Johan Verjans *ISBI 2024* Distal radius fractures (DRFs) are common and their instability assessment is crucial for treatment decisions, affecting recovery and costs. We propose a deep learning-based landmark detection method using anatomical landmarks from X-rays to measure distances and angles. These features are used in an XGBoost model for DRF instability classification, validated on a large Dutch dataset.
	Can Rotational Thromboelastometry Rapidly Identify Theragnostic Targets in Isolated Traumatic Brain Injury? Abhiram Hiwase, Christopher Ovenden, Lola Kaukas, Mark Finnis, Zeyu Zhang, Stephanie O'Connor, Ngee Foo, Benjamin Reddi, Adam Wells, Daniel Ellis *EMA 2024* This study evaluates the prognostic utility of ROTEM sigma in isolated traumatic brain injury (TBI). ROTEM sigma, a point-of-care assay, demonstrated faster turnaround times and comparable accuracy to standard coagulation tests in predicting head injury-related deaths. The findings suggest ROTEM sigma effectively detects coagulopathy in isolated TBI cases.
	BHSD: A 3D Multi-class Brain Hemorrhage Segmentation Dataset Biao Wu, Yutong Xie, Zeyu Zhang, Jinchao Ge, Kaspar Yaxley, Suzan Bahadir, Qi Wu, Yifan Liu, Minh-Son To^✉ *MLMI 2023* The Brain Hemorrhage Segmentation Dataset (BHSD) is a comprehensive 3D multi-class ICH dataset with pixel-level and slice-level annotations designed to support supervised and semi-supervised ICH segmentation tasks, addressing the lack of existing public datasets for multi-class ICH segmentation.

Research Experience

	Visiting Student Researcher MIT CSAIL Jun 2024 - Present Worked on physically compatible 3D generation in MIT CSAIL HCIE group, working with Assoc. Prof. Stefanie Mueller (MIT CSAIL) and Mr. Faraz Faruqi (MIT CSAIL).
	Research Assistant Zhejiang University Aug 2024 - Present Worked on efficient generative models, working with Prof. Bohan Zhuang (ZJU).
	Visiting Student Researcher Peking University July 2024 - Present Worked on 3D human motion generation, working with Asst. Prof. Hao Tang (PKU).
	Research Intern Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) May 2024 - June 2024 Worked on unsupervised classification of cellular structures based on cryo-electron tomography (cryo-ET), working with Assoc. Prof. Min Xu (CMU, MBZUAI) and Prof. Ian Reid (MBZUAI, AIML).
	Research Assistant La Trobe University Apr 2024 - Present Worked on 3D generation and AI for Heath, working with Dr. Yang Zhao (La Trobe University).
	Research Assistant Monash University Feb 2024 - May 2024 Worked on 3D/4D generative learning, specifically focusing on text-guided human motion and avatar generation, working with Prof. Reza Haffari (Monash University), and Prof. Bohan Zhuang (ZJU, Monash University).
	Research Intern National Computational Infrastructure (NCI) Feb 2023 - Jun 2023 Worked on long tail large scale multi-label text classification, working with Dr. Jingbo Wang (NCI).
	Visiting Student Researcher Australian Institute for Machine Learning (AIML) Nov 2022 - Jan 2024 Worked on 3D medical imaging analysis, with a particular focus on semantic segmentations of tumors, hemorrhages, and organs at risk, working with Prof. Ian Reid (MBZUAI, AIML), Dr. Bowen Zhang (AIML), Dr. Yutong Xie (AIML), and Dr. Qi Chen (AIML).
	Research Assistant Flinders Health and Medical Research Institute (FHMRI) Nov 2022 - Present Worked on 3D medical imaging analysis, particularly in the realms of 2D and 3D medical representation learning and explainable AI, working with Dr. Minh-Son To (FHMRI).
	Student Researcher The Australian National University (ANU) Jul 2022 - Nov 2022 Worked on diabetes diagnosis in deep learning, working with Dr. Md Zakir Hossain (ANU, Curtin University, CSIRO Data61), Dr. Khandaker Asif Ahmed (CSIRO), Mr. Md Rakibul Hasan (Curtin University, Brac University), and Prof. Tom Gedeon (Curtin University, ANU, Óbuda University).

Education

Bachelor of Science (Advanced) (Honours)
The Australian National University (ANU)
Jul 2021 - Jun 2025 (Expected)
Major: Computer Science, Minor: Mathematics

Visiting Student
Imperial College London
Jul 2022
Quantitative Sciences Research Institute (QSRI)

Visiting Student
University College London (UCL)
Jul 2022

Visiting Student
Shanghai Jiao Tong University (SJTU)
Dec 2021 - Jan 2022

Honors & Awards

NRF Vacation Scholarship, NeuroSurgical Research Foundation, Oct 2023.
Flinders Summer Research Scholarship, Flinders University CMPH, Nov 2022.
UNSW Science Vacation Research Scholarship, The UNSW Sydney, Oct 2022.

Academic Services

Conference: CVPR 2025, ICLR 2025, VR 2025, MIUA 2024, BIBM 2024.
Journal: EBM, ACO, MCET, PSEN.

Talks

(07/22/2024) Motion Mamba: Efficient and Long Sequence Motion Generation @ miHoYo, Shanghai. You can find our slides here.