Introduction

The Multimodal Data Fusing and Processing Laboratory at Peking University is an essential part of the National Software Engineering Research Center. In this collaborative environment, faculty, students, and software engineers jointly develop advanced algorithms for multimodal data analysis and big data storage solutions. Our research primarily focuses on large-scale multimodal language models, including pre-training and fine-tuning, LLM evaluation, and various real-world applications. Driven by practical objectives, we have successfully deployed our work in large-scale, real-world projects in fields such as medical artificial intelligence and power grid technology.

The laboratory has designed a verifiable, high-reliability data management model tailored to precision medicine data and knowledge. Leveraging this technology, we created a robust, reusable, high-quality clinical big data platform (data.ncrc.org.cn). This platform integrates comprehensive clinical data collection and management, follow-up services, and continuously adopts cutting-edge information technologies to enhance the efficiency, quality, and accessibility of medical services. Consequently, it facilitates ongoing progress and advanced developments in healthcare. Currently supporting over 1,000 medical institutions and managing more than 1,000 online queues for over 1 million people, the platform has generated significant social and economic benefits. To date, it has resulted in over 50 research publications, including more than 20 focused on clinical phenotypes, imaging, and genetic data analysis. In addition, it has secured over 20 patents, registered more than 20 software copyrights, and received a second-place award from the Ministry of Education and the Peking University Industry-University-Research Collaboration Award.

Papers

The two latest AAAI papers by Wang Jieyi and Zhang Rongchao have been added. (Details to be updated)
Lou, Y., Xu, D., Zhang, R., Zhang, J., Cao, Y., Wang, H., Huang, Y.* MR Image Quality Assessment via Enhanced Mamba: A Hybrid Spatial-Frequency Approach, International Conference on Bioinformatics and Biomedicine, December 2024. (CCF B)
Zhang, R., Huang, Y.*, Lou, Y., Ding, W., Cao, Y., Wang, H. Synergistic Attention-Guided Cascaded Graph Diffusion Model for Complementarity Determining Region Synthesis, IEEE Transactions on Neural Networks and Learning Systems, November 2024. (SCI-indexed, Q1, Impact Factor 10.2, CCF B)
Xu, D., Chen, Y., Wang, J., Huang, Y., Wang, H., Jin, Z., Wang, H., Yue, W., He, J., Li, H., Huang, Y.* MLeVLM: Improve Multi-level Progressive Capabilities based on Multimodal Large Language Model for Medical Visual Question Answering [C] //Findings of the Association for Computational Linguistics: ACL 2024. 2024
Lou, Y., Zhang, J., Xu, D., Cao, Y., Wang, H., Huang, Y.* No-Reference MRI Quality Assessment via Contrastive Representation: Spatial and Frequency Domain Perspectives [C] //2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2024. (CCF B recommended)
Zhang, J., Li, H., Xu, D., Lou, Y., Ran, M., Jin, Z., Huang, Y.* Decouple and Decorrelate: A Disentanglement Security Framework Combining Sample Weighting for Cross-Institution Biased Disease Diagnosis. IEEE Internet of Things Journal, 2024. (Q1, SCI-indexed, Impact Factor 11.1)
Zhu, H., Xu, D., Huang, Y.*, Jin, Z., Ding, W., Tong, J., Chong, G. Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024. (Q1, SCI-indexed, Impact Factor 5.3)
Zhang, R., Lou, Y., Xu, D., Cao, Y., Wang, H., Huang, Y.* A Learnable Discrete-Prior Fusion Autoencoder with Contrastive Learning for Tabular Data Synthesis [C] //Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(15): 16803-16811. (CCF A recommended)
Zhang, J., Xu, D., Lou, Y., Huang, Y.* A Novel Multi-Atlas Fusion Model Based On Contrastive Learning For Functional Connectivity Graph Diagnosis [C] //ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024: 2295-2299. (CCF B recommended)
Lou, Y., Chen, Y., Xu, D., Zhou, D., Cao, Y., Wang, H., Huang, Y.* Refining the Unseen: Self-supervised Two-stream Feature Extraction for Image Quality Assessment [C] //2023 IEEE International Conference on Data Mining (ICDM). IEEE, 2023: 1193-1198.
Xu, D., Chen, Y., Zhang, J., Lou, Y., Wang, H., He, J., Huang, Y.* Radiology Report Generation via Structured Knowledge-Enhanced Multi-modal Attention and Contrastive Learning [C] //2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2023: 2320-2325.
Xu, D., Zhu, H., Huang, Y., et al. Vision-knowledge fusion model for multi-domain medical report generation. Information Fusion, 2023, 97: 101817. (Q1, IF: 18.6)
Lu, Z.#, Zhang, Y.#, Sun, Y., Liao, Y., Kang, Z., Feng, X., Yan, H., Li, J., Wang, L., Lu, T., Zhang, D., Huang, Y.*, Yue, W.* The positive association between antipsychotic-induced weight gain and therapeutic response: New biotypes of schizophrenia. Psychiatry Res. 2023 Jun;324:115226. doi: 10.1016/j.psychres.2023.115226. (Q1, IF: 11.3)
Zhao, G., Lu, Z., Sun, Y., Kang, Z., Feng, X., Liao, Y., Sun, J., Zhang, Y.*, Huang, Y.*, Yue, W.* Dissecting the causal association between social or physical inactivity and depression: a bidirectional two-sample Mendelian Randomization study. Transl Psychiatry. 2023 Jun 8;13(1):194. doi: 10.1038/s41398-023-02492-5. (Q1, IF: 6.8)
Lou, Y., Huang, Y.*, Xing, X., Cao, Y., Wang, H. MTS-LSTDM: Multi-Time-Scale Long Short-Term Double Memory for power load forecasting. Journal of Systems Architecture. Volume 125, April 2022. (CCF B, SCI-indexed)
Qin, Y., Lou, Y., Huang, Y.*, Chen, R., Yue, W. An Ensemble Deep Learning Approach Combining Phenotypic Data and fMRI for ADHD Diagnosis. Journal of Signal Processing Systems, 19 September 2022. (CCF B, SCI-indexed)
Mao, Z., Su, Y., Xu, G., Wang, X., Huang, Y.*, Yue, W.*, Sun, L., Xiong, N. Spatio-temporal deep learning method for ADHD fMRI classification. Information Science, vol. 499, 1-12, 2019. (Q1, CCF B, SCI-indexed, cited 50+ times)
Dou, C., Zhang, S., Wang, H., Sun, L., Yue, W.*, Huang, Y.* (Yu Huang). ADHD fMRI short-time analysis method for edge computing based on multi-instance learning. JOURNAL OF SYSTEMS ARCHITECTURE, 2020. (Brain imaging big data analysis, CCF B, SCI-indexed, IF 2.55)
Huang, Y. A Node-oriented discrete event scheduling algorithm based on finite resource model. Journal of Organizational and End User Computing, vol.31(3), 2019. (SCI-indexed)
Lin, J. Y., Huang, Y.* Association between Perceived Stressfulness of Stressful Life Events and the Suicidal Risk in Chinese Patients with Major Depressive Disorder. Chinese Medical Journal, vol.131(8), 2018. (SCI-indexed)
Jing, Y., Wang, H., Huang, Y.* A modeling language to describe massive data storage management in cyber-physical systems, Journal of Parallel and Distributed Computing, Volume 103, May 2017, Pages 113-120. (SCI-indexed)
Wang, X., Su, Y., Yan, H., Huang, Z., Huang, Y.* (Yu Huang), Yue, W.* Association Study of KCNH7 Polymorphisms and Individual Responses to Risperidone Treatment in Schizophrenia. Front Psychiatry. 2019 30;10:633. (Genomic big data analysis, IF 3.161)
Lin, J. Y., Huang, Y. (co-first author), Su, Y.-A., Yu, X., Lyu, X.-Z., Liu, Q., Si, T.-M. Association between Perceived Stressfulness of Stressful Life Events and the Suicidal Risk in Chinese Patients with Major Depressive Disorder. Chinese Medical Journal, vol.131(8), 2018. (Big data analysis, SCI-indexed, 8 external citations)
Jing, Y., Wang, H., Huang, Y.* A Modeling Language for MapReduce Programming in a Storage System Perspective. Journal of Signal Processing Systems, September 2018, Volume 90, Issue 89, pp 1133-1150.

Teachers

Huang Yu

Male, born on January 12, 1979

Doctor of Science, Doctoral Supervisor, Research Professor at the National Engineering Research Center for Software Engineering, Peking University

Project Leader of Key Research and Development Programs of the Ministry of Science and Technology

Dr. Huang Yu focuses on big data, artificial intelligence, and related software applications. Over the past five years, he has published over 50 papers, obtained 20+ patent authorizations, filed 30 patent applications, and registered 20 software copyrights. He also contributed to two national defense reports.

Dr. Huang has received several prestigious honors, including the "Science and Technology Rising Star" award from the Beijing Municipal Government, a second-place prize from the Ministry of Education, and the Peking University Industry-University-Research Award.

He has led two projects funded by the National Natural Science Foundation of China and one Chief Scientist Project under the Ministry of Science and Technology’s "Biology and Information Fusion Special Project." Additionally, Dr. Huang has participated in multiple key national and regional initiatives, such as:

1 major special project of the Ministry of Science and Technology
1 key research and development project funded by the Ministry of Science and Technology
4 sub-projects under the Ministry of Science and Technology’s Key Research and Development Plan
1 sub-project of the Science and Technology Support Plan
1 doctoral program fund from the Ministry of Education
1 pre-research fund from the Ministry of Education
1 project funded by the Beijing Municipal Education Commission
2 pre-research projects for final assembly
2 projects funded by the Beijing Municipal Science and Technology Commission
3 clinical research projects funded by Peking University
3 projects focused on the transformation of scientific and technological achievements

Dr. Huang’s work has notably advanced big data, artificial intelligence, and the practical implementation of these technologies.

Cooperating Doctors

Li Hang

Chief Physician, Professor, and Doctor of Medicine. Currently serving as the Vice President and Director of the Department of Dermatology at Peking University First Hospital, and Executive Vice Director of the National Clinical Research Center for Dermatology and Immunological Diseases. He is the General Secretary of the Dermatology Branch of the Chinese Medical Association, Chairman of the Dermatology Professional Committee, and Vice Chairman and Chairperson-elect of the Dermatology and Beauty Branch of the Chinese Leprosy Association. Additionally, he is a Standing Committee Member of the Beauty Dermatology Branch of the Chinese Plastic and Cosmetic Association and a member of the National Health Science Popularization Expert Database.

Gao Ying

Doctor of Medicine, doctoral supervisor, member of the Youth Committee of the Endocrinology Branch of the Chinese Medical Association, and a member of the Thyroid and Basic Endocrinology Groups. She also serves in the Endocrinology Branch of the Beijing Medical Association. Her primary research centers on autoimmune thyroid diseases. Dr. Gao has received multiple grants from the National and Beijing Natural Science Foundations and has published nearly 50 papers in Chinese and English. She has contributed to research projects recognized with prestigious awards, including the third prize of the Chinese Medical Science and Technology Award and the first prize of the Ministry of Education's Excellent Scientific Research Achievement Award (Science and Technology) in Natural Science for Higher Education Institutions. Dr. Gao has been honored with Peking University School of Medicine’s "Young Scholar" award, the "Science and Technology Star" program of Beijing, and the "New Century Excellent Talent Support Program" of the Ministry of Education. She has also received the "Young Researcher Award" at the Huaxia Endocrinology Conference and the Huang Tingfang/Xinhe Young Outstanding Scholar Award from Peking University. Currently, Dr. Gao serves as a reviewer for JCEM, CEI, and CMJ.

He Jing

Associate Professor and Chief Physician of the Department of Rheumatology and Immunology at Peking University People's Hospital; Deputy Director of the Rheumatology Research Institute at the same institution; Clinical Doctor at Peking University School of Medicine. Recognized as a Beijing Science and Technology Rising Star, she is a Youth Member of the Rheumatology and Immunology Specialist Branch of the Chinese Medical Association, a member of the Rheumatology and Immunology Branch of the Health Management Association, a Youth Member of the Basic Immunology Society Branch of the Asia Pacific Medical Biological Immunology Society, and a Youth Member of the Chinese Immunology Society. Dr. He also serves as Deputy Editor-in-Chief of Clin Rheum.

Yue Weihua

Doctor of Medicine, Professor, Researcher, and Doctoral Supervisor. Recipient of both the National Outstanding Youth Science Fund and the National Excellent Youth Science Fund. He serves as Deputy Director (in charge of scientific research) of the Institute of Mental Health at Peking University Sixth Hospital and Deputy Director of the National Clinical Research Center for Mental and Psychological Disorders. His primary research focuses on clinical and applied basic studies of common mental illnesses, with particular expertise in diagnosis, treatment, and genetic counseling for schizophrenia and depression. Dr. Yue completed his bachelor’s degree at Jining Medical College in 1996, obtained his MD from Central South University in 2003, and was a senior visiting scholar at Harvard Medical School in 2012. He holds multiple academic positions, including membership in the Basic and Clinical Psychiatry Branch of the Chinese Society of Neuroscience, and serves as Vice Editor-in-Chief of Front Psychiatry, the Chinese Journal of Psychiatry, the Chinese Journal of Mental Health, and the Chinese Journal of Behavioral Medicine and Neuroscience. As a principal investigator, Dr. Yue has received funding from the National Natural Science Foundation of China, the National Key R&D Program (Chronic Disease Special Project), the National 863 Program, the Beijing Natural Science Foundation, and the Capital Health Development Research Special Project. He was named a “Beijing Science and Technology Rising Star in 2009, selected for the Ministry of Education New Century Excellent Talents Program in 2012, and received second prize from the Ministry of Education in 2019.

Chen Dongxue

Director of the Research Office and Associate Researcher at the National Clinical Research Center for Mental Disorders at Peking University Sixth Hospital.

Chen Hongguang

Associate Researcher at Peking University Sixth Hospital, specializing in psychiatry and psychology. With extensive experience in clinical practice and research, he has significantly contributed to the diagnosis and treatment of mental illnesses and psychological crisis intervention.

Wang Xueping

Assistant Researcher at the National Clinical Research Center for Mental and Psychological Disorders (Peking University Sixth Hospital). Her research focuses on the molecular biology mechanisms underlying mental illnesses, especially the role of ion channels in the pathogenesis of these conditions. She has led multiple research projects, including the Capital Science and Technology Innovation Voucher Project on designing and developing a mental health scale and process (October 2018–December 2019) and the School-Level Youth Cultivation Fund Project on the molecular regulation mechanism of the potassium ion channel KCNH2 in schizophrenia (January 2018–December 2018).

Wang Hongxing

Chief Physician, Associate Professor, PhD, and Postdoctoral Fellow at Xuanwu Hospital, Capital Medical University. He serves as the Vice Chairman of the 2nd Youth Committee of the Psychosomatic Medicine Branch of the Chinese Medical Association and the Vice Chairman of the Psychosomatic Medicine Branch of the Beijing Medical Association. Dr. Wang is also a Youth Committee Member of both the Basic and Clinical Psychiatry Branch of the Chinese Society of Neuroscience and the 4th Committee of the same branch. Additionally, he is an Executive Committee Member of the Middle-aged and Elderly Healthcare Branch of the China Association for the Promotion of International Exchange in Healthcare.