Lu Dong (董璐)

Ph.D. Student

Department of Computer Science and Engineering
University at Buffalo, SUNY (UB)
301 Davis Hall, Buffalo, New York, US, 14260

Email: ludong@buffalo.edu

About Me

I am a Ph.D. student (2021-Present) at the Department of Computer Science and Engineering (CSE), University at Buffalo-The State University of New York (UB) , working with Prof. Ifeoma Nwogu at the Human Behavior Modeling Lab. Prior to my Ph.D. studies, I obtained a Master's degree in Computer Science and Technology from the School of Computer Science and Technology at Xi'an Jiaotong University (XJTU) in China, where I was advised by Prof. Xinyu Yang at the YLab .

My research focuses on computer vision (CV), large language models (LLMs), 3D human body and face modeling, AI-generated content (AIGC), and their applications in various aspects of human behavior, including sign language, children's education, and VR/AR. I am open to postdoctoral positions, faculty roles, and research-focused opportunities in industry.

News []

2024.10: We are launching an initiative called the Research Acceleration Program , aimed at helping early-stage AI researchers accelerate their growth, focusing on AIGC, Human Motion Generation, with applications in children's educational interactions, sign language synthesis, social intelligence, and VR/AR. UB master's and undergraduate students are encouraged to apply. If you're interested, please feel free to send me your resume.
2024.10: Happy to be invited as a speaker for a panel discussion on 'AI Research and Career Development' at UB.
2024.09: One paper on 3D sign language motion generation has been accepted by EMNLP 2024 .
2024.09: I’m the Local Student Chair for the IJCB 2024 Conference Organizing Team.
2024.08: Two papers have been accepted by ECCV 2024: one on enhancing facial expression inference, and the other on open-domain multi-person motion generation.
2024.08: I am honored to share that my work has been recognized by National AI Institute for Exceptional Education (AI4ExceptionalEd), the Institute for Artificial Intelligence and Data Science (IAD) and UB-CSE. [Read] [More].
2024.05: I will present my work at FG 2024 Conference on May 27 and 30. Welcome to come by for a chat.
2024.04: Two papers on sign language generation have been accepted by FG 2024.
2024.03: I participated in the AI4EE: Year 1 Annual Review Site Visit and presented our work in the event.
2023.09: The National AI Institute for Exceptional Education (AI4EE) will support my research aimed at analyzing and modeling children's non-verbal behaviors, such as facial expressions and body gestures, to enhance educational performance.
2023.08: I was invited to give a talk on Internships in the course CSE501, Fall2023, UB.
2023.07: One paper on human motion synthesis has been accepted by ACM MM 2023.
2023.06: I successfully passed my Oral Qualifying Exam (OQE) and became a PhD candidate at UB.
2023.05: Started my research intern journey at InnoPeak Technology (OPPO US Research) in Seattle, WA. (on-site)
2022.11: I was invited to be a judge of the 2022 UB Hacking Competition!
2022.10: I was invited to give a talk in the course CSE501, Fall2022, UB.
2022.05: Started my research intern journey at InnoPeak Technology (OPPO US Research) in Palo Alto, CA. (on-site)
2021.08: Started my Ph.D. journey following my advisor at UB, Buffalo, NY.

Selected Research

Word-Conditioned 3D American Sign Language Motion Generation
Lu Dong, Xiao Wang, Ifeoma Nwogu.
The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)
DOI PDF BibTeX Project Webpage

Ig3D: Integrating 3D Face Representations in Facial Expression Inference
Lu Dong*, Xiao Wang*, Srirangaraj Setlur, Venu Govindaraju, Ifeoma Nwogu.
The 18th European Conference on Computer Vision (ECCV 2024) Workshop
DOI PDF BibTeX Project Webpage

Towards Open Domain Text-Driven Synthesis of Multi-Person Motions
Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifeoma Nwogu, Guo-Jun Qi, Mitch Hill.
The 18th European Conference on Computer Vision (ECCV 2024)
DOI PDF BibTeX Project Webpage

SignAvatar: Sign Language 3D Motion Reconstruction and Generation
Lu Dong, Lipisha Chaudhary, Fei Xu, Xiao Wang, Mason Lary, Ifeoma Nwogu.
The 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2024.
DOI PDF BibTeX Project Webpage

A Comparative Study of Video-based Human Representations for American Sign Language Alphabet Generation
FeiXu, Lipisha Chaudhary, Lu Dong, Srirangaraj Setlur, Venu Govindaraju, Ifeoma Nwogu.
The 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG) Workshop, 2024.
DOI PDF BibTeX

Language-guided Human Motion Synthesis with Atomic Actions
Yuanhao Zhai, Mingzhen Huang, Tianyu Luan, Lu Dong , Ifeoma Nwogu, Siwei Lyu, David Doermann, Junsong Yuan
ACM Multimedia (MM), 2023.
Please check title link for more details.
DOI PDF BibTeX Project Webpage

Exploring the General Melodic Characteristics of XinTianYou Folk Songs
Juan Li, Lu Dong, Jianhang Ding, Xinyu Yang
12th Sound and Music Computing Conference(SMC)
DOI PDF BibTeX Project Webpage

Selected Project

A smooth 3D Human Pose Estimation pipeline for video in the wild
The majority of current models have been trained on limited close set data, which often results in subpar performance when applied to real-world video data. One of the major challenges in this context is the issue of self-occlusion, which refers to instances where parts of a subject's body obstruct or cover other parts. This results in a less smooth and precise performance. This project aims to tackle this issue.
Please check title link for more details.

A Training ChatBot to facilitate medical knowledge dissemination in underdeveloped regions
This project is undertaken in providing for support for a non-profit organization with the aim of addressing the lack of medical knowledge in underdeveloped regions of India. This shortfall leads to an alarming number of preventable fatalities each year. We have developed a training chatbot to enhance the understanding of medical knowledge and local resources by volunteers and the local population. The chatbot will serve as an educational tool to improve medical literacy and ultimately, save lives.
Please check title link for more details.

A Covid Analysis Search Engine: Dissecting Twitter data to analyze government & public attitude towards Covid and Vaccines
Do authoritative figures have any influence on the trend of COVID-19, what is the attitude of the general public toward COVID-19 and vaccines, whether changes in public attitudes are influenced by authoritative speech? The aim of this project is to design and develop a full-stack search engine that integrates advanced functionalities for retrieval, translation, sentiment analysis, attitude tracking, topic & area statistics, and disinformation detection from multiple dimensions.
Please check title link for more details.

Experience

Research Assitant National AI Institute for Exceptional Education, University at Buaffalo SUNY (UB), Sep 2023- Now.
Research Internship InnoPeak Technology, Seattle, WA, On-Site, June 2023- Aug 2023.
Research Internship InnoPeak Technology, Palo Alto, CA, On-Site, May 2022- Aug 2022.
Research Assitant Human Behavior Modeling Lab, University at Buaffalo SUNY (UB), Aug 2021- Now.
Research Assitant Rochester Institute of Technology(RIT), Aug 2020- May 2021.
Senior Data Analyst Shaanxi Haina Electronic Technology Co.,LT, Sep 2016- Apr 2020.
Research Assitant XI'An Jiaotong University (XJTU), Aug 2013- May 2016.

Selected Awards & Honors

Awards

Outstanding Leadership Award, Shaanxi Haina Electronic Technology Co.,LT. 2018
National Graduate Academic Scholarship , Xi'an Jiaotong University (XJTU), 2013-2016.
Silver Metal for Women's Hurdle at the Intercollegiate Athletic Meet，Xi'an Jiaotong University (XJTU), 2014.
National Encouragement Scholarship, Northeast Electric Power University (NEEPU), 2010.(top 5%)
The Academic Scholarship , Northeast Electric Power University (NEEPU), 2010. (top 5%)
Champion & MVP , Women's Basketball of NEEPU, 2010.

Honors

Excellent Postgraduate Student , Xi'an Jiaotong University (XJTU), 2014 & 2015. (top 10%)
Outstanding Leadership among Students, Northeast Electric Power University (NEEPU), 2010. (top 3 out of 120)
Excellent Undergraduate Student ,Northeast Electric Power University (NEEPU).(top 5%)

Academic Services

Conference Organization

2024.09 I am the Local Student Chair for IJCB 2024 Conference @ Buffalo, NY.

Competition

2022.11.06 I was invited as a Judge for 2022 UB Hacking Competition.

Conference Reviewer

ACM International Conference on Multimedia (ACM MM): 2023, 2024.

Journal Reviewer

Springer Nature, Machine Vision and Applications, 2024.

Membership

IEEE Student Member, IEEE Biometrics Council Membership.

Invited Talks

2023.8: I was invited to give a talk on Internships in the course CSE501, Fall2023, UB.
2022.10: Delivering a talk, introducing Human Pose Estimation and Ph.D Life Guidance in the course CSE-501, Fall2022.