Oral and Maxillofacial Reconstructive Surgery at The University of Hong Kong, Powered by Large Multimodal Model for Medicine

HONG KONG, July 18, 2024 /PRNewswire/ -- In June 2024, a surgical team led by Professor Richard Y. Su and Dr. Jane J. Pu from the Faculty of Dentistry at The University of Hong Kong, in collaboration with United Imaging Intelligence (UII), has achieved a groundbreaking AI-assisted oral and maxillofacial reconstructive surgery using uAI MERITS platform. uAI MERITS platform, which stands for Metaverse Ecosystem for Robotic Intervention, Therapy, and Surgery, is powered by a large multimodal model for medicine.

Overcoming Challenges in Oral and Maxillofacial Reconstructive Surgery 

This surgery helped a patient who had lost part of her mandible due to cancer by successfully reconstructing the mandible and restoring normal aesthetics and functions. The surgery involved the transplantation of a free fibular flap from the patient's lower leg, which encompassed bone, and soft tissue, to reconstruct the missing mandible and oral mucosa. At the same time, dental implants were placed in the newly transplanted bone, restoring the patient's masticatory functions, anatomical structures, and facial aesthetics.

Maxillofacial reconstruction surgeries have been constrained by the complexity of anatomical structures, the high demands for aesthetics and functionality, and the necessity for surgical precision. In this case, the accurate identification and localization of perforator vessels within the soft tissue were pivotal to the success of mandibular reconstruction using a free fibula flap.

In traditional approaches, it is imperative for surgeons to possess an exceptionally high-level of expertise and extensive surgical experience to minimize errors when localizing perforator vessels. Historically, physicians have relied on auxiliary tools such as ultrasound to estimate the location of these perforator vessels, a method that often lacks precision and fails to achieve optimal surgical outcomes. Moreover, the surgical process demands a significant expenditure of time and effort from the surgeons, who must manually compare and delineate between conventional cross-sectional CT images and the surgical site. This process poses considerable challenges to the efficiency and accuracy of the surgery.

AI-Assisted Oral and Maxillofacial Reconstructive Surgery Achieves Outstand Results

For the first time, Professor Su's team successfully completed an oral and maxillofacial reconstructive surgery with the assistance of a large multimodal model for medicine. The innovation of this technology lies in its ability to address the clinical challenge of surgeons relying on empirical knowledge and best estimates to locate perforating vessels during free flap harvesting surgery. This solution is achieved through a robust, large-scale transformer model trained on a diverse array of medical images for precise segmentation preoperatively, and a large multimodal model for 3D image and video registration, prospective projection, and dynamic visual tracking intraoperatively. This integrated approach significantly improves surgical efficiency and accuracy.

Empowered by AI, the mandibular reconstruction surgery utilized the UII Discover - Runoff CTA system, an advanced intelligent system for evaluating the arteries of the lower extremity. This system facilitated rapid and automatic reconstruction of a comprehensive 3D model of the lower limb's arteries, bones, and skin, offering a multimodal, 360° rotational view. It intelligently identified and outlined the perforator vessels in the leg, and displayed them in various colors, which streamlined the preoperative planning process and enhanced surgical efficiency.

During the surgery, the uAI MERITS system, which integrates advanced AI technologies such as large multimodal models and digital twins, played a pivotal role in enhancing precision and efficiency. It provided real-time projection of anatomy that intelligently and dynamically aligned the 3D reconstruction from the CTA system with the patient's surgical site, seamlessly adapting to patient movement without compromising the accuracy of the procedure. This goggle-free approach allowed for the rapid and precise delineation of the surgical field, significantly improving the precision and success rate of the surgery.

The success of this oral and maxillofacial reconstructive surgery is a testament to the integration of cutting-edge technologies, such as AI and large multimodal models, with real-world medical use cases. It also marks the world's first oral and maxillofacial reconstructive surgery driven by a large multimodal model for medicine.

Currently, the team has successfully completed three oral and maxillofacial reconstructive surgeries using the uAI MERITS platform. Notably, distinct from the first two free fibula flap surgeries, the third surgery marked a milestone by introducing another groundbreaking technique of harvesting the anterolateral thigh flap for the first time.

At the International Society for Oral and Maxillofacial Rehabilitation Conference held in Hong Kong in June, Professor Su shared a successful case of using uAI MERITS for oral and maxillofacial reconstructive surgery, which garnered widespread attention from the attending experts and clinicians. Additionally, James J. Xia, M.D., Ph.D., Chief Medical Officer at United Imaging Intelligence, shared the latest advances of AI technology in the field of medicine. He expressed that the company will continue to collaborate with medical professionals to discover more clinical use cases and develop more clinical applications leveraging large model technology, thereby expanding possibilities for physicians and patients worldwide.

Disclaimer: Products and features mentioned herein may not be commercially available in all countries. Their future availability cannot be guaranteed.

source: United Imaging Intelligence