About this work package
The third version of the prototype.
Fetal experts operate under tight space constraints on small and delicate tissues. Work Package 3 elaborates novel instrument concepts to simplify the expert’s task, allowing them to navigate easily, more intuitively and with greater precision inside the uterus. Improvements are targeted to the level that new treatment methods can be considered.
The keyhole through which surgeons insert their instruments significantly restrains instrument motion. In twin-to-twin transfusion syndrome (TTTS), this restriction prevents the alignment of the instrument perpendicular to the placenta, complicating ablation. To alleviate this problem, the first steerable flexible fetoscopes were developed in June 2016. To improve the usability of the new line of instruments, we developed haptic guidance features in April 2018 and automatic control features in April 2020 were developed, allowing the surgeon to focus on the ablation task without requiring attention to steer the flexible section themselves. In the meanwhile, a virtual reality trainer was developed and validated June 2018 for training TTTS ablation.
Further, a steady hand is needed for precise tissue ablation. At the same time, the surgeon should compensate for physiological patient motion and avoid applying too much tension on the uterine membranes. Robotic co-manipulation technology and dedicated synergistic control strategies have been developed for TTTS surgery in October 2019. These leverage the above constraints while offering the surgeon full control over the navigation task. A paper featuring automatic instrument tracking with robotic co-manipulation is currently under preparation.
Building upon above technology, new surgical approaches can be considered. We showed how flexible fetoscopy and robotic techniques can come together and developed an innovative 11mm diameter multi-arm surgical robotic system for myelomeningocele (MMC) treatment in November 2019. Where minimally invasive MMC treatment is currently a technique in development, the multi-arm system could potentially reduce invasiveness even further by allowing single port access approaches that could reduce post-operative complications.
The goal of this work package is to develop steerable instruments that offer the surgeon better and more intuitive access to the uterine cavity. The available access (keyhole) to the cavity is extremely tight. This massive constraint pushes us to come up with new approaches in terms of actuation, sensing, planning and control to produce a new generation of steerable instruments. We are motivated to work on better tools as we are keen to see what expert surgeons could achieve if they were not limited by yesterday’s technology.
Emmanuel Vander Poorten, Prof. dr. ir.
Work package tasks
Development of a self-aware synergetic stabiliser
During laser treatment of TTTS, surgeons have to coagulate anastomoses on the placenta to equalize blood supply and nutrition between fetuses. Long and slender instruments are used to gain inner access to the uterus. Besides the well-known technical difficulties associated with MIS, this particular intervention imposes additional constraints as vision is further deteriorated by the turbid amniotic fluid. This might lead to dangerous situations where the surgeon can puncture delicate tissue with the sharp instruments. In order to keep the laser equipped fetoscope at a safe distance from the placenta, the instrument is equipped with an all-optical ultrasound (AOUS) sensor and placed on a robotic comanipulator (Virtuose 6D developed by Haption SA). The AOUS allows to measure the distance between the fetoscope and the placenta. This information is fed back to the comanipulator which counteracts the user’s motion when exceeding a distance threshold, which is implemented as a virtual fixture (VF).
The result is a robotic comanipulation setup that allows for safe execution of TTTS while not intervening the surgical workflow in a negative way. No prior knowledge about the surgical site is required as the system is able to (locally) model the environment intraoperative.
Control of synergetic stabiliser featuring intra-operatively adjustable compliance
Experimental setup for verifying the feasibility of the supervised autonomy mode for synergistic endoscope guidance.
Development of single-arm instruments with augmented dexterity
Flexible fetoscopes have been introduced in prior literature as a new surgical tool to limit the stresses applied on the fetal membrane during fetoscopic laser surgery and therefore avoid iatrogenic preterm premature rupture of membranes. However, previous flexible fetoscopes are either too large or do not foresee all the tools that are essential to the success of the targeted procedure. The proposed solution is a single-handed active flexible fetoscope. The instrument is equipped with a camera, a light source, and a working channel while remaining compact (350 mm length and 30 mm wide, with a shaft of 3 mm diameter) and lightweight (87 g).
Shared control approach with autonomous alignment of the flexible tip using Deep Learning
The proposed flexible fetoscope with active distal tip comes with usability challenges and a steep learning curve due to augmented degrees of freedom. Foreseeing these challenges, a shared control approach is proposed in which the actuated flexible segment is controlled autonomously while the gross motion of the instrument is controlled by the surgeon. This shared control approach reduces stress for the surgeon as they may focus on the ablation task and navigation, while the autonomous tip maintains an optimal angle for effective ablation. Deep learning techniques are proposed to estimate the distance and orientation of the placenta with respect to the camera, which acts as feedback for the autonomous control. Using intrinsic camera of the fetoscope allows us to avoid adding extra sensors into the scope keeping the diameter of the instrument shaft at 3 mm.
The functionality of the proposed approach has been demonstrated on the instrument using images from a virtual reality simulator due to unavailability of a usable flexible image source. A new chip on tip camera will be integrated into the proposed instrument to demonstrate the working of full hardware and software pipeline.
Development of a multi-arm single-port access instrument with augmented dexterity
MMC repair is a recent surgery with high rates of complications. The complexity of this procedure and the governing space constraints in the uterus necessitate a small and agile robot to an extent that was not found in the literature. A novel multi-arm robot of 11mm diameter is in development. It is based on a hybrid macro-micro approach of concentric tube robots for the triangulation platform (macro) and flexible bending segments for the end effectors (micro). The fluidic actuation is advantageous as it minimally interferes with the triangulation platform. The triangulation platform, on the other hand, provides a stable base for the end-effectors such that large distal actuation bandwidth can be achieved. The dexterity of the robot would restore intuitive manipulation and comfort to the surgeon, while its small diameter would allow to strongly reduce postoperative complications as the uterus would be less damaged.
The results include five prototype generations, from the fourth including full Degrees of Freedom capability; the development of a novel hybrid needle driver/tissue manipulator, flexible segments optimisation and early-stage teleoperation.
Success stories
- With their thesis on “Development of a Synthetic Uterus Model with Embedded Sensing for Multiport Spina Bifida Surgery“, supervised by Awais Ahmad, Lara and Olivier were nominated with the three best engineering master thesis for 2018-2019 by Agoria. In the subsequent competition, they won the popular vote with as much as 1259 votes they reached as much of 49.2% of the votes. Read more.
- “Active Handheld Flexible Fetoscope–Design and Control Based on a Modified Generalized Prandtl-Ishlinski Model.” by Julie Legrand et al. was a best paper finalist (among the top five) at 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2020).
- “Hybrid Visual Serving for Synergistic Endoscope Guidance” and “Deep Learning for Environment Pose Estimation in Robot Assisted Surgery” won the best presentation award at the Key Technologies for Autonomous Robotic Surgery workshop held at the 19th International Conference on Advanced Robotics (ICAR 2019) in Brazil, shared between Jef De Smet (Caspar) and Awais Ahmad.
- As part of the Operating Theatre of the Future at New Scientist Live (ExCel London, 2019) a “Mixed reality surgical trainer for fetal laser minimally invasive surgery” was demonstrated. The exhibition was an overwhelming success with 10,000 visitors at the stand over four days. Read more about the event.
Key publications
A mixed-reality surgical trainer with comprehensive sensing for fetal laser minimally invasive surgery. Javaux, A., Bouget, D., Gruijthuijsen, C., Stoyanov, D., Vercauteren, T., Ourselin, S., Deprest, J., Denis, K., Vander Poorten, E. (2018). International Journal of Computer Assisted Radiology and Surgery, 13 (12), 1949-1957. doi: 10.1007/s11548-018-1822-7 Open Access
Haptic Guidance based on All-Optical Ultrasound Distance Sensing for Safer Minimally Invasive Fetal Surgery. Gruijthuijsen, C., Colchester, R., Devreker, A., Javaux, A., Efthymios, M., Noimark, S., Xia, W., Stoyanov, D., Reynaerts, D., Deprest, J., Ourselin, S., Desjardins, A., Vercauteren, T., Vander Poorten, E. (2018). Journal of Medical Robotics Research, 03 (03n04), 1841001-1841001. doi: 10.1142/S2424905X18410015
Deep learning-based monocular placental pose estimation: towards collaborative robotics in fetoscopy. Ahmad, M.A., Ourak, M., Gruijthuijsen, C., Deprest, J., Vercauteren, T., Vander Poorten, E. (2020). International Journal of Computer Assisted Radiology and Surgery. doi: 10.1007/s11548-020-02166-3 Open Access
From a Disposable Ureteroscope to an Active Lightweight Fetoscope-Characterization and Usability Evaluation. Legrand, J., Ourak, M., Javaux, A., Gruijthuijsen, C., Ahmad, M., Van Cleynenbreugel, B., Vercauteren, T., Deprest, J., Ourselin, S., Vander Poorten, E. (2018). IEEE Robotics and Automation Letters, 3 (4), 4359-4366. doi: 10.1109/LRA.2018.2866204 Open Access
Macro-Micro Multi-Arm Robot for Single-Port Access Surgery. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (425-432). Vandebroek, T., Ourak, M., Gruijthuijsen, C., Javaux, A., Legrand, J., Vercauteren, T., Ourselin, S., Deprest, J., Vander Poorten, E. (2019). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems, Macao, China, 03 Nov 2019-08 Nov 2019. doi: 10.1109/IROS40897.2019.8968219 Open Access
Synergistic Robotic Surgery: Novel Techniques to Harmonize Surgeon and Robot. Gruijthuijsen, C., Reynaerts, D. (sup.), Vander Poorten, E. (cosup.) (2019). Open Access