Thackray Medical Museum Exhibits simplified Robotic Colonoscopy Platform

Thanks to the effort of Andrew Jackson from Mech Eng at Uni Leeds, a simplified version of our robotic colonoscopy platform is now on display at the Thackray Medical Museum. The exhibit allows visitors to control a robot arm to steer a magnetic capsule in order to detect polyps inside a simulated colon.

The Thackray Medical Museum houses a collection of over 47,000 objects from medical history which date from Roman times to the present day, along with 15,000 trade catalogues and 9,000 books on medicine and healthcare.

Autonomy in SurgicalRobotics

Starting from the autonomotive field, the topic of autonomy in robotics has been largely debated by the research community. In particular, the last 20 years have seen the rise of autonomous surgical robots as a result of an increasing interest by the community and the availability of robotic research platform.

In this paper, starting from the definition of levels of autonomy reported in [1], a thorough analysis of the current state of the art is carried out. Our aim is to further specify the boundaries for each level of autonomy, categorising the currently available state-of-the-art approaches giving examples of both commercially available systems and research prototypes. With this analysis we define what are the already available technologies in surgical robotics, what are the next ones to be implemented in future systems and which ones are still far from being commercialised. Concluding, the paper deals with the ethical and regulatory issues which may arise adopting autonomous technologies that tend to swap the responsibility of an intervention from the human operator to a machine.

[1] Yang, G. Z., Cambias, J., Cleary, K., Daimler, E., Drake, J., Dupont, P. E., Hata, N., Kazanzides, P., Martel, S., Patel, R. V., Santos, V. J., & Taylor, R. H. (2017). Medical robotics-Regulatory, ethical, and legal considerations for increasing levels of autonomy. Science Robotics, 2(4), [eaam8638]. 

https://www.annualreviews.org/doi/full/10.1146/annurev-control-062420-090543

Closed-loop Control Approach for Soft Continuum Manipulators under Tip Follower Actuation in IJRR

Soft continuum manipulators, inspired by nature, facilitate motion within complex environments where traditional rigid robots may be ineffective. This has driven demand for new control schemes designed to precisely control these highly flexible manipulators, whose kinematics may be sensitive to external loads, such as gravity. In this paper, published in the International Journal of Robotics Research (IJRR), we propose coupling Cosserat rod-based modelling with integrated sensing and efficient numerical determination of the Jacobian matrix at each time step to deliver practically viable, real-time closed-loop control for our waterjet-actuated soft continuum manipulator (the HydroJet). Using this control approach, we demonstrate a reduction orientation error and increased system stability, even under the influence of gravity. The stable closed-loop path following, as demonstrated, has the potential to enable semi and fully autonomous manipulation tasks in the next generation of soft continuum robots actuated via a follower wrench and experiencing external loading.

Guidelines for Robotic Flexible Endoscopy at the Time of COVID-19

In this paper, we review the state of flexible gastrointestinal endoscopy during COVID-19 and the significant changes that it has undergone over the last few months. One of the most challenging aspects has been related to potential generation of aerosols during endoscopy procedures and how this presents an increased risk to healthcare workers. This resulted in a lengthy pause in most diagnostic, non-urgent care procedures during the height of the COVID-19 pandemic and has major long-term implications such as a large increase in excess deaths. For this reason, we have decided to probe how robotics can help guide the field to pre-pandemic capacity and deal with the back log that has resulted from the pause. Robotic platforms have the capability of creating separation (physical distancing) between patients and healthcare workers as well as minimizing the number of people in the room during the process. This would vastly improve the safety of the procedures. We also set the scene for the urgent need of a study to quantify aerosol generation during endoscopy procedures. The development and adoption of a robotic flexible endoscopy platform that is pandemic safe could revolutionize the field.

A Comparative Study of Spatio-Temporal U-Nets for Tissue Segmentation in Surgical Robotics

This paper compares the performance of three different neural network structures based on the U-Net for tissue segmentation. The models subject of this study comprise temporal layers such as Long Short Term Memory cells and Attention Gate block. Results show that the model benefits from the implementation of temporal layers along with attention-based layers, even in case of a limited dataset. The proposed method allows to extract fundamental features from the scene which can be fed to a system to perform autonomous surgical gestures such as tissue retraction, suturing and ablation.

In order to achieve a robust model for tissue segmentation, Long Short term memory cells have been implemented to model the temporal dependencies between the subsequent frames of an endoscopic video. Additionally, Attention Gate blocks have been adopted with the aim of further enhancing the model’s performance compared to a standard straight-forward model (i.e. the U-Net).

doi: 10.1109/TMRB.2021.3054326. https://ieeexplore.ieee.org/document/9335948

On the Observability and Observer Design on the Special Orthogonal Group Based on Partial Inertial Sensing

We recently demonstrated the feasibility in estimating the rotation of a rigid body (attitude), by the only mean of accelerometer and gyroscope. Find our paper on IEEE Transactions on Automatic Control: https://ieeexplore.ieee.org/document/9308988

These results are fundamental for localization in presence of strong magnetic fields, which provoke saturation or artefacts in magnetometers.


Saturations in the magnetometer are observed in magnetic localization methods, as well as in many other examples such as drones (electrical motors induced field), indoors cenarios (electromagnetic devices). This work is a fundamental step for demonstrating we can estimate the attitude without magnetometer and guarantee a reliable localization in every environment.

Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation

The magnetic flexible endoscope team have just published their latest work entitled ‘Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation’ – the culmination of 12 years of research by an international team of scientists led by the University of Leeds. The research has been published in the scientific journal Nature Machine Intelligence (https://www.nature.com/articles/s42256-020-00231-9) and reported in The Times, New Scientist and The Daily Mail.

Our system has been developed to reduce the pain and discomfort associated with colonoscopy, an important component in the move to make colonoscopy more widely available – essential if colorectal cancer is to be identified early. To achieve this, our system uses a magnetic, capsule-shaped device which is connected to a highly flexible tether. The device is then guided through the colon, not by the doctor or nurse pushing (the main source of pain), but by a magnet on a robotic arm positioned over the patient. This magnet on the outside of the patient then interacts with the capsule inside the body, navigating it through the colon.

For this latest work, we have developed and compare the performance of increased levels of intelligent and autonomous control for guiding the magnetic endoscope. These levels are:

  • Direct robot control. This is where the operator has direct control of the robot via a joystick. In this case, there is no assistance.
  • Intelligent teleoperation. The operator focuses on where they want the capsule to be in the colon, leaving the robotic system to calculate the movements of the robotic arm necessary to get the capsule into place.
  • Semi-autonomous navigation. The robotic system autonomously navigates the capsule through the colon, using computer vision – although this can be overridden by the operator.

Benchtop and porcine in-vivo trials showed that the introduction of more intelligent control strategies reduced procedure times, made the system easier for the user to control, allowed the device to more consistently reach the end of the large colon, and enabled a large portion of the colon to be autonomously navigated.

These techniques developed to conduct colonoscopy examinations could be applied to other endoscopic devices, such as those used to inspect the upper digestive tract or lungs.

With these results, we hope that robotic colonoscopy can increase the number of providers who are able to perform the procedure and allow for greater patient access to colonoscopy. We are currently aiming to perform patient trials using the system, beginning next year or in early 2022.

(https://www.nature.com/articles/s42256-020-00231-9)

Parallel Helix Actuators for Soft Robotic Applications

Multi-chamber soft pneumatic actuators are useful for many applications, including medical and surgical robotic devices. However, existing methods used for their manufacture often require multiple manual steps, leading to reduced precision and an increase in size and complexity. In this new work, we present Parallel Helix Actuators (PHAs) which use helical interlocking cores to significantly reduce fabrication complexity, and allow for single material, small (< 1 cm) designs with 3D mobility. We hope this approach will facilitate new and improved soft robot designs.

https://www.frontiersin.org/articles/10.3389/frobt.2020.00119/full