Soft continuum manipulators, inspired by nature, facilitate motion within complex environments where traditional rigid robots may be ineffective. This has driven demand for new control schemes designed to precisely control these highly flexible manipulators, whose kinematics may be sensitive to external loads, such as gravity. In this paper, published in the International Journal of Robotics Research (IJRR), we propose coupling Cosserat rod-based modelling with integrated sensing and efficient numerical determination of the Jacobian matrix at each time step to deliver practically viable, real-time closed-loop control for our waterjet-actuated soft continuum manipulator (the HydroJet). Using this control approach, we demonstrate a reduction orientation error and increased system stability, even under the influence of gravity. The stable closed-loop path following, as demonstrated, has the potential to enable semi and fully autonomous manipulation tasks in the next generation of soft continuum robots actuated via a follower wrench and experiencing external loading.

In this paper, we review the state of flexible gastrointestinal endoscopy during COVID-19 and the significant changes that it has undergone over the last few months. One of the most challenging aspects has been related to potential generation of aerosols during endoscopy procedures and how this presents an increased risk to healthcare workers. This resulted in a lengthy pause in most diagnostic, non-urgent care procedures during the height of the COVID-19 pandemic and has major long-term implications such as a large increase in excess deaths. For this reason, we have decided to probe how robotics can help guide the field to pre-pandemic capacity and deal with the back log that has resulted from the pause. Robotic platforms have the capability of creating separation (physical distancing) between patients and healthcare workers as well as minimizing the number of people in the room during the process. This would vastly improve the safety of the procedures. We also set the scene for the urgent need of a study to quantify aerosol generation during endoscopy procedures. The development and adoption of a robotic flexible endoscopy platform that is pandemic safe could revolutionize the field.

This paper compares the performance of three different neural network structures based on the U-Net for tissue segmentation. The models subject of this study comprise temporal layers such as Long Short Term Memory cells and Attention Gate block. Results show that the model benefits from the implementation of temporal layers along with attention-based layers, even in case of a limited dataset. The proposed method allows to extract fundamental features from the scene which can be fed to a system to perform autonomous surgical gestures such as tissue retraction, suturing and ablation.

In order to achieve a robust model for tissue segmentation, Long Short term memory cells have been implemented to model the temporal dependencies between the subsequent frames of an endoscopic video. Additionally, Attention Gate blocks have been adopted with the aim of further enhancing the model’s performance compared to a standard straight-forward model (i.e. the U-Net).

doi: 10.1109/TMRB.2021.3054326. https://ieeexplore.ieee.org/document/9335948

The magnetic flexible endoscope team have just published their latest work entitled ‘Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation’ – the culmination of 12 years of research by an international team of scientists led by the University of Leeds. The research has been published in the scientific journal Nature Machine Intelligence (https://www.nature.com/articles/s42256-020-00231-9) and reported in The Times, New Scientist and The Daily Mail.

Our system has been developed to reduce the pain and discomfort associated with colonoscopy, an important component in the move to make colonoscopy more widely available – essential if colorectal cancer is to be identified early. To achieve this, our system uses a magnetic, capsule-shaped device which is connected to a highly flexible tether. The device is then guided through the colon, not by the doctor or nurse pushing (the main source of pain), but by a magnet on a robotic arm positioned over the patient. This magnet on the outside of the patient then interacts with the capsule inside the body, navigating it through the colon.

For this latest work, we have developed and compare the performance of increased levels of intelligent and autonomous control for guiding the magnetic endoscope. These levels are:

• Direct robot control. This is where the operator has direct control of the robot via a joystick. In this case, there is no assistance.
• Intelligent teleoperation. The operator focuses on where they want the capsule to be in the colon, leaving the robotic system to calculate the movements of the robotic arm necessary to get the capsule into place.
• Semi-autonomous navigation. The robotic system autonomously navigates the capsule through the colon, using computer vision – although this can be overridden by the operator.

Benchtop and porcine in-vivo trials showed that the introduction of more intelligent control strategies reduced procedure times, made the system easier for the user to control, allowed the device to more consistently reach the end of the large colon, and enabled a large portion of the colon to be autonomously navigated.

These techniques developed to conduct colonoscopy examinations could be applied to other endoscopic devices, such as those used to inspect the upper digestive tract or lungs.

With these results, we hope that robotic colonoscopy can increase the number of providers who are able to perform the procedure and allow for greater patient access to colonoscopy. We are currently aiming to perform patient trials using the system, beginning next year or in early 2022.

(https://www.nature.com/articles/s42256-020-00231-9)

Multi-chamber soft pneumatic actuators are useful for many applications, including medical and surgical robotic devices. However, existing methods used for their manufacture often require multiple manual steps, leading to reduced precision and an increase in size and complexity. In this new work, we present Parallel Helix Actuators (PHAs) which use helical interlocking cores to significantly reduce fabrication complexity, and allow for single material, small (< 1 cm) designs with 3D mobility. We hope this approach will facilitate new and improved soft robot designs.

https://www.frontiersin.org/articles/10.3389/frobt.2020.00119/full

A literature review on the Gastrointestinal tract has just been published by the STORM Lab UK in Progress of Biomedical Engineering.

The inspection of the GI tract is fundamental for the early detection and diagnosis of GI diseases. In the last decade, miniaturized robots for gastrointestinal inspection have been investigated with the aim of developing innovative, more sophisticated, and minimally invasive technologies to access this part of the body. Despite much progress, the need for innovation is stronger than ever due to the combination of a growing disease prevalence and the harsh, difficult-to-access environment of the gut. To address limitations and develop innovative and more sophisticated technologies for diagnoses and therapy of the GI tract, capsule engineers need to understand the complex environment of the GI tract. Therefore, the purpose of this review is to provide engineers in this field a comprehensive reference manual of the GI environment and its complex physical, biological, and chemical characteristics. The work reviews and summarizes a broad spectrum of literature covering the main anatomical and physiological properties; each organ in the GI is discussed in this context, including the main mechanisms of digestion, chemical and mechanical processes that could impact devices, and GI motor behaviour and resultant forces that may be experienced by objects as they move through the environment of the gut.

https://iopscience.iop.org/article/10.1088/2516-1091/abab4c

A new podcast talking about robotics has launched; episode 1 features Dr James Chandler a Research Fellow in STORM Lab who is working on low-cost, soft robotic technology for intravascular and endoscopic applications.

“It’s not an exaggeration to say that robotics research is driving life-saving innovations – robots are already making a difference in clinics and operating theatres around the world. In our first episode, we’ll chat to researchers about how robotics is revolutionising medicine and surgery.

In our first episode, Claire is joined by Dr James Chandler (STORM Lab, University of Leeds) and Dr Matina Giannarou (Hamlyn Centre, Imperial College London) to talk about the exciting world of medical and surgical robotics and find out what they’ve been working on.”

https://ukrobotics.libsyn.com/episode-one-medicinal-robots

This paper presents a novel approach for semi-autonomous tissue retraction in minimally invasive surgery; taking advantage of neural networks to detect the candidate flaps for retraction from depth maps of the surgical scenario. The proposed method allows to plan and execute consistent and repeatable tool trajectories to enhance the surgeons vision during navigation in the patient’s anatomy.

Aleks Attanasio

In order to achieve semi-autonomous tissue retraction in minimally invasive surgery, a U-Net is trained to extract the tissue flap profile from the scene. To this end, the 3D reconstruction of the scene is evaluated from stereo images captured by a Da Vinci endoscope. Once the geometry of the tissues is defined, the 3D position of the flaps is used to plan and execute retraction following experienced surgeons guidelines.

https://ieeexplore.ieee.org/document/9158395

Our new work on teleoperation and contact detection of a waterjet-actuated soft continuum manipulator has been published on RA-L. This work presents a method to enable intuitive articulation of a waterjet-actuated soft continuum robot in the confined space of the stomach and describes in detail the integration of contact detection within the teleoperation scheme to prevent integral windup of the feedback controller.

Federico Campisano

Gastric cancer is the third leading cause of cancer deaths worldwide, with most new cases occurring in low and middle income countries, where access to screening programs is hindered by the high cost of conventional endoscopy. The waterjet-actuated HydroJet endoscopic platform was developed as a low-cost, disposable alternative for inspection of the gastric cavity in low-resource settings. In this work, we present a teleoperation scheme and contact detection algorithm that work together to enable intuitive teleoperation of the HydroJet within the confined space of the stomach. Using a geometrically accurate stomach model and realistic anatomical inspection targets, we demonstrate that, using these methods, a novice user can complete a gastroscopy in approximately the same amount of time with the HydroJet as with a conventional endoscope.

https://ieeexplore.ieee.org/document/9157939