Comparing neural networks with a traditional method for identifying the vanishing points of surgical tools

Robot-assisted minimally invasive surgery (RMIS) offers numerous advantages, such as higher precision and shorter patient recovery time, leading to more successful outcomes. Force feedback from the surgical tool is essential for the surgeon to control the pressure applied. Recently, computer vision techniques have been explored as an alternative to expensive sensors for force feedback. A crucial step is identifying the vanishing point of the surgical tool, where the two edges of the tool seem to converge, but this becomes challenging when the tool is obstructed by tissue (occlusion). We hypothesized that neural networks could achieve higher accuracy than traditional techniques such as the minimum area enclosing triangle method in finding the vanishing point, particularly with occlusion. To test our hypothesis, we trained a neural network for vanishing point detection using datasets of synthetic images of surgical tools with and without occlusion, and then tested the triangle method on the same datasets. We then evaluated and compared their performances. The results partially supported the hypothesis as the neural network performed much better than the triangle method for unjointed tools, both with and without occlusion. For jointed tools, the triangle method surprisingly performed better than the neural network, although both methods require improvement in accuracy. Overall, the neural network’s consistency regardless of the level of occlusion, as compared to the triangle method’s poor performance when there is occlusion, demonstrates the neural network’s potential to improve computer vision for force feedback in RMIS, leading to better outcomes for patients.