Academia.eduAcademia.edu

Figure 3 – uploaded by Alexandre Krupa

See full PDFdownloadDownload figure
Fig. 3. Robust detection of optical markers. only other source of vertical high frequency components in the image is motion as shown in Fig. 4.

Figure 3 Robust detection of optical markers. only other source of vertical high frequency components in the image is motion as shown in Fig. 4.

Related Figures (17)

Fig. 1. System confguration.
Fig. 1. System confguration.
Fig. 2. The endoscopic laser pointing instrument: holder. SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
Fig. 2. The endoscopic laser pointing instrument: holder. SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
Fig. 4. Suppression of artifacts due to motion. In this example, the left pattern in the original image is produced by a blinking marker and the right pattern is in the image. After high-pass filtering and thresholding, matching algorithm. The horizontal pattern around the detec are next to this pixel. If they match, then the pixel is remov This matching is very fast since it is limited to the detec pixels. blinking marker is detected as expected but also the moving edge. The artifacts due to the moving edge are removed by a produced by the image of an edge moving from left to right the ted pixel is compared with the horizontal pattems in the lines that ed. ted
Fig. 4. Suppression of artifacts due to motion. In this example, the left pattern in the original image is produced by a blinking marker and the right pattern is in the image. After high-pass filtering and thresholding, matching algorithm. The horizontal pattern around the detec are next to this pixel. If they match, then the pixel is remov This matching is very fast since it is limited to the detec pixels. blinking marker is detected as expected but also the moving edge. The artifacts due to the moving edge are removed by a produced by the image of an edge moving from left to right the ted pixel is compared with the horizontal pattems in the lines that ed. ted
Fig. 5. Detection of laser spots. a) original interlaced image. - b) high-pass fltering and thresholding on even frame. - c) matching. - d) localization of center of mass (square).
Fig. 5. Detection of laser spots. a) original interlaced image. - b) high-pass fltering and thresholding on even frame. - c) matching. - d) localization of center of mass (square).
CMS LEDs markers are turned on during the odd feld and tured off during the even feld. Edge detection is applied on the result of high-pass fltering and matching in the odd feld. Edges detector always yields contours with many pixels of thickness. Thinning operations are performed on the extracted set of pixels based on the comparison of gradient magnitude and direction of each pixel with their neighbors (non maxima suppression) producing a 1-pixel wide edge: this thinning is required to apply hysteresis thresholding and an edge tracking algorithm. Then, contours are merged by using a method called mutual favorite pairing [6] that merges neighboring contour chains into a single chain. Finally, the contours are fitted by ellipses (see Fig. 6). Fig. 6. - (left) Detection of optical markers and laser spots (+) - (right) Contours detection of optical markers (odd frame).
CMS LEDs markers are turned on during the odd feld and tured off during the even feld. Edge detection is applied on the result of high-pass fltering and matching in the odd feld. Edges detector always yields contours with many pixels of thickness. Thinning operations are performed on the extracted set of pixels based on the comparison of gradient magnitude and direction of each pixel with their neighbors (non maxima suppression) producing a 1-pixel wide edge: this thinning is required to apply hysteresis thresholding and an edge tracking algorithm. Then, contours are merged by using a method called mutual favorite pairing [6] that merges neighboring contour chains into a single chain. Finally, the contours are fitted by ellipses (see Fig. 6). Fig. 6. - (left) Detection of optical markers and laser spots (+) - (right) Contours detection of optical markers (odd frame).
where a and 7 depend only on the known relative position of P,, P. and P;. Similar computations lead to the same rela- tionship between dz and another cross-ratio defned with the points P,, P2, P3,Og and their respective projections provided
where a and 7 depend only on the known relative position of P,, P. and P;. Similar computations lead to the same rela- tionship between dz and another cross-ratio defned with the points P,, P2, P3,Og and their respective projections provided
Fig. 7. The basic geometry involved. SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS that o,, the perspective projection of the incision point Og, can be recovered. Since Og is generally not in the camera feld of view, this can be achieved by considering a displacement of the surgical instrument between two confgurations yielding straight lines 6 and 5’ in the image. Then, o, is the intersection of these lines since Og is motionless. Finally:
Fig. 7. The basic geometry involved. SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS that o,, the perspective projection of the incision point Og, can be recovered. Since Og is generally not in the camera feld of view, this can be achieved by considering a displacement of the surgical instrument between two confgurations yielding straight lines 6 and 5’ in the image. Then, o, is the intersection of these lines since Og is motionless. Finally:
Fig. 8. Standard deviation o4, of the estimated depth do (mm) as a function of the standard deviation o, of the markers image coordinates (pixel) for several geometrical confgurations.
Fig. 8. Standard deviation o4, of the estimated depth do (mm) as a function of the standard deviation o, of the markers image coordinates (pixel) for several geometrical confgurations.
Fig. 9. The full visual servoing scheme.
Fig. 9. The full visual servoing scheme.
Fig. 10. Visual servoing scheme used for experiments. and the stability of the visual feedback loop is guaranteed as long as J.,[J.,]~! remains positive defnite [2]. In our application, if the camera and the incision point are motionless, the stability is ensured in a workspace much larger than the region covered during experiments. To quantify the stability properties, we have modelled the organ as an ellipsoid. The estimated Jacobian J., is constant and correspond to a nominal configuration. We have then computed J, when the laser spot is moved accross the organ surface, and computed the eigen- values of J.,[J..]~! in the different confgurations. Unsafe
Fig. 10. Visual servoing scheme used for experiments. and the stability of the visual feedback loop is guaranteed as long as J.,[J.,]~! remains positive defnite [2]. In our application, if the camera and the incision point are motionless, the stability is ensured in a workspace much larger than the region covered during experiments. To quantify the stability properties, we have modelled the organ as an ellipsoid. The estimated Jacobian J., is constant and correspond to a nominal configuration. We have then computed J, when the laser spot is moved accross the organ surface, and computed the eigen- values of J.,[J..]~! in the different confgurations. Unsafe
Fig. 12. Stability analysis - laser spot surface area delimiting the stability of the control vision loop for a constant identified Jacobian. Fig. 11. Initial on-line identifcation of the interaction matrix (displacements around the image center) and image-based visual servoing along a square using this identifcation. (1 mm ~ 25 pixels)
Fig. 12. Stability analysis - laser spot surface area delimiting the stability of the control vision loop for a constant identified Jacobian. Fig. 11. Initial on-line identifcation of the interaction matrix (displacements around the image center) and image-based visual servoing along a square using this identifcation. (1 mm ~ 25 pixels)
SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
Fig. 13. The experimental setup. Step 4: descent of the instrument by applying a velocity reference signal v* in open loop until it
Fig. 13. The experimental setup. Step 4: descent of the instrument by applying a velocity reference signal v* in open loop until it
The robot vision system presented in this paper automatically positions a laparoscopic surgical instrument by means of laser pointers and optical markers. To add Fig. 14. Experimental measurements for the identifcation procedure of the interaction matrix. (top) Slow identifcation procedure (averaging the effect of breathing) and (bottom) fast identifcation procedure (short time interval between two regular breathings). (1 mm ~ 25 pixels)
The robot vision system presented in this paper automatically positions a laparoscopic surgical instrument by means of laser pointers and optical markers. To add Fig. 14. Experimental measurements for the identifcation procedure of the interaction matrix. (top) Slow identifcation procedure (averaging the effect of breathing) and (bottom) fast identifcation procedure (short time interval between two regular breathings). (1 mm ~ 25 pixels)
TIME PERFORMANCES OF THE RECOVERING AND POSITIONING TASKS FOR A SET OF 10 EXPERIMENTS.
TIME PERFORMANCES OF THE RECOVERING AND POSITIONING TASKS FOR A SET OF 10 EXPERIMENTS.
Fig. 16. New desired respective positions (of the surgical instrument with respect to the pointed organ) specifed in the image by the surgeon (step 6). Responses (a)-(b) are obtained on living tissue. Responses (c)-(d) are obtained by the use of an endo-trainer with no disturbance due to breathing. (1 mm = 25 pixels) SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
Fig. 16. New desired respective positions (of the surgical instrument with respect to the pointed organ) specifed in the image by the surgeon (step 6). Responses (a)-(b) are obtained on living tissue. Responses (c)-(d) are obtained by the use of an endo-trainer with no disturbance due to breathing. (1 mm = 25 pixels) SUBMITTED TO IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION - SPECIAL ISSUE ON MEDICAL ROBOTICS
Alexandre Krupa was born on April 20, 1976 in Strasbourg, France. He received the M.S. degree (DEA) in Control Science and Signal Processing from the National Polytechnic Institute of Lorraine, Nancy, France in 1999. He is currently working to- ward the completion of his Ph.D. degree in robotics in the Laboratoire des Sciences de |’Image, de l'Informatique et de la Télédétection (LSIIT) at the University of Strasbourg. His research interests include medical robotics, visual servoing of robotic manipulators, computer vision and micro-robotics.
Alexandre Krupa was born on April 20, 1976 in Strasbourg, France. He received the M.S. degree (DEA) in Control Science and Signal Processing from the National Polytechnic Institute of Lorraine, Nancy, France in 1999. He is currently working to- ward the completion of his Ph.D. degree in robotics in the Laboratoire des Sciences de |’Image, de l'Informatique et de la Télédétection (LSIIT) at the University of Strasbourg. His research interests include medical robotics, visual servoing of robotic manipulators, computer vision and micro-robotics.
Related topics:
Mechanical EngineeringRoboticsRobot VisionLaparoscopic SurgeryMedical RoboticsManufacturing EngineeringElectrical and Electronic EngineeringMinimal Invasive Surgery
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved