3.3. Examples

3.3.1. Tutorial: How to integrate a SensoPart camera with Python

Introduction

This tutorial will demonstrate how a SensoPart V20 camera manufactured by SensoPart Industriesensorik GmbH can be integrated using Python. The SensoPart cameras communicate using the TCP/IP protocol. Once a job is configured on the camera, it can be triggered via TCP/IP.

To configure a SensoPart camera, additional software is necessary. Please refer to the Useful links section for the software and its documentation.

Getting Started

Useful links

VISOR Software documentation

VISOR PC software

Requirements

1. Download the VISOR software.

2. Connect the SensoPart camera to your computer and open the VISOR software to configure the camera according to the documentation.

3. Configure the job (Fig. 69/➊) of the camera using your preferred detector (Fig. 69/➋) according to the documentation.

4. Set the trigger mode to “Trigger” like shown in Fig. 70/➊

Fig. 69 SensoPart configuration overview

Fig. 70 SensoPart job configuration panel

Implementation Steps

Configuring the data output

1. Once the job is properly configured, open the Telegram tab (Fig. 71/➊). In this tab, the data output can be configured.

2. Set the Start identifier (Fig. 71/➋) to “(” and the Trailer identifier (Fig. 71/➌) to “)”.

3. Set the Separator (Fig. 71/➍) to “,”.

4. In the payload on the right side, tick the boxes for the Pos. X, Pos. Y and Angle Z (Fig. 71/➎).

5. Please refer to the VISOR Software documentation for further information on how to configure the data output.

6. If everything is properly configured (see Fig. 71), click on Start sensor (Fig. 69/➌) to upload the job to the connected camera. Afterwards the camera can be disconnected from your computer and be moved somewhere else.

Fig. 71 SensoPart output configuration panel

Writing a Python TCP/IP client for the camera

The code section below (Basic SensoPart Python class.) provides a client implementation to communicate with a SensoPart camera via TCP/IP. The main functionalities include establishing a connection, triggering a job, and retrieving the results.

Listing 1 Basic SensoPart Python class.

import socket
import string


class V20:
    def __init__(self, device_host: string, device_port: int = 6001, timeout: int = 10):
        """
        Initialize the client for the sensopart TCP socket.

        Args:
            ipAddress (string): _description_
        """
        self.device_host = device_host
        self.device_port = device_port
        self.timeout = timeout
        self.is_connected = False
        self._attempt_connection_if_necessary()

    def _connect(self) -> None:
        # Create a socket object (IPv4 address and TCP protocol)
        try:
            self.client_socket.connect((self.device_host, self.device_port))
            self.is_connected = True
            print(
                f"Connection to the TCP socket {self.device_host}:{self.device_port} was successful!"
            )
        except socket.error as exc:
            print(f"Failed to establish connection with the Sensopart camera: {exc}")

    def disconnect(self):
        self.client_socket.close()

    def _attempt_connection_if_necessary(self):
        if not self.is_connected:
            # Socket needs to be reopened, cannot attempt connect twice
            self.client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.client_socket.settimeout(self.timeout)
            self._connect()

    def trigger_job(self):
        self._attempt_connection_if_necessary()
        try:
            # Send the "trigger" command
            trigger_command = "TRG"
            self.client_socket.sendall(trigger_command.encode("utf-8"))

        except socket.error as exc:
            print("Error in trigger_job(): ", exc)

    def read_results(self):
        self._attempt_connection_if_necessary()
        try:
            response = ""
            while ")" not in response:
                # Receive a chunk of data
                chunk = self.client_socket.recv(1).decode("ascii")

                response += chunk

            # Check if the received response is valid
            # if ")" in response and any(val != "0" for val in response.strip("()").split(",")):
            # Extract values from the response string
            object_pos = response.strip("()").split(",")

            # Convert values to floats and rename variables
            x_pos, y_pos, z_rot = map(lambda v: float(v) / 1000, object_pos)

            # Print values with units
            print(f"x_pos: {x_pos:.3f} mm, y_pos: {y_pos:.3f} mm, z_rot: {z_rot:.3f}°")

            return x_pos, y_pos, z_rot

        except socket.error as exc:
            print("Error in read_results(): ", exc)
            raise exc

• Line 5: This class contains the TCP/IP client for the SensoPart camera.

• Line 6: The __init__ constructor initializes the client with the provided device host, port, and timeout settings. The initialization is executed automatically when an instance of the class is created.

• Line 19: This private method establishes a connection to the SensoPart camera.

• Line 30: This method closes the connection to the SensoPart camera. This is especially useful since most cameras only allow to be accessed by one resource at the time. If the connection is not closed properly, the user might need to restart the camera before they can reestablish a connection.

• Line 33: This method checks if a connection is established and attempts to connect if not. Useful to avoid redundant connection attempts.

• Line 40: This method triggers a job on the SensoPart camera.

• Line 50: This method reads the results of a job triggered on the SensoPart camera. The response syntax has been configured in Configuring the data output before. It continuously receives data until it encounters a closing parenthesis “)”. It then parses the received data, extracts the object’s position values, converts them to floating-point values in millimeters and degrees, and prints them with units. Upon successful execution, it returns a tuple containing the object’s x and y position and z rotation.

If any errors occur during the connection process or while reading results, error messages are printed and exceptions are raised.

The following code section gives an example on how to use the V20 class during runtime:

#Establish connection
client = V20(device_host = "192.168.100.100", device_port = 2006)

client.trigger_job()

(x_pos, y_pos, z_rot) = client.read_results()

print(f"Pose of detected object: x:{x_pos}mm, y:{y_pos}mm, z:{z_rot}°")

client.disconnect()

Troubleshooting

Table 1 Common mistakes, raised errors and how to handle them

Error Message	Comment
Failed to establish connection with the SensoPart camera: timed out	Check the cameras network settings in the VISOR software and the used IP address and port.
Error in trigger_job()/read_results():	Check whether the connection was interrupted.
The program is stuck in the read_results() method	Check if the data output was configured correctly in the VISOR software. Did you miss a closing “)”?
The results are all zeros	No object was detected. Try a different illumination of the scene and make sure that the job is configured correctly in the VISOR software

3.3.2. Tutorial: How to integrate a SensoPart camera with the voraus.operator

Introduction

In Tutorial: How to integrate a SensoPart camera with Python you learned how to communicate with a SensoPart V20 camera manufactured by SensoPart Industriesensorik GmbH using Python. This tutorial will demonstrate how the previously written Python code can be integrated directly into a new program with the voraus.operator.

Getting Started

Note

The SensoPart camera used for this tutorial was configured with the IP address 192.168.100.200. The port 2006 was used for the request channel and the port 2005 for the response channel. This might be different for your setup, depending on your camera network settings. Just replace these values in the tutorial according to your needs. It is also possible to configure the SensoPart camera in a way that one port is used for both channels. Please refer to the documentation mentioned in Tutorial: How to integrate a SensoPart camera with Python.

Requirements

1. A running voraus.core with the voraus.operator. Follow the installation instructions of your voraus.core version in voraus.core.

2. You followed the Tutorial: How to integrate a SensoPart camera with Python. Especially the configuration of the SensoPart camera with the VISOR software was already successful.

3. You already know the basics of how to use the voraus.operator, e.g. how to create a new program and how to add commands.

Implementation Steps

1. Create and load a new program in the voraus.operator called “SensoPart Integration Example”.

2. For the first command, add a Python code block containing the base class for the camera communication.

   1. Navigate to the LOGIC tab (Fig. 72/➊) and select the Run Script command (Fig. 72/➋)

      

      Fig. 72 Adding a Python code block to the program

   2. Copy and paste the content of the Python base class described in Tutorial: How to integrate a SensoPart camera with Python into the text field (Fig. 73/➊). Afterwards click on Apply (Fig. 73/➋)

      

      Fig. 73 Defining the base class

3. Next, add a Move To motion command to the starting pose of the program. The robot should be moved in such a way that the view of the camera is not obscured (Fig. 74).

   

   Fig. 74 Adding a starting pose

4. Before including another Python code block for detecting the object, add a Wait command, to wait for the robot to stop moving. Otherwise, the detection might trigger, although the starting pose was not reached yet.

   1. Navigate to the LOGIC tab (Fig. 75/➊) and select the Wait command (Fig. 75/➋)

      

      Fig. 75 Adding a wait command

   2. Inside the command, set the waiting criterion to Wait until robot stopped (Fig. 76/➊). Afterwards, click on Apply (Fig. 76/➋)

      

      Fig. 76 Selecting the waiting criterion

5. As in Fig. 72, add another Python code block in which the detection is triggered and the result is parsed (Fig. 77). The content (Fig. 77/➊) looks like the following:

       #Establish connection
       request_client = V20(device_host = "192.168.100.200", device_port = 2006)
       response_client = V20(device_host = "192.168.100.200", device_port = 2005)
   
       request_client.trigger_job()
   
       (x_pos, y_pos, z_rot) = response_client.read_results()
   
       logger.info(f"Pose of detected object: x:{x_pos}mm, y:{y_pos}mm, z:{z_rot}°")
   
       request_client.disconnect()
       response_client.disconnect()
   
       if x_pos == 0.0:
           return
   

   In this code block, two instances of the base class are initialized, one for the request channel and one for the response channel (Lines 2 & 3). Afterwards, the detection is triggered from the request client (Line 5) and the detection result is read out with the response client (Line 7). The connection to both clients is closed in Lines 11 & 12. In case that no object was detected, x_pos, y_pos, and z_rot will be all zero, terminating the program. This way, the robot does not try to move to the invalid pose (Line 14).

   After the command is configured, click on Apply (Fig. 77/➋).

   

   Fig. 77 Adding a code block for detecting the object

6. Add an additional Python code block that saves the detection result to USERCS1 (Fig. 78). The content (Fig. 78/➊) looks like the following:

       import numpy as np
   
       pose = robot_arm.Pose((x_pos/1000, y_pos/1000, 0, np.deg2rad(-180),0, np.deg2rad(z_rot)), enums.UserCS.ROBOTCS.value)
       logger.info(f"Resulting Pose: {pose}")
       self.robot.set_user_cs(pose, enums.UserCS.USERCS1.value)
       time.sleep(0.1)
   

   Note

   The x-rotation of the USERCS1 is set to -180° (Line 3) because the tool coordinate system is flipped by 180° with respect to to the robot coordinate system.

   The time.sleep(0.1) at the end (Line 6) ensures that the coordinate system is saved before it is used.

   

   Fig. 78 Adding a code block for setting a UserCS

   After the command is configured, click on Apply (Fig. 78/➋).

7. Depending on the workspace of the robot, it might be necessary to add some intermediate poses, so that the robot can move from the starting pose to the object pose without causing a collision with the environment. Add a Move To command containing the intermediate pose (Fig. 79).

   

   Fig. 79 Adding an intermediate pose

8. Now move to the detected object using the USERCS1 which was defined earlier. Add a Move To command and select the User CS 1 as the coordinate system (Fig. 80/➊). If you want to move to the center of the object (the detected coordinates represent the object’s center), keep the x Translation and y Translation values at 0.0. In this example, it is desired to move 300 mm above the object. Therefore, set the z Translation to -300 mm (Fig. 80/➋).

   Note

   Setting the z Translation to 0.0 could be used to grasp the object.

   Note how negative values for the z Translation have to be used, due to the rotation of the USERCS1 by 180° around the x-axis.

   

   Fig. 80 Adding a pose above the object

9. Finally, add two more Move To commands. One moving back to the intermediate position and one moving back to the starting position. The final program can now be saved (Fig. 81/➊).

   

   Fig. 81 Saving the program

Troubleshooting

Table 2 Common mistakes, raised errors and how to handle them

Error Message	Comment
Program is idling for some time, then the program terminates with an error.	Timeout when trying to establish a connection to the camera. Make sure the network settings of the camera are correct.
The program terminates without an error, but the robot does not move.	No object was detected, make sure the camera’s detection job is configured correctly.