📷Sensors

Sensors (RoarPySensor) consist of the main inputs for any controller.

The ROAR_PY repository provides several standard interfaces to interact with different sensors, including a camera, lidar, GNSS (GPS), accelerometer, gyro, compass, etc.

The CARLA implementation provides a set of sensors that can be attached to the vehicle via the various RoarPyCarlaActor attach sensor methods.

ROARPyAccelerometerSensor

The accelerometer sensor returns a roar_py_interface.RoarPyAccelerometerSensorData. The wrapper contains the acceleration array following the Coordinate System in meters.

The following sample code creates an accelerometer sensor in CARLA, attaches it to a vehicle, and prints the data to the terminal.

#create an object and attach
accelerometer = vehicle.attach_accelerometer_sensor(...) # Parameter should include location, rotation of the sensor, etc.

#print accel data
try:
    while True:
        await carla_world.step()
        accel: roar_py_interface.RoarPyAccelerometerSensorData = await accelerometer.receive_observation()
        print(accel)
finally:
    roar_py_instance.close()
    

RoarPyCameraSensor

The camera sensor can return multiple types of data.

Depending on what kind of data is actually supported by the underlying hardware, the camera sensor returns one of the following data types: RoarPyCameraSensorDataRGB, RoarPyCameraSensorDataDepth, RoarPyCameraSensorDataSemanticSegmentation, RoarPyCameraSensorDataGreyscale.

All of the above data types are sub-classes of the base abstract type RoarPyCameraSensorData, which all have the method get_image() that converts the internal data into a human-viewable image typed by PIL.Image.

The converted PIL.Image is not guaranteed to be the same as the original internal data. It is just there for the convenience of viewing and debugging

To attach a camera in the CARLA implementation, we need to specify the sensor data, and the position and rotation relative to the binded actor/world.

For detailed data specs returned by those different camera data, refer to RoarPySensor.get_gym_observation_spec().

RoarPyCollisionSensor

The collision sensor returns a roar_py_interface.RoarPyCollisionSensorData. The wrapper contains an array of the following information:

• The current actor that reports the collision

• The other actor involved in the collision

• An array of normal impulses (x, y, z) resulting from the collision in Newton * second units.

1. (Optional) current actor that reports the collision

2. (Optional) the other actor that is involved in the collision

3. an array of normal impulse (x, y, z) resulting from the collision in Newton * second units.

RoarPyGnssSensor

The GNSS Sensor returns real-world aligned GPS data that reports the latitude(deg), longitude(deg), altitude(m) of the sensor location.

If you want to retrieve the location data with respect to the RoarPyWorld object then please use RoarPyLocationInWorld sensor, not GNSS sensor.

RoarPyGyroscopeSensor

The gyroscope sensor reports the angular velocity (in rad/s) of the binded actor in the (roll[x], pitch[y], yaw[z]) order.

RoarPyLiDARSensor

The LiDAR sensor returns a roar_py_interface.RoarPyLiDARSensorData. The wrapper contains:

• The number of lasers shot (channels)

• The horizontal angle in radians of the LiDAR sensor is rotated at the time of the measurement

• A 4-Dimension array that contains the point in XYZ coordinates as well as the computed intensity for this point as a scalar value between [0.0, 1.0].

RoarPyFrameQuatSensor / RoarPyRollPitchYawSensor

The RoarPyFrameQuatSensor and RoarPyRollPitchYawSensor returns the rotation data of the binded actor using either unit quaternion in the wxyz order or euler angles in roll, pitch, yaw order in radians.

RoarPyLocationInWorldSensor

The RoarPyLocationInWorldSensor returns the location of the binded actor in standard Coordinate System measurements (X, Y, Z) with respect to the underlying RoarPyWorld.

Remember the standard unit in ROAR_PY is meters.

RoarPyOccupancyMapSensor

RoarPyOccupancyMapSensor is the sensor that returns a grayscale image, centered around the location of the vehicle, representing the road information available provided by the RoarPyWorld.maneuverable_waypoints property.

Example Occupancy Map on the right, left is the image produced by a RoarPyCameraSensor

RoarPyRadarSensor

The radar sensor returns a roar_py_interface.RoarPyRadarSensorData. The wrapper contains a 4-Dimensional array that records the details for each point in the point cloud in:

• Altitude angle in radians

• Azimuth angle in radians

• Depth in meters

• Velocity in meters per second