Time of Flight Sensor as Camera Component

[chilly crown]

Hi there! Thanks for having me at the Open Lab last night - super fun and helpful.

I’m building a little rover to deliver notes - it will have a thermal receipt printer mounted on it and the idea is to click on a map to send the rover over to deliver a note.

The rover will not have lidar or camera on it - I’m going low budget and using a VL53L0X Time of Flight sensor (possibly two) [1]. I’d like to use the time of flight sensor to produce a point cloud that will feed into the SLAM Cartographer Service. (I recognize that I might be headed into danger and the ToF sensor will probably not produce nearly enough data for SLAM. 😂🤷🏼)

The ToF sensor has a Python client so I’m implementing the camera component using the Python SDK. I originally implemented it as a sensor component [2] but am trying to implement it as a Camera instead.

I’m stuck on how to implement get_point_cloud. I want to make the point cloud by hand, which will only be a single point. It looks like the bytes are encoded as a PCD. Is there a way to manually build the point cloud and encode it as a PCD?

async def get_point_cloud(self, *, timeout: Optional[float] = None, **kwargs) -> Tuple[bytes, str]:
    // Return a point cloud in the form of [[range, 0, 0]] - the single sensor reading from the ToF sensor.

[1] https://learn.adafruit.com/adafruit-vl53l0x-micro-lidar-distance-sensor-breakout/overview
[2] https://github.com/ethanlook/palbot/blob/main/vl53l0x_sensor.py

[queen shell]

Hey Ethan, it was good talking to you about this last night. I'm tagging @robust steeple in on this since he has more context here

[chilly crown]

Thanks, Ray! And thank you in advance, Bijan!

[robust steeple]

Nice to meet you @chilly crown ! So as you pointed out, you'll have to send out a point cloud in PCD format using the python SDK as a server. The common library in C++ to mess around with pointcloud is the Point Cloud Library (PCL) . There are python bindings for the code here at this repo: https://github.com/strawlab/python-pcl

I admit, I have not used this library before, but pcl should be able to read/write to PCD format

[chilly crown]

Awesome, thanks for the pointer! I'll give it a shot and let you know.

[robust steeple]

Actually, it looks like that library hasn't been updated for quite a few years! Open3D looks like its more recently maintained: http://www.open3d.org/docs/release/tutorial/geometry/file_io.html?highlight=pcd

http://www.open3d.org/docs/release/python_api/open3d.io.write_point_cloud.html#open3d.io.write_point_cloud

[chilly crown]

Thanks! I'll give that a try - documentation for Open3D seems a little sparse and it's not clear how to construct a point cloud without reading from a file.

Not sure if this makes sense outside of my use case, but it would be convenient to send the point cloud as a serialized proto instead of working with PCDs.