Search and Rescue
Telstra has built a UAV swarm in-house, for surveying and search and rescue applications. The swarm is made up of 9 drones which fly together in concert.
With such rich aerial data sets and video available in real time, how might we better augment emergency response operations?
Myself and a colleague Ravi Singh were asked to evaluate different technologies options to enable search and rescue operations for the swarm.
We kicked off our explorations with a full day workshop together with Microsoft's AI experts, Julian Lee and Ananth Prakash. We dived into the Azure platform offerings, the pre-configured deep learning virtual machines, and worked through various custom AI algorithms.
We collected video footage from an altitude of 20m at Caulfield Racecourse, and generated an initial dataset of 200 images.
Our first model is shown below, using a Faster RCNN model:
The Telstra UAV team headed out to shoot footage of the swarm in operation. My team went along to test out the Search and Rescue algorithm, pulling video live from a Phantom 4, through the DJI SDK. The code for the video integration was written by one of the graduates in our team, Ke Yi Ren.
The video shown below is the same model as above, trained for only 1 hour, and despite the more challenging environment it performs very well.
I have been collecting a wide range of additional training data, and using Active Learning to build it out with the help of a data trainer.
We are conducting ongoing experiments to evaluate the effectiveness of the AI algorithm from various altitudes.
Opportunities exist to leverage the next generation of Telstra's mobile network, 5G, through the use of high bandwidth video (4K, 8K). This allows the drones to fly twice as high, and still see the same detail on the ground, enabling faster search and rescue operations.
This technology will help augment emergency response teams. Rather than watching hours of video feeds and suffering fatigue, looking for very small objects from high altitude, the AI can search through the video feeds and alert the operator when a detection is made.
The video shot from the day was posted to Telstra's LinkedIn page, showcasing the work of the Telstra UAV program and UAV swarm team. It is a fantastic video, and a few shots of the computer vision video made it to the final cut!
