Drone mapping by Hovermap
Among a growing list of customers in more than 30 countries are some of the world’s top mining companies, such as BHP, Rio Tinto, Barrick, Anglo American, Newmont, Glencore, Oz Minerals and Newcrest, as well as major businesses involved in infrastructure, defence, construction and mapping.
The reason?
Hovermap – technology that provides fast, accurate, 3D mapping in a product that can be hand-held, carried in a backpack, mounted on a vehicle or flown on a drone.
Moreover, Hovermap can operate entirely autonomously, and works with or without access to GPS, that is, in a GPS-denied environment such as an underground mine.
Wildcat SLAM generated 3D models for navigation
At Hovermap’s heart is Wildcat SLAM, a suite of algorithms for simultaneous location and mapping (SLAM) that CSIRO has spent more than 15 years developing.
Wildcat SLAM enables a drone or vehicle equipped with an omnidirectional LiDAR (light detection and ranging) sensor – the laser-light equivalent of a bat’s echolocation system – to generate a 3D model of its surrounds almost instantaneously.
This map can then be used to navigate the drone or other vehicle autonomously to survey unknown regions, even beyond the line of sight.
And it’s paired with sophisticated collision avoidance software, that keeps the conveyance from crashing into walls or any other objects in its path.
Technology for dangerous and hard to reach places
The result is a tool that can be used to survey, monitor or explore areas in underground mines where it would be difficult, dangerous or downright impossible for humans to venture, for example, where there are potentially toxic gases or the possibility of a wall or floor collapse.
Above ground, the technology can be used to inspect bridges, powerlines and buildings or to assess dangerous areas or even piles of material.
Hovermap not only can perform these tasks quickly and efficiently, but it can provide data for future inspections, comparisons and planning that can be gathered in no other way. The mining and resources industry sees the technology in Hovermap as a potential key to unlocking future automation.
“From a pure technology perspective, Hovermap is at the forefront,” says Richard Cheung, Digital Robotics & Automation Architect at Oz Minerals which operates copper mines in Australia and Brazil.
To work autonomously, robots need to be able locate themselves in space.
This is a fundamental problem of robotics, and one on which CSIRO has been working for decades—initially in two dimensions but for the past 15 years, with the vast increase in the sophistication of sensors and computing, in 3D.
“CSIRO is uniquely positioned in that we’ve been able to invest in this technology area over a very long period of time,” says Fred Pauling, team leader for Wildcat technology at Data61.
“Now it’s probably the best technology to hit the market.”
Autonomous functioning robots
For nine of those years Emesent’s chief technical officer, Farid Kendoul and chief executive office, Stefan Hrabar were part of the CSIRO team that worked on developing robots that could function autonomously.
“From 2015 on, we started seeing start-up companies raising millions and millions of dollars promoting themselves as world leaders in drone and robotic automation,” Farid says.
“It was frustrating because we knew we had developed better technology than them.”
So they struck out on their own and, with CSIRO’s backing, founded Emesent – the name is a portmanteau word derived from “emerging sentience”.
The partnership also works at another level – a two-way flow of information.
While Emesent can provide CSIRO with feedback on the performance of the technology and on future industry needs, CSIRO can provide Emesent and its industry partners with a vision of what’s possible in future.
Levels of autonomy
At present Hovermap comes in three different forms or products, says Farid.
They are based on the level of autonomy built in.
AL-0 (Autonomy Level 0) is solely a mapping system used by an operator, hand held, in a back pack or mounted on a vehicle.
AL-1 can be carried on a drone or robotic vehicle with an operator in control, but it has enough autonomy for collision avoidance, so it will not crash into obstacles or the ground.
With AL-2, however, the user simply defines its mission, and the drone takes over. The drone can execute the mission autonomously, even beyond line of sight and communication.
On the drawing board already are AL-3 and AL-4, devices that can be located permanently in a mine or other facility and conduct routine monitoring and surveying completely on its own, without each individual mission having to be defined.
Against this background of future development, many of the present applications of Hovermap in the resources industry do not even use its capability in an active mining area.
They capitalise on Hovermap’s ease and efficiency of data gathering in other facilities.
At one large iron ore development, for instance, a major mining company uses Hovermap to monitor the volume of explosive material in its storage sheds, something that is otherwise tricky and dangerous to measure.
Hovermap underground and on the moon
Even underground, Hovermap is being used in routine monitoring of tunnel convergence or deformation.
At the same time, it can detect things like missing roof bolts or pick up useful geotechnical information.
And the data it accumulates can be stored and processed later into highly accurate maps, or used for comparison with previous scans.
Some of Hovermap’s potential applications are not even on Earth.
"At a time when NASA has begun to revive space exploration beginning with the recent successful mission of the Artemis rocket to the Moon, Emesent is already discussing a potential role for Hovermap in exploring extra-terrestrial bodies," Farid says.
Put to the test in the DARPA robot olympics
A team comprising CSIRO Data61, Emesent and Georgia Tech recently competed with 10 others from the world’s top robotics institutes and companies – including NASA’s Jet Propulsion Laboratory – in the US Defense Advanced Research Projects Agency (DARPA) Subterranean Challenge.
The three-year competition involves mapping and locating objects in GPS-denied environments. These include:
- a tunnel system (a coal mine near Pittsburgh)
- an urban underground environment (an uncommissioned nuclear power station in Washington State), and
- a natural cave system.
The CSIRO Data61-Emesent-Georgia Tech team came second of 11 (ahead of JPL) and its solution incorporating CSIRO’s Wildcat technology won the 'Single Most Accurate Report' award.
Nearly half of the people employed at Emesent are research engineers. They represent the future.
“With the exception of Wildcat SLAM, we have developed and own most of the technology in our products,” Farid says.
“From the beginning Hovermap was developed as a versatile plug and play system, to make it easy for our different customers to integrate it with their technology and workflows.”
This integration can take many forms.
For instance, depending on the sensors available, the LiDAR ranging data Hovermap uses to construct its 3D model of the real world can be integrated with information from other sensors, such as cameras and various other means of sensing the surrounding materials.
One of the physical problems to solve is the trade-off between the flight time and range of drones with the weight of the payload of sensors and equipment they can carry—the heavier the payload the more limited the flight time.
Emesent researchers are currently working on ways of miniaturising sensors, while at the same time drone manufacturers are developing ever more powerful models of their equipment.
Meanwhile, CSIRO researchers are looking even further into the future with discussions underway for applications for Wildcat in agriculture, defence, security, and critical infrastructure.
But the real advances will be made in automation, using teams of robots both on the ground and in the air to manage farms or mines.
“During the DARPA challenge, each of the team of robots we used was able to develop a common, shared map of what all had seen,” says Jason Williams, team leader of robot autonomy at CSIRO Data61.
“That enables multi-agent coordination.
“A big part of the promise of autonomous robots is in productivity benefits. But if you have one person controlling a robot remotely, that’s not necessarily a great gain. There is a great gain, however, if that person can control ten or more robots and all know where each of them is.”