Operating robots with virtual reality

Bagus Manuaba is researching at the ANU how to remote control a machine using virtual reality. Devices such as bomb disposal robots are normally operated by remote control, with an operator looking at a live video display and pushing buttons and operating a joystick. The operator is only a few hundred metres from the robot and so there is little delay transmitting video from the robot to the operator and the operators instructions back. However, as the distance in increases the, delay does also.

Where the operator is on one side of the world and the robot the other, the delay prevents fine control and the robot has to be semi-autonomous. In addition, looking at a flat 2d image makes it hard to control a robot.

Some systems use stereoscopic video, such as the Airbus Military Aerial Refuelling Boom System (ARBS) fr the Airbus A330 Multi Role Tanker Transport (MRTT) aircraft being acquired by the Royal Australian Air Force. With this system the operator sits in the aircraft cockpit and fly the boom while looking at a stereoscopic image. Previous systems, such as the KC-135, require the operator to be in the tail of the aircraft. It might be interesting to enhance the Airbus system with virtual reality.

Teleoperation System with Supervisory Control in a Mixed Reality Environment

Bagus Manuaba (SoCS CECS)

CS HDR MONITORING Info & Human Centred Computing Research Group

DATE: 2010-04-15
TIME: 11:30:00 - 12:00:00
LOCATION: Ian Ross Seminar Room
CONTACT: Michelle.Moravec@anu.edu.au

ABSTRACT:
Mixed reality is a new innovation of virtual reality. By combine this innovation with supervisory control in teleoperation system is expected can solve any limitations that occur in teleoperation system today. In this presentation I will give a brief description about teleoperation, supervisory control and mixed reality, and also bring several examples of research that related in this area. In addition, it is also showing the focus and the expected outcomes from my current research.

Australian Robot Aircraft Launched from US Stealth Warship

The Australian developed Aerosonde UAV has been succesfully launced and recorved from the US stealth warship M80 Stletto, according to Janes International Defence Review ("Aerosonde Mark 4.7 UAS proves shipborne capability", March 2010). The AAI Aerosonde Mk 4.7 is one option for the US Navy/US Marine Corp's Small Tactical UAS (STUAS)/Tier II programme. The Aerosone carries visable and infrared sensors and a laser range finder/pointer. The Aerosonde is small enough to be lifted by one person but can fly for 12 hours and an earlier model flew accross the Atlantic Ocean.

Controlling robots via the Internet

Professor Sandra Hirche will talk on "Human-oriented telerobotic control over the Internet" at the ANU in Canberra, 16 February 2010:

SYSTEMS AND CONTROL SERIES

Human-oriented telerobotic control over the Internet

Prof. Sandra Hirche (Technical University of Munich (TUM))

DATE: 2010-02-16
TIME: 11:00:00 - 12:00:00
LOCATION: RSISE Seminar Room, ground floor, building 115, cnr. North and Daley Roads, ANU

ABSTRACT:
Packet switched communication networks, as e.g. the Internet, are very attractive for the transmission of haptic information between the operator and the teleoperator side in a telerobotic systems. As a result of the haptic coupling a global control loop is closed through the packet switched communication network with the human and the environment being part of it. Without further control measures the intrinsic communication unreliabilities render the haptic telepresence system unstable. Similar challenges arise in the closely related field of networked control systems~(NCS), however, in haptic telepresence systems extended stabilization methods are required due to the largely unknown human/environment dynamics. Human factors play a major role in the evaluation of the control performance in terms of transparency.

This talk focuses on the stability and transparency of haptic telepresence systems in packet switched communication networks. Conjointly control theoretic, communication related, and human haptic perception aspects are considered. Based on the passivity framework methods to stabilize the system with time delay, packet loss and haptic data compression are introduced. The transparency for all the considered control approaches is evaluated by an extended transparency measure taking into account human haptic perception. The obtained results are validated in experimental user studies.

BIO:
Sandra HIRCHE (*1974) is professor and director of the Information-Oriented Control Laboratory associated with the Institute of Automatic Control Engineering in the Faculty of Electrical Engineering and Information Technology, Technische Universität München, Munich, Germany. She received the diploma engineering degree in mechanical engineering and transport systems in 2002 from the Technical University Berlin, Germany, and the Doctor of Engineering degree in Electrical Engineering in 2005 from the Technische Universität München, Munich, Germany. Being awarded a postdoctoral research fellowship from the Japanese Society for the Promotion of Science (JSPS) she has been a PostDoc at Fujita Lab, Tokyo Institute of Technology, Japan from 2005 to 2007. Her research interests include networked and distributed control systems, cooperative control, human-robot interaction, multimodal telepresence systems, and perception-oriented control.

Robots at war

"Wired for war : the robotics revolution" (Singer, P. W. 2009) is a very readable book about the use of robots in modern warfare. It suffer from having a very US centric view of the field and providing a few too many anecdotes. But it is an easy read for someone needing an overview.

Singer starts with anecdotes about the use of bomb disposal robots in Iraq. He describes the origins and different development philosophies of the two major companies supplying the US defence department. Having attended a seminar by Professor Rodney Brooks, an Australian from Adelaide and one of the founders of iRobot, I can see that Singer's analysis is insightful.

What is missing is the discussion of the development of robotics outside the USA and the role of the scientific research community. As an example, the Aerosonde UAV, which is now being marketed to the military, was developed in Melbourne, (Australia), for taking meteorological measurements (thus the name "Aero-sonde". The aircraft was later adapted for other remote sensing and military applications.

Aerosonde pioneered small long endurance autonomous UAVs (having flow across the North Atlantic). Previously it was assumed that UAVs small enough for a person to lift would only have a range of a few hours.

One problem with innovation is having something too different from the competition. Aerosonde faced this with their early models which were controlled from an ordinary laptop computer. This removed the need for specialised control units. But rather than being seen as an advantage, this counted as a disadvantage in the logic of military procurement. With the Aerosonde Mark 4.7, released in March, there has been effort to provide compatibility with military UAV systems, such as NATO STANAG 4586 standards for UAV ground stations (also see the STANAG-4586 LinkedIn Group).

Aerosonde also pioneered the idea of UAVs being provided as a service, rather than individual aircraft purchased by the customer. This idea is yet to take off with UAV customers, but with widespread use, it appears an idea who's time has come. This concept is not discussed in Singer's book.

Earlier in the year the Australian and US Defence Departments announced the Multi Autonomous Ground-robotic International Challenge (MAGIC 2010). This is a competition researchers to build a fleet of cooperating autonomous ground vehicle systems (robots) for military and civilian emergency use. These will be tested in Australia in November 2010.

Robot Cameras for Parliament

The Department of Parliamentary Services has issued a Request for Tender for "Provision of a Camera Robotics Solution" (DPS09014, 14-Jul-2009) for 47 remote control cameras, with seven control units. These are used to record parliament, committees and some conferences. There are detailed documents provided, describing the requirement, technical architecture and standards. The parliament also recently issued a RFT for a Video Archiving System.

Australia US Military Robot Competition

The Multi Autonomous Ground-robotic International Challenge (MAGIC 2010) is a Australian and US Defence Department competition. Teams of researchers will each be given small research grants of up to $100,000 to build a fleet of cooperating autonomous ground vehicle systems (robots) for military and civilian emergency use. These will be tested in Australia in November 2010.

The Multi Autonomous Ground-robotic International Challenge (MAGIC 2010) is jointly sponsored by the Australian and US Departments of Defence to attract innovative proposals from worldwide research organisations to develop next-generation fully autonomous ground vehicle systems that can be deployed effectively in military operations and civilian emergency situations.

What’s the challenge?
The challenge requires competitors to submit proposals demonstrating the use of multi-vehicle robotic teams that can execute an intelligence, surveillance and reconnaissance mission in a dynamic urban environment.
Shortlisted competitors will need to field cooperatives of unmanned vehicle prototypes with the ability to autonomously and dynamically coordinate, plan and carry out tasks against changing priorities.
To complete the challenge competitors must: (i) accurately and completely explore and map the challenge area; (ii) correctly locate, classify and recognise all simulated threats; and (iii) complete all phases within 3.5 hours. The challenge event will be conducted in Australia during November 2010.

Eligibility

The challenge is open to national and international organisations from industry and academia. Government organisations are not eligible.

Time Table

MAGIC 2010 will be conducted in two phases:

1. Initial shortlisting of ten teams based on their proposals – 19 October 2009.
2. Grand challenge event – 8-14 November 2010.

Date	Event
1 July 2009	Challenge announced
31 July 2009	Participants Conference, Adelaide ( Australia )
4 Aug 2009	Participants Conference, Frankfurt (Germany)
6 Aug 2009	Participants Conference, TARDEC Warren, Michigan ( USA )
4 Sep 2009	Intention to Compete forms – closing date
2 Oct 2009	Closing date for receipt of technical proposals from competitors
19 Oct 2009	Ten shortlisted teams notified
2 – 13 Nov 2009	Site visits by judging panel for detailed technical assessments of ten shortlisted competitors
16 Nov 2009	Five competitors (out of ten) selected and research contracts awarded - 50% first payment initiated
7-21 June 2010	Site visits to shortlisted competitors by assessment panel. Progress assessments - balance 50% payment initiated
22 Oct 2010	Closing date for submitting MAGIC 2010 Technical Paper for Land Warfare Conference
8-13 Nov 2010	Grand Challenge Event at undisclosed location in South Australia
15-19 Nov 2010	Land Warfare Conference, Brisbane (winners announced).

Research awards

Of the ten shortlisted teams, five teams will be selected to receive US$100,000 each to develop and build their unmanned vehicle prototypes.

After successfully completing the challenge the top three ranked teams will receive US$750,000, US$250,000 and US$100,000 respectively.

Judging

A panel of international experts in the field of robotics, military operations, academia, and defence R&D will be appointed. The judges will be responsible for verifying compliance with the rules and determining the winners.

Proposal presentation

The selected competing teams will be required to submit a detailed technical paper covering all aspects of their technology for presentation and publication in the proceedings of the Land Warfare Conference 2010 being held in Brisbane, Australia from 15 to 19 November 2010.

More information
Detailed information on challenge rules, entry requirements, judging criteria and forms can be accessed here...

From: Multi Autonomous Ground-robotic International Challenge (MAGIC 2010), DSTO, 2009

Australian Robot Software Wins Award

A team of students Australian National University have won the AIIA 2009 iAward in the Tertiary Student Category for their Linux Robotics Framework. This can be used to develop low cost robots for everything from vacuum cleaners to bomb disposal.

The team's demonstration robot is called Buzz , controlled by a NGW100 network gateway low-powered development board with a Atmel AVR32 processor. This could allow for the development of disposable military robots. Currently each military robot used by the US DoD and the Australian Department of Defence costs more than $100,000.

The Linux Robotics Framework ... can be used to develop robotic systems of varying levels of professionalism. It streamlines the development process for hobbyists and commercial developers alike. The LRF allows for the development of robotics applications on small low-cost system architectures with specific focus on support for the Atmel AVR32 microprocessor. It does this by providing a collection of component modules comprising a reusable and extensible robotics framework. The framework includes interface definitions and the implementation of specific drivers and libraries. The framework is extensible, providing a mechanism for adding new hardware and software drivers.

From: Linux Robotics Framework, ANU Linux Robotics Framework Team, 2009

As noted in Bomb Squad Diary (Glenn Zorpette, IEEE Spectrum, October 2008), the threat from Improvised Explosive Devices (IEDs) is changing rapidly and equipment needs to be constantly modified to adapt to the changes in the tatitics and techniques of the bomb makers. Being able to rapidly modify the software and the hardware of the robots would be an advantage.

Simplified Robotics Framework for Bomb Disposal

A team of students at the ANU have developed a Linux Robotics Framework to simplify development of robotics applications. This should be particularly useful for bomb disposal robots, where the requirement change rapidly due to changes in the threat.

As noted in Bomb Squad Diary (Glenn Zorpette, IEEE Spectrum, October 2008), the threat from Improvised Explosive Devices (IEDs) is changing rapidly and equipment needs to be constantly modified to adapt to the changes in the tatitics and techniques of the bomb makers. Being able to rapidly modify the software and the hardware of the robots would be an advantage.

The team's demonstration robot is called Buzz , controlled by a NGW100 network gateway low-powered development board with a Atmel AVR32 processor. This could allow for the development of disposable military robots. Currently each military robot used by the US DoD and the Australian Department of Defence costs more than $100,000.

Australian Robotic Weapon Platform

Metal Storm have displayed their high speed mortar on a military PacBot robot from an iRobot. The Metal Storm gun is electrically fired, with a minimum of moving parts, making it a good match for a robot. However, a cluster of off the shelf single shot, electrically primed mortars would be almost as effective, a lot cheaper and able to be fielded without years of development.

MetalStorm's FireStorm four barrel 40mm system has four rounds in each barrel , giving 14 shots total. The platform has electric motors to rotated and elevate the system. However, the iRobot itself can rotate using differential steering of the tracks and and elevate using a scissor action of the “flippers” (axillary tracks). So a much simpler arrangement would be possible using sixteen one shot 40mm grenade barrels fixed on the robot platform, using ammunition of the M203, or similar and aimed using the tracks.

Also the MetalStorm people need to keep in mind that they are just a programing change away from breaching the land mine treaty. A robotic mortar which fires automatically, without human intervention, would technically be a land mine and so illegal.

Payload weight, reliability and the ability to render the weapon harmless are critical requirements when arming unmanned robotic platforms and systems. In addition to meeting weight restrictions, the weapon system cannot be permitted to jam or need manual intervention or the mission may be jeopardised. Also if the weapon is captured it should not be capable of being fired back at friendly forces.

Metal Storm has unique characteristics that make it ideally suited to unmanned air or ground robotic platforms. In particular it has the following features:

• Extremely lightweight
• No moving parts (no jamming)
• Precision firing rates - single shot to rapid burst automatic
• Selectable munitions; lethal, non lethal, marker rounds
• Low velocity or medium velocity mitigating recoil for smaller lightweight UAVs or platforms
• High velocity for larger UAVs or robotic platforms
• Electronic fire control and sighting integrates with UAV or robotic platform electronics
• Weapon is rendered harmless if captured as fire control system is locked out.

Small robotic platforms can carry single or multi-barrel Metal Storm weapons that can be attached to existing structures (such as a robotic arm or wing), or can be integrated with specialised mounts that can target independently of the movement and direction of travel of the robotic platform.

Metal Storm has already conducted live firings from several robotic platforms including the Dragonfly rotary wing UAV, the iRobot Warrior UGV and the Talon UGV. ...

From: Unmanned Systems/Robotic Platforms, Metal Storm

FireStorm is a lightweight multi barrel 40mm electronic weapon system that brings cutting edge solutions to the Defence, Security and Law Enforcement communities.

The system can be mounted to fixed or mobile platforms to provide mission support for operations to include:

• Military Operations on Urban Terrain (MOUT)
• Reconnaissance Patrol
• Border Patrol
• Critical Infrastructure Protection
• Crowd Control

FireStorm has successfully demonstrated its capabilities under separate contracts for the U.S. Navy and U.S. Army. Recent demonstrations have proven the system capable of delivering High Explosive (HE) and a range of less lethal munitions. FireStorm brings the operational community a total force continuum application.

• No Moving Parts - All Metal Storm guns have no moving parts for high reliability.
• Safe Operation - FireStorm features a number of mechanical and electrical interlocks making it one of the safest guns to operate.
• Cost Effective - FireStorm is inexpensive to acquire, use and maintain.
• Plug & Play Operation - FireStorm requires only DC power and an ethernet connection for operations.
• Unattended All Weather Operation - FireStorm is environmentally sealed allowing for all-weather operations for extended periods without continuing maintenance requirements.

From: FireStorm, Metal Storm

From robot vision to Solar Energy in Australia in NZ

Chris Johnson from ANU will be talking next week about solar energy and robot research in an attempt to get New Zealand students to come and study at the Australian National University.

I suggested he simply point out that several ANU students and graduates have got research jobs at Microsoft and Google. ;-)

Also ANU has run an innovation course to teach researchers how to commercialize their discoveries. Also people turn up at ANU and ask for help getting things to work, such as the F-35 Joint Strike Fighter.

Building the Future: from robot vision to Solar Energy

Associate Professor Chris Johnson, from the ANU College of Engineering and Computer Science, spins a vision of the future that is now in view, from automated high speed parking of cars, pervasive computing in the home of the future, Big Dish solar power, and affordable, efficient Small Trough photovoltaics. Chris will also give an overview of the world class research fields available to PhD students and undergraduates in the ANU College of Engineering and Computer Science.

where and when:
all sessions are 4-7pm for the whole ANU event - I am on at various times in this slot as noted, 45 minute subslots

Mon 21 July 4pm Christchurch Hotel Grand Chancellor, Cashel St
Tue 22 July 5.30pm Dunedin Otago Museum
Wed 23 July 4pm Auckland The Langham hotel, 83 Symonds St
Thu 24 July 5.30pm Wellington Wellington Convention Centre, 111 Wakefield St

Associate Professor Chris W. Johnson, PhD, MIEEE, MACS
Acting Deputy Dean (Education)
ANU College of Engineering and Computer Science
R205 Ian Ross Building, North Rd
The Australian National University
Canberra ACT 0200 Australia

Gutter Cleaning Robot

iRobot is a US company which makes small household appliance robots. Unlike Robbie the robot, these are each designed to do one specific task, such Vacuuming, Floor Washing, or Pool Cleaning. The iRobot Looj Gutter Cleaning Robot is the latest (September 2007) and appears to be one of their less successful products and the least "robotic". Essentially it is a radio control tiny tractor which drives along the gutter with a flail at the front to throw the leaves out.

In 2006 I attended a talk by Professor Rodney Brooks, Director of the MIT Computer Science and Artificial Intelligence Laboratory, at the ANU in Canberra. He argues that low cost flexible robots which can be easily set up by workers (not robot experts) are becoming feasible. Robots need not be able to do everything a human can do to be useful, they can do the easy (for them) more repetitive tasks.

However, it is difficult to see the difference between the iRobot Verro 300 Pool-Cleaning Robot, for example and previous automatic pool cleaners.

Driverless Electric Taxi and Parcel Van?

I have finally found something useful to do with a Segway personal transporter. Here is a half serious proposal for an university research project:

Driverless Electric Taxi and Parcel Van

A Segway personal transporter, or similar small electric vehicle, would be equipped for autonomous operation around a campus. The unit could be used a one person driver-less taxi and equipped with a lockable box for the unattended delivery of packages.

A minimum of mechanical changes would be made to the vehicle. A GPS receiver, small computer and wireless communications would be added to allow the vehicle to automatically steer a set course around the campus and into buildings. A GPS equipped smartphone might provide sufficient computing power. The vehicle could open power operated doors and use lifts. The unit could be summoned from a mobile phone or computer web page.

As the vehicle would be relatively light weight and low speed, there would be limited risk from collisions. Its two wheel format would allow access to narrow corridors. Equipped with bright lights it could be safely used at night.

This would not be the first robot based on a Segway. MIT's Cardea is designed to be able to open ordinary doors:

Researchers from Massachusetts Institute of Technology have crossed a robotic arm with the bottom half of a Segway to make a robot named Cardea that can traverse hallways and open doors.

Cardea, named after the Roman goddess of thresholds and door pivots, is the one-armed first prototype of a robot designed to have three arms and the ability to safely interact with humans at eye level. ...
From:Segway robot opens doors, By Eric Smalley and Kimberly Patch, Technology Research News, November 19/26, 2003

The Cardera is built on the Segway RMP (Robotic Mobility Platform). This is a family of specially made Segways designed carry cargo:

The Segway RMP (Robotic Mobility Platform) is a new mobile platform based on a Segway Human Transporter (HT). It is faster, cheaper, and can carry a greater payload than existing comparable robotic platforms. DARPA IPTO initiated this effort, with SSC San Diego serving as the technical agent. We provided technical oversight, coordination, and tracking of the conversion by Segway, LLC, of fifteen Segway HT I-series machines into RMPs.
From: Segway RMP, SPAWAR, USN, 29 December 2004.

However, a much simpler approach is possible. The standard model Segway has optional lights and panniers for carrying cargo. This could be equipped with a very modest computer and navigation system to allow it to travel around a defined track on a campus. The vehicle could be restricted to defined bicycle paths and particular corridors in buildings. GPS would be sufficient for outdoor navigation and could be supplemented fort indoor use. Simple sensors designed for burglar alarms could be used to detect a person or object in the path, with the vehicle simply stopping. The vehicle could operate automatic doors by simply moving slowly in sensor range. Lifts could be operated by an interface to the lift computer.

Ethics of Killer Robots

Philip Argy, president of the Australian Computer Society, wrote a thought provoking article "Dilemma in Killer Bots" (The Australian, January 16, 2007):

'WHEN science fiction writer Isaac Asimov developed his Three Laws of Robotics back in 1940, the first law was: "A robot may not harm a human being, or, through inaction, allow a human being to come to harm." ... reports out of Korea of newly developed guard robots capable of firing autonomously on human targets are raising concerns about their potential uses. ...'.

Roger Clarke wrote an article "Asimov's Laws of Robotics - Implications for Information Technology" for IEEE Computer Magazine in 1993.

There are already some domestic robots available which could raise safety concerns, even if not designed to deliberately harm people. Professor Rodney Brooks, Director of the MIT Computer Science and Artificial Intelligence Laboratory, gave a talk in Canberra last year about the potential for low cost robots.

The Roomba robot vacuum cleaner he talked about and the Lego robot kit are examples of low cost robots and kits available.

Hacking Robot Vacuum Cleaners and Lego Robots

Back in August I attended a lecture by Professor Rodney Brooks, Director of the MIT Computer Science and Artificial Intelligence Laboratory, on Robotics. He used the example of the iRobot Roomba robot vacuum cleaner. There has been research about the vacuum cleaners (they can run the Linux operating system).

Amazon sell remanufactured models of the Roomba and books on reprogramming them.

Hacking Roomba: ExtremeTech, Tod E. Kurt, 2006:

A gizmo as cool as Roomba just begs to be hacked. Now, with this book and the official ROI specification furnished by iRobot®, you can become the robotic engineer you've always dreamed of being. Build a Bluetooth interface for your Roomba. Turn it into an artist. Install Linux on it and give it a new brain. Some hacks are functional, others are purely fun. ...

The new LEGO Mindstorms NXT robot kit with a 32 bit processor is also available. These are popular at universities as well as for hobbyists.

LEGO Mindstorms NXT robot kit:

Set includes: NXT programmable LEGO brick, ultrasonic sensor, sound sensor, light sensor, touch sensor, 3 interactive servo motors, USB cable, 7 6-wire cables, building instructions, user guide, Quickstart guide, easy to use software and test panel. 6 AA batteries not included.

LEGO MINDSTORMS NXT Hacker's Guide, Dave Prochnow, 2006:

• Your First Robot • Stupid RCX Tricks • Save Your RIS • As Smart as a Brick • MOVE IT! With Servo Motors • Hmm, I Sense Something • Yes, But I Don't Know How to Program • Testing, Testing; Oh, Trouble Shoot • Katherine's Best Hacking Projects • Katherine's Design Fun House • NXT Programming Language Guide • NXT Elements • NXT Resources

See also: Robot store, with kits, books and videos.