AI - 1 Robot Stalker

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

One of the most useful things you can tell anyone when teaching them something is "follow me". The aim of this project is to give an ActiveMedia Pioneer robot the ability to follow a person around a home or office environment. This isn't as easy as it sounds. Firstly, the robot has to be able to use vision or other sensors to distinguish a person from his or her surroundings. The robot must then be able to recognise the motion of the person and plan a path to follow that motion. It must take into account that it has a different shape to a human and therefore has to avoid obstacles that present no problems for a human but may for a robot.

This is part of a larger effort to develop effective techniques for human-robot interaction in a domestic setting. Much of robotics research focuses on the robot itself and pays little attention to the fact that humans will be interacting with robots.

An expected outcome of the research is a software system for a pioneer robot that enables it to recognise and follow a human teacher. The student will learn about computer vision, planning algorithms and robot control. This is one of the exercises in the RoboCup@home competition: http://www.robocupathome.org/.

For further information, please contact the supervisor or undertake general reading on computer vision and motion detection in any text on computer vision.

AI - 2 Getting a robot to thread a (very big) needle

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

Many skills that we take for granted are difficult for a robot. For example, our hand-eye coordination allows us to perform complex manipulations such as threading a needle or inserting a screw into a hole. In the latter case, we use vision to align the screw but also use our sense of touch to fine tune the location of the screw. We do similar things when inserting a key into a lock.

This project will make use of our new humanoid robot (torso only) that consists of two robot arms, each equipped with three-fingered grippers with force-torque sensors and a stereo camera head. The aim of the project will be to program the robot to insert a peg into a hole using a combination of vision and touch.

This is a lot harder than you might think!

Despite a large body of research in robotics, manipulation is one area that has not received as much attention as it deserves. This is particularly of multi-modal sensing, as described above. A potential outcome of the research is a software system for controlling a humanoid robot to perform dextrous manipulation tasks. The student will learn about computer vision and robot control. The project may also involve some machine learning.

For further information, please contact the supervisor. Our robot is loosely based on MIT's Cog, so also consider http://www.ai.mit.edu/projects/humanoid-robotics-group/cog/

AI - 3 Teaching a robot how to drive over rough terrain

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

It is very difficult to program a robot to drive over rough terrain. Choosing the right line to follow to avoid obstacles or climb over hills without rolling, etc, requires considerable skill. Rather than programming driving strategies by hand, this project aims to give a robot the ability to learn from observing a human driving the robot.

We have a very capable tracked robot that can be driven by remote control using a fairly intuitive user interface. We can capture the human operator's actions, as well as the information from a variety of cameras onboard the robot. This information is used as training data for a machine learning program that abstracts the human's responses to different situations to produces a set of control rules.

Some preliminary experiments have already been conducted with promising results. This project will conduct more thorough experiments ands explore different learning algorithms to try to improve the reliability of the learned control rules.

This is a highly innovative application of machine learning. If successful, this project is likely to lead to publication in an international conference. A potential outcome of the research is a system for learning driving control rules; a working set of rules for the DT-3 robot that are capable of handling a random step field. The student will learn about machine learning, robot perception and control.

For further information, please contact the supervisor or consider the following:

An early paper on behavioural cloning: http://www.cse.unsw.edu.au/~claude/research/papers/ML9.pdf

A paper on the preliminary learning to drive experiments: http://rescue.web.cse.unsw.edu.au/papers/nips05.pdf

The rescue robot platform: http://rescue.web.cse.unsw.edu.au/

AI - 4 Talking Robot

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

The aim of this project is to implement a conversational agent on the PeopleBot robot. The PeopleBot stand approximately at human height and has a touch screen mounted on top with cameras and stereo microphones. We wish to be able issue commands by talk to the robot or by pointing to the touch screen. The robot should also be able to respond with speech.

Software tools exist to help create scripts for conversational interactions. This project will also require learning about speech recognition systems and interfacing to natural language dialogue programs as well as robot control.

Several tools for spoken natural language interaction have been developed in CSE over a number of years but they have not yet been implemented on a robot which can perform physical actions in response to commands. This project involves significant research in human-robot interaction. A potential outcome of the research is a robot that responds to voice commands to perform simple tasks around a home or office. The student will learn about speech and natural language processing, as well as robot control.

For further information, please contact the supervisor or consider the following:

A QuickTime video of a conversational agent developed at CSE: http://www.cse.unsw.edu.au/~claude/inca/inca_stream.mov

A paper on conversational agents: http://www.ida.liu.se/ext/epa/cis/2001/027/tcover.html

AI - 5 Games that talk to you

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

Over the past five years, researchers at the School of Computer Science and Engineering have been developing tools for creating conversational agents. These programs can communicate with the user by speech and natural language. The aim of this project is to use these tools to build characters in a game engine, such as Unreal Tournament or half life. The idea is to have characters with distinct personalities that can talk to you about who they are and what they know.

The scholar will learn to use scripting languages for multi-modal user interaction, how to program speech interfaces and how to integrate these with a game engine. He or she will work with researchers in the Smart Internet Technology Cooperative Research Centre and the ARC Centre of Excellence for Autonomous Systems. It is related to another Taste of Research project on communicating with robots.

This project will make contributions to natural language processing and games design. The intention is to develop novel types of games with much richer interactions with the user. In this project, the student will learn methods of artificial intelligence, including natural language processing and speech processing. More broadly, this project will contribute to our understanding of human-machine interaction and a successful implementation may lead to a prototype interactive game.

For further information, please contact the supervisor or consider the following:

http://www.cse.unsw.edu.au/~claude/projects/nlp.html

AI - 6 Smart Room

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

A meeting room in CSE has been fitted out with microphones and cameras so that people in the room can interact with services using speech and gestures. For example, someone may "talk to the room" to book a time for a seminar. The occupants may tell the room to download a presentation onto the computer connected to the projector and start the presentation. Occupants may use gesture to point to devices and speech to tell the room to turn them on and off, etc.

Some very preliminary programs have been developed to provide such services but there are many more that need to be programmed. This is an example of an "intelligent environment" to combines AI technology with sensor networks. At the end of the project, it should be possible to talk to the meeting room to perform tasks such as those described above. The student will learn about speech and gesture recognition, middleware for multi-modal user interfaces and scripting for user interfaces.

For further information, please contact the supervisor or consider the following:

Some papers on our software, developed for the Smart Internet Technology CRC:

ftp://ftp.cse.unsw.edu.au/pub/users/waleed/perware.pdf ftp://ftp.cse.unsw.edu.au/pub/users/waleed/pricai04.pdf

AI - 7 Robot navigation around the home

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au) and Dr. Waleed Kadous (waleed@cse.unsw.edu.au)

For a mobile robot to be able to do something useful around the home, it must first be able to create a map of the environment and be able to localise itself in the map. This is known as Simultaneous Localisation and Mapping (SLAM). The aim of this project is to construct SLAM software that can be run on CSE's Pioneer robots so that they can map the robotics lab and be able to navigate around it. This is needed so that the robot can obey commands such as to go to a particular location or to retrieve some object and return to its starting location.

This is largely a software project but it involves controlling the sensors and motors of an ActiveMedia Pioneer robot.

While there has been extensive research on SLAM, there are very few working pieces of software that can easily be incorporated into a complete robot architecture for accomplishing tasks in an office or home environment.

The outcome should be a robot that can be set loose in an unfamiliar environment and following an exploration phase, we produce an accurate map of its environment. The student will learn about algorithms for SLAM, exploration and navigation. He or she will also learn how to write programs that control a mobile robot

For further information, please contact the supervisor or consider the following:

Some simple implementations can be found in http://playerstage.sourceforge.net/

Mathematical background http://www.cas.kth.se/SLAM/Presentations/slam_nebot.pdf

AI-8 Teaching an Old Dog New Tricks

CSE Project - Professor Claude Sammut (claude@cse.unsw.edu.au)

RoboCup is an international competition which aims to advance robotics research. One of the competitions requires teams of Sony Aibo robots to play soccer. UNSW/NICTA's team, rUNSWift, has been the most successful in this league over the past seven years.

Programming robots to play a good game of soccer is not easy. The aim of this project is to develop programs that will allow the robots to learn soccer tactics by being shown examples by a human trainer, much as a coach would instruct a human player. When the player does something wrong, the coach points out the mistake and the player adjusts his behaviour.

More broadly, this project will contribute to our understanding of machine learning and, in particular, how a robot can learn complex tasks by interacting with a human. This should result in robots that are more adaptable than current systems. Some course in AI is desirable but not essential. Good programming skills in C or Java.

For further information, please contact the supervisor or consider the following:

http://www.cse.unsw.edu.au/~robocup
http://www.robocup.org

[Top of Page]