Robotics Rodeo at Ft. Hood

I had the good fortune to attend the Robotics Rodeo at Ft. Hood last week- a rodeo of unmanned ground robotics hosted by U.S. Army Tank Automotive Research, Development and Engineering Center (TARDEC) and Fort Hood III Corps (go Phantom Warriors!). A wonderful experience and many thanks to the CO Gen. Ricky Lynch (he has MS in robotics from MIT).

Some pictures are above and some thoughts about the commercialization of robotics...

According to a good friend, Bill Kearns, at the turn of the last century, there were over 200 car manufacturers in North America. (His family’s business was one of them.) Each manufacturer had something special, a starter motor, independent suspension, what have you. An amazing array of advances, some redundant, many brilliant.

But the problem was, they weren’t on the same car. Who wanted the latest, greatest engine on a car that you had to use with a hand crank?

Durant and Ford were credited with manufacturing but sometimes it is missed that they it wasn’t just that they mastered mass production, they mastered mass production of the right thing. They were among the first to view the cars as the sum of its parts. The superior technology of a component (usually invented by the owner) was not the reason for existence but rather a marketable feature of a desirable whole. As was stressed in one of my mechanical engineering courses, automobile companies are manufacturing companies, which make things for people to buy, not engineering companies, which create or investigate ideas for someone else to make into things for people to buy. Automotive companies at the turn of the century were really about engineering, not about the car. A similar pattern of scattered developments which were consolidated into systems happened in the aerospace industry.

The Robotics Rodeo reinforced my opinion that ground robotics is in the same state. Interesting pieces, some brilliant engineering, lots of duplication, and few useful systems.

One conclusion is that this is the Natural Order of Things and will sort itself out over time. This line of reasoning is: perhaps some duplication will result in lesser technologies occasionally trumping superior technologies and some dollars will be wasted. But this should be tolerated since the duplication and competition is usually efficient overall and reduces purchase prices, right? Besides, premature standards or regulations can kill off an emerging technology.

The Natural Order of Things philosophy has problems. In asymmetric warfare, do we have time or dare risk being beta-maxed by an adversary? And in days of trillion dollar deficits, will we be able to afford the cost of duplication? Remember, the government is subsidizing the UGV market (either through DoD or law enforcement) whereas automotive industry was private capital. There is no real consumer market for these devices. Is UGV development is in fact regulated by the invisible hand of capitalism or being de facto regulated by current defense acquisition processes. If so, is that a good or a bod thing? I don't know...

Wildland firefighting, UGVs, and UAVs

I think UAVs for wildland firefighting is a good thing, honest!

I'm at the AUVSI North America conference-- yesterday I gave two papers, one on the wildland firefighting descriptive analysis that we did with Lockheed Martin on the use of ground robots and one on our Rollover Pass, Texas, response. The wildland firefighting paper made the Flight Global Daily newsletter today (probably as the token application du jour that didn't involve weaponization). I'm quoted giving a list of problems with UAVs for wildland firefighting- that was the list of problems from the focus group of subject matter experts.

There may be a killer UAV with my name on it... please, please, call them off. I love UAVs, honest!

The list of problems is based on what they've seen in UAVs to date, not what's possible or what is even available. Sadly the disconnect between what exists and what the response community has access to remains depressingly high. Bob Roth and Tom Zajkowski with the Forestry Service are working hard, with Greg Walker's group at Alaska and Brian Argrow's team at Colorado combining research and fieldwork.

But the poor firefighters often only see and interact with vendors who come out of nowhere at a disaster and claim to have the best technology; while well-meaning, the technology is often a poor match because there is no understanding of what the responders really need. Trust me, it's not covered in any of the movies, you actually have to talk with them. Before a disaster. During, they are way too busy and are justifiably deeply suspicious of anything outside of their network of relationships..

Which reminds me about the time a group of technologists were told by an agency that their technology wasn't needed, but showed up at the disaster anyway (I warned them not to do that), and were jailed and their gear impounded. Yep, interfering with a response is an offense. And the incident commander makes the call as to what constitutes interference.

Mismatched technology plus bad manners = deep abiding negative view of robots.

Anyway, ground robots good, aerial robots good, all good for wildland firefighting when applied appropriately! But we've got to educate the firefighters about what's out there and ourselves about what they need. Don't shoot the messenger ;-)

Making my point for me: Beyond Asimov

Thanks to fellow Texas A&M Professor Walter Dougherity for pointing out that a Swedish company was fined for their robot injuring a worker.

Beyond Asimov; The Three Laws of Responsible Robotics

IEEE Intelligent Systems just printed our (David Woods, OSU, and my) article about "Beyond Asimov: The Three Laws of Responsible Robotics" and put "Beyond Asimov" as one of the articles on the cover... and the hate mail has started!

So what does Asimov and his Three Laws of Robotics have to do with rescue robotics? The Three Laws are being taken seriously as a framework for discussing human-robot interaction. Rescue robotics has humans behind the robot and humans in front of the robot- it's about as human-centric as you can get. I became one of the early drivers of the human-robot interaction community (I co-chaired the seminal 2001 DARPA/NSF study) precisely because I found through my fieldwork that the poor interaction was the limiting factor. No matter how bad the rescue robots were in terms of locomotion, communications, sensing- the horrible mismatch between the robots and the human cognitive abilities for the environment was the limiting factor. It isn't just interfaces, it is the set of fundamentally, pervasively wrong assumptions about how people interact with robots.

Whenever I hear some grad student talking about wanting to design robots which meet Asimov's Three Laws and thereby provide perfect human-robot interaction I get ill. One year I heard a researcher telling the press that their robot met the First Law of Robotics (a robot may not injure a human) because it was able to avoid people. Except that it was simply avoiding heat sources and people happen to be warm.

To say AI researchers tend to be technological optimists is an understatement.

The paper came about when I began to read Moral Machines (David texted me that I had to stop whatever it was I was doing and go read it now, he was so put out by the book). I next-day-ed the book, and between that and the Living Safely with Robots tome, shouted "enough with the Asimov's Laws already as some sort of gold standard for robot ethics. It was a literary device. Let it go!" My family tends to find things to do away from the house at time like that. I thought the Moral Machines actually made a strong, though unintentional, argument for why Asimov would get sued if he were a robot manufacturer.

So I whipped up a draft on alternative laws one Saturday morning. Leila Takayama and Victoria Groom from Cliff Nass' group at Stanford read it, make great suggestions, and included in their HRI workshop. I sent it to David to read and he came back with excellent ideas, tons of experiences and examples of how autonomy and automation fails, and way better prose. I insisted that we stay with three laws and that they had to be symmetric with Asimov's-- sticking with literary convention to make a point. I agree with David, if you really want laws, it'd be better to start over. Anyway, we put a version in an IEEE ICRA workshop (thanks Cindy for presenting!) and continued to refine. We ran it past Robert Hoffman who saw the possibilities of getting a more informed discussion going and after a rapid edit cycle and a discussion with Jeff Bradshaw, it's in print. (I'm sharing names in a Good Way, please don't go yell at them if you hate the paper.)

Hopefully, besides hate mail, we'll get a real intellectual discussion going instead of extreme quotes in the media. AI robotics is capable of so many things, I hate to handicap true progress by adherence to a cute literary devices designed to create problems.

RoboCup Rescue

I’m in Graz for the RoboCup Rescue- my talk on Friday was well-received and it was great to see so many students doing such great work!

I finally got to meet Amir Soltanzadeh, my Facebook buddy, and leader of the AriAnA team- the team gave me their uniform- thanks! Iran has always been very active in RoboCup Rescue and AAAI rescue competitions- it’s certainly a practical application given the earthquakes in that area, though the real “killer app” is using robots to remove land mines. I was very impressed with how quickly the US and Iran set aside differences to allow the US international team to help at the 2003 Bam earthquake. Colleagues in Iran tried to arrange for CRASAR to attend but it would have taken 2 days by commercial air- too late to be of much help and too expensive to just check things out- and we could only get permission from the State department to fly with the military over, not back.



The competition was well attended by about 20 teams from Europe and Asia (none from the US)- you can check out robocup.org for more details. The competition arena designed by Adam Jacoff at NIST has to satisfy many constraints- it has to be cheap, shippable to different venues, repeatable so everyone can build their own, open and visible so that spectators can see, and still present a challenge! In the early years of the competition when it was held at IJCAI, Chief Ron Rogers of Florida Task Force 3 was involved and put up tarps to black out areas and created some water hazards. More realistic, but quite the barrier for new teams to get involved and hard on the spectators!

I haven’t been to RoboCup Rescue in several years. There was a surprising homogeneity between platforms. Almost all of the ground platforms have converged to a Packbot or Talon style with flippers and treads and a similar size, with manipulator arms. I didn’t see any innovative platforms such as snakes, legs (such as RHex), or even the wheel/leg combinations you see from Case Western. Most of the platforms were large enough to be mistaken for bomb squad or law enforcement robots.

Another common touch was the addition of a camera on a mount behind the manipulator arm (if there is one). This is to give the operator exproprioceptive information and compensate for the lack of sensing in robotics. Rescue robots always have a camera- that provides exteroceptive sensing- sensing of the world around the robot. Usually they have proprioceptive sensing- sensing of the robot’s internal position- but not always, the lack of proprioception on the mine crawler at the Crandal Canyon Utah mine disaster was a big problem. Many of the Operator Control Units had icons representing the relative position of the flippers, taking advantage of the proprioception. But there’s a third category of sensing- exproprioception: where is the body relative to the world? Am I stuck? About to fall over? Is my arm under a rock? Exproprioception is clearly important. And extremely difficult to do without a “skin” and good spatial reasoning.

So the teams are trying to get exproprioception through exteroception. That’s common in bomb squads and I’ve heard it referred to as the “we’ll just stick another camera on it” non-solution. That leads to challenges in the operator’s attention and situation awareness- which camera to look at, when? Also note that the higher the camera, the better the view. But the higher the camera, the less likely it can be used for a real response where the voids are less than 1m high and anything that sticks up or out snags. It's a tough problem and hopefully one of these groups will find a more optimal solution.

Sadly, I saw perhaps 2 women total on the teams. We had hoped that the societal relevance of rescue robotics would help attract women to computer science and engineering but there was no evidence here. Hopefully, the lack of women is a fluke.

Collaboration at a Distance! TU Delft

What a great phrase from the researchers at TU Delft that captures a major aspect of emergency informatics: collaboration at a distance! Given that robots and embedded sensors provide a remote presence into places that people can’t get to (or get to quickly enough), the question is how to use it? Which leads to collaboration at a distance!

As we noted in a recent article (see ``From Remote Tool to Shared Roles," in IEEE Robotics and Automation Magazine, special issue on New Vistas and Challenges for Teleoperation, 15:4, Dec.08, pp. 39-49), the real human-robot ratio isn’t having 1 person controlling a 1,000 robots but rather having a 1,000 people be able to use the data from a robot, without necessarily knowing that it’s from a robot. And certainly those 1,000 people won’t all know each other and may be working independently (and at cross-purposes), which David Woods at Ohio State calls polycentric control. Plus in order to use the data, we expect some ad-hoc teams to form and that they will use the visual data as a common ground (as per Jenny Burke’s PhD).

I was at the Technical University of Delft (TU Delft) on Wednesday to attend Maarten van Zomeran’s MS defense, as I had been invited to serve on his committee. Maarten did a great job with the Rubbleviewer and his MS was well attended- I am very proud. His thesis was chock full of interesting information beyond the Rubbleviewer, including a comparison of US FEMA search methods and information representation with United Nations INSARAG. Maarten has participated in two full scale exercises, one with the Czech team and one with the Netherland team in an exercise in Dubai- plus with the responders at Disaster City. An amazing grounding for his work in better representations and visualization!

It also gave me an opportunity to meet in person the research team there- they don't focus on emergency response but that's becoming an area of interest. A lot of great work in artificial intelligence, HCI, and especially software agents. Prof. Dr. Catholijn Jonker, head of Man Machine Interaction dept (and another right thinking woman!) proposed a way forward for continued work on the Rubbleviewer and collaboration in general. TJ de Greef was a great host (few things are better than great conversations over Dutch white beer in an outdoor plaza!) and I admire his industrial research expertise and research savvy. What a great group of people! Check out http://mmi.tudelft.nl/index.php

I sat in on a undergraduate student capstone project presentation and was impressed not only by the topic and competence, but also that it was delivered in perfect English.

All that remains... return to Cologne

I'm on travel this week in Europe. My first start was Cologne to meet with the gang from the Franhofer Institute and to present plaques to them and the Cologne Fire Department, thanking them for allowing us to participate in the State Archive Building Collapse. Hartmut, Sebastian, and Thorsten came down from Bonn for dinner and a walk by the collapse site, now leveled, waiting the conclusion of lawsuits, new officials, etc. It is hard to believe that it has only been 3 months. Not only has it only been 3 months, but the city had erected a temporary roof (like those used at stadiums) and removed it.

BTW, I was told the oldest, most valuable manuscripts were among the 20% forever lost.

Prof. Thomas Christaller was receiving a prestigious medal, so the timing was bad, but Harmut arranged a "mini-symposia" at the Franhofer Institute with Capt. Rorhle and me giving talks. All of Capt. Rorhle's slides were in English, so despite him talking in German, it was totally fascinating. Perhaps the most fascinating was to see the timeline of events, from getting a call 2 minutes before the collapse throughout the first hours. The flow of information (and mis-information) is apparently the same there as it is in the US-- which really emphasizes the need for emergency informatics.

Prof. Stefan Wrobel attended and my hats off to him and Thomas for an amazing place! It's a lovely combination of old (a castle) and new (the buildings and especially the robotics high bay lab) with an artistic and eco sensibility ("green" roofs).

So looking at the cleared site, similar to the WTC site, it is hard to tell that two people lost their lives there, that a physical connection to the past was lost as well. But given that Cologne appears in some ways defines itself by the bombings from WWII, I suspect every resident can feel the tortured earth and have added it to their long memories.

Summer Institutes and being an Alpha Geek

While it's been longer than I realized since my last post. Part of the delay was the usual end of the semester scrambling, but also much more exciting event our first mini Summer Institute On emergency informatics at Disaster City. A Summer Institute as created by David Woods at Ohio State is and amazing innovation workshop, where the various stakeholders in a system -- in this case academics responders, agencies, and industry, give together to learn each other's language, conduct hands-on missions, and participate in envisioning exercises. We had 35 participants from TEES, TEEX, and our academic and industry partners. Faculty from 5 engineering departments (CE, CSE, ECE, ME, and ENTC) plus Architecture participated along with Carnegie Mellon, Ohio State, and TAMU Corpus Christi. TCAT led the industry affiliates program discussion- attended by 4 industries (Lockheed Martin, AirRobot, SA Technologies, and Velodyne), and the US Army Research Lab. One major outcome was a model of information flow and structural inspection tasks-- none of this had any idea how much rescue, recovery and reentry hinges on the structural specialist, building inspectors, insurance adjusters, and contractors. Look for a publication with the model soon!

But the other big news is that Damon Tabor at WIRED did a short piece on us under the Alpha Geek heading. Click the title to see. Nice job- he spent over a year on it, we had wanted him to embed with us but couldn't work it out.

Italian Earthquake: The Gardens of L’Aquila

I joined the team led by Prof. Daniele Nardi (Universita’ di Roma La Sapienza) for a one day visit to L’Aquila, Italy, the cultural epicenter of the April 6 earthquake that killed 300 and displaced thousands of others.

It was a long day- I flew into Rome on an overnight flight, arriving Wed at 7:45AM to a quick change into my response gear, a warm welcome from Daniele and his colleagues, and a one and a half hour drive to the mountains. Daniele had gotten the local fire rescue department to give us a heartbreaking tour and permission to fly his group’s small Ascending Technologies Quadrotor on-site. I was along as an observer on this trip, so I didn’t bring any robots, just cameras and my field notebook.

As we drove east from Rome, the mountains were a carpet of spring green dotted with the purple blooms of the Judas trees and the outskirts of town a maze of gardens of tulips and wisteria in full bloom, ins sharp contrast to the maze of collapsed buildings just a few kilometers ahead.

L’Aquila is a historical town in the mountains, along a ridge in a valley, famous for saffron. Think of a 15th century New Orleans French Quarter, the Canal Business District, and the Garden District all along a ridge, providing a mixture of very old, very new, commercial, apartment buildings, and villas. The town has a large university with the flavor of Tulane.

Our first stop was the ruins of an apartment building where a 24 year old student was pulled out alive.

Our second stop was an apartment building where 24 people died.

And everywhere, toys, stuffed animals, legos, and baby blankets peeking out from the rubble causing my heart to clench in fear, “what about the children?”

The team flew four flights among the ancient villas and then a flight downtown along buildings tumbling into the narrow streets, notice the Italian flag in the background. The flight conditions were perfect, no wind and the rain clouds eliminated glare. While the team had no mission, the rubble served as natural targets for trying out camera configurations and flying strategies. The environment offered many examples the close quarters of the types of clutter that make flying in urban areas so challenging: trees, flags, telephone lines, etc.

The displaced will remain so for a long time. The aftershocks are expected to persist through August and it is difficult to assess the structural condition of the buildings and even more difficult to remove or repair structures. We passed two large tent cities on the outskirts of town.

As with hurricanes in the US, the recovery from earthquakes may be harder than the response. To me, it cries out for embedded sensors and smart structures to measure the impact- either before or inserted after a quake in anticipation of aftershocks. It also suggests that we need to explore ways to develop new sensors and use humans and unmanned systems better to more quickly inspect buildings, perhaps using the internet to send data to experts all through out the country rather than having 1 or 2 inspectors or claims adjusters act as a bottleneck. And I wonder how much information is known to the population but unharnessed because we still lag in exploiting connectivity and the “wisdom of the crowd” showing up in twitter, flickr, and other social software?

There’s work to be done…

Italy Earthquake: while waiting, victim management thoughts

I saw the first announcements of the Italy earthquake as I arrived in Seattle at 1AM for the Unmanned Unlimited workshop at AIAA... Wow. Stunned. Scrambled to offer our services, offer prayers for families and responders, got some sleep, waiting for replies.

In this case, it's not only rescue robots (including Choset's snake) but also being prepared to put into play what we've learned about victim management.

There's how to do triage with a robot-- using the protocol developed by Prof. Carolina Chang from Simon Bolivar University in Venezuela based on work by Dr. Dawn Riddle

There's the use of robots to carrying IV tubing to victims-- thus giving them water, hot air, or liquid medication- part of the work with did with Eric Rasmussen, MD FACP, now with InSTEDD

And there's the work with Cliff Nass' CHIME lab at Stanford on the robots serving as a Survivor Buddy.

Hopefully we can help.

Analysis: Caterpillars and Snakes

The Cologne deployment highlights the need for true robot snakes that can work in wet environments and interior of mixed rubble. The ASC is actually a caterpillar. The important point is just like real caterpillars and snakes, the mechanical versions have different ecological niches- I want both. We’re missing a snake from our arsenal of ground robots. Never forget that the US&R ecological niche requires a self-cleaning camera (the equivalent eyelids and tears) to permit rescuers to see as dust, dirt, and water collect on the lens.

The dear ASC (which we’ve nicknamed “Catey”) provides the smallest size on a fieldable robot I’ve ever seen, on par with those big fat hairy caterpillars that either delighted or grossed you out as a child. Like a real caterpillar, it is slow and has to “go with the flow” as it can bend but can’t necessarily climb. Unlike a caterpillar, it is pretty stiff, as only the “head” can bend- the forward motion comes not from undulation but from vibration, which is extraordinarily brilliant but provides less propulsion. Think of the current ASC as a partially paralyzed caterpillar that can’t blink, as if it evolved in the desert- which is great for a fairly dry pancake collapse- like when we used it so successfully at the Berkman Plaza II collapse. I’d like to see the ASC adapt to become a rainforest caterpillar, able to blink and work in mud (the latter may be impossible due to clogging of the cilia). But don’t get me wrong, I’ll take the ASC caterpillar just the way she is!

Snakes, on the other are usually bigger than caterpillars, but more powerful and have eyes that blink. A lot of rattlers have simple thermal sensors for better targeting that poisonous bite (which could be transferred to finding survivors). A robot snake that slithered (technically traverse motion) could climb and more aggressively attempt to penetrate irregular rubble. Notice that snakes are smooth. Many mechanical snakes (see http://www.engin.umich.edu/research/mrl/00MoRob_6.html) are tracked- lumpy and exposed. And big, just under the size of the Inuktun Extremes. Exposed segmented mechanical snakes (where the snake is a series of miniature tracks or wheels on articulated joints) collect mud and debris interfering with movement. And if there is a way for a device to jam or get tangled, it will. That’s why I am very excited about Howie Choset’s smooth, highly articulated snake (see http://www.cs.cmu.edu/~biorobotics/serpentine/serpentine.html ). He has been part of the CRASAR team since 2003 and it has been interesting watching his ideas about snakes for US&R evolve with each field exercise he and has students participate in. Anyway, what I want is the mechanical equivalent of a Texas brown snake, a snake that likes to burrow in moist compost (or a mixed rubble collapse).

Preliminary Post Hoc Analysis

Clint and I are back in the USA! Here's a picture of the two robots with Satoshi and me. Below is a video that shows the challenges for robots. Note that risk to human operators near the pile and also the lack of voids. The trick is not building a robot to climb ON the rubble, but rather to be able to burrow IN the rubble. Also note the rubble is from 3 sources: an older residential building, a modern commercial building, and road/subway/infrastructure. Really, really challenging!

I'll post more as I continue to work up my notes and edit video/photos to illustrate the challenges and opportunities for rescue robotics!

CRASAR heading home

We're in the airport now (painfully early in the morning at the Cologne airport, making our way home. The firemen found one of the victims early Sunday morning, the old fashioned way by slow, tedious extrication. As of this morning (Monday), it doesn't look like the second victim has been found. We were not called out on Sunday but continue to learn more about robots that would be suitable for these conditions. It was a bit embarrassing, the Mayor, Deputy Mayor, and the Chief all shook our hands and were very enthusiastic, despite us doing nothing tangible. Indeed, we are very grateful because we learned so much.

The one photo that "got away" was that of a dozen women with cardboard boxes working in the light rain, picking up documents from the rubble a few meters from the rescuers carefully excavating for victims in the dangerous, unstablized rubble. The women looked like they about to weep from the lost and damage to the Archives, while the fire rescue workers were grim. A sad moment

Looking forward to sleeping on the plane!

Day 1.5 at Cologne

Quick notes before getting some sleep... we have been on standby since going to the site on Friday around noon. City of Cologne and fire department (still fuzzy on naming translations) have been terrific and the Fraunhofer Institute IAIS incredibly supportive. 

The Archives buildings was a new, modern multistory commercial building surrounding by multistory residential buildings. When it collapsed into the hole, some of the surrounding buildings were damaged. The one where the two victims are missing is an older (like more than a 100 years old looking) brick building. When brick buildings collapse, the brick crumbles into sand and small pebbles, filling every possible void. Even the ASC couldn't get in. 

The robots were requested this afternoon for the mixed rubble from the houses and the Archives, but it wasn't a good fit. There were small voids but we couldn't stand at the face of the rubble due to safety reasons- and the ASC requires us to be right there. The voids big enough for the larger Extreme, which we could operate from the safe location ~10m away, were shallow and thus didn't require a robot. More notes on how to build better robots.

We're back on site tomorrow- a huge crane is being brought in to do more excavations and more voids may open. It looks like the old fashioned tedious manual rubble removal is the best technology for this job for now.

Lots of pictures but haven't gotten permission to release them yet, here's link to pics from the media and gratuitous coverage ;-) Can't seem to embed the URL so please cut and paste

Local news in English: http://www.thelocal.de/national/20090307-17863.htm

Yahoo!News: http://news.yahoo.com/nphotos/Building-collapses-Cologne/ss/events/wl/030309colognecollaps

CRASAR off to Cologne, Germany building collapse