banner National Institute of Standards and Technology
ISD Products and Services
ISD home About ISD ISD Research Areas ISD's Products and Services What's New in ISD Search button
 

Performance Metrics for Technology Evaluation

Task Analysis for Driving
Calibrated Data
Performance Metrics
Human Robot Interaction
Publications
Related Links
Home
 

Goal

Measure performance in terms of efficiency, effectiveness, precision, and reliability   (e.g., interventions per unit of success)

Approach

bullet
Use human performance as a benchmark
bullet
Evaluate technologies to ascertain their performance characteristics in various environments and scenarios
bullet
Develop visualization and analysis tools for viewing execution, world models, states, and plans
bullet
Compare approaches to behavior generation (graph-based, constraint-based, finite-state, neural nets vs. algorithmic, arbitration vs. planning)
bullet
Compare performance of model-based, deliberative, reactive, hybrid, and behaviorist approaches in calibrated application domains
bullet
Compare various approaches to perception, cognition, reasoning, and planning in various calibrated environments

Performance Metrics for Intelligent Systems

An annual workshop dedicated to research and practice of measuring performance of intelligent systems. Web site for upcoming workshop PerMIS'03

Web sites for past workshops

PerMIS'00
PerMIS'01
PerMIS'02

Testbeds for Evaluating and Measuring Performance of Mobile Robots

NIST Urban Search and Rescue Facility used in the AAAI/IJCAI Mobile Robot Rescue and the RoboCup Rescue Competitions.

More information is available at the USAR web page.

 


Date Created: 1/17/2003
Last Updated: 3/24/2003