Performance Metrics
Performance Metrics for Intelligent Systems
PerMIS '03
The proceedings are available in PDF format. You can view each paper below. If you do not have Adobe Acrobat Reader
you can download it for free from Adobe.
NIST Special Publication 1014
September 16 - 18, 2003
Edited by: E.R. Messina and A.M. Meystel
The Proceedings are available in PDF format. You can view each paper
or the entire proceedings below. If you do not have Adobe Acrobat Reader
you can download it for free from Adobe.
| Session TuAM1 - Machine Intelligence: Measures & Issues
I (Chairs: R. Madhavan, J. Spall) |
| Madhavan, R., Messina, E., Quantifying
Uncertainty Towards Information-Centric Unmanned Navigation |
| Spall, J., The Fisher Information Matrix:
Performance Measure and Monte Carlo-Based Computation |
| Horst, J., Exercising a Native Intelligence
Metric on an Autonomous On-Road Driving System |
| Krishna, K., Hexmoor, H., Towards Quantification
of the Need to Cooperate between Robots |
| Session TuAM2 - Metrics in Control and Planning (Chairs:
K. Pathak, A. Suri) |
| Pathak, K., Agrawal, S., Messina, E., A Computationally
Efficient Scheme for Hierarchical Predictive Control |
| Joshi, S., Surianarayan, A., Calibration of
a 6-DOF Cable Robot Using Two Inclinometers |
| Guo, Y., Qu, Z., Wang, J., A New Performance-Based
Motion Planner for Nonholonomic Mobile Robots |
| Roy, A., Iqbal, K., Contributors to Postural
Stabilization: A Modeling - Simulation Study |
| Session TuPM1 - Invited Session: Performance Metrics for
Perception Systems (Chairs: T. Hong and M. Shneier) |
| Dufourd, D., Dalgalarrondo, A., Results
and Lessons Learned from the Quantitative Evaluation of Road Detection
and Tracking Algorithms |
| Phillips, P., Grother, P., Micheals, R., Blackburn, D., Tabassi
, E., Bone, M., Face Recognition Vendor Test
2002 |
| Cheok, G., Stone, W., Witzgall, C., Some
Issues Relating to Performance Evaluation of LADARs |
| Chalidabhongse, T., Kim, K., Hardwood, D., Davis, L., A
Perturbation Method for Evaluating Background Subtraction Algorithms |
| Hong, T., Chang, T., Takeuchi, A., Cheok, G., Scott, H., Shneier,
M., Performance Evaluation
of Sensors on Mobile Vehicles Using a Large Data Repository and
Ground Truth |
| Session TuPM2 - Measuring Autonomy of a System (Chairs: A.
Yavnai and H. Huang) |
| Huang, H., Messina, E., Albus, J., Toward
a Generic Model for the Autonomy Levels for Unmanned Systems (ALFUS) |
| Huang, H., Messina, E., Albus, J., Autonomy
Level Specification for Intelligent Autonomous Vehicles: Interim
Progress Report |
| Proud, R., Hart, J., Mrozinski, R., Methods
for Determining the Level of Autonomy to Design into a Human Spaceflight
Vehicle: A Functional Specific Approach |
| Yavnai, A., An Information-Based Approach
for System Autonomy Metrics: Part I: Metrics Definition |
| Session WeAM1 - Machine Intelligence: Measures & Issues
II (Chairs: L. Arata and R. Cottam) |
| Gunderson, J., Gunderson, L., Mom! The
Vacuum Cleaner is Chasing the Dog Again! |
| Ulinwa, C., MIQ: Understanding a Machine
through Multiple Perspectives Analysis |
| Berg-Cross, G., A Pragmatic Approach
to Discussing Intelligence in Systems |
| Cottam, R., Ranson, W., Vounckx, R., Abstract
or Die: Life, Artificial Life and (v)Organisms |
| Arata, L., Interactive Measures and Innovation |
| Session WeAM2- Invited Session: Technology Readiness Level
Assessment of an Intelligent System (Chairs: A. Jacoff and A. Lytle) |
| Schipani, S., An Evaluation of Operator
Workload, During Partially-Autonomous Vehicle Operations |
| Sholtz, J., Antonishek, B., Young, J., Evaluation
of Operator Interventions in Autonomous Off-road Driving |
| Witzgall, C., Cheok, G., Gilsinn, D., Terrain
Characterization from Ground-Based LADAR |
| Session WePM1 - Invited Session: Performance Metrics for
Driving Systems (Chairs: C. Schlenoff and J. McKnight) |
| McKnight, J., McKnight, S., Motor Vehicle
Technology: Automation of Driving Tasks |
| Parent, M., Yang, M., Performance Metrics
for Cybercars |
| Barbera, A., Horst, J., Schlenoff, C., Wallace, E., Aha, D., Developing
World Model Specification and Metrics for Sensory Processing for
On-road Driving Tasks |
| Zimmerman, N., Schlenoff, C., Balakirsky, S., Wray, R., Performance
Evaluation of Tools & Techniques for Representing Cost-Based
Decision Criteria for On-Road Autonomous Navigation |
| Session WePM2 - Invited Session: Modeling and Simulation
Support for Performance Evaluation (Chairs: S.Balakirsky and M. Fields) |
| Headden, W., Maloof, M., Evaluating
Rules Learned from Simulated Environments |
| Fields, M., Haug, T., Developing a Chemical
Reconnaissance Behavior for Unmanned Ground Vehicles Using the OneSAF
Battlefield Simulation Tool |
| Fields, M., Developing an Overwatching
Fires Mission for a Team of Unmanned Ground Vehicles |
| Session ThAM1 - Evaluation of Human and Robot Interactions
(Chairs: J. Drury) |
| Crandall, J., Goodrich, M., Measuring the
Intelligence of a Robot and its Interface |
| Drury, J., Riek, L., Christiansen, A., Eyler-Walker, Z., Maggi,
A., Smith, D., Evaluating Human-Robot Interaction
in a Search-and-Rescue Context |
| Olson, D., Goodrich, M., Metrics for Evaluating
Human-Robot Interactions |
| Session ThAM2 - Evaluating Algorithms & Tools for Intelligence
(Chairs: SB. VerDuin and A. Guez) |
| Kothamasu, R., Huang, S., VerDuin, W., Analysis
of Performance Evaluation Metrics to Combat the Model Selection
Problem |
| Balakirsky, S., Kramer, T., NOT (Faster
Implementation ==> Better Algorithm), A Case Study |
| Hutchison, D., Spall, J., Stopping Stochastic
Approximation |
| Guez, A., A Multiple Objectives Optimization
Approach To Robotic Teams’ Analysis, Design and Control |
| Jarrold, W., Using an Ontology to Evaluate
a Large Rule Based Ontology: Theory and Practice |
| Session ThPM1 - Applied Performance Evaluation (Chairs: A.
Clerentin and W. McBride) |
| Clerentin, A., Delahoche, L., Brassart, E., Izri, S., Imprecision
and Uncertainty Quantification for the Problem of Mobile Robot Localization |
| Jacoff, A., Weiss, B., Messina, E., Evolution
of a Performance Metric for Urban Search and Rescue Robots (2003) |
| McBride, W., Longoria, R., Krotkov, E., Measurement
and Prediction of the Off-Road Mobility of Small Robotic Ground
Vehicles |
| Chin, D., Real Time 3-D Discrimination of
Buried Object in Subsurface Soil |
Contact
Ms.
Elena Messina
NIST
Intelligent Systems Division
100 Bureau Drive, Mail Stop 8230
Gaithersburg, MD 20899-8230
Phone: 301-975-3510
Fax: 301-990-9688
E-mail: elena.messina@nist.gov
Date created: 4/27/04
Last updated: April 27, 2004
|
