• Metrics for measuring sustainability such as energy, carbon footprint, water usage, product and process performance
• Sustainable indicators for manufacturing
• Product standards and performance metrics
• Data modeling for sustainability
• Methodologies and tools which incorporate environmental manufacturing standards
• Design for environment tools
• Integration of sustainable parameters in life cycle assessment tools
• Integration and interoperability among tools that promote sustainable manufacturing

 

SS2 Test and Evaluation of Unmanned and Autonomous Systems
Organizers: Mauricio Castillo-Effen and Nikita Visnevski, GE Global Research Center

Unmanned and Autonomous Systems (UAS) with ever growing levels of sophistication are being developed to assist humans in a variety of application fields. Robots have proved their usefulness assisting first responders finding survivors in disaster situations. A variety of UAS are essential assets in the arsenal of the Armed Forces of the world. Furthermore, recent developments in autonomous driving have also shown that vehicles can navigate complex urban environments without human assistance.

Yet, questions remain open as how to perform relevant developmental and operational Test and Evaluation (T&E) of UAS efficiently. Novel T&E methods and tools are needed that will allow for assessing reliability and usefulness of UAS in an unequivocal, timely, and economic manner. Test and evaluation is a key process that assists decision makers in planning and executing the acquisition of UAS for all their envisioned applications.
The military T&E community has started efforts in developing methods and tools that will assist testers in coping with challenges imposed by systems with growing levels of machine cognition. In addition, a roadmap for T&E of UAS for the military domain is being developed. Nevertheless, more input from the scientific community is needed, which may help shaping this emerging multidisciplinary research field. Researchers from all disciplines are invited to submit contributions and report relevant research in the following areas:

 

SS3 Is an Agent Theory of Mind (ToM) Valuable for Adaptive, Intelligent Systems?
Organizer: Gary Berg-Cross, Knowledge Strategies

Theory of Mind (ToM) is a way of looking at special cognitive capacity by which a cognitive agent understands and predicts external behavior of others by attributing unobservable mental states, such as beliefs, desires and intentions. As part of the 9th iteration of the Performance Metrics for Intelligent Systems workshop (PerMIS'09) we are organizing a session that more directly addresses a "Theory of Mind". We believe this session will help address scientific and practical goals towards a "Scientific Theory of Mind" to "extend the frontiers of human knowledge to include a scientific understanding of the processes in the human brain that give rise to the phenomenon of mind.” The session will address philosophical, research and robotic implementations central to the topic. This includes prediction and imitation of behavior of others by attributing internal states, such as knowledge, beliefs, and intentions as well as how these may affect performance, resilience and adaptability of intelligent systems.

Important topic areas include:

 

SS4 An Ontology for Robotics Science and Systems
Organizers: Erwin Prassler and Herman Bruyninckx

The field of robotics research has reached a state of matureness which makes a well-defined, commonly agreed terminology indispensable.
Apart from a few sub-domains, in which first attempts have been made to establish a common terminology and to elaborate on the semantical dependencies between specific terms, there is no established ontology for robotics.

The prime objective of this special session is to kick-off a community activity towards developing an ontology for robotics science and systems. In the session we will discuss issues such as:

 

SS5 TRANSTAC: Performance Evaluation of Speech Translation Systems for Military Applications
Organizers: Craig Schlenoff and Brian Weiss, NIST

Designing and implementing a performance evaluation of an emerging technology to present a broad picture of technology performance in its typical operating environment is a very challenging goal. Intelligent systems tend to be complex and non-deterministic, involving numerous components that are jointly working together to accomplish an overall goal. As intelligent systems emerge and take shape, it is important to understand their capabilities and limitations. Evaluations are a means to assess both quantitative technical performance and qualitative end-user utility.

The System, Component and Operationally Relevant Evaluation (SCORE) Framework has been developed at the National Institute of Standards and Technology (NIST) over the past three years to provide formative evaluations of advanced military technologies that are still under development. SCORE is built around the premise that, in order to get a true picture of how a system performs in the field, it must be evaluated at the component level, the system level, and within operationally-relevant environments. To date, SCORE has been used to evaluate a wide range of advanced technologies, including Soldier-worn sensor systems, technologies allowing real-time multimedia information sharing among Soldiers in the field and autonomous robotic platforms. It has been the foundation for ten technology evaluations involving Soldiers and Marines from around the country.

TRANSTAC is a DARPA advanced technology and research program whose goal is to demonstrate capabilities to rapidly develop and field free-form, two-way speech-to-speech translation systems enabling English and foreign language speakers to communicate with one another in real-world tactical situations where an interpreter is unavailable. Several prototype systems have been developed under this program for numerous military applications including force protection and medical screening.

DARPA asked NIST to assess the performance of the TRANSTAC systems using the SCORE framework. In particular, NIST was asked to assess the usability and performance of the overall translation system and to individually assess the technical performance of each component of the system (the speech recognition, the machine translation, and the text-to-speech). NIST has served as the independent evaluation teams for the TRANSTAC program over the past three years.

 

SS6 Performance Measurements Towards Improved Forklift Safety
Organizer: Roger Bostelman, NIST

Important topic areas include:

Within the above topics, we will discuss associated forklift and/or sensor performance measurements and standards.