An Intelligent Systems Architecture for Manufacturing
(ISAM)
A Reference Model Architecture
for
Intelligent Manufacturing Systems
Abstract
The Intelligent Systems Architecture for Manufacturing (ISAM) addresses the application of intelligent systems to the manufacturing enterprise at three degrees of abstraction: 1) a conceptual framework for developing metrics, standards, and performance measures, 2) a reference model architecture for conceptual design of intelligent systems, and 3) a set of engineering guidelines for the implementation of manufacturing applications.
ISAM consists of a hierarchically layered set of intelligent processing nodes organized as a nested series of control loops. In each node, tasks are decomposed, plans are generated, world models are maintained, feedback from sensors is processed, and control loops are closed. In each layer, nodes have a characteristic span of control, with a characteristic planning horizon, and corresponding level of detail in space and time. Nodes at the higher levels deal with corporate and production management, while nodes at lower levels deal with machine coordination and process control. ISAM integrates and distributes deliberative planning and reactive control functions throughout the entire hierarchical architecture, at all levels, with different spatial and temporal scales at each level.
The ISAM model addresses the application of intelligent systems to the manufacturing enterprise at three degrees of abstraction:
1. At a high degree of abstraction, ISAM provides a conceptual framework for the entire manufacturing enterprise, including machines, processes, tools, facilities, computers, software, and human beings operating over time on materials to produce products.
2. At a middle degree of abstraction, ISAM provides a reference model architecture for the design of manufacturing systems and software. ISAM addresses the integration of perception, cognition, knowledge representation, simulation, reasoning, and planning with reactive control. ISAM applies to any modular structure with well defined interfaces.
3. At a low degree of abstraction, ISAM provides engineering guidelines for implementing specific instances of manufacturing systems, including machine tools, robots, inspection machines, and material handling systems organized into workstations, cells, shops, and factories.
At all levels of abstraction, ISAM provides a framework for developing metrics, information exchange standards, and performance measures.
I. ISAM as a Conceptual Framework
The ISAM conceptual framework spans the entire range of manufacturing operations, from those that take place over time periods of microseconds and distances of microns to those that take place over time periods of years and distances of many kilometers. The ISAM model is intended to allow for the representation of activities that range from detailed dynamic analysis of a single actuator in a single machine to the combined activity of thousands of machines and human beings in hundreds of plants comprising the operations of a multinational corporation.
In order to span this wide range of activities, ISAM adopts a hierarchical layering, with different range and resolution in time and space at each level. It defines functional units at each level within the enterprise such that each unit can view its particular responsibilities and priorities at a level of spatial and temporal resolution that is understandable and manageable to itself. At any level within the hierarchy, functional units receive goals and priorities from above, and observe situations in the environment below. In each functional unit at each level, there are decisions to be made, plans to be formulated, and actions to be taken that affect peers at the same level and subordinates at a level below. In each unit, information must be processed, situations analyzed, and status reported to peers at the same level and a supervisor above. Each functional unit has access to a model of the world that enables situation analysis, decision making, and planning to be carried out despite the uncertainties and noise that exists in the real world. At each level, there are values (sometimes not explicitly stated) that set priorities and guide decision making.
For example, every manufacturing enterprise has a top level where there is typically a board of directors. This is where corporate policy is set and strategic goals are established. The board decides what kind of business the corporation is to engage in, what types of products will be produced, what kind of business and labor policies will be pursued, and what are the values that determine priorities and shape corporate decisions. These values typically include criteria of success, which may be defined in terms of profits, market share, stock prices, etc. In successful corporations, there typically are well defined corporate goals with a time table and a business plan for achieving them. The CEO provides the highest level executor function.
At the top of the manufacturing hierarchy, categorical imperatives such as <make a profit> are decomposed into prioritized tasks and goals that determine behavior throughout the enterprise. At intermediate levels, tasks with goals and priorities are received from the level above, and subtasks with subgoals and priorities are output to the level below.
Typically, the corporation is organized into management units such that each unit of management consists of a group of intelligent agents (humans or machines), each of which possesses a particular combination of knowledge, skills, and abilities, and each of which has a job description that defines duties and responsibilities. Each management unit accepts tasks from higher level management units, and issues subtasks to subordinate management units. Within each management unit, agents are given job assignments and allocated resources with which to carry out their assignments. Within each management unit, intelligent agents schedule their activities so as to achieve the goals of the jobs assigned to them. Each agent is expected to make local executive decisions in order to keep operations on schedule by solving problems and compensating for minor unexpected events. Major problems that cannot be handled locally are referred up through the hierarchy to higher levels of management.
Typically, each unit of management has a model of the environment in which it functions. This world model is a representation of the state of the environment, the entities that exist in the environment, the events that take place in the environment, the attributes and behavior of entities and events, and the relationships among them. The world model also typically includes a set of rules that describes how the environment is expected to behave under various conditions. Each unit of management also has access to sources of information that keep its world model current and accurate. Finally, each management unit has a set of values, priorities, or cost functions, that it uses to analyze the state of the world and evaluate plans for future actions.
Every enterprise also has a bottom level, where physical actions take place and sensors measure phenomena in the environment. Typically the bottom level actuators and sensors are grouped into operational units or subsystems, and those subsystems are controlled by subsystem controllers. Subsystems are typically grouped into machines, machines into workstations, workstations into cells, cells into shops, shops into factories, etc. At each level of grouping, a controller may be assigned to coordinate and control group activities. This controller typically has the responsibility of decomposing tasks into job assignments for each subsystem, assigning resources to each subsystem, and developing a schedule of subtasks for each subsystem to accomplish its job. In many cases, especially at higher levels in the enterprise, coordination and control functions are performed by organizational units consisting of one or more humans that are schooled in the knowledge and trained in the skills required to perform the required control activity. Increasingly, computers can be programmed to perform these functions. Typically, computer controllers have operator interfaces whereby human operators can access the system to gather information, make decisions, and take action. Operators usually can select modes (such as manual, single-step, or automatic) and often can override automatic operations (for example by changing feed-rate, or commanding halt, or resume.)
Every corporation has some sort of organizational chart that describes the functional responsibilities of each management unit and defines the flow of command and control. Of course, no corporate management chart ever shows all the communication pathways by which information flows throughout the enterprise. In any organization, much information flows horizontally between peer agents, within management units and between units, through both formal and informal channels. Furthermore, the flow of command and control is not always strictly hierarchical. For example, a product design developed in one division of a manufacturing organization may be used to develop process plans in an entirely different part of the corporation, or even in a different company. Process plans developed in one organizational unit may be used to develop routing plans for material flow in a different unit. NC machine code developed in one corporate entity may be used by machinists in an entirely different facility for making parts. A programmer writing NC code may not even belong to the same company as the user who runs the code.
The ISAM model is designed to accommodate all of these possibilities. The fundamental property of the ISAM model is that it consists of modular nodes that correspond to organizational units. Each node contains one or more agents that have clearly defined functional responsibilities, knowledge, skills, and abilities. Each node also has clearly defined interfaces and communication channels that can be established with other nodes in the system. Thus, although most examples of the ISAM model are presented here in terms of a strictly hierarchical organization, ISAM can be adapted to fit any modular organization, so long as there are clearly defined roles and responsibilities for each of the modules and well defined interfaces between modules.
ISAM can be used to describe current industry practice, but it is designed to anticipate and address the future needs of U.S. industry. For the most part, current industry practice assumes that manufacturing consists of highly predictable processes that can safely proceed without the benefit of, or need for, on-line measurement and real-time feedback control. Most on-line adjustments to manufacturing processes are made by human operators who often use intuition and experience to tune parameters. In control parlance, most production processes operate “open-loop,” with little or no consideration given to the need for real-time planning or replanning, automatic error recovery, on-line optimization, or adaptability to changing conditions. On-line schedule changes and process modifications are handled mostly by manual ad-hoc methods.
Current data exchange standards such as the Initial Graphics Exchange Specification (IGES) [1] and the Standard for Exchange of Product model data (STEP) [2, 3] are limited to static data. Most communication protocol standards such as the Manufacturing Message Specification (MMS) [4, 5, 6, 7] and TCP/IP do not address the semantics or logic of the processes being controlled. A few steps toward standards for interfaces that convey meaning such as the Common Object Request Broker (CORBA) [8] and Component Object Model (COM) [9] have recently appeared, but these do not support control systems that require hard real-time response. Open architecture interface standards such as Open Modular Architecture Controls (OMAC) [10] that are designed to both specify functionality and operate in real-time are still under development. The Metrology Automation Association [11] has supported early development efforts in interface standardization for Coordinate Measuring Machines (CMMs). A recent effort, IEEE Std 1471-2000, Recommended Practice for Architectural Description of Software-Intensive Systems, focuses on how to describe such systems, and what should be included in the descriptions [12].
ISAM is intended to provide a theoretical foundation for the next generation of physical and informational measurements and standards. ISAM is designed to address future manufacturing practices that will depend heavily on in-process measurements and highly sophisticated computer-based control techniques. The assumption is that the future will be shaped by the demands of agile manufacturing, including rapid response to changing customer requirements, concurrent design and engineering, low-cost small-volume production, out-sourcing of supply, distributed manufacturing, just in-time delivery, real-time planning and scheduling, increased demands for precision and quality, reduced tolerance for error, in-process measurement, and feedback control. These demands generate requirements for adaptability and on-line decision making that cannot be met with current practice.
ISAM incorporates intelligent control concepts that in many cases are being developed outside of the field of industrial engineering. Intelligent control includes concepts from artificial intelligence, operations research, game theory, pattern recognition, neural nets, fuzzy logic, and control theory. It borrows heavily from cognitive psychology, semiotics, neuroscience, and computer science. Intelligent control closes the loop between sensing and acting through perception, world modeling, planning, and control. Intelligent control addresses problems of large complex systems with many sensors and actuators that are designed to pursue and achieve sophisticated goals in uncertain, competitive, and sometimes hostile environments. Intelligent control deals with systems designed to analyze the past, perceive the present, and plan for the future. Intelligent control enables systems to assess the cost, risk, and benefit of past events and future plans, and to make intelligent real-time dynamic choices between alternative courses of action in the face of uncertainty. The ISAM conceptual framework brings intelligent control concepts to bear on the domain of manufacturing so as to enable the full range of agile manufacturing concepts.
II. ISAM as a Reference Model Architecture
At a middle degree of abstraction, ISAM is intended to provide a reference model architecture for the design of intelligent systems and software and for the development of future standards and performance measures. The goal of the ISAM reference model architecture is to provide a sound theoretical foundation for:
1. metrics, measures, and procedures that can quantitatively measure and evaluate the performance of intelligent control systems.
2. interface standards that can support dynamic real-time interactions between production goals and sensed conditions in the industrial environment.
3. open architecture standards that can enable manufacturing system software from a variety of vendors to work together without extensive effort required for software integration.
To illustrate the types of issues that are addressed by the ISAM reference model, an example of a seven level ISAM hierarchy for a machine shop is given below. The particular functional assignments and numerical values given in the example are for illustration purposes only. A methodology for applying the ISAM architecture to specific manufacturing applications will be addressed in a later section.
Level 7 -- Shop (≈5 h planning horizon)
The shop level plans activities and allocates resources for one or more manufacturing cells for a period of five to eight hours. At the shop level, orders for products are sorted into batches and a production schedule is generated for the cells to process the batches. At the shop level, the world model maintains a knowledge database containing names, contents, and attributes of batches and the inventory of tools and materials required to manufacture them. Maps may describe the location of, and routing between, manufacturing cells within the factory. Sensory processing algorithms compute information about the flow of parts, the level of inventory, and the operational status of all the cells in the shop. Value judgment computes the cost and benefit of various batching and routing options and evaluates statistical quality control parameters. An operator interface allows human operators to visualize the status of orders and inventory, the flow of work, and the overall situation within the entire shop facility. Operators can intervene to change priorities and redirect the flow of materials and tools. Executors monitor how well plans are being followed, and modify parameters as necessary to keep production on schedule. Output commands from shop level nodes consist of work flow assignments for specific cells.
Level 6—Cell (≈30 minute planning horizon)
The cell level plans activities and allocates resources for one or more workstations for a period of about 30 minutes into the future. Batches of parts and tools are scheduled into particular workstations. The world model symbolic database contains names and attributes of batches of parts, and the tools and materials necessary to manufacture them. Maps describe the location of, and routing between, workstations within the cell. Sensory processing determines the location and status of trays of parts and tools. Value judgment evaluates routing options for moving batches of parts and tools. An operator interface allows human operators to visualize the status of batches and the flow of work through and within the cell. Operators can intervene to change priorities and reorder the plan of operations. Executors monitor how well plans are being followed, and modify parameters as necessary to keep work flow on schedule. The output commands from the cell level nodes consists of tasks assigned to specific workstation controllers to perform specific machining, inspection, or material handling operations on specific batches or trays of parts.
Level 5—Workstation (≈3 minute planning horizon)
The workstation level schedules tasks and controls activities within each workstation. A workstation may consist of a group of machines, such as one or more closely coupled machine tools, robots, inspection machines, material transport devices, and part and tool buffers. Plans are developed and commands are issued to equipment to operate on material, tools, and fixtures in order to produce parts. The world model symbolic database contains names and attributes of parts, tools, and buffer trays in the workstation. Maps describe the location of machines, robots, and part trays within the workstation. Sensory processing computes the position and orientation of parts and tools in trays and buffers. Value judgment evaluates plans for sequencing machining and parts handling operations within the workstation. An operator interface allows human operators to visualize the status of parts and tools within the workstation, or to intervene to change priorities and reorder the sequence of operations within the workstation. Executors keep track of how well plans are being followed, and modify parameters as necessary to keep on plan. Output commands are issued to controllers for particular machine tools, robots, and tray buffers to perform specific operations on specific parts, tools, and fixtures.
Level 4—Equipment task (≈20 s planning horizon)
The equipment level schedules tasks and controls the activities of each machine within a workstation. (Tasks that take much longer than 20 seconds may be broken into several ≈20 s segments at the workstation level.) Level 4 decomposes each equipment task into elemental moves for the subsystems. Plans are developed that sequence elemental movements of tools and grippers, tool changers, and pallet shuttle systems. Commands are formulated to move tools and grippers so as to approach, grasp, move, fixture, cut, drill, mill, or measure parts. The world model symbolic database contains names and attributes of parts, such as their size and shape (dimensions and tolerances) and material characteristics (mass, color, hardness, etc.). Maps consist of plan drawings that illustrate part shape and the relative positions of features on parts, or parts in assemblies. Sensory processing computes estimates of part shape and dimensions, and estimates differences between desired and measured properties. Value judgment evaluates part quality and supports planning for part handling and fixturing sequences. An operator interface allows human operators to visualize the status of operations of the machine, or to intervene to change priorities or modify the sequence of operations. Executors monitor how well plans are being followed, and modify parameters as necessary to keep on plan. Output commands consist of elemental movement commands (such as <GoAlongPath>, <MoveToPoint>, <DrillHole>, <MillFace>, <MeasureSurface>) that are issued to machine subsystem controllers for controlling elementary movements or behaviors for machining, manipulating, and inspecting part features. Output motion commands are referenced to a coordinate frame defined in the object or part being operated upon.
Level 3—Elemental move (≈2 s planning horizon)
The Elemental move (E-move) level decomposes elemental movement commands into a series of trajectory segments defined in machine coordinates. (Complex movements that require significantly more than one second are broken up at the task level into several E-moves.) Plans are developed and commands are issued defining safe path way points or reference trajectories for tools, manipulators, and inspection probes so as to avoid collisions and singularities, and assure part quality and process safety. The world model symbolic database contains names and attributes of part features such as surfaces, holes, pockets, grooves, threads, chamfers, burrs. Maps consist of drawings that illustrate feature shape and dimension and the relative positions of feature boundaries. Sensory processing computes estimated attributes of part features such as position, shape, dimensions, and surface properties. Value judgment supports planning of machine motions and evaluates feature quality. An operator interface allows a human operator to visualize the state of the machine, or to intervene to change mode or interrupt the sequence of operations. Executors monitor how well plans are being followed, and modify parameters as necessary to keep on plan. Output consists of commands to motion sequencers to move along trajectory segments between waypoints defined in machine coordinates.
Level 2—Primitive (200 ms planning horizon)
The primitive level plans paths for tools, manipulators, and inspection probes so as to minimize time and optimize performance. It computes tool or gripper acceleration and deceleration profiles taking into consideration dynamical interaction between mass, stiffness, force, and time. The world model symbolic database contains names and attributes of linear features such as lines, trajectory segments, and vertices. Maps (when they exist) consist of perspective projections of linear features such as edges, lines, or tool or end-effector trajectories. Sensory processing applied to machine axis data computes estimated motions of tools and grippers. Sensory processing applied to touch probes and cameras may compute part feature attributes such as position, shape, size, and orientation. Value judgment supports trajectory optimization. An operator interface allows a human operator to visualize the state of the machine, the position of the tool, or to intervene to change mode or override the feed rate. Executors monitor how well plans are being followed, and modify parameters as necessary to keep part dimensions within tolerance. Output consists of commands to coordinated axis controllers to move tools or grippers at desired velocities and accelerations in tool tip coordinates, or to exert desired forces, or maintain desired stiffness parameters.
Level 1—Servo level (20 ms planning horizon)
The servo level transforms commands from tool tip to joint actuator coordinates. Planners interpolate between primitive trajectory points for each actuator. The world model symbolic database contains values of state variables such as joint positions, velocities, and forces, proximity sensor readings, position of discrete switches, state of touch probes, as well as image attributes associated with camera pixels. Maps consist of camera images and displays of sensor readings. Sensory processing scales and filters data from sensors that measure actuator positions, velocities, forces, torques, and touch. This information is provided as estimated state feedback to actuator controllers. Sensory processing applied to data from touch probes may measure the position of points on part features. Sensory processing applied to camera data may compute attributes of pixels in an image. An operator interface allows a human operator to visualize the position and velocity of individual axes or to intervene to change mode or jog individual axes. Executors servo individual actuators and motors to follow interpolated trajectories. Position, velocity, or force servoing may be implemented, and in various combinations. Output consists of commands to power amplifiers that specify desired actuator torque or power. Outputs are produced every 2 ms (or whatever rate is dictated by the machine dynamics and servo performance requirements). The servo level also commands switch closures that control discrete actuators such as relays and solenoids.
At the Servo and Primitive levels, the output command rate is typically clock driven on a regular cycle. At the E-Move level and above, the output command rate becomes irregular because it is event driven.
The above example shows how the complexity inherent in a large manufacturing enterprise can be managed through hierarchical layering. Hierarchical decomposition of tasks is a well developed method for organizing complex systems. It has been used in many different types of organizations throughout history for effectiveness and efficiency of command and control. In a hierarchical organization, higher level nodes have broader scope and longer time horizons, with less concern for detail. Lower levels have narrower scope and shorter time horizons, with more focus on detail. Similarly, ISAM nodes at the upper levels in the hierarchy are responsible for long range plans consisting of major milestones, while at lower levels, ISAM nodes successively refine the long range plans into short term tasks with detailed activity goals. At lower levels, information from sensors is computed over local neighborhoods and short time intervals, while at higher levels, sensory information is integrated over large spatial regions and long time. At low levels, knowledge is short term and fine grained, while at the higher levels knowledge is broad in scope and generalized. At every level, feedback loops are closed to provide reactive behavior. Lower levels have high-bandwidth loops with fast-response reactions, whereas higher levels integrate feedback over longer time intervals and react more deliberately to slower trends.
ISAM defines a computational node as shown in Figure 1 consisting of five basic processes: sensory processing (SP), world modeling (WM), behavior generation (BG), value judgment (VJ), and a knowledge database process (KD). All these processes may have input and output connections to an Operator Interface. The arrows between processes indicate the flow of information within the ISAM node among SP, WM, BG, VJ processes, the KD, and the operator interface. The pathway from sensory processing through world modeling to behavior generation closes a reactive feedback control loop between observed input and commanded actions. The pathway from behavior generation through world modeling and value judgment back to behavior generation enables deliberative planning and reasoning about future actions. The pathway from sensory processing through value judgment to world modeling enables situation evaluation and learning. The looping interconnection between sensory processing and world modeling enables recursive estimation and knowledge acquisition. Thus, ISAM integrates real-time planning and execution of behavior with dynamic world modeling, knowledge representation, and sensory perception within each node.
Input from the Operator Interface enables a human supervisor to provide commands, to override or modify system behavior, to perform various types of teleoperation, to switch control modes (e.g., automatic, teleoperation, single step, pause). Output to the Operator Interface enables a human to observe the values of state variables, images, maps, and entity attributes. The Operator Interface can also be used for maintenance, programming, and debugging.
Figure 1. A node in the ISAM reference
model architecture. The elemental
processes of an intelligent system are behavior generation (planning and control),
sensory processing (filtering, detection, recognition, and interpretation),
world modeling (store and retrieve knowledge and predict future states), and
value judgment (compute cost, benefit, importance, and uncertainty).
These are supported by a knowledge database and a system architecture
that interconnects the elemental processes and the knowledge database.
This collection of processes and their interconnections make up a generic
node in the ISAM reference model architecture. Each process in the node may have an operator
interface.
Figure 2 shows a more detailed view of the ISAM node. In Figure 2, a BG unit accepts task command input from an Executor in a higher-level BG unit. Within the BG unit there is a task decomposition planner that decomposes each commanded task into set of a tentative plans for subordinate BG units. These tentative plans are submitted to a WM simulator/predictor which generates expected results. Value Judgment computes the cost and benefit of each tentative plan and its expected results, and reports this evaluation back to the task decomposition planner. The planner then selects the best set of tentative plans to give to Executors for execution. For each subordinate BG unit, there is an Executor that compares each step in its plan with feedback from the Knowledge Database. Each Executor then outputs subtask commands to a subordinate BG unit and monitors its behavior. The KD is kept up-to-date by processed sensory information. The SP, WM, and VJ processes that supply the KD with the information needed by the BG unit for planning and control are also shown in Figure 2.
Figure 2. Interfaces between BG, WM, SP, and VJ. The Task Decomposition Planner within the Behavior Generation BG process selects or generates tentative plans. These are sent to the WM simulator/predictor to generate expected results. The Value Judgment process computes cost and benefits of tentative plans and expected results. The planner selects the tentative plan with the best cost/benefit ratio and places it in plan buffers within the Executors. Each Executor cycles through its plan, compares feedback with planned results, and issues subtask command output to lower level BG modules.
The ISAM generic node illustrated in Figures 1 and 2 can be used to construct a hierarchical reference model architecture such as shown in Figure 3.
Figure 3. An ISAM reference model architecture for a machining
center. Processing nodes are
organized such that the BG processes form a command tree. On the right, are examples of the functional
characteristics and planning horizon of the BG processes at each level. On the left, are examples of the type of
entities recognized by the SP processes and stored by the WM in the KD
knowledge database at each level. (KD processes are not shown in this
figure.) Sensory feedback data flowing
up the hierarchy typically form a graph, not a tree. VJ processes are hidden behind WM processes. An operator interface may provide input to,
and output from BG, WM, SP, and VJ processes in every node.
Each node in the ISAM architecture shown in Figure 3 corresponds to an organizational unit in a manufacturing enterprise. At the top of Figure 3 is the Shop Control node responsible for generating plans for one or more Cell Control nodes out to a planning horizon of about 5 h. At the Cell level, Cell Control nodes are responsible for generating plans for one or more Workstation Control nodes out to a planning horizon of about 30 minutes. At the Workstation level, Workstation Control nodes are responsible for generating plans for one or more Machine Control nodes out to a planning horizon of about 3 minutes. At the Machine level, each Machine Control node is responsible for generating plans for each of its E-move nodes out to a planning horizon of about 20 s. The Machine node in Figure 3 supervises three E-move nodes. One is for an Inspection subsystem that computes trajectories for touch probes and pan and tilt motors for cameras. A second is for a Tool Motion subsystem that computes motion trajectories for several axes of continuous tool path and spindle control. A third is for a Discrete operations subsystem that controls a conveyor, buffer, pallet load and unload, tool change, and coolant on/off. Coordination between E-move nodes is accomplished through peer-to-peer communication between agents in the Machine node and between world model processes at the E-move level. The horizontal curved lines between WM processes represent the sharing of state information between nodes within subsystems. Each E-move Control node is responsible for generating plans for its Primitive nodes out to a planning horizon of about 2 s. The Tool Motion E-move node has one or more Primitive Control nodes that compute dynamic tool trajectories out to a planning horizon of about 200 ms. The Inspection subsystem E-move node may have one or more Primitive Control nodes that compute camera pointing trajectories, or touch probe trajectories out to a planning horizon of 200 ms. The Discrete Operations E-move node typically has no Primitive Control node but connects directly to one or more Servo nodes. This is because discrete operations typically consist of simple switch closures. Finally, at the Servo level, tool motions are decomposed into signals for each actuator. For each actuator there is an Executor that sends it commands. Each sensor sends signals to a sensory processing process that provides updates to the world model knowledge database.
At all levels, ISAM nodes close reactive loops. At the Servo level, ISAM nodes route feedback from observed sensory input through world modeling to behavior generation in a tight feedback control loop. The Servo level knowledge database contains estimated and predicted state variables. Servo level sensory processing and world modeling interact to provide Kalman filtering. The Servo feedback loop for a machine tool typically operates at a rate of 10,000 Hz. For a robot, the Servo loop may operate at 1,000 Hz or even slower. At higher levels, reactive loops are slower because sensory information is processed, filtered, and clustered into higher level abstractions as it ascends the sensory processing hierarchy.
Each ISAM node acts as an intelligent controller. Depending on where the ISAM node resides in the architecture, it might serve as an intelligent controller for a set of actuators, a subsystem, a machine, a workstation, a manufacturing cell, a shop, factory, or upper management organizational unit. The functionality of any ISAM node (or a set of processes within a node) can be implemented as a computer controller, or as a human management unit, at any level in the manufacturing enterprise.
Within each ISAM node, World Modeling, Sensory Processing, and Value Judgment processes provide Behavior Generation processes with the information needed for decision making and control. Within each node, Behavior Generation processes decompose tasks into subtasks for subordinate Behavior Generation processes. At each level World Model knowledge is shared between Knowledge Databases at the same level, and relational pointers are established between Knowledge Data structures at both higher and lower levels.
At each level, Sensory Processing processes accept sensory observations from Sensory Processing processes at lower levels, and output processed and clustered sensory observations to Sensory Processing processes in higher level nodes. At all levels, SP processes compare and correlate predictions from the WM with observations from lower level SP processes. Differences are used to update the estimated state of the World stored in the world model Knowledge Database. Correlation is used to recognize correspondence between what is stored in the internal Knowledge Database and what is observed in the external real World.
At each level, knowledge is represented in a form, and with a spatial and temporal resolution, that meets the processing requirements of the nodes at that level. At each level, state variables, entities, events, and maps are maintained to the resolution in space and time that is appropriate to that level. At each successively lower level in the hierarchy, detail geometrically increases while range geometrically decreases. Temporal resolution increases while the span of interest decreases. This produces a computational load within each node that remains relative constant throughout the hierarchy. As a result, behavior generating functions make plans of roughly the same length in time at each level. For many this will seem strange, because in current practice machine tool paths may consist of thousands of steps, or only a single step. ISAM handles this by making planners at each level look ahead to relatively constant time horizons. Thus, a tool path that is 1000 commands long may be partitioned into 100 paths of roughly 10 steps, each of which take about the same period of time.
Sensory perception functions compute entities that contain roughly the same number of sub entities. At higher levels, plans, perceived entities, and world modeling simulations are more complex, but there are less of them and there is more time available between replanning intervals for processes to run. Thus, hierarchical layering keeps the amount of computing resources needed in each node within reasonable limits and relatively constant.
At each level, there is a characteristic loop bandwidth, a characteristic planning horizon, a characteristic type of task decomposition, a characteristic range of temporal integration of sensory data, and a characteristic window of spatial integration. At each level, information is extracted from the sensory data stream in order to keep the world model knowledge database accurate and up to date.
At each level, sensory data is processed, entities are recognized, world model representations are maintained, and tasks are deliberatively decomposed into parallel and sequential subtasks, to be performed by cooperating sets of agents within the BG processes. At each level, feedback from sensors reactively closes a control loop allowing each agent to respond and react to unexpected events. At each level, tasks are decomposed into subtasks and subgoals, and agent behavior is planned and controlled. The result is a system that combines and distributes deliberative and reactive control information throughout the entire hierarchical architecture, with both planned and reactive capabilities tightly integrated at all levels of space and time resolution.
At each level, global goals are refined and focused onto more narrow and higher resolution sub goals. At each level, attention is focused into a more narrow and higher resolution view of the world. The effect of each hierarchical level is thus to geometrically refine the detail of the task and the view of the world, while only linearly increasing the computational power required of the overall system.
At the bottom of the hierarchy and external to the control system, are actuators that act on the world environment, and sensors that transform events in the world into information signals for the control system. The external world environment contains a variety of real objects, such as materials, tools, machines, and fixtures as well as other intelligent agents, and forces of nature, all of which have states and may cause events and situations to occur.
Some definitions
To provide a solid foundation for interface standards and performance measures, the language used to define the reference model must be precise, so that there is no ambiguity and the reference model precisely reflects physical design and software architecture within the intelligent system. Therefore, we provide the following set of definitions for terminology used in describing ISAM.
a description of the functional elements, the
representation of knowledge, and the flow of information within the system
the assignment of functions to subsystems and
the specification of the interfaces between subsystems
Df: reference model architecture
an architecture in which the entire
collection of entities, relationships, and information units involved in
interactions between and within subsystems are defined and modeled
To be considered as a reference model for the study and design of intelligent manufacturing systems, an architecture should have the following properties:
1. It should define the functional elements, subsystems, interfaces, entities, relationships, and information units involved in intelligent manufacturing systems.
2. It should support the selection of goals, the generation of plans, the decomposition of tasks, the scheduling of sub tasks, and provide for feedback to be incorporated into control processes so that both deliberative and reactive behaviors can be combined into a single integrated system.
3. It should support the processing of signals from sensors into knowledge of situations and relationships, and the storage of knowledge in representational forms that can support reasoning, decision making, and intelligent control.
4. It should provide both static (long term) and dynamic (short term) means for representing the richness and abundance of knowledge necessary to describe the manufacturing environment.
5. It should support the transformation of information from sensor signals to symbolic representations of objects, events, and situations; and from iconic (pictorial) to descriptive (symbolic) forms, and vice versa.
6. It should support the acquisition (or learning) of new information, and the integration and consolidation of newly acquired data into long term memory.
7. It should provide for the representation of values, the computation of costs and benefits, assessment of uncertainty and risk, the evaluation of plans and behavioral results, and the optimization of performance.
a system with the ability to act
appropriately in an uncertain environment
that which maximizes the probability of success
the achievement or maintenance of behavioral
goals
a desired state of the environment that a
behavior is designed to achieve or maintain
the fundamental computational processes from
which the system is composed
Axiom: The functional elements of an intelligent
system are sensory processing, world modeling, value judgment, and behavior
generation.
a series of actions or operations that accept
input and produce output
Df: sensory processing
sensory processing is a set of processes by
which sensory data interacts with prior knowledge to detect or recognize useful
information about the world
Sensory processing accepts signals from sensors that measure properties of the external world or conditions internal to the system itself. Sensory processing scales, windows, and filters data, computes observed features and attributes, and compares them with predictions from internal models. Correlation between sensed observations and internally generated expectations are used to detect events and recognize entities and situations. Variance between sensed observations and internally generated predictions are used to update internal models. Sensory processing also computes attributes of entities and events, and it clusters, or groups, recognized entities and detected events into higher-order entities and events.
In general, sensors do not directly measure the state of the world. Sensors typically only measure phenomena that depend on the state of the world. Signals generated by sensors may be affected by control actions that cause the sensors to move through the world. Sensor output signals are also corrupted by noise. The set of equations that describe how sensory output depends on the state of the world, the control action, and sensor noise is called a measurement model.
A measurement model is typically of the form
y = H(x, u, h)
where
y = signals from sensors
x = state of the world
u = control action
h = sensor noise
H = a function that relates sensor output to world state, control action , and noise
A linearized form of the measurement model is typically of the form
y = Cx + Du
+ h
where
C is a matrix that defines how sensor signals depend on the world state
D is a matrix that defines how sensor signals depend on the control action
Df: world modeling
world modeling is a process that constructs
and maintains a world model that can be used for feedback control as well as
for the generation of predictions of sensory signals and simulation of plans
World modeling performs four principal
functions:
1. It generates and maintains a best estimate of the state of the world that can be used for controlling current actions and planning future behavior. This best estimate resides in a knowledge database describing the state and attributes of objects, events, classes, agents, situations, and relationships. This knowledge database has both iconic and symbolic structures and both short-term and long-term components.
2. It predicts (possibly with
several hypotheses) sensory observations based on the estimated state of the
world. Predicted signals can be used by
sensory processing to configure filters, masks, windows, and schema for
correlation, model matching, recursive
estimation, and focusing attention.
3. It acts as a database server
in response to queries for information stored in the knowledge database.
4. It simulates results of possible future plans based on the estimated state of the world and planned actions. Simulated results are evaluated by the value judgment system in order to select the “best” plan for execution.
Df: world model
an internal representation of the world
The world model is
the intelligent system's best estimate of the world. The world model may include models of portions of the
environment, as well as models of objects and agents. It also includes a system model that represents the internal
state of the intelligent system itself. The world model is stored in a dynamic
distributed knowledge database that is maintained by world modeling
processes. Knowledge stored in the
world model is distributed among computational nodes in the ISAM reference
architecture. The world model in each
node contains knowledge of the world with range and resolution that is
appropriate for control functions in the behavior generation process in that
node. The ISAM concept of a world model
is closely related to the control theory of a system model.
a set of differential equations
(for a continuous system) or difference equations (for a discrete system) that
predict how a system will respond to a given input
A system model is
typically of the form
dx/dt = F(x, u, z)
where
x = state of the system
dx/dt
= rate of change in the system state
u = control action
F = function that defines how the system
state changes over time in response to control actions
z
= error in the
system model
A linearized form
of the above system model is of the form
dx/dt = A x + B u + z
where
A
is a matrix that defines how the system state evolves over time without
control action
B is a matrix that defines how the control action affects the system
state
Learning may enable
the world model to acquire a system model.
This type of learning is often called “system identification.” Learning of world model parameters may be
implemented by neural nets, adaptive filtering, or system identification
techniques.
the data structures and the
static and dynamic information that collectively form the world model
The knowledge
database is a store of information about the world in the form of structural
and dynamic models, state variables, attributes and values, entities and
events, rules and equations, task knowledge, images, and maps.