After receiving the mission to be performed from the operator, the Central Decision unit of the autonomous system (CD, top) runs the mission planning software (top left) taking potential side constraints into account. Here, the mission is decomposed into a sequential list of mission elements according to a sequence of behavioral capabilities available for mission performance. This list of mission elements determines how to proceed in the mission as long as there is no unexpected event perceived that requires immediate response for safety reasons or a re-planning of the rest of the mission.
Before a behavior is activated, a check is performed exploiting the connectivity links in the capability networks that indicate which perceptual and behavioral skills are needed and whether they are actually available ( M.2 Action, behavioral components); this is done in separate tracks for gaze and attention (BDGA, center left, below; see M.2.5) and for locomotion (BDL, center right; see M.2.4).
In the example given for ground vehicle guidance, the behavioral capability ‘Follow road, lane’ (top left in central rectangle) requires two perceptual and one locomotion capabilities. OVB means ‘Optimization of Visual Behavior’, and RDT means the perceptual capability of ‘Road Detection and Tracking’. RLR as locomotion capability means ‘Road or Lane Running’. OVB needs the basic gaze skills of ‘fixation’ and performing a ‘saccade’ (fast change in gaze direction). RDT needs two cameras, one for the local environment (nearby) and one for larger distances (far).
CRDT represents the perceptual capability of ‘Cross Road Detection and Tracking’ (top center in central rectangle); for clarity, further links are not shown. The capability of ‘following a sequence of waypoints’ (top right in central rectangle) needs the four stereotypical capabilities indicated by dashed arrows: Beside OVB these are ‘3-D Surface recognition’ (3-DS), ‘Obstacle Detection and Tracking’ (ODT), and WPN standing for ‘Way-Point Navigation’.
The figure is intended to get the basic ideas across; it is not at all complete.