List of contents of book:

Dynamic Vision for Perception and Control of Motion

by E.D.Dickmanns, Springer 2007


1   Introduction

1.1    Different Types of Vision Tasks and Systems

1.2    Why Perception and Action?

1.3    Why Perception and Not Just Vision?

 1.4    What Are Appropriate Interpretation Spaces?

1.4.1    Differential Models for Perception ‘Here and Now’

1.4.2    Local Integrals as Central Elements for Perception

1.4.3    Global Integrals for Situation Assessment

1.5    What Type of Vision System is Most Adequate?

1.6    Influence of the Material Substrate on System Design: Technical vs. Biological Systems

1.7   What Is Intelligence? A Practical (Ecological) Definition

1.8   Structuring of Material Covered


2     Basic Relations: Image Sequences ­– “the World”

2.1   Three-dimensional (3-D) Space and Time

2.1.1    Homogeneous Coordinate Transformations in 3-D Space

2.1.2    Jacobian Matrices for Concatenations of HCMs

2.1.3    Time Representation

2.1.4    Multiple Scales

2.2     Objects

2.2.1    Generic 4-D Object Classes

2.2.2    Stationary Objects, Buildings

2.2.3    Mobile Objects in General

2.2.4    Shape and Feature Description

2.2.5    Representation of Motion

2.3   Points of Discontinuity in Time

2.3.1    Smooth Evolution of a Trajectory

2.3.2    Sudden Changes and Discontinuities

2.4   Spatiotemporal Embedding and First-order Approximations

2.4.1    Gain by Multiple Images in Space and/or Time for Model Fitting

2.4.2    Role of Jacobian Matrix in the 4-D Approach to Vision


3     Subjects and Subject Classes

3.1    General Introduction: Perception – Action Cycles

3.2    A Framework for Capabilities

3.3    Perceptual Capabilities

3.3.1    Sensors for Ground Vehicle Guidance

3.3.2    Vision for Ground Vehicles

3.3.3    Knowledge Base for Perception Including Vision

3.4    Behavioral Capabilities for Locomotion

3.4.1    The General Model: Control Degrees of Freedom

3.4.2    Control Variables for Ground Vehicles

3.4.3    Basic Modes of Control Defining Skills

3.4.4    Dual Representation Scheme

3.4.5    Dynamic Effects in Road Vehicle Guidance

3.4.6    Phases of Smooth Evolution and Sudden Changes

3.5    Situation Assessment and Decision-Making

3.6    Growth Potential of the Concept, Outlook

3.6.1    Simple Model of Human Body as a Traffic Participant

3.6.2    Ground Animals and Birds


4     Application Domains, Missions, and Situations

4.1   Structuring of Application Domains

4.2   Goals and Their Relations to Capabilities

4.3   Situations as Precise Decision Scenarios

4.3.1    Environmental Background

4.3.2    Objects/Subjects of Relevance

4.3.3    Rule Systems for Decision-Making

4.4   List of Mission Elements


5     Extraction of Visual Features

5.1    Visual Features

5.1.1    Introduction to Feature Extraction

5.1.2    Fields of View, Multifocal Vision, and Scales

5.2    Efficient Extraction of Oriented Edge Features

5.2.1    Generic Types of Edge Extraction Templates

5.2.2    Search Paths and Subpixel Accuracy

5.2.3    Edge Candidate Selection

5.2.4    Template Scaling as a Function of the Overall Gestalt

5.3    The Unified Blob-edge-corner Method (UBM)

5.3.1    Segmentation of Stripes Through Corners, Edges, and Blobs

5.3.2    Fitting an Intensity Plane in a Mask Region

5.3.3    The Corner Detection Algorithm

5.3.4    Examples of Road Scenes

5.4    Statistics of Photometric Properties of Images

5.4.1    Intensity Corrections for Image Pairs

5.4.2    Finding Corresponding Features

5.4.3    Grouping of Edge Features to Extended Edges

5.5    Visual Features Characteristic of General Outdoor Situations


6     Recursive State Estimation

6.1    Introduction to the 4-D Approach for Spatiotemporal Perception

6.2    Basic Assumptions Underlying the 4-D Approach

6.3    Structural Survey of the 4-D Approach

6.4    Recursive Estimation Techniques for Dynamic Vision

6.4.1    Introduction to Recursive Estimation

6.4.2    General Procedure

6.4.3    The Stabilized Kalman Filter

6.4.4    Remarks on Kalman Filtering

6.4.5    Kalman Filter with Sequential Innovation

6.4.6    Square Root Filters

6.4.7    Conclusion of Recursive Estimation for Dynamic Vision


7     Beginnings of Spatiotemporal Road and Ego-state Recognition

7.1    Road Model

7.2    Simple Lateral Motion Model for Road Vehicles

7.3    Mapping of Planar Road Boundary into an Image

7.3.1    Simple Beginnings in the Early 1980s

7.3.2    Overall Early Model for Spatiotemporal Road Perception

7.3.3    Some Experimental Results

7.3.4    A Look at Vertical Mapping Conditions

7.4    Multiple Edge Measurements for Road Recognition

7.4.1    Spreading the Discontinuity of the Clothoid Model

7.4.2    Window Placing and Edge Mapping

7.4.3    Resulting Measurement Model

7.4.4    Experimental Results


8     Initialization in Dynamic Scene Understanding

8.1    Introduction to Visual Integration for Road Recognition

8.2    Road Recognition and Hypothesis Generation

8.2.1    Starting from Zero Curvature for Near Range

8.2.2    Road Curvature from Look-ahead Regions Further Away

8.2.3    Simple Numerical Example of Initialization

8.3    Selection of Tuning Parameters for Recursive Estimation

8.3.1    Elements of the Measurement Covariance Matrix R

8.3.2    Elements of the System State Covariance Matrix Q

8.3.3    Initial Values of the Error Covariance Matrix

8.4    First Recursive Trials and Monitoring of Convergence

8.4.1    Jacobian Elements and Hypothesis Checking

8.4.2    Monitoring Residues

8.5    Road Elements To Be Initialized

8.6    Exploiting the Idea of Gestalt

8.6.1    The Extended  Gestalt Idea for Dynamic Machine Vision

8.6.2    Traffic Circle as an Example of Gestalt Perception

8.7    Default Procedure for Objects of Unknown Classes


9     Recursive Estimation of Road Parameters and Ego State While Cruising

9.1    Planar Roads with Minor Perturbations in Pitch

9.1.1    Discrete Models

9.1.2    Elements of the Jacobian Matrix

9.1.3    Data Fusion by Recursive Estimation

9.1.4    Experimental Results

9.2    Hilly Terrain, 3-D Road Recognition

9.2.1    Superposition of Differential Geometry Models

9.2.2    Vertical Mapping Geometry

9.2.3    The Overall 3-D Perception Model for Roads

9.2.4    Experimental Results

9.3    Perturbations in Pitch and Changing Lane Widths

9.3.1    Mapping of Lane Width and Pitch Angle

9.3.2    Ambiguity of Road Width in 3-D Interpretation

9.3.3    Dynamics of Pitch Movements: Damped Oscillations

9.3.4    Dynamic Model for Changes in Lane Width

9.3.5    Measurement Model Including Pitch Angle, Width Changes

9.4    Experimental Results

9.4.1    Simulations with Ground Truth Available

9.4.2    Evaluation of Video Scenes

9.5    High-precision Visual Perception

9.5.1    Edge Feature Extraction to Subpixel Accuracy for Tracking

9.5.2    Handling the Aperture Problem in Edge Perception


10    Perception of Crossroads

10.1 General Introduction

10.1.1 Geometry of Crossings and Types of Vision Systems Required

10.1.2 Phases of Crossroad Perception and Turnoff

10.1.3 Hardware Bases and Real-world Effects

10.2 Theoretical Background

10.2.1 Motion Control and Trajectories

10.2.2 Gaze Control for Efficient Perception

10.2.3 Models for Recursive Estimation

10.3 System Integration and Realization

10.3.1 System Structure

10.3.2 Modes of Operation

10.4 Experimental Results

10.4.1 Turnoff to the Right

10.4.2 Turnoff to the Left

10.5 Outlook


11    Perception of Obstacles and Other Vehicles

11.1 Introduction to Detecting and Tracking Obstacles

11.1.1 What Kinds of Objects Are Obstacles for Road Vehicles?

11.1.2 At Which Range Do Obstacles Have To Be Detected?

11.1.3 How Can Obstacles Be Detected?

11.2 Detecting and Tracking Stationary Obstacles

11.2.1 Odometry as an Essential Component of Dynamic Vision

11.2.2 Attention Focusing on Sets of Features

11.2.3 Monocular Range Estimation (Motion Stereo)

11.2.4 Experimental Results

11.3 Detecting and Tracking Moving Obstacles on Roads

11.3.1 Feature Sets for Visual Vehicle Detection

11.3.2 Hypothesis Generation and Initialization

11.3.3 Recursive Estimation of Open Parameters and Relative State

11.3.4 Experimental Results

11.3.5 Outlook on Object Recognition


12    Sensor Requirements for Road Scenes

12.1 Structural Decomposition of the Vision Task

12.1.1 Hardware Base

12.1.2 Functional Structure

12.2 Vision under Conditions of Perturbation

12.2.1 Delay Time and High-frequency Perturbation

12.2.2 Visual Complexity and the Idea of Gestalt

12.3 Visual Range and Resolution Required for Road Traffic Applications

12.3.1 Large Simultaneous Field of View

12.3.2 Multifocal Design

12.3.3 View Fixation

12.3.4 Saccadic Control

12.3.5 Stereovision

12.3.6 Total Range of Fields of View

12.3.7 High Dynamic Performance

12.4 MarVEye as One of Many Possible Solutions

12.5 Experimental Result in Saccadic Sign Recognition


13    Integrated Knowledge Representations for Dynamic Vision

13.1 Generic Object/Subject Classes

13.2 The Scene Tree

13.3 Total Network of Behavioral Capabilities

13.4 Task To Be Performed, Mission Decomposition

13.5 Situations and Adequate Behavior Decision

13.6 Performance Criteria and Monitoring Actual Behavior

13.7 Visualization of Hardware/Software Integration


14    Mission Performance, Experimental Results

14.1 Situational Aspects for Subtasks

14.1.1 Initialization

14.1.2 Classes of Capabilities

14.2 Applying Decision Rules Based on Behavioral Capabilities

14.3 Decision Levels and Competencies, Coordination Challenges

14.4 Control Flow in Object-oriented Programming

14.5 Hardware Realization of Third-generation EMS vision

14.6 Experimental Results of Mission Performance

14.6.1 Observing a Maneuver of Another Car

14.6.2 Mode Transitions Including Harsh Braking

14.6.3 Multisensor Adaptive Cruise Control

14.6.4 Lane Changes with Preceding Checks

14.6.5 Turning Off on Network of Minor Unsealed Roads

14.6.6 On- and Off-road Demonstration with Complex Mission Elements


15   Conclusions and Outlook


Appendix A Contributions to Ontology for Ground Vehicles

A.1  General environmental conditions

A.2  Roadways

A.3  Vehicle

A.4  Form, Appearance, and Function of Vehicles

A.5  Form, Appearance, and Function of Humans

A.6  Form, Appearance, and Likely Behavior of Animals

A.7  General Terms for Acting “Subjects” in Traffic


Appendix B Lateral dynamics

B.1  Transition Matrix for Fourth-order Lateral Dynamics

B.2  Transfer Functions and Time Responses to an Idealized Doublet in Fifth-order Lateral Dynamics


Appendix C Recursive Least–squares Line Fit

C.1  Basic Approach

C.2  Extension of Segment by One Data Point

C.3  Stripe Segmentation with Linear Homogeneity Model

C.4  Dropping Initial Data Point






More information from