A.5 Aircraft landing approach

Since this task is very demanding even for trained pilots, it seemed very unlikely in the early 1980’s that external funding could be obtained for developing a machine vision system that is capable of handling this task autonomously. Therefore, the first round of basic research was done on internal funding with the Hardware-In-the-Loop (HIL) simulation facility

 H.4.2 HIL-Sim AirVehGuidance (1982 – 1987).

After solid foundations based on the 4-D approach to dynamic machine vision with video demonstrations had been achieved, the second round of development towards practically useful systems has been funded by the German Science Foundation (DFG, 1987 – 1992). For flight guidance in landing approach there is no way around the full six degrees of freedom formulation with second-order dynamics in each of them; that means, there are 12 state variables for describing the process. Aircrafts have four control inputs: throttle, elevator, aileron and rudder, the first two of which control motion in the plane of symmetry (two translations and one rotation). Ailerons are designed to generate rotation around the longitudinal axis, but they also generate some motion around the vertical axis, usually. Similarly, the rudder is designed to generate torque around the vertical axis, but has some effect also on rotation around the longitudinal axis and on lateral acceleration. In landing approach, several kinds of flaps may be set in addition for generating more lift and for allowing slower speed for touch down. Control laws for automatic landing with microwave systems or with GPS signals taking inertial sensor data into account are well known. Exploiting human capabilities in visual feedback and conventional control techniques for automatic landing, autonomous landings by machine vision were considered to be in reach. Using the knowledge captured in dynamic models for simulation and flight analysis is mandatory for success. The 4-D approach integrates all these components right from the beginning.

Show upper part of Picture

Show bottom part of Picture

G. Eberl did the first formulation of the 4-D approach for aircraft landing by computer vision (1982 – 1987) based on the standardized terminology for flight applications [LN 9300]:

  • Easily scalable coordinates including transformations by 4x4 Homogeneous coordinate Transformation Matrices (HTM), developed in the field of computer graphics, have been used right from the beginning.

  • HIL- simulation was a big help in setting up and testing the basic closed loop vision system with real edge feature extraction by a

    H.1.1 FirstGenerationBVVx

    (without inertial sensing).

  • A gaze controllable (active) camera was used that fixated the runway, counteracting aircraft motion in pitch and yaw.

    The block diagram (left) shows the very simple beginnings in 1982 with four evaluation windows on edges of the landing strip and on the horizon.

Only with the ‘4-D approach’ introducing physically correct spatio-temporal models a solution was possible.

 

 

First HIL-simulation results for landing approach

  • Calligraphic grayscale computer image generation;

  • initially, a single camera has been used;

  • All hardware for real-time visual perception was included in the closed simulation loop.

  • The figure (at left) shows a sequence of snapshots (1a, b, c, 2a, b, c, 3a, b, c) during approach till touch down.

This set-up did not allow handling simulated larger disturbances by crosswinds or gusts.

References

Eberl G (1987). Automatischer Landeanflug durch Rechnersehen. Dissertation, UniBwM / LRT

Dickmanns ED (1988). Computer Vision for Flight Vehicles. Zeitschrift für Flugwissenschaft und Weltraumforschung (ZFW), Vol. 12 (88), pp 71-79

 

Combined visual-inertial sensing

Show Graphs with improved resolution

F.-R. Schell (1987 – 1992) introduced real inertial sensors in the simulation loop; this was an essential step, since their data have very small delay times (a few milliseconds (ms)) compared to visual perception (a few hundred ms, like in the vestibular / ocular interaction in vertebrate systems developed by Nature). He developed visual / inertial data fusion in a hybrid approach. Real inertial sensors allow

  • picking up the relatively high-frequency perturbations on the own body (and the cameras, their own motion from the pointing platform added) with typical noise characteristics and integrating them to angular pose;

  • own body and camera states thus obtained were used for visual interpretation (taking different delay times into account).

  • Low-frequency inertial drifts can be compensated by visual feedback from feature sets stemming from objects sufficiently far away (for example the horizon line).

The hybrid solution could not only be verified in simulation including perturbations by gusts and crosswinds with full control application till touch-down, but also in real flight experiments. However, for safety reasons the real flights with a twin-propeller aircraft Do 128 of the University of Brunswick were done in 1991 without control implementation, i.e. the perception part only, and without final touch down; these flight experiments were funded by the German Science Foundation DFG, (see photo and graphs on the left).



video 12 AircraftLandingApproach 1982– 92

Contrary to Global Positioning Systems (like GPS) as additionally available devices nowadays, the autonomous sense of vision also allows detecting obstacles on the runway and is independent of availability of GPS [Fuerst et al. 1997]. It also can serve as additional device for taxiing on airports. On top of this, landmark navigation and fixation of objects of special interest (other vehicles in the air or constructions of special importance on the ground) may be valuable inputs to safety and mission performance. These topics have been investigated in the framework of helicopter guidance by machine vision (next subsection).

 

References

LN 9300 (1970). Luftfahrtnorm 9300 zur Flugmechanik. Anhang, Beuth-Vertrieb GmbH, Koeln

Schell FR (1992). Bordautonomer automatischer Landeanflug aufgrund bildhafter und inertialer Meßdatenauswertung, Dissertation, UniBwM / LRT

Schell FR (1992). Computer Vision for Autonomous Flight Guidance and Landing, IFAC Symposium Aerospace Control '92, Ottobrunn

Schell FR, Dickmanns ED (1992). Autonomous Landing of Airplanes by Dynamic Machine Vision. Proc. IEEE-Workshop on 'Applications of Computer Vision', Palm Springs,

Fürst S, Werner S, Dickmanns D, Dickmanns ED (1997). Landmark navigation and autonomous landing approach with obstacle detection for aircraft. AeroSense ’97, SPIE Proc. Vol. 3088, Orlando FL, April 20-25, pp 94-105.

Fürst S, Werner S, Dickmanns ED (1997). Autonomous Landmark Navigation and Landing Approach with Obstacle Detection for Aircraft. 10th European Aerospace Conference ‘Free Flight’, Amsterdam, NL, pp 36.1 – 36.11