TABLE 1: COMPARISON OF FUSION LEVELS (ADAPTED FROM [53])
TABLE 2: CONFLICTS BETWEEN SENSING TASKS AND CAMERA MOTION
TABLE 3: ROOM EXPERIMENT DATA
FIGURE 1: AUDIO-VISUAL SENSOR FUSION IN THE BARN OWL
FIGURE 2: SENSOR CONFIGURATION
FIGURE 3: COMPONENT INTERCONNECTIONS
FIGURE 4: DATA FLOW BLOCK DIAGRAM
FIGURE 5: SOUND LOCALIZATION GEOMETRY
FIGURE 6: ONSET SIGNAL GENERATION
FIGURE 7: DIRECT AND ECHO PATH LENGTHS
FIGURE 8: ECHO/DIRECT SOUND VOLUME RATIO DECAY RATES
FIGURE 9: STEREO SAMPLING OF THE WORD "TESTING"
FIGURE 10: ENVELOPE SIGNALS FROM "TESTING"
FIGURE 11: ONSET SIGNALS FROM "TESTING"
FIGURE 12: CROSS-CORRELATION OF ONSET SIGNALS AND ORIGINAL SIGNALS
FIGURE 13: GAIN OF THE ONSET GENERATION PROCESS
FIGURE 14: ROOM LAYOUT WITH SPEAKER AND MICROPHONE LOCATIONS
FIGURE 15: MIRRORED-ROOM METHOD OF ECHO IMAGING
FIGURE 16: FIR CONVOLUTION OPERATOR FOR LEFT MICROPHONE, SPEAKER LOCATION 20, 50% WALL ABSORPTION
FIGURE 17: ONSET CORRELATION FOR TIME-DOMAIN ECHO SIMULATION, LOCATIONS 5(8, 50% ABSORPTION
FIGURE 18: RAW CROSS-CORRELATION FOR SPEAKER LOCATIONS 1(8, 50% ABSORPTION
FIGURE 19: SOUND ABSORPTION OF CARPET (SOLID) AND CEILING TILE (DOTTED)
FIGURE 20: ONSET CORRELATION FOR FREQUENCY-DOMAIN SIMULATION, LOCATIONS 17(20
FIGURE 21: RAW CORRELATION FOR FREQUENCY DOMAIN SIMULATION, LOCATIONS 17(20
FIGURE 22: ONSET CORRELATION FOR ROOM EXPERIMENT, LOCATIONS 5(8
FIGURE 23: RAW CORRELATION FOR ROOM EXPERIMENT, LOCATIONS 5(8
FIGURE 24: LOCALIZATION ERRORS FOR ONSET AND RAW CORRELATION
FIGURE 25: PERFORMANCE FOR TWO SPEAKERS
FIGURE 26: ORIGINAL IMAGE
FIGURE 27: CURRENT IMAGE
FIGURE 28: PIXELS CHANGED
FIGURE 29: ORIGINAL IMAGE
FIGURE 30: INTER-FRAME DIFFERENCE
FIGURE 31: DETECTED HUMAN
FIGURE 32: ORIGINAL CAMERA IMAGE
FIGURE 33: ORIGINAL REFERENCE IMAGE
FIGURE 34: OBJECT BEING ASSIMILATED INTO IMAGE
FIGURE 35: FINAL REFERENCE IMAGE
FIGURE 36: SPHERE UNDER ILLUMINATION
FIGURE 37: SPECTRAL CLUSTERING
FIGURE 38: CHROMATICITY SPACE
FIGURE 39: SKIN TONE CHROMATICITY SAMPLE POPULATION
FIGURE 40: IMAGE OF THE AUTHOR
FIGURE 41: DETECTION OF SKIN PIXELS
FIGURE 42: DETECTION OF FACES BY CHOOSING THE LARGEST SKIN TONE-COLORED REGION
FIGURE 43: PIXEL-LEVEL AUDIOVISUAL FUSION FOR DETECTING A TALKING FACE
FIGURE 44: IMAGES FROM SOUND LOCALIZATION DATA
FIGURE 45: AUDIOVISUAL FUSION GEOMETRY
FIGURE 46: FUSION OF SOUND LOCALIZATION WITH SKIN TONE
FIGURE 47: TARGET DETECTION AFTER FILTERING AND SEGMENTATION.
FIGURE 48: COLOR IMAGE
FIGURE 49: DETECTION OF PERSON SPEAKING
FIGURE 50: TRACKING IN THREE DIMENSIONS
FIGURE 51: MODEL OF TARGET DYNAMICS
FIGURE 52: NOISE ENTERING THE PLANT
FIGURE 53: TRACKING A PERSON WALKING
FIGURE 54: POSITION ESTIMATE WHILE TRACKING A FACE
FIGURE 55: TARGET TRACKING OF A FACE IN 3D
FIGURE 56: EXAMPLE MEMBERSHIP FUNCTIONS
FIGURE 57: CENTROID FOR FUZZY OUTPUT
FIGURE 58: CENTROID FOR SINGLETON OUTPUT
FIGURE 59: RULE COMPONENTS
FIGURE 60: COMBINATION OF RULES
FIGURE 61: ACCEPTABLE COMPROMISES
FIGURE 62: MUTUALLY EXCLUSIVE OPTIONS
FIGURE 63: UNACCEPTABLE CENTROIDS
FIGURE 64: FUSION OF LOCAL AGENTS BY A FUZZY MULTIPLEXER
FIGURE 65: SOUND-FOLLOWING BEHAVIOR
FIGURE 66: CAMERA MOTION FOLLOWING SOUND
FIGURE 67: BEHAVIOR FOR FOLLOWING A FACE
FIGURE 68: INTEGRATION OF FACE AND SOUND TRACKING
FIGURE 69: FACE SELECTION AND CAMERA RESPONSE DUE TO SOUND DIRECTION
FIGURE 70: CAMERA BEHAVIOR WHILE FOLLOWING A CONVERSATION
FIGURE 71: CAMERA PAN BEHAVIOR WHEN TRACKING A FACE
FIGURE 72: FOLLOW-MOVING-TARGET BEHAVIOR COMPONENTS
FIGURE 73: CAMERA PAN BEHAVIOR WHEN TRACKING A BODY
FIGURE 74: BEHAVIOR COMPONENTS FOR VIDEOCONFERENCING
FIGURE 75: GEOMETRY OF FACE WIDTH ANGLE
FIGURE 76: CAMERA BEHAVIOR FOR VIDEOCONFERENCING
FIGURE 77: BEHAVIOR INTEGRATION FOR SURVEILLANCE
FIGURE 78: CAMERA TRACKING MOTION FOR SURVEILLANCE