A system for video-based analysis of face motion during speech

During face-to-face interaction, facial motion conveys

information at various levels. These include a person's emotional

condition, position in a discourse, and, while speaking, phonetic

details about the speech sounds being produced. Trivially, the

measurement of face motion is a prerequisite for any further analysis

of its functional characteristics or information content. It is

possible to make precise measures of locations on the face using

systems that track the motion by means of active or passive markers

placed directly on the face. Such systems, however, have the

disadvantages of requiring specialised equipment, thus restricting the

use outside the lab, and being invasive in the sense that the markers

have to be attached to the subject's face.

To overcome these limitations we developed a video-based system to

measure face motion from standard video recordings by deforming the

surface of an ellipsoidal mesh fit to the face. The mesh is

initialised manually for a reference frame and then projected onto

subsequent video frames. Location changes (between successive frames)

for each mesh node are determined adaptively within a well-defined

area around each mesh node, using a two-dimensional cross-correlation

analysis on a two-dimensional wavelet transform of the

frames. Position parameters are propagated in three steps from a

coarser mesh and a correspondingly higher scale of the wavelet

transform to the final fine mesh and lower scale of the wavelet

transform. The sequential changes in position of the mesh nodes

represent the facial motion. The method takes advantage of inherent

constraints of the facial surfaces which distinguishes it from more

general image motion estimation methods and it returns measurement

points globally distributed over the facial surface contrary to

feature-based methods.

Weitere Episoden von „Fakultät für Sprach- und Literaturwissenschaften - Digitale Hochschulschriften der LMU“