Knowledge-dependent patterning of spatio-temporal characteristics of lip motion during smile

Objectives:

To generate the codes representing the spatio-temporal features of lip motions during posed and natural smile on the basis of quantitative descriptions of expertise knowledge as well as vector quantization and to develop a mathematical model that identifies the smile types.

Methods:

A total of 30 female subjects performed natural smile induced by comic video clips (5 trials) and posed smile (5 trials). The 3-dimensional motion of four markers adhered on the corners of the mouth and the upper and lower mid-lip points during smile were recorded at a frequency of 100 Hz by a motion capturing system (ProReflex Motion Capture Unit, Qualisys, Gothenburg, Sweden). The four reference markers on the face were recorded simultaneously to minimize the effects of head movements during smile. The displacements of each marker in x, y and z directions corresponding to each half of the period from the rest until the peak of smile were normalized with respect to the distance between the corners of the lips at rest and combined to generate 24-dimensional feature vector for each trial. Feature vectors for both types of smile (3 trials each) were classified into n codes (template vectors) (n=2, 6, 10, 20, 30, 50 and 180) by vector quantization. The remaining feature vectors were used as training data. Template matching with the weight patterns determined on the basis of expertise knowledge was performed to distinguish the types of smile.

Results:

The model with the 10 codes and the double weights for the vector elements representing the lateral motion in template matching provided the same success rate of 76 % as that of the model with all the 180 codes.

Conclusions:

The codes well representing the spatio-temporal features of lip motion during smile was successfully generated and the model for identifying the smile types was developed.