3Q’s: Facial Recognition is the New Fingerprint

September 20, 2012

Ear­lier this month, the FBI began rolling out a $1 bil­lion update to the national fin­ger­printing data­base. Facial-​​recognition sys­tems, DNA analysis, voice iden­ti­fi­ca­tion and iris scan­ning will all con­tribute to the government’s arsenal of Next Gen­er­a­tion Iden­ti­fi­ca­tion (NGI) data. We asked Ray­mond Fu, a new assis­tant pro­fessor with joint appoint­ments in the Col­lege of Engi­neering and the Col­lege of Com­puter and Infor­ma­tion Sci­ence, to explain the sci­ence behind one of these new tech­nolo­gies: facial-​​recognition software.

Face- ​​recognition research has been pop­ular for more than two decades. Great advances have been made from researchers from a broad com­mu­nity, such as bio­met­rics, com­puter vision and machine learning. The state-​​of-​​the-​​art tech­niques have been applied to real-​​world sys­tems for appli­ca­tions in sur­veil­lance, secu­rity and foren­sics. Face recog­ni­tion is a tech­nology that requires high accu­racy, espe­cially when secu­rity and foren­sics fac­tors are con­sid­ered. The cur­rent chal­lenges are scal­a­bility of data­bases; large vari­a­tion fac­tors in dif­ferent envi­ron­ments; aging, makeup and pose fac­tors of faces; and faces in social-​​media spaces.

Face-​​recognition sys­tems start with face detec­tion and tracking. Com­pu­ta­tional algo­rithms detect face posi­tions and poses in an image and then extract them for pro­cessing and analysis. During this pipeline, a couple of major chal­lenges create bot­tle­necks for the per­for­mance of real-​​world sys­tems. Facial expres­sions, aging and makeup are key vari­a­tions that cannot be easily removed. Tech­niques of 3-​​D mor­phable mod­eling and local fea­tures have been devel­oped to mit­i­gate such vari­a­tions. Lighting vari­a­tions can sig­nif­i­cantly affect the recog­ni­tion accu­racy espe­cially when a system is used out­side. Bench­mark data­bases have been col­lected from well-​​controlled lighting sources for devel­oping lighting insen­si­tive fea­ture extrac­tion and ana­lyt­ical mod­eling for such purposes.

The increasing acces­si­bility of the social-​​media space presents yet a new chal­lenge to devel­oping a large-​​scale iden­tity data­base. Con­fu­sion of sim­ilar appear­ances, over­load com­pu­ta­tions and mul­tiple data sources bring up uncer­tain­ties in modern face recog­ni­tion. Addi­tion­ally, new trends of soft-​​biometrics, big data and mul­ti­modality face recog­ni­tion have opened up new research thrusts.

Face recog­ni­tion and iden­ti­fi­ca­tion are two dif­ferent prob­lems. Face recog­ni­tion is to match a person’s face against a set of known faces and iden­tify who he or she is. For example, in a crim­inal inves­ti­ga­tion, a detec­tive may want to ID a sus­pect from a face image cap­tured on a sur­veil­lance camera.

Iden­ti­fi­ca­tion is to val­i­date the match of a given face and the claimed ID. For example, if an employee wants to access a secured area in a clas­si­fied depart­ment, she shows her ID card to the sensor while a camera cap­tures her face to match it with the record retrieved from the ID card input. If the match passes, the door will open automatically.

Face recog­ni­tion can be either pas­sive or active. In the air­port, for example, the sur­veil­lance cam­eras are taking videos in real time. Pas­sen­gers’ faces are cap­tured in a pas­sive way. Online social-​​media spaces, like Face­book, pro­vide public domains for users to share their photos in an active way. Both may involve pri­vacy issues. How to bal­ance the pri­vacy issues and the public needs of secu­rity and human-​​computer inter­ac­tion are new research topics in this era.

In my research group, we have been funded by Air Force Office of Sci­en­tific Research, IC Postdoc Fel­low­ship and Google Research on these issues. Our research is mainly focused on under­standing social status and net­working of social-​​media users and their pri­vacy con­cerns. We are working on new com­pu­ta­tional method­olo­gies that could well ana­lyze the visual con­tent of social media and pro­vide auto­matic solu­tions for human-​​computer inter­ac­tion that could advance future social-​​network ecosystems.