Facial recognition: Everything you need to know

What is facial recognition?

Facial recognition technology analyzes pictures or videos of a person’s face, examining specific details like the distance between their eyes, the shape of their jawline, and the size and shape of their nose and mouth. Then, it compares these characteristics to a database of known faces to find a match.

Facial recognition helps law enforcement quickly and accurately detect criminals. It is also widely used in security systems, video surveillance, and passenger screening. Wondering if you’ve ever encountered face recognition technology yourself? Chances are, you use it daily to unlock your phone.

How does facial recognition work?

Let’s see how facial recognition captures, analyzes, and identifies our most unique feature — our face.

• Face detection. Facial recognition uses computer vision to detect faces in photos or videos. Advanced algorithms then distinguish the face from the crowd, standardize its format, and align key features like the eyes, nose, and mouth. This way, the system focuses on unique facial characteristics instead of being distracted by variations in pose or orientation.
• Feature analysis. The software maps out your facial features — eyes, nose, mouth, and all the unique contours. It measures the distance between your eyes, the width of your nose, the depth of your eye sockets, and the shape of your cheekbones. It captures every detail that makes you you.
• Face matching. Next up, the system compares your faceprint against a huge database of known faces. If your face lines up with someone in the database, bingo!
• Recognition. Finally, we have recognition — the moment of truth when your face logs you into your device or unlocks your door.

Although facial recognition uses 80 key points to identify individuals, the outcome depends on the initial photo or video quality. If the face is obscured or in profile, recognition becomes challenging. However, the technology is continuously improving. Since 1993, the false acceptance ratio in facial recognition systems has decreased by half.

Where is facial recognition used?

Facial recognition enhances security at airports and shops, personalizes shopping, simplifies check-ins, and helps those with disabilities. Let’s explore in more detail how facial recognition is making its mark:

Unlocking smartphones

Passwords are so 2010. Thanks to the face recognition technology in the latest phones, you can unlock your phone with just a glance. It’s fast, and you don’t have to remember a numeric combination or a lock pattern. Also, your face protects your data better than any code if your phone gets stolen. Apple claims the odds of a random stranger unlocking your phone with Face ID are one in a million.

Law enforcement

Facial recognition in CCTVs is an extra eye on the streets, keeping people safe and sound. In fact, it’s becoming essential in law enforcement. When someone gets arrested, the police add their mugshot to local, state, and federal facial recognition databases. This information becomes invaluable in future criminal investigations.

Border control and airport security

Facial recognition makes traveling easier by changing how we move through airports from check-in to boarding. Airports worldwide implement automated ePassport controls where travelers can use biometric passports for quicker security checks. Just scan your face, and you’re good to go. Plus, this technology can swiftly identify passengers and detect potential threats.

Finding missing people

Every second counts when finding missing people or victims of human trafficking. Physical searches by police in crowded areas can be nearly impossible. However, by using photos in a facial recognition database, the system can match faces in real time and identify missing individuals in airports, retail stores, and other public areas.

Reducing shoplifting

Facial recognition software keeps shelves stocked. Stores use this software to spot repeat shoplifters and organized retail criminals. The system compares photographs of these individuals with a database of fraudsters and alerts security guards if it finds a match. The accuracy of image recognition also reduces the likelihood of misidentification and saves retailers some trouble if they encounter a criminal.

Banking operations

Passwordless authentication is the future. Banks now use facial recognition technology to protect your accounts — just one glance at your phone or computer, and you’re authorized. Even if a hacker gets a photo of you and tries to break into your account, the “liveness” detectors in facial recognition have your back.

Healthcare

Hospitals use facial recognition to improve patient care. It helps healthcare workers access patient health records (and avoid mix-ups), speed up registration, recognize emotions and pain, and even identify genetic diseases. Certain healthcare apps use facial recognition to make sure patients take their medication according to their treatment plan.

Monitoring attendance of students and employees

Do you remember roll calls and punch cards? They’re long gone now. These days, schools and workplaces use facial recognition to track attendance. The system scans faces, matches them with photos in a database, and confirms identities. This technology makes it super easy for school administrators and employers to know who’s where and when they’re supposed to be there.

Gambling addiction monitoring

Casinos and gambling venues are now using facial recognition as a proactive measure to help people with gambling addictions. This technology lets companies spot individuals registered as gambling addicts and monitor their activity. That way, staff can step in and suggest it’s time to take a break when needed.

Advantages of facial recognition

Facial recognition provides multiple benefits for society — from improved security at public places and airports to enhanced customer experiences at concerts and sporting events — so let’s take a closer look.

Improved security

Facial recognition technology isn’t just for governments tracking terrorists or criminals in airports, at border control, or in public spaces. It’s also great for boosting security in private businesses and public areas by detecting potential threats and ensuring that only authorized people enter sensitive areas.

Lower crime rates

Face recognition is a game-changer for tracking down burglars, thieves, and trespassers. Just the mere fact that a facial recognition system is in place can scare off criminals. Besides providing physical security, companies also use this software for cybersecurity. Companies swap passwords for face recognition to access computers, making it almost unhackable since there’s nothing to steal or change.

Meanwhile, for law enforcement, facial recognition helps quickly identify criminal suspects and find missing people. It solves cases where suspects have evaded capture for years, detecting unchanging face data points even as they age and their facial features change.

Improved customer experience

Facial recognition is revolutionizing customer experiences. We already unlock our phones with it but imagine using facial recognition to access accounts and make transactions without passwords or pins. At airports, cruises, concerts, or theme parks, your face could become your ticket, speeding up check-ins and controlling access. Moreover, facial recognition empowers people with limited mobility to control devices and access services through facial gestures.

Disadvantages of face recognition

However, facial recognition systems have limitations that we should take into account.

Mass data surveillance

Facial recognition systems gather and use gigantic biometric databases full of your personal data, and they’re owned by private companies run with little to no oversight. How much can we trust a private company that holds huge databases of our biometric data ? What’s stopping them from selling our data?

Exploiting these biometric databases would be at the top of every hacker’s list, and once your identity has been leaked, how difficult would it be to get it back? How difficult would it be to prove that you didn’t commit a crime if your fingerprints or face scans were stolen? How hackers can abuse these systems is simply disastrous and far too risky.

Inaccuracy

Several independent studies have confirmed that facial recognition systems can exhibit racial and gender biases. The National Institute of Standards and Technology (NIST) found that for one-to-one matching, most systems had higher false positive rates for Asian and African American compared to Caucasian faces. NIST also found that when studying one-to-many matches, the systems had the worst false positive rates for African American women, which puts entire populations at greater risk of being falsely accused of a crime.

Clearview AI, a facial recognition company, was recently exposed for collecting over 3 billion images from social media without users' consent to feed its facial recognition database. Meanwhile, the UK government has installed live facial recognition cameras across London, ready to be integrated into everyday policing. While the city’s police say only individuals on “bespoke” watch lists will be flagged, independent researchers found that four out of five people identified by the Metropolitan Police’s facial recognition software were innocent.

Breach of privacy

With global databases full of our images, officials could use live facial recognition cameras as an overt surveillance tool to track anyone at any time. If these databases got into the hands of authoritarian governments in oppressive countries, the data could be used to target, frame, blackmail, or even imprison innocent civilians.

How would this data be stored, and under which state laws would it be regulated? Can we opt out? After being convicted, how long would a person be included on a “bespoke” watch-list and flagged? These are all privacy and security issues that have quietly slipped under the radar at the convenience of private corporations.

While governments and law enforcement openly monitor our lives through optical lenses, some are turning to anti-surveillance fashion to disrupt image recognition. These innovative solutions include camouflaging clothing, signal-blocking and absorbent fabrics, and reflective materials that bend light, concealing people from surveillance cameras.

The future of facial recognition

Is normalized surveillance our future ? Before we introduce facial recognition into society, we must take the following precautions to avoid becoming a surveillance state run by law enforcement:

• Demand tighter regulations: We must consider the civil rights violations that facial recognition technology might cause. We will need deeper insight into where and how long our data is being stored, gathered, and used. Even if regulations are passed, we shouldn’t ignore the fact that fragmented US privacy laws left huge data loopholes for Facebook and others to exploit.
• Wait for the technology to improve: With all of the unanswered questions surrounding the use of facial recognition, it’s obvious that the technology is still in its infancy. While airports have embraced the technology to speed up the boarding process, big tech companies like Google, Amazon, and IBM have backed a one-year moratorium on using facial recognition technology while others push for a nationwide facial recognition law. These are all opportunities to improve its accuracy and restrict its scope.

The power that private companies have could redesign civilization as we know it. We hope Western legislation can ensure that this technology respects our rights to security and privacy, but right now, we wouldn’t bet on it.