Recently Apple announced iPhone X which had a revolutionary facial recognition unlocking system allowing you to use your face as a password.
Many apple fanatics ran to buy the phone as soon as it was released, but for a science nerd like me, it piqued my interest to find out what algorithm is Apple using (Also, is it leaking information to the NSA?). Let me first take you’ll through the original system of facial recognition and how it works and then we get into the algorithm adopted by Apple.
The picture above shall help me to explain this process in a much simpler fashion. In 1966 when the traditional method of facial recognition was being adopted for “educational purposes” (It was for military applications) the scientists applied a complicated mathematical principle to it. They used certain distinguishable landmarks marked such as the center of the pupil from which distances to other parts of the face were measured. If you follow in the image each point is connected by a line to another point to signify its perpendicular distance from the landmarks.
Now, this seems pretty simple, calculate the distance and store it in a database for the computers pursual, but what if the face was turned or at an angle. The simple solution to this problem was to first find the degree of turn or the angle of the bend and then apply the math to revert those changes thus figuring out the original perpendicular distance and cross-checking with the database.
Identix®, a company based in Minnesota, is one of many developers of facial recognition technology. Its software, FaceIt®, can pick someone’s face out of a crowd, extract the face from the rest of the scene and compare it to a database of stored images.
Every face has a couple of distinguishable landmarks, the different peaks and valleys that make up our face. This software tends to recreate the face in 2D using those landmarks as nodal points. Face it utilises about 80 distinguishable landmarks which include the following:
- The distance between the eyes
- The width of the nose
- The depth of the eye sockets
- The shape of the cheekbones
- The length of the jawline
This system turned out to be flawed since in most cases it was required that the image should be taken with the face almost looking at the camera as well as lighting had to be maintained perfectly which rendered the system ineffective.
This system sounds revolutionary for its age and advancement, in fact, in 2001 the Tampa police department installed surveillance systems equipped with facial recognition software to catch offenders with ease. Sadly, in 2003 the system had to be scrapped due to the unexpected ineffectiveness of the system. Apparently, people started wearing masks to cover their faces, who could have thought of that?
The Era of 3D Facial Recognition
In this recently adopted method, a 3D model of the face is graphed by using videos or 2D images recorded from surveillance devices. It is easier to use a system which adopts 3D facial recognition as it renders problems such as angle of face and degree of turn redundant. Furthermore, a 3D image makes the chances of errors in detection less as well as increases the probability of detection.
One paramount advantage of this method is the fact that this system can work even in the darkest of segments, this was a huge improvement over the previous methods.
Â
One major drawback of this system is that the current databases which are available store images in 2D thus rendering matching of data a little difficult. New technology is addressing this challenge. When a 3D image is taken, different points (usually three) are identified. For example, the outside of the eye, the inside of the eye and the tip of the nose will be pulled out and measured. Once those measurements are in place, an algorithm (a step-by-step procedure) will be applied to the image to convert it to a 2D image. After conversion, the software will then compare the image with the 2D images in the database to find a potential match.
The ERA of iPhone X
The technology that enables Face ID is some of the most advanced hardware and software that we’ve ever created. The TrueDepth camera captures accurate face data by projecting and analyzing over 30,000 invisible dots to create a depth map of your face and also captures an infrared image of your face. A portion of the A11 Bionic chip’s neural engine — protected within the Secure Enclave — transforms the depth map and infrared image into a mathematical representation and compares that representation to the enrolled facial data.
Face ID automatically adapts to changes in your appearance, such as wearing cosmetic makeup or growing facial hair. If there is a more significant change in your appearance, like shaving a full beard, Face ID confirms your identity by using your passcode before it updates your face data. Face ID is designed to work with hats, scarves, glasses, contact lenses, and many sunglasses. Furthermore, it’s designed to work indoors, outdoors, and even in total darkness.
In simple words, Face ID does exactly what the 3D image recognition systems do but using more number of vectors and landmarks at a much faster rate. Faster being the operative word. It was extraordinary to notice how Apple updates and analyzes images in such a small time when in reality 3D analysis of facial features can take up to 3 to 4 minutes to find accurate results.
This feature is courtesy of the A11 bionic chip as well as local storage of data. Instead of storing your verification details on a cloud and sending and receiving data every time you log in, apple simply localizes the process by storing relevant information on your phone itself thus allowing the verification time to be cut by 99%.
Furthermore, the TrueDepth camera is intelligently activated; for example, by raising to wake your iPhone X, tapping to wake your screen, or from an incoming notification that wakes the screen. Each time you unlock your iPhone X, the TrueDepth camera recognizes you by capturing accurate depth data and an infrared image. This information is matched against the stored mathematical representation to authenticate.
It is obvious that technology has truly reached the paramount of reality and that day isn’t far when there would be very little difference between technology and magic. With the availability of Convolution Neural networks to apply to object detection, classification, we are close to achieving technological greatness but with all the good comes the bad too. We cannot be exactly sure as to how and where this information may be used and who will use it.
While all the examples above work with the permission of the individual, not all systems are used with your knowledge. Many feel that privacy infringement is too great with the use of these systems, but their concerns don’t end there. They also point out the risk involved with identity theft. Even facial recognition corporations admit that the more use the technology gets, the higher the likelihood of identity theft or fraud. As with many developing technologies, the incredible potential of facial recognition comes with some drawbacks, but manufacturers are striving to enhance the usability and accuracy of the systems.
Further Reading: