A common industry standard for deep learning hardware is the use of neural network quantization, as noted by an MIT study. Neural networks are computation and memory intensive. Quantization cuts down on the usage of these resources by stripping an input of details which won’t gravely affect the information conveyed.

As an example, consider image quantization. Creating GIFs involves compressing photographs containing potentially thousands of colors to an image with a maximum of 256 colors. While there’s just a limited number of colors, humans can still discern what is being conveyed by the original image. With less information to contain, a GIF file occupies less memory space. Moreover, it is easier to send over networks. This makes it suitable for instant messaging.

Neural network quantization can also be used to automate tasks such as driving. By compressing the signals received by an deep learning machine, quick and accurate decisions can be made on the whim.

The catch

The problem is, standard quantization procedures are vulnerable to adversarial attacks. Adversarial attacks are slight disturbances indiscernible to humans. For deep learning models however, it can cause confusion. This leads to incorrect responses. Needless to say, this is  an alarming security threat since it could be easily taken advantage by attackers to disrupt the models, potentially endangering users.

To make things worse, increase in layers of processing translates to error amplification. Now we see that there must be a balance between attaining efficiency and maintaining robustness.If we are headed towards a future which heavily relies on the faculties of deep learning models, we ought to solve this matter.

The fix

In the earlier mentioned MIT study authored by Ji Lin, Chuang Gan, and Song Han, a possible solution is presented. This comes in the form of the introduction of a mathematical constraint known as the Lipschitz constraint during quantization to keep the error amplification at bay. Their study reveals that the control over the Lipschitz constraint results to performance gains.

The researchers intend to expand the application of this technique to larger data sets and a wider range of models.

“Deep learning models need to be fast and secure as they move into a world of internet-connected devices,” Gan remarked in an interview.

Truly enough, resolving this matter will open doors to a technology which will make our lives simultaneously easier and secure.