Last week I attended an interesting lecture by Ran Gilad Bachrach from MSR. Ran presented CryptoNets who was reported in ICML 2016. CryptoNets allows to score trained deep learning models on encrypted data. They use homomorphic encryption a well known mechanism which allows computing encrypted products and sums. So the main trick is to limit the neural net operations to include only sums and products. To overcome this problem CryptoNet is using the square function as the only non-linear operation supported (vs. sigmoids, ReLU etc.)
On the up side, CryptoNets reports 99% accuracy on MNIST data which is the toy example everyone is using for deep learning. On the downside, you can not train a network but just score on new test data. Scoring is quite slow - around 5 minutes, although you can batch up to a few thousands scoring operations together at the same batch. Due to increasing complexity of the represented numbers the technique is also limited to a certain number of network layers.
I believe that in the coming few years additional research effort will be invested for trying to tackle the training of neural networks on private data without revealing the data contents.
Anyone who is interested in reading about other primitives who may be used for performing similar computation is welcome to take a look at my paper: D. Bickson, D. Dolev, G. Bezman and B. Pinkas
Secure Multi-party Peer-to-Peer Numerical Computation. Proceedings of the 8th IEEE Peer-to-Peer Computing (P2P'08), Sept. 2008, Aachen, Germany - where we use both homomorphic encryption but also Shamir Secret Sharing to compute a similar distributed computation (in terms of sums and products).
No comments:
Post a Comment