Google open source GPipe, Training Large-scale Neural Network Models
Google has open sourced a distributed machine learning library, GPipe, a library for efficient training of large-scale neural network models. GPipe uses synchronous random gradient descent and pipelines parallel training for any DNN consisting...
