Amazon Enters The Open-Source Deep Learning Fray Amazon Enters The Open-Source Deep Learning Fray
The Synergy Research Group’s last report of 2015 attributed 31% of the cloud computing market to Amazon’s Amazon Web Services (AWS), nearly four times... Amazon Enters The Open-Source Deep Learning Fray

The Synergy Research Group’s last report of 2015 attributed 31% of the cloud computing market to Amazon’s Amazon Web Services (AWS), nearly four times as much as its nearest competitor, Microsoft. This would come as no surprise to any programmer, Data Engineer, or Data Scientist, AWS is a mainstay when it comes to working at scale. It is integral to the internet’s infrastructure. In light of this, Amazon’s recent analytics offering is surprisingly banausic. Competitors like Microsoft and IBM released dedicated analytics platforms to support Data Scientists and have open-sourced deep learning frameworks such as CNTK and TensorFlow. Comparatively, Amazon has been resting on its laurels. Yes, there is AWS IOT, AWS EMR, AWS Kinesis, and AWS Machine Learning, but these seem to cater more to engineering rather than Data Science. Outside of pre-built instances like the Data Science Toolbox – Python, Apache Jupyter, and Apache Spark – Data Scientists using AWS tools have to go through the process of setting up every component they need from scratch. This may be about to change with the recent open-sourcing of Amazon’s Deep Scalable Sparse Tensor Network Engine. It’s quite a mouthful and the abbreviation isn’t much better, DSSTNE. Amazon says you can just call it… Destiny.

Like other frameworks of its ilk, Amazon Destiny facilitates the training and deploying of deep learning models on GPUs, and can be run locally, on a Docker container, or on AWS on a GPU instance. Unlike similar frameworks, Amazon Destiny is orientated towards search and recommendations – no surprise given its parent company. The release notes claim that Destiny is the fastest deep learning library available, and that it’s the best at using multiple GPUs on a single server. At the other hand, Destiny is a bit bare-bones. The documentation is sparse, the library only works with data in the NetCDF format, and it has no support for Convolutional or Recurrent Neural Networks. (The addition of CNN’s and RNN’s is apparently next in the pipeline.)

Time and I will tell if this release is a token open-source offering or thoughtful gift to the wider analytics community. You can find the project here.


©ODSC 2016.

Gordon Fleetwood

Gordon Fleetwood

Gordon studied Math before immersing himself in Data Science. Originally a die-hard Python user, R's tidyverse ecosystem gradually subsumed his workflow until only scikit-learn remained untouched. He is fascinated by the elegance of robust data-driven decision making in all areas of life, and is currently involved in applying these techniques to the EdTech space.


Notice: Undefined variable: logo_content in /home/odsc1733/public_html/opendatascience.com/wp-content/plugins/mobile-menu/includes/class-wp-mobile-menu-core.php on line 310