Attention and Memory in Deep Learning and NLP

Attention and Memory...

A recent trend in Deep Learning are Attention Mechanisms. In an interview, Ilya Sutskever, now the research director of OpenAI, mentioned that Attention Mechanisms are one of the most exciting advancements, and that they are here to stay. That sounds exciting. But what are Attention Mechanisms? Attention Mechanisms in Neural Networks are (very) loosely based […]

An Intuitive Explanation of Convolutional Neural Networks

An Intuitive Explana...

Convolutional Neural Networks (ConvNets or CNNs) are a category of Neural Networks that have proven very effective in areas such as image recognition and classification. ConvNets have been successful in identifying faces, objects and traffic signs apart from powering vision in robots and self driving cars. Figure 1: Source [1] In Figure 1 above, a ConvNet is able […]

Don’t Panic: Deep Learning will be Mostly Harmless

Don’t Panic: D...

This is a blog post summarizing topics covered in a talk at the Dagstuhl workshop on “New Directions in Kernels and Gaussian Processes”. Thoughts on New Directions in Kernels and Gaussian Processes The field of machine learning is entering an interesting era, where due to success we are seeing a large expansion in our leading […]

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

Deep Learning Part 1...

Background and Approach This blog series is based on my upcoming talk on re-usability of Deep Learning Models at the Hadoop+Strata World Conference in Singapore. This blog series will be in several parts – where I describe my experiences and go deep into the reasons behind my choices. Deep learning is an emerging field of research, […]

Introduction: Deep Learning for Chatbots, Part 2

Introduction: Deep L...

IMPLEMENTING A RETRIEVAL-BASED MODEL IN TENSORFLOW Retrieval-Based bots In this post we’ll implement a retrieval-based bot. Retrieval-based models have a repository of pre-defined responses they can use, which is unlike generative models that can generate responses they’ve never seen before. A bit more formally, the input to a retrieval-based model is a context \(c\) (the […]

Training MNIST Neural Networks Using Lasagne

Training MNIST Neura...

The past few weeks, I have been experimenting with the latest-and-greatest deep learning networks, all written in python, to decide which framework I could dive into an become an expert in. After looking at hebel, keras, chainer, and Lasagne, I decided to go with Lasagne because of the documentation and tutorials available online. The other […]

Deep Learning Applications: Smart Cities

Deep Learning Applic...

A. When we speak of Machines – the media dramatizes the issue.  Yet,  city officials and planners plan for ten to twenty years in the future. They will have to consider many of these issues in a pragmatic way B. Deep Learning / Artificial Intelligence will impact many aspects of Smart cities. We decided to […]

Introduction: Deep Learning for Chatbots, part 1

Introduction: Deep L...

Chatbots, also called Conversational Agents or Dialog Systems, are a hot topic. Microsoft is making big bets on chatbots, and so are companies like Facebook (M), Apple (Siri), Google, WeChat, and Slack. There is a new wave of startups trying to change how consumers interact with services by building consumer apps like Operator or x.ai, bot […]

Amazon Enters The Open-Source Deep Learning Fray

Amazon Enters The Op...

The Synergy Research Group’s last report of 2015 attributed 31% of the cloud computing market to Amazon’s Amazon Web Services (AWS), nearly four times as much as its nearest competitor, Microsoft. This would come as no surprise to any programmer, Data Engineer, or Data Scientist, AWS is a mainstay when it comes to working at […]