Nearest Neighbor Methods and Vector Models – part 1

Nearest Neighbor Met...

This is a blog post rewritten from a presentation at NYC Machine Learning. It covers a library called Annoy that I have built that helps you do (approximate) nearest neighbor queries in high dimensional spaces. I will be splitting it into several parts. This first talks about vector models, how to measure similarity, and why […]

Breaking Linear Classifiers on ImageNet

Breaking Linear Clas...

You’ve probably heard that Convolutional Networks work very well in practice and across a wide range of visual recognition problems. You may have also read articles and papers that claim to reach a near “human-level performance”. There are all kinds of caveats to that (e.g. see my G+ post on Human Accuracy is not a […]

A Quick Introduction to Neural Networks

A Quick Introduction...

An Artificial Neural Network (ANN) is a computational model that is inspired by the way biological neural networks in the human brain process information. Artificial Neural Networks have generated a lot of excitement in Machine Learning research and industry, thanks to many breakthrough results in speech recognition, computer vision and text processing. In this blog post we will try to […]

Attention and Memory in Deep Learning and NLP

Attention and Memory...

A recent trend in Deep Learning are Attention Mechanisms. In an interview, Ilya Sutskever, now the research director of OpenAI, mentioned that Attention Mechanisms are one of the most exciting advancements, and that they are here to stay. That sounds exciting. But what are Attention Mechanisms? Attention Mechanisms in Neural Networks are (very) loosely based […]

An Intuitive Explanation of Convolutional Neural Networks

An Intuitive Explana...

Convolutional Neural Networks (ConvNets or CNNs) are a category of Neural Networks that have proven very effective in areas such as image recognition and classification. ConvNets have been successful in identifying faces, objects and traffic signs apart from powering vision in robots and self driving cars. Figure 1: Source [1] In Figure 1 above, a ConvNet is able […]

RNNs in Tensorflow, a Practical Guide and Undocumented Features

RNNs in Tensorflow, ...

In a previous tutorial series I went over some of the theory behind Recurrent Neural Networks (RNNs) and the implementation of a simple RNN from scratch. That’s a useful exercise, but in practice we use libraries like Tensorflow with high-level primitives for dealing with RNNs. With that using an RNN should be as easy as […]

Feature Learning Escapades

Feature Learning Esc...

Act I: Google Research Summer Internship 2011 Disclaimer: The following post was brought out of the archives on Andrej Karpathy’s blog for your viewing here.  Hanging out at Google in 2011 The thread of this paper begins in Summer of 2011, when I accepted a summer internship offer from Google Research. My project involved Deep Learning for videos, as […]

Recurrent Neural Network Tutorial #4 – Implementing  a GRU/LSTM RNN with Python and Theano

Recurrent Neural Net...

The code for this post is on Github. This is part 4, the last part of the Recurrent Neural Network Tutorial. The previous parts are: Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs Recurrent Neural Networks Tutorial, Part 2 – Implementing a RNN with Python, Numpy and Theano Recurrent Neural Networks Tutorial, Part 3 – Backpropagation […]

Neural Networks 2: Evaluation

Neural Networks 2: E...

In last week’s post, I introduced the Artificial Neural Network (ANN) algorithm by explaining how a single neuron in a neural network behaves. Essentially, we can think of a neuron as a classification algorithm with a number of inputs that correspond to the coordinates of data points and a single output that corresponds to the […]