Denny Britz, Deep Learning Specialist - Google Brain

Denny Britz, Deep Learning Specialist - Google Brain

Deep Learning Specialist

Bio: I'm currently a Brain Resident at Google, working on Deep Learning techniques for Machine Translation, Conversational Models ("Chatbots") and Reinforcement Learning. Previously I've founded several startups, worked on Distributed System Infrastructure (such as Apache Spark), and developed software for Knowledge Basic Construction and Probabilistic Graphical Models (such as DeepDive). I was born in Germany and I've lived in Japan, Korea and Thailand for extended periods of time. What I'm looking for: Opportunities in Asia, especially Japan and Korea. Areas I'm excited about include Deep Learning and Reinforcement Learning, Infrastructure/DevOps, eSports, Distributed Teams, and data-driven Venture Capital.

Implementing a CNN for Text Classification in Tensorflow

Implementing a CNN for Text Classification in Tensorflow

The full code is available on Github. In this post we will implement a model similar to Kim Yoon’s Convolutional Neural Networks for Sentence Classification. The model presented in the paper achieves good classification performance across a range of text classification tasks (like Sentiment Analysis) and has since become a standard baseline for new text […]

Attention and Memory in Deep Learning and NLP

Attention and Memory in Deep Learning and NLP

A recent trend in Deep Learning are Attention Mechanisms. In an interview, Ilya Sutskever, now the research director of OpenAI, mentioned that Attention Mechanisms are one of the most exciting advancements, and that they are here to stay. That sounds exciting. But what are Attention Mechanisms? Attention Mechanisms in Neural Networks are (very) loosely based […]

RNNs in Tensorflow, a Practical Guide and Undocumented Features

RNNs in Tensorflow, a Practical Guide and Undocumented Features

In a previous tutorial series I went over some of the theory behind Recurrent Neural Networks (RNNs) and the implementation of a simple RNN from scratch. That’s a useful exercise, but in practice we use libraries like Tensorflow with high-level primitives for dealing with RNNs. With that using an RNN should be as easy as […]

Learning Reinforcement Learning (With Code, Exercises and Solutions)

Learning Reinforcement Learning (With Code, Exercises and Solutio...

Skip all the talk and go directly to the Github Repo with code and exercises. WHY STUDY REINFORCEMENT LEARNING Reinforcement Learning is one of the fields I’m most excited about. Over the past few years amazing results like learning to play Atari Games from raw pixels and Mastering the Game of Go have gotten a […]

Understanding Convolutional Neural Networks for NLP

Understanding Convolutional Neural Networks for NLP

When we hear about Convolutional Neural Network (CNNs), we typically think of Computer Vision. CNNs were responsible for major breakthroughs in Image Classification and are the core of most Computer Vision systems today, from Facebook’s automated photo tagging to self-driving cars. More recently we’ve also started to apply CNNs to problems in Natural Language Processing and gotten some interesting […]

Recurrent Neural Network Tutorial #4 – Implementing  a GRU/LSTM RNN with Python and Theano

Recurrent Neural Network Tutorial #4 – Implementing a GRU/LSTM...

The code for this post is on Github. This is part 4, the last part of the Recurrent Neural Network Tutorial. The previous parts are: Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs Recurrent Neural Networks Tutorial, Part 2 – Implementing a RNN with Python, Numpy and Theano Recurrent Neural Networks Tutorial, Part 3 – Backpropagation […]

Recurrent Neural Networks Tutorial, Part 3 – Backpropagation through time and vanishing gradients

Recurrent Neural Networks Tutorial, Part 3 – Backpropagatio...

In the previous part of the tutorial we implemented a RNN from scratch, but didn’t go into detail on how Backpropagation Through Time (BPTT) algorithms calculates the gradients. In this part we’ll give a brief overview of BPTT and explain how it differs from traditional backpropagation. We will then try to understand the vanishing gradient […]

Intro to Recurrent Neural Networks #2 – Implement an RNN

Intro to Recurrent Neural Networks #2 – Implement an RNN

This the second part of the Recurrent Neural Network Tutorial. The first part is here. Code to follow along is on Github. In this part we will implement a full Recurrent Neural Network from scratch using Python and optimize our implementation using Theano, a library to perform operations on a GPU. The full code is […]

Implementing a Neural Network from Scratch in Python – an Introduction

Implementing a Neural Network from Scratch in Python – an Intro...

Get the code: To follow along, all the code is also available as an iPython notebook on Github. In this post we will implement a simple 3-layer neural network from scratch. We won’t derive all the math that’s required, but I will try to give an intuitive explanation of what we are doing. I will […]