Hello there! I'm Jason, a Ph.D. candidate (almost finished!) in Computer Science at Cornell and a Machine Learning scientist at a little startup called Geometric Intelligence. My research focuses on training and understanding neural networks for computer vision and robotics. At various times through my PhD I've worked with Hod Lipson (at the Creative Machines Lab), Jeff Clune, Yoshua Bengio (at U. Montreal's MILA), and Google DeepMind. My work is supported by a NASA Space Technology Research Fellowship.

Research

Conv3 matches

Convergent Learning: Do different neural networks learn the same representations?

ICLR 2016 arXiv (pdf) | code | video of talk and more info »

Deep neural networks have recently been working really well, which has prompted active investigation into the features learned in the middle of the network. The investigation is hard because it requires making sense of millions of learned parameters. But it’s also valuable, because any understanding we acquire promises to help us build and train better models. In this paper we investigate the extent to which neural networks exhibit what we call convergent learning, which is when the representations learned by multiple nets converge to a set of features which are either individually similar between networks or where subsets of features span similar low-dimensional spaces. We probe representations by training multiple networks and then comparing and contrasting their individual, learned features at the level of neurons or groups of neurons. This initial investigation has led to several insights which you will find out if you read the paper. Read more »

Deep Vis Toolbox

Understanding Neural Networks Through Deep Visualization

ICML DL Workshop paper | video | code and more info »

Recent years have produced great advances in training large, deep neural networks (DNNs), including notable successes in training convolutional neural networks (convnets) to recognize natural images. However, our understanding of how these models work, especially what computations they perform at intermediate layers, has lagged behind. Here we introduce two tools for better visualizing and interpreting neural nets. The first is a set of new regularization methods for finding preferred activations using optimization, which leads to clearer and more interpretable images than had been found before. The second tool is an interactive toolbox that visualizes the activations produced on each layer of a trained convnet. You can input image files or read video from your webcam, which we've found fun and informative. Both tools are open source. Read more »

Example fooling images

Deep Neural Networks are Easily Fooled

CVPR paper | code | more »

Deep neural networks (DNNs) have recently been doing very well at visual classification problems (e.g. recognizing that one image is of a lion and another image is of a school bus). A recent study by Szegedy et al. showed that changing an image (e.g. of a lion) in a way imperceptible to humans can cause a network to label the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise static is a lion). We show methods of producing fooling images both with and without the class gradient in pixel space. The results shed light on interesting differences between human vision and state-of-the-art DNNs. Read more »

Plot of convnet transfer performance

How Transferable are Features in Deep Neural Networks?

NIPS paper | code | more »

Many deep neural networks trained on natural images exhibit a curious phenomenon: they all learn roughly the same Gabor filters and color blobs on the first layer. These features seem to be generic — useful for many datasets and tasks — as opposed to specific — useful for only one dataset and task. By the last layer features must be task specific, which prompts the question: how do features transition from general to specific throughout the network? In this paper, presented at NIPS 2014, we show the manner in which features transition from general to specific, and also uncover a few other interesting results along the way. Read more »

Generative Stochastic Networks (GSN) example

Generative Stochastic Networks

First arXiv paper | ICML paper | Latest arXiv paper

Unsupervised learning of models for probability distributions can be difficult due to intractable partition functions. We introduce a general family of models called Generative Stochastic Networks (GSNs) as an alternative to maximum likelihood. Briefly, we show how to learn the transition operator of a Markov chain whose stationary distribution estimates the data distribution. Because this transition distribution is a conditional distribution, it's often much easier to learn than the data distribution itself. Intuitively, this works by pushing the complexity that normally lives in the partition function into the “function approximation” part of the transition operator, which can be learned via simple backprop. We validate the theory by showing several successful experiments on two image datasets and with a particular architecture that mimics the Deep Boltzmann Machine but without the need for layerwise pretraining.

Endless Forms shapes

EndlessForms.com

Watch the two minute intro video. Users on EndlessForms.com collaborate to produce interesting crowdsourced designs. Since launch, over 4,000,000 shapes have been seen and evaluated by human eyes. This volume of user input has produced some really cool shapes. EndlessForms has received some favorable press. Evolve your own shape »

Aracna

Aracna

(read the paper) Many labs work on gait learning research, but since they each use different robotic platforms to test out their ideas, it is hard to compare results between these teams. To encourage greater collaboration between scientists, we have developed Aracna, an open-source, 3D printed platform that anyone can use for robotic experiments.

Cornell Chatbots

AI vs. AI

As part of a class project, Igor Labutov and I cobbled together a speech-to-text + chatbot + text-to-speech system that could converse with a user. We then hooked up two such systems, gave them names (Alan and Sruthi), and let them talk together, producing endless robotic comedy. Somehow the video became popular. There was an XKCD about it, and Sruthi even told Robert Siegel to “be afraid” on NPR. Dress appropriately for the coming robot uprising with one of our fashionable t-shirts.

QuadraTot

Gait Learning on QuadraTot

(read the paper) Getting robots to walk is tricky. We compared many algorithms for automating the creation of quadruped gaits, with all the learning done in hardware (read: very time consuming). The best gaits we found were nearly 9 times faster than a hand-designed gait and exhibited complex motion patterns that contained multiple frequencies, yet showed coordinated leg movement. More recent work blends learning in simulation and reality to create even faster gaits.

For more information, see the project website (with video), project Trac, and code on GitHub.

Nevermind all this, just show me the videos!
Or, if you prefer, here's a slightly outdated CV.

Selected Papers and Posters more »

  1. first page of paper
    (pdf)
    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks?. Advances in Neural Information Processing Systems 27 (NIPS '14), pages 3320 - 3328. 8 December 2014.
    See also: earlier arXiv version. Oral presentation (1.2%).
    abstract▾ | bib▾
  2. first page of paper
    (pdf)
    Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8 June 2015. Oral presentation (3.3%), CVPR 2015 Community Top Paper Award.
    abstract▾ | bib▾
  3. first page of paper
    (pdf)
    Yoshua Bengio, Éric Thibodeau-Laufer, Guillaume Alain, Jason Yosinski. Deep Generative Stochastic Networks Trainable by Backprop. Proceedings of the International Conference on Machine Learning. 21 June 2014.
    See also: supplemental section, earlier and later arXiv versions.
    abstract▾ | bib▾
  4. first page of paper
    (pdf)
    Jeff Clune, Jason Yosinski, Eugene Doan, and Hod Lipson. EndlessForms.com: Collaboratively Evolving Objects and 3D Printing Them. Proceedings of the 13th International Conference on the Synthesis and Simulation of Living Systems. 21 July 2012.
    Winner of Best Poster award.
    abstract▾ | bib▾
  5. first page of paper
    (pdf)
    Sara Lohmann, Jason Yosinski, Eric Gold, Jeff Clune, Jeremy Blum and Hod Lipson. Aracna: An Open-Source Quadruped Platform for Evolutionary Robotics. Proceedings of the 13th International Conference on the Synthesis and Simulation of Living Systems. 19 July 2012.
    Winner of Best Presentation award.
    abstract▾ | bib▾
  6. first page of paper
    (pdf)
    Jason Yosinski, Jeff Clune, Diana Hidalgo, Sarah Nguyen, Juan Cristobal Zagal, and Hod Lipson. Evolving Robot Gaits in Hardware: the HyperNEAT Generative Encoding Vs. Parameter Optimization. Proceedings of the 20th European Conference on Artificial Life, Paris, France. pp 890-897. 8 August 2011.
    abstract▾ | bib▾

Google scholar | see all 26 papers and posters »

Selected Pressmore »

XKCD: AI Sept 7, 2011

BBC: First 'chatbot' conversation ends in argument
(Video interview with Igor and I)
Sept 8, 2011

New Scientist: One Percent: Evolve your own objects for 3D printing
(on homepage)
19 August 2011

Through the Wormhole with Morgan Freeman: Through the Wormhole with Morgan Freeman: Are Robots the Future of Human Evolution? See my walking robots from 7:00 - 7:45 and 9:40 - 11:10. (Season 4, episode 7. unreliable video link) July 10, 2013

see more press »

Miscellaneous

Before coming to Cornell, I did my undergrad at Caltech and then worked on estimation at a research-oriented applied math startup for a couple years.