Stuff by Saki Shinoda on machine learning, scientific computing, data science, programming, etc.

Paper Notes
Saki Shinoda

I'm a research scientist in deep learning and reinforcement learning for Prediction Machines, based in London, UK. I hold an undergraduate degree from the University of Cambridge in Natural Sciences (Physical Sciences) and a postgraduate MSc from University College London in Computational Statistics and Machine Learning.

Latest post:

Virtual Adversarial Ladder Networks for Semi-Supervised Learning

2017-11-28 18:49:00 +0000

Authors: Saki Shinoda, Daniel E. Worrall, Gabriel J. Brostow

This paper was the outcome of my thesis for the MSc Computational Statistics and Machine Learning at University College London.


Semi-supervised learning (SSL) partially circumvents the high cost of labeling data by augmenting a small labeled dataset with a large and relatively cheap unlabeled dataset drawn from the same distribution. This paper offers a novel interpretation of two deep learning-based SSL approaches, ladder networks and virtual adversarial training (VAT), as applying distributional smoothing to their respective latent spaces. We propose a class of models that fuse these approaches. We achieve near-supervised accuracy with high consistency on the MNIST dataset using just 5 labels per class: our best model, ladder with layer-wise virtual adversarial noise (LVAN-LW), achieves 1.42% +/- 0.12 average error rate on the MNIST test set, in comparison with 1.62% +/- 0.65 reported for the ladder network. On adversarial examples generated with L2-normalized fast gradient method, LVAN-LW trained with 5 examples per class achieves average error rate 2.4% +/- 0.3 compared to 68.6% +/- 6.5 for the ladder network and 9.9% +/- 7.5 for VAT.