This page was last significantly updated in 2015! There’s been a lot of neat stuff since then. You should especially check out Neural Tangents.
Sum of Functions Optimizer (SFO) - An optimizer combining the benefits of quasi-Newton and stochastic gradient approaches.
https://github.com/Sohl-Dickstein/Sum-of-Functions-Optimizer
Python and MATLAB code implementing quasi-Newton minibatch optimizer and reproducing figures from paper.
Sohl-Dickstein, Jascha, Ben Poole, and Surya Ganguli. "Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods." Proceedings of the 31st International Conference on Machine Learning (ICML-14). 2014. http://arxiv.org/abs/1311.2115
Look Ahead Hamiltonian Monte Carlo (LAHMC) - An HMC sampler which does not rely on detailed balance.
Python and MATLAB code implementing LAHMC sampler and reproducing figures from paper.
Sohl-Dickstein, Jascha, Mayur Mudigonda, and Michael DeWeese. "Hamiltonian Monte Carlo without detailed balance." Proceedings of the 31st International Conference on Machine Learning (ICML-14). 2014. http://jmlr.org/proceedings/papers/v32/sohl-dickstein14.pdf
Hamiltonian Annealed Importance Sampling (HAIS) - A method for computing log likelihood by combining HMC and AIS.
https://github.com/Sohl-Dickstein/Hamiltonian-Annealed-Importance-Sampling
MATLAB code implementing HAIS for HMC sampling, partition function estimation, and importance weight estimation.
Sohl-Dickstein, Jascha, and Benjamin J. Culpepper. "Hamiltonian annealed importance sampling for partition function estimation." arXiv preprint arXiv:1205.1925 (2012). http://arxiv.org/abs/1205.1925
Minimum Probability Flow learning (MPF) - A parameter estimation method for unnormalized probabilistic models.
https://github.com/Sohl-Dickstein/Minimum-Probability-Flow-Learning
MATLAB code implementing MPF for an Ising model and for an RBM.
Sohl-dickstein, Jascha, Peter Battaglino, and Michael R. Deweese. "Minimum Probability Flow Learning." Proceedings of the 28th International Conference on Machine Learning (ICML-11). 2011. http://arxiv.org/abs/0906.4779
Sohl-Dickstein, Jascha, Peter B. Battaglino, and Michael R. DeWeese. "New method for parameter estimation in probabilistic models: minimum probability flow." Physical review letters 107.22 (2011) 220601. http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.107.220601
Estimating functional connectivity in neural data using an Ising model
MATLAB code implementing MPF to train an Ising model and estimate functional connectivity for neural data.
Hamilton LS, Sohl-Dickstein J, Huth AG, Carels VM, Deisseroth K, Bao S. "Optogenetic Activation of an Inhibitory Network Enhances Feedforward Functional Connectivity in Auditory Cortex." Neuron (2013). http://www.cell.com/neuron/abstract/S0896-6273(13)00751-4