Ilya Loshchilov

Home * People * Ilya Loshchilov

Ilya Loshchilov

Ilya Gennadyevich Loshchilov,
a Russian computer scientist in the field of evolutionary algorithms (EA), optimization and machine learning. He worked with Marc Schoenauer and Michèle Sebag at Paris-Sud 11 University and INRIA within their TAO Project Team on multi-objective optimization, adaptive coordinate descent and CMA-ES, where he defended his Ph.D. with the title Surrogate-Assisted Evolutionary Algorithms in 2013 . As postdoc at University of Freiburg, he continued his research with Frank Hutter combining, evolutionary algorithms with deep learning techniques.

SGDR

Loshchilov's and Hutter's stochastic gradient descent with warm restarts (SGDR) as introduced in their 2016 paper , is mentioned as method for learning rate scheduling to train Leela Chess' third party network Leelenstein , which is combined with Allie for the AllieStein chess playing entity.

Selected Publications

2010 ...

• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2010). Dominance-Based Pareto-Surrogate for Multi-Objective Optimization. SEAL 2010, pdf
• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2010). A mono surrogate for multiobjective optimization. GECCO 2010, pdf
• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2011). Adaptive coordinate descent. GECCO 2011, pdf
• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2012). Self-Adaptive Surrogate-Assisted Covariance Matrix Adaptation Evolution Strategy. arXiv:1204.2356
• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2012). Alternative Restart Strategies for CMA-ES. arXiv:1207.0206
• Ilya Loshchilov, Marc Schoenauer, Michèle Sebag (2013). KL-based Control of the Learning Schedule for Surrogate Black-Box Optimization. arXiv:1308.2655
• Ilya Loshchilov (2013). Surrogate-Assisted Evolutionary Algorithms. Ph.D. thesis, Paris-Sud 11 University, advisors Marc Schoenauer and Michèle Sebag
• Ilya Loshchilov (2013). CMA-ES with Restarts for Solving CEC 2013 Benchmark Problems. CEC 2013, pdf
• Ilya Loshchilov (2014). A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization. arXiv:1404.5520

2015 ...

• Ilya Loshchilov (2015). LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization. arXiv:1511.00221
• Ilya Loshchilov, Frank Hutter (2015). Online Batch Selection for Faster Training of Neural Networks. arXiv:1511.06343
• Ilya Loshchilov, Frank Hutter (2016). CMA-ES for Hyperparameter Optimization of Deep Neural Networks. arXiv:1604.07269
• Ilya Loshchilov, Frank Hutter (2016). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv:1608.03983
• Ilya Loshchilov, Tobias Glasmachers (2016). Anytime Bi-Objective Optimization with a Hybrid Multi-Objective CMA-ES (HMO-CMA-ES). GECCO 2016
• Ilya Loshchilov, Frank Hutter (2017). Decoupled Weight Decay Regularization. arXiv:1711.05101
• Ilya Loshchilov, Tobias Glasmachers, Hans-Georg Beyer (2017). Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization. arXiv:1705.06693
• Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter (2018). Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari. arXiv:1802.08842
• Ilya Loshchilov, Tobias Glasmachers, Hans-Georg Beyer (2019). Large Scale Black-Box Optimization by Limited-Memory Matrix Adaptation. IEEE Transactions on Evolutionary Computation, Vol. 23, No. 2

External Links

• Ilya Loshchilov
• loshchil (Ilya Loshchilov) · GitHub

GitHub - loshchil/SGDR

GitHub - loshchil/AdamW-and-SGDW: Decoupled Weight Decay Regularization (ICLR 2019)

References

Up one Level