Bamdev Mishra

Applied Machine Learning Researcher

Research

My research interests are in nonlinear optimization and machine learning. A specific focus has been on developing efficient numerical algorithms for large-scale problems with nonlinear manifold constraints, which are ubiquitous in machine learning applications. Below are some of the projects that I have been actively working on with my collaborators across industry and academia.

Industrial Research

  • Kaizala and Office Lens [Microsoft]. Working with multiple teams to integrate data-driven intelligent features into the Kaizala enterprise and Office Lens apps. The use cases that we tackle revolve around improving user experience on the apps by leveraging latest machine learning tools.
  • Multilingual embedding [Microsoft]. The aim is to generate embeddings for words in the target language that are learnt from a small set of dictionary mappings. An initial version of the work is at [arXiv:1808.08773].
  • Product recommendation [Microsoft]. Working on developing algorithms/codes for a Microsoft customer to enable ML driven product recommendations.
  • Competitive pricing of products [Amazon.com]. We estimate the competitive prices of 3p unique products, i.e., Amazon is not a retail player for these products. The large-scale nature of the project demands to work on simple algorithmic implementations that are readily scalable. Currently, the price estimates are generated worldwide for all of the Amazon marketplaces. The proposed model is currently deployed internally and consumed by multiple customers. (Patent​ filed.)
  • Style recommendation for fashion [Amazon.com]. Worked on cold-start product recommendation by exploiting user and product attributes. Our algorithm​s have been pushed online to internal customers. The online results have shown significant lift of performance over other baselines. The proposed algorithms are being productionized for internal launch.
  • Tensor decomposition [Amazon.com]. We exploit the problem structure to propose natural gradient learning algorithms for tensor decomposition. The proposed algorithm is part of internal production systems to detect anomalies in seller and reviewer activities. The work has been submitted [arXiv:1804.03836].
  • Demand forecasting [Amazon.com]. As part of an internship project, we study low-rank tensor factorization models for demand forecasting problems.

Research themes

  • Optimization on Riemannian manifolds. Understanding the (Riemannian) geometry of structured constraints is of particular interest in machine learning. Conceptually, it allows to translate a constrained optimization problem into an unconstrained optimization problem on a nonlinear search space (manifold). Building upon this point of view, one research interest is to exploit manifold geometry in nonlinear optimization.
    We have been actively involved in both matrix and tensor applications. Recently, we have looked into domain adaptation and generating multilingual embedding tasks. Specific papers include [arXiv:1211.1550]
    [arXiv:1306.2672] [arXiv:1405.6055] [arXiv:1605.08257] [arXiv:1712.01193]
    [arXiv:1808.08773][arXiv:1804.10834].
  • Decenetralized and stochastic optimization algorithms. We explore recent advances in stochastic gradient algorithms on manifolds. Specific papers include [arXiv:1605.07367][arXiv:1603.04989][arXiv:1703.04890][AISTATS2018].
    We exploit consensus learning on manifolds in the context of large-scale distributed algorithms on problems like matrix factorization and multitask learning. An initial work is in [arXiv:1605.06968][arXiv:1705.00467].
  • Low-rank optimization with structure. We develop efficient algorithms for problems in big data systems by exploiting low-rank and sparse decomposition. Papers include [arXiv:1604.04325][arXiv:1607.07252].
    The work [arXiv:1704.07352] proposes a generic framework for tackling low-rank optimization with constraints by exploiting a variational characterization of nuclear norm.
  • Deep learning. We study geometric algorithms for modern deep networks that are robust to invariances of the parameters. An initial work is in [arXiv:1511.01754]. Currently, we are exploring the duality between learning “complex architecture with simple algorithms” and “simple architecture with sophisticated algorithms”.

Optimization tools

  • McTorch. Currently working on the Python toolbox McTorch, which is a manifold optimization library for deep learning .
  • Manopt. I am also involved in the development and promotion of the Matlab toolbox Manopt for optimization on manifolds.
Example