McTorch

McTorch, a manifold optimization library for deep learning

McTorch is a Python library that adds manifold optimization functionality to PyTorch.

McTorch:

– Leverages tensor computation and GPU acceleration from PyTorch.

– Enables optimization on manifold constrained tensors to address nonlinear optimization problems.

– Facilitates constrained weight tensors in deep learning layers.

McTorch builds on top of PyTorch and supports all PyTorch functions in addition to manifold optimization. This is done to ensure researchers and developers using PyTorch can easily experiment with McTorch functions without writing a single line of extra code.

McTorch’s manifold implementations and optimization methods are derived from the Matlab toolbox Manopt and the Python toolbox Pymanopt.

McTorch currently supports the following with more in the pipeline.

Manifolds: Stiefel, PositiveDefinite, Hyperbolic.

All manifolds support k multiplier as well. Different uses of the above two manifold definitions generate most frequently used manifolds used in deep learning.

Optimizers: SGD, Adagrad, ConjugateGradient.

Layers:

– Linear

– Conv1d, Conv2d, Conv3d

– Conv1d\_transpose, Conv2d\_transpose, Conv3d\_transpose.

If you are using McTorch, please cite the following.

@techreport{meghwanshi2018mctorch,
title={McTorch, a manifold optimization library for deep learning},
author={Meghwanshi, Mayank and Jawanpuria, Pratik and Kunchukuttan, Anoop and Kasai, Hiroyuki and Mishra, Bamdev},
institution={arXiv preprint arXiv:1810.01811},
year={2018}
}