This post is a short summary of equivalent first order optimality conditions collected from several papers for the smooth optimization problem over Stiefel manifold. Throughout this post, we consider the optimization problem \( \min f(X) \), s.t. \( X \in \mathbf{St}(n,r) \), where \( X \in \mathbb{R}^{n \times r} \) and \( f \) is smooth. Let \( \nabla f(X) \) be the Euclidean gradient of \( f \) at \( X \). The following statements are equivalent:

1. \( X \) is a first-order stationary point of the problem.
2. \( \operatorname{grad} f(X) \triangleq \nabla f(X) - \frac{X}{2} (X^{\top}\nabla f(X) + \nabla f(X)^{\top}X) = 0 \) (by definition).
3. \( (I - XX^{\top}) \nabla f(X) = 0 \) and \( X^{\top} \nabla f(X) = \nabla f(X)^{\top}X \) (Gao et al. 2018).
4. \( \nabla f(X) - X \nabla f(X)^{\top}X = 0 \) (Wen and Yin 2013; Gao et al. 2018).
5. \( \nabla f(X) X^{\top} = X \nabla f(X)^{\top} \) (Wen and Yin 2013).
6. \(\nabla f(X) - X(2\rho \nabla f(X)^{\top} X + (1-2\rho) X^{\top} \nabla f(X) ) = 0\), where \(\rho > 0\) is a constant (Jiang and Dai 2015).

Note that the above conditions implies that \( \nabla f(X)^{\top}X = X^{\top} \nabla f(X) \), that is, it is a necessary condition. When \( n = r \) (the feasible region is an orthogonal group), this condition becomes an equivalent condition.

References