Stochastic quasigradient methods, Information theory
Abstract
In this paper, we extend the stochastic quasigradient method when there is prior information on the region where descent directions are likely to be found. Our extension uses maximum entropy and minimum cross-entropy subgradient estimators that incorporate prior information in the form of expectations. We also analyze a number of prior information patterns and provide the convergence conditions for the proposed method. Finally, we obtain a limiting distribution representation for the expected information, which is provided by the sequence of subgradient estimators generated by the proposed method.
