我正在使用scipy.optimize.fmin_l_bfgs_b来解决一个高斯混合问题。混合分布的均值由回归模型建模,其权重必须使用EM算法进行优化。
sigma_sp_new, func_val, info_dict = fmin_l_bfgs_b(func_to_minimize, self.sigma_vector[si][pj],
args=(self.w_vectors[si][pj], Y, X, E_step_results[si][pj]),
approx_grad=True, bounds=[(1e-8, 0.5)], factr=1e02, pgtol=1e-05, epsilon=1e-08)
但有时我会在信息字典中收到“ABNORMAL_TERMINATION_IN_LNSRCH”的警告:
func_to_minimize value = 1.14462324063e-07
information dictionary: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'funcalls': 147, 'grad': array([ 1.77635684e-05, 2.87769808e-05, 3.51718654e-05,
6.75015599e-06, -4.97379915e-06, -1.06581410e-06]), 'nit': 0, 'warnflag': 2}
RUNNING THE L-BFGS-B CODE
* * *
Machine precision = 2.220D-16
N = 6 M = 10
This problem is unconstrained.
At X0 0 variables are exactly at the bounds
At iterate 0 f= 1.14462D-07 |proj g|= 3.51719D-05
* * *
Tit = total number of iterations
Tnf = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip = number of BFGS updates skipped
Nact = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F = final function value
* * *
N Tit Tnf Tnint Skip Nact Projg F
6 1 21 1 0 0 3.517D-05 1.145D-07
F = 1.144619474757747E-007
ABNORMAL_TERMINATION_IN_LNSRCH
Line search cannot locate an adequate point after 20 function
and gradient evaluations. Previous x, f and g restored.
Possible causes: 1 error in function or gradient evaluation;
2 rounding error dominate computation.
Cauchy time 0.000E+00 seconds.
Subspace minimization time 0.000E+00 seconds.
Line search time 0.000E+00 seconds.
Total User time 0.000E+00 seconds.
我并非每次都收到这个警告,但有时会出现。(大多数是“收敛: 迭代梯度范数≤PGTOL”或“收敛: F的相对减少≤FACTR*EPSMCH”)。
我知道这意味着在这次迭代中可以达到最小值。我谷歌了一下这个问题。有人说经常发生这种情况是因为目标和梯度函数不匹配。但是我这里没有提供梯度函数,因为我使用了'approx_grad'。
我应该调查哪些可能的原因?什么是“舍入误差占据计算主导地位”的意思?
======
我还发现对数似然不单调增加:
########## Convergence !!! ##########
log_likelihood_history: [-28659.725891322563, 220.49993177669558, 291.3513633060345, 267.47745327823907, 265.31567762171181, 265.07311121000367, 265.04217683341682]
通常在第二或第三次迭代时开始下降,即使没有出现'ABNORMAL_TERMINATION_IN_LNSRCH'问题。我不知道这个问题是否与以前的问题有关。