IPOPTのprint levelによる出力の詳細度合い

※記事内に商品プロモーションを含むことがあります。

はじめに

非線形最適化ソルバIPOPTのprint_levelオプションによる、最適化計算の出力の詳細度合いについて解説する。
print_levelは0～12の13段階で指定でき、数値が大きいほど詳細になる。デフォルト値は5である。

本記事では、Pythonの最適化モデリングツールであるCasADiを使ってIPOPTを実行したときを例に示す。使用したPythonとライブラリのバージョンは以下の通り。

ライブラリ	バージョン
Python	3.8.5
CasADi	3.5.5
IPOPT	3.12.3

pipでCasADiをインストールすると、IPOPTも同時にインストールされる。

pip install casadi

リンク

print levelオプションの概要

前述のように、print_levelは0～12の範囲で指定する。数値を増やすごとに、下表のように出力される要素が追加される。
なお、反復回数によるが、print_levelを6以上にするとログが膨大な量になるため注意。

`print_level`	追加される主な出力
0～2	出力なし
3, 4	最適化問題の情報と最適化結果を出力
5	各反復の結果を1行でまとめて出力
6	各反復の変数などの無限大ノルム（最大値）
7	各反復の収束判定の詳細・ステップ幅の無限大ノルム
8	変数や勾配ノルムの各要素
9	RHSの要素
10	行列`KKT`の情報
11, 12	`KKT SYSTEM`の値

実行したスクリプト

2変数の制約付き最小化問題を解く。print_levelの値を変えると、IPOPTのログの詳細度合いも変化する。
また、p_optsを{'print_time': False}にすることで、CasADiのログを出力しないようにしている。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import casadi
opti = casadi.Opti()
x = opti.variable(2) # 変数
opti.set_initial(x, 2)
opti.subject_to( x >= 1 )

OBJ = casadi.sum1(x**2) # 目的関数
opti.minimize(OBJ)

p_opts = {'print_time': False}
s_opts = {'print_level': 5}
opti.solver('ipopt', p_opts, s_opts)
sol = opti.solve()

最適化結果を出力しない (level 0～2)

print_levelが0～2のとき、IPOPTは何も出力しない。

最適化問題の情報と最適化結果を出力 (level 3～4)

print_levelが3～4のとき、IPOPTは以下を出力する。

最適化問題の情報（変数の数など）
収束までの反復回数
最適化結果（最適値、計算時間など）

Total number of variables............................:        2
                     variables with only lower bounds:        0
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:        0
Total number of inequality constraints...............:        2
        inequality constraints with only lower bounds:        2
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0


Number of Iterations....: 5

                                   (scaled)                 (unscaled)
Objective...............:  1.9999999650154130e+000   1.9999999650154130e+000
Dual infeasibility......:  2.5091040356528538e-014   2.5091040356528538e-014
Constraint violation....:  0.0000000000000000e+000   0.0000000000000000e+000
Complementarity.........:  2.5077064649767179e-009   2.5077064649767179e-009
Overall NLP error.......:  2.5077064649767179e-009   2.5077064649767179e-009


Number of objective function evaluations             = 6
Number of objective gradient evaluations             = 6
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 6
Number of equality constraint Jacobian evaluations   = 0
Number of inequality constraint Jacobian evaluations = 6
Number of Lagrangian Hessian evaluations             = 5
Total CPU secs in IPOPT (w/o function evaluations)   =      0.003
Total CPU secs in NLP function evaluations           =      0.000

EXIT: Optimal Solution Found.

各反復の結果を1行でまとめて出力（level 5）

print_levelがデフォルト値の5のとき、IPOPTは以下を追加で出力する。

ヤコビアン・ヘッシアンの非ゼロ要素の数
各反復の結果（評価関数や制約違反量など）

This is Ipopt version 3.12.3, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...:        0
Number of nonzeros in inequality constraint Jacobian.:        2
Number of nonzeros in Lagrangian Hessian.............:        2

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0 8.0000000e+000 0.00e+000 1.50e+000  -1.0 0.00e+000    -  0.00e+000 0.00e+000   0
   1 2.0402000e+000 0.00e+000 3.58e-001  -1.0 1.30e+000    -  1.00e+000 7.62e-001f  1
   2 2.0396310e+000 0.00e+000 2.00e-007  -1.7 1.41e-004    -  1.00e+000 1.00e+000f  1
   3 2.0006763e+000 0.00e+000 1.50e-009  -3.8 9.69e-003    -  1.00e+000 1.00e+000f  1
   4 2.0000038e+000 0.00e+000 1.84e-011  -5.7 1.68e-004    -  1.00e+000 1.00e+000f  1
   5 2.0000000e+000 0.00e+000 2.51e-014  -8.6 9.49e-007    -  1.00e+000 1.00e+000f  1

変数などの無限大ノルム (level 6)

print_levelが6のとき、IPOPTは以下を追加で出力する。

ユーザが指定したオプション
最適化のスケーリングに用いる係数
変数などの無限大ノルム（最大値）

List of options:

                                    Name   Value                # times used
                             print_level = 6                         2

Scaling parameter for objective function = 1.000000e+000
objective scaling factor = 1
No x scaling provided
No c scaling provided
No d scaling provided
Initial values of x sufficiently inside the bounds.
Initial values of s sufficiently inside the bounds.
MUMPS used permuting_scaling 5 and pivot_order 5.
           scaling will be 77.
Number of doubles for MUMPS to hold factorization (INFO(9)) = 12
Number of integers for MUMPS to hold factorization (INFO(10)) = 64
Factorization successful.
Least square estimates max(y_c) = 0.000000e+000, max(y_d) = 2.500000e+000

**************************************************
*** Update HessianMatrix for Iteration 0:
**************************************************



**************************************************
*** Summary of Iteration: 0:
**************************************************

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0 8.0000000e+000 0.00e+000 1.50e+000  -1.0 0.00e+000    -  0.00e+000 0.00e+000   0 y

**************************************************
*** Beginning Iteration 0 from the following point:
**************************************************

Current barrier parameter mu = 1.0000000000000001e-001
Current fraction-to-the-boundary parameter tau = 9.8999999999999999e-001

||curr_x||_inf   = 2.0000000000000000e+000
||curr_s||_inf   = 2.0000000000000000e+000
||curr_y_c||_inf = 0.0000000000000000e+000
||curr_y_d||_inf = 2.5000000000000000e+000
||curr_z_L||_inf = 0.0000000000000000e+000
||curr_z_U||_inf = 0.0000000000000000e+000
||curr_v_L||_inf = 1.0000000000000000e+000
||curr_v_U||_inf = 0.0000000000000000e+000


***Current NLP Values for Iteration 0:

                                   (scaled)                 (unscaled)
Objective...............:  8.0000000000000000e+000   8.0000000000000000e+000
Dual infeasibility......:  1.5000000000000000e+000   1.5000000000000000e+000
Constraint violation....:  0.0000000000000000e+000   0.0000000000000000e+000
Complementarity.........:  1.0000000099999999e+000   1.0000000099999999e+000
Overall NLP error.......:  1.5000000000000000e+000   1.5000000000000000e+000




**************************************************
*** Update Barrier Parameter for Iteration 0:
**************************************************

Optimality Error for Barrier Sub-problem = 1.500000e+000
Barrier Parameter: 1.000000e-001

**************************************************
*** Solving the Primal Dual System for Iteration 0:
**************************************************

Number of doubles for MUMPS to hold factorization (INFO(9)) = 12
Number of integers for MUMPS to hold factorization (INFO(10)) = 64
Factorization successful.
Number of trial factorizations performed: 1
Perturbation parameters: delta_x=0.000000e+000 delta_s=0.000000e+000
                         delta_c=0.000000e+000 delta_d=0.000000e+000
residual_ratio = 1.189524e-016
Factorization successful.
residual_ratio = 7.930161e-017

**************************************************
*** Finding Acceptable Trial Point for Iteration 0:
**************************************************

--> Starting line search in iteration 0 <--
Mu has changed in line search - resetting watchdog counters.
The current filter has 0 entries.
minimal step size ALPHA_MIN = 0.000000E+000
Starting checks for alpha (primal) = 7.62e-001
trial_max is initialized to 1.000000e+004
trial_min is initialized to 1.000000e-004
Checking acceptability for trial step size alpha_primal_test=7.615383e-001:
  New values of barrier function     = 2.9612340152016166e+000  (reference 8.0000019980000197e+000):
  New values of constraint violation = 0.0000000000000000e+000  (reference 0.0000000000000000e+000):
Checking Armijo Condition...
Succeeded...
Checking filter acceptability...
Succeeded...

各反復の収束判定の詳細・ステップ幅の無限大ノルム (level 7)

print_levelが7のとき、IPOPTは以下を追加で出力する。

各反復の収束判定の詳細
各反復のステップ幅の無限大ノルム

Convergence Check:
  overall_error = 1.5000000000000000e+000   IpData().tol()   = 1.0000000000000000e-008
  dual_inf      = 1.5000000000000000e+000   dual_inf_tol_    = 1.0000000000000000e+000
  constr_viol   = 0.0000000000000000e+000   constr_viol_tol_ = 1.0000000000000000e-004
  compl_inf     = 1.0000000099999999e+000   compl_inf_tol_   = 1.0000000000000000e-004
obj val update iter = 0
Acceptable Check:
  overall_error = 1.5000000000000000e+000   acceptable_tol_             = 9.9999999999999995e-007
  dual_inf      = 1.5000000000000000e+000   acceptable_dual_inf_tol_    = 1.0000000000000000e+010
  constr_viol   = 0.0000000000000000e+000   acceptable_constr_viol_tol_ = 1.0000000000000000e-002
  compl_inf     = 1.0000000099999999e+000   acceptable_compl_inf_tol_   = 1.0000000000000000e-002
  curr_obj_val_ = 8.0000000000000000e+000   last_obj_val                = -1.0000000000000001e+050
  fabs(curr_obj_val_-last_obj_val_)/Max(1., fabs(curr_obj_val_)) = 1.2500000000000001e+049 acceptable_obj_change_tol_ = 1.0000000000000000e+020
test iter = 0

||delta_x||_inf   = 1.3000003380000009e+000
||delta_s||_inf   = 1.3000003380000009e+000
||delta_y_c||_inf = 0.0000000000000000e+000
||delta_y_d||_inf = 1.1000006760000021e+000
||delta_z_L||_inf = 0.0000000000000000e+000
||delta_z_U||_inf = 0.0000000000000000e+000
||delta_v_L||_inf = 4.0000032399999785e-001
||delta_v_U||_inf = 0.0000000000000000e+000

変数や勾配ノルムの各要素 (level 8)

print_levelが8のとき、IPOPTは以下を追加で出力する。

各反復における、変数や勾配ノルムの各要素
最適化結果における、変数や勾配ノルムの各要素

No search direction has been computed yet.
DenseVector "curr_x" with 2 elements:
curr_x[    1]=2.0000000000000000e+000
curr_x[    2]=2.0000000000000000e+000
DenseVector "curr_s" with 2 elements:
curr_s[    1]=2.0000000000000000e+000
curr_s[    2]=2.0000000000000000e+000
DenseVector "curr_y_c" with 0 elements:
DenseVector "curr_y_d" with 2 elements:
curr_y_d[    1]=-2.5000000000000000e+000
curr_y_d[    2]=-2.5000000000000000e+000
DenseVector "curr_slack_x_L" with 0 elements:
DenseVector "curr_slack_x_U" with 0 elements:
DenseVector "curr_z_L" with 0 elements:
Homogeneous vector, all elements have value 1.0000000000000000e+000
DenseVector "curr_z_U" with 0 elements:
Homogeneous vector, all elements have value 1.0000000000000000e+000
DenseVector "curr_slack_s_L" with 2 elements:
curr_slack_s_L[    1]=1.0000000099999999e+000
curr_slack_s_L[    2]=1.0000000099999999e+000
DenseVector "curr_slack_s_U" with 0 elements:
DenseVector "curr_v_L" with 2 elements:
Homogeneous vector, all elements have value 1.0000000000000000e+000
DenseVector "curr_v_U" with 0 elements:
Homogeneous vector, all elements have value 1.0000000000000000e+000

DenseVector "grad_f" with 2 elements:
grad_f[    1]=4.0000000000000000e+000
grad_f[    2]=4.0000000000000000e+000
DenseVector "curr_c" with 0 elements:
DenseVector "curr_d" with 2 elements:
curr_d[    1]=2.0000000000000000e+000
curr_d[    2]=2.0000000000000000e+000
DenseVector "curr_d - curr_s" with 2 elements:
curr_d - curr_s[    1]=0.0000000000000000e+000
curr_d - curr_s[    2]=0.0000000000000000e+000

RHSの要素 (level 9)

print_levelが9のとき、RHSの要素が出力される（RHSはRight Hand Sideの略だと思うが、詳細不明）。

CompoundVector "RHS[ 0]" with 4 components:

Component 1:
  DenseVector "RHS[ 0][ 0]" with 2 elements:
  RHS[ 0][ 0][    1]=-4.0000000000000000e+000
  RHS[ 0][ 0][    2]=-4.0000000000000000e+000

Component 2:
  DenseVector "RHS[ 0][ 1]" with 2 elements:
  RHS[ 0][ 1][    1]=1.0000000000000000e+000
  RHS[ 0][ 1][    2]=1.0000000000000000e+000

Component 3:
  DenseVector "RHS[ 0][ 2]" with 0 elements:
  Homogeneous vector, all elements have value 0.0000000000000000e+000

Component 4:
  DenseVector "RHS[ 0][ 3]" with 2 elements:
  Homogeneous vector, all elements have value 0.0000000000000000e+000

行列KKTの情報 (level 10)

print_levelが10のとき、行列KKTの情報が出力される。

CompoundSymMatrix "KKT" with 4 rows and columns components:
Component for row 0 and column 0:

  SumSymMatrix "KKT[0][0]" of dimension 2 with 2 terms:
  Term 0 with factor 0.0000000000000000e+000 and the following matrix:

    SymTMatrix "Term: 0" of dimension 2 with 2 nonzero elements:
    Uninitialized!
  Term 1 with factor 1.0000000000000000e+000 and the following matrix:

    DiagMatrix "Term: 1" with 2 rows and columns, and with diagonal elements:
      DenseVector "Term: 1" with 2 elements:
      Homogeneous vector, all elements have value 1.0000000000000000e+000
Component for row 1 and column 0:
This component has not been set.
Component for row 1 and column 1:

  DiagMatrix "KKT[1][1]" with 2 rows and columns, and with diagonal elements:
    DenseVector "KKT[1][1]" with 2 elements:
    Homogeneous vector, all elements have value 1.0000000000000000e+000
Component for row 2 and column 0:

  GenTMatrix "KKT[2][0]" of dimension 0 by 2 with 0 nonzero elements:
Component for row 2 and column 1:
This component has not been set.
Component for row 2 and column 2:

  DiagMatrix "KKT[2][2]" with 0 rows and columns, and with diagonal elements:
    DenseVector "KKT[2][2]" with 0 elements:
    Homogeneous vector, all elements have value -0.0000000000000000e+000
Component for row 3 and column 0:

  GenTMatrix "KKT[3][0]" of dimension 2 by 2 with 2 nonzero elements:
  KKT[3][0][    1,    1]=1.0000000000000000e+000  (0)
  KKT[3][0][    2,    2]=1.0000000000000000e+000  (1)
Component for row 3 and column 1:

  IdentityMatrix "KKT[3][1]" with 2 rows and columns and the factor -1.0000000000000000e+000.
Component for row 3 and column 2:
This component has not been set.
Component for row 3 and column 3:

  DiagMatrix "KKT[3][3]" with 2 rows and columns, and with diagonal elements:
    DenseVector "KKT[3][3]" with 2 elements:
    Homogeneous vector, all elements have value -0.0000000000000000e+000

KKT SYSTEMの値 (level 11, 12)

print_levelが11, 12のとき、KKT SYSTEMの値が出力される

******* KKT SYSTEM *******
(0) KKT[1][1] =  0.000000000000000e+000
(1) KKT[2][2] =  0.000000000000000e+000
(2) KKT[1][1] =  1.000000000000000e+000
(3) KKT[2][2] =  1.000000000000000e+000
(4) KKT[3][3] =  1.000000000000000e+000
(5) KKT[4][4] =  1.000000000000000e+000
(6) KKT[5][1] =  1.000000000000000e+000
(7) KKT[6][2] =  1.000000000000000e+000
(8) KKT[5][3] = -1.000000000000000e+000
(9) KKT[6][4] = -1.000000000000000e+000
(10) KKT[5][5] = -0.000000000000000e+000
(11) KKT[6][6] = -0.000000000000000e+000
Right hand side 0 in TSymLinearSolver:
Trhs[    0,    0] = -4.0000000000000000e+000
Trhs[    0,    1] = -4.0000000000000000e+000
Trhs[    0,    2] = 1.0000000000000000e+000
Trhs[    0,    3] = 1.0000000000000000e+000
Trhs[    0,    4] = 0.0000000000000000e+000
Trhs[    0,    5] = 0.0000000000000000e+000