Solving joint chance constrained problems using regularization and Benders’ decomposition

Abstract

We consider stochastic programs with joint chance constraints with discrete random distribution. We reformulate the problem by adding auxiliary variables. Since the resulting problem has a non-regular feasible set, we regularize it by increasing the feasible set. We solve the regularized problem by iteratively solving a master problem while adding Benders’ cuts from a slave problem. Since the number of variables of the slave problem equals to the number of scenarios, we express its solution in a closed form. We show convergence properties of the solutions. On a gas network design problem, we perform a numerical study by increasing the number of scenarios and compare our solution with a solution obtained by solving the same problem with the continuous distribution.

Appendices

A New result for sufficient optimality conditions for hierarchical problems

In this short section, we present a new result which may play a crucial role in deriving sufficient optimality conditions for hierarchical problems such as bilevel problems, mathematical problems with equilibrium/complementarity/vanishing constraints and so on. In these problems, the feasible set is usually rather nasty but may be written as a finite union of nice sets. We show that if these nice sets are convex, then strong stationary points are immediately local minima.

Theorem 5

Consider a convex differentiable function $f:{\mathbb {R}}^n\rightarrow {\mathbb {R}}$, a set $X\subset {\mathbb {R}}^n$, a point $\bar{x}\in X$ and an optimization problem

$$\begin{aligned} \begin{aligned} \hbox {minimize}&f(x)\\ \hbox {subject to}\,&x\in X. \end{aligned} \end{aligned}$$

(23)

Assume that $\bar{x}$ is its S-stationary point, thus a point with

$$\begin{aligned} 0\in \nabla f(\bar{x}) + \hat{N}_X(\bar{x}), \end{aligned}$$

where $\hat{N}_X(\bar{x})$ stands for the Fréchet normal cone of X at $\bar{x}$. If X can be locally around $\bar{x}$ written as a union of a finite number of (possibly overlapping) convex sets, then $\bar{x}$ is a local minimum of problem (23).

Proof

From the theorem statement, there are convex sets $X_i$, $i=1,\ldots ,I$ such that locally around $\bar{x}$ we have that X coincides with $\cup _{i=1}^IX_i$. First, we realize that

$$\begin{aligned} \hat{N}_X(\bar{x}) = (T_X(\bar{x}))^* = (\cup _{i=1}^I T_{X_i}(\bar{x}))^* = \cap _{i=1}^I (T_{X_i}(\bar{x}))^* = \cap _{i=1}^I N_{X_i}(\bar{x}). \end{aligned}$$

Since $\bar{x}$ is a S-stationary point of (23), we have

$$\begin{aligned} 0\in \nabla f(\bar{x}) + \hat{N}_X(\bar{x}) = \nabla f(\bar{x}) + \bigcap _{i=1}^I N_{X_i}(\bar{x}) = \bigcap _{i=1}^I\Big (\nabla f(\bar{x}) + N_{X_i}(\bar{x})\Big ). \end{aligned}$$

Fix now any i. From the equation above we obtain that $\bar{x}$ is a stationary point of

$$\begin{aligned} \begin{aligned} {\hbox {minimize}}&f(x)\\ \hbox {subject to}\,&x\in X_i. \end{aligned} \end{aligned}$$

Due to the data convexity, it is a local minimum of the above problem and thus for all $x\in X_i$ sufficiently close to $\bar{x}$ we have $f(x)\ge f(\bar{x})$. But since i was chosen arbitrarily, we obtain that $\bar{x}$ is a local minimum of problem (23). $\square$

B Proofs

In this section, we collect the skipped proofs from Sect. 2.

Proof

(Proposition 1) It is sufficient to follow the proof of Lemma 3.1 in Adam and Branda (2016). $\square$

Proof

(Theorem 1) Denote the feasible set of problem (2) by Z and consider a point $\bar{x}\in Z$. Note that Z can be written as

$$\begin{aligned} Z = \bigcup _{I\subset \{1,\ldots ,S\}, \sum _{i\in I}p_i\ge 1-\varepsilon } {\left\{ x:\begin{aligned} g_k(x,\xi _i)&\le 0,\ i\in I,\ k=1,\ldots ,K \\ h_j(x)&\le 0,\ j\in J \end{aligned} \right\} }. \end{aligned}$$

Then Z coincides locally around $\bar{x}$ with

$$\begin{aligned} \bigcup _{I\in \mathscr {I}(\bar{x})} Z_I := \bigcup _{I\in \mathscr {I}(\bar{x})} {\left\{ x:\begin{aligned} g_k(x,\xi _i)&\le 0,\ i\in I_0(\bar{x})\cap I,\ k \in K_{0}^i(\bar{x})\\ h_j(x)&\le 0,\ j\in J_0 \end{aligned} \right\} }, \end{aligned}$$

(24)

which means that

$$\begin{aligned} \hat{N}_Z(\bar{x}) = (T_Z(\bar{x}))^* = \left( \bigcup _{I\in \mathscr {I}(\bar{x})}T_{Z_I}(\bar{x})\right) ^* = \bigcap _{I\in \mathscr {I}(\bar{x})}\hat{N}_{Z_I}(\bar{x}). \end{aligned}$$

By (Rockafellar and Wets 1998, Theorem 6.12) we obtain that $0\in \nabla f(\bar{x})+\hat{N}_Z(\bar{x})$ is a necessary optimality condition for chance constrained problem (2). To obtain the first statement, it suffices to realize that $Z_I$ can be due to (24) written as $F(x)\le 0$ for some function F combining $g_k$ and $h_j$ and to use chain rule (Rockafellar and Wets 1998, Theorem 6.14).

The proof of the second part goes in a similar way. Due to (Rockafellar and Wets 1998, Theorem 6.12), the necessary optimality conditions for problem (4) read

$$\begin{aligned} 0\in \begin{pmatrix}\nabla f(\bar{x})\\ 0\end{pmatrix}+ \hat{N}_Z(\bar{x},\bar{y}) \end{aligned}$$

(25)

where Z is the feasible set of problem (4). For the computation of the normal cone, realize first that Z locally around $(\bar{x},\bar{y})$ coincides with the union of $Z_I:=Z_I^x\times Z_I^y$ with respect to all $I\subset I_{00}(\bar{x},\bar{y})$, where

$$\begin{aligned} \begin{aligned} Z_I^x&:= {\left\{ x:\begin{aligned}&g_k(x,\xi _i)\le 0,\ i\in I\cup I_{0+}(\bar{x},\bar{y}),\ k \in K_0^i(\bar{x})\\&h_j(x)\le 0,\ j\in J_0 \\ \end{aligned} \right\} }\\ Z_I^y&:= {\left\{ y:\begin{aligned}&p^\top y\ge 1-\varepsilon \\&y_i\in [0,1],\ i\in I\cup I_{0+}(\bar{x},\bar{y})\cup {\left\{ i:\textstyle {\max _k}\ g_k(\bar{x},\xi _i)<0\right\} } \\&y_i=0,\ i\in (I_{00}(\bar{x},\bar{y})\setminus I)\cup {\left\{ i:\textstyle {\max _k}\ g_k(\bar{x},\xi _i)>0\right\} } \end{aligned} \right\} }. \end{aligned} \end{aligned}$$

As before, we have

$$\begin{aligned} \hat{N}_Z(\bar{x},\bar{y})&= \bigcap _{I\subset I_{00}(\bar{x},\bar{y})}\hat{N}_{Z_I}(\bar{x},\bar{y}) = \bigcap _{I\subset I_{00}(\bar{x},\bar{y})}\hat{N}_{Z_I^x}(\bar{x})\times \bigcap _{I\subset I_{00}(\bar{x},\bar{y})}\hat{N}_{Z_I^y}(\bar{y}). \end{aligned}$$

Since zero always belongs to a normal cone, the optimality condition (25) is equivalent to

$$\begin{aligned} 0\in \nabla f(\bar{x}) + \bigcap _{I\subset I_{00}(\bar{x},\bar{y})}\hat{N}_{Z_I^x}(\bar{x}). \end{aligned}$$

(26)

To finish the proof, it suffices to realize that the intersection in (26) is attained for $I=\emptyset$ and to use either (Ioffe and Outrata 2008, Proposition 3.4) (if the first part of Assumption 1 holds true) or (Rockafellar and Wets 1998, Theorem 6.14) (if the second part of Assumption 1 holds true). $\square$

The proof of Theorem 3 is more complicated. For notational simplicity, we consider only the case of $J_0(\bar{x})=\emptyset$. First, we write down the stationarity conditions of (8), then show two preliminary lemmas and only then proof the theorem itself.

The necessary optimality conditions for problem (8) at a point $(\bar{x}^t,\bar{y}^t)$ read as follows: there exist multipliers $\alpha ^t\in {\mathbb {R}}$, $\beta ^t\in {\mathbb {R}}^S$ and $\gamma ^t\in {\mathbb {R}}^{SK}$ such that the first-order optimality conditions

$$\begin{aligned} 0&= \nabla f(\bar{x}^t) - \sum _{i=1}^S\sum _{k=1}^K \gamma _{ik}^t \phi _t'(g_k(\bar{x}^t,\xi _i))\nabla _x g_k(\bar{x}^t,\xi _i), \end{aligned}$$

(27a)

$$\begin{aligned} 0&= -\alpha ^t p_i + \beta _i^t + \sum _{k=1}^K \gamma _{ik}^t,\ i=1,\ldots ,S \end{aligned}$$

(27b)

and the complementarity conditions

$$\begin{aligned} \alpha ^t(1-\varepsilon -p^\top \bar{y}^t)&= 0, \end{aligned}$$

(28a)

$$\begin{aligned} \beta _i^t&{\left\{ \begin{array}{ll} \ge 0 &{}\text {if }\bar{y}_i^t = 1,\\ = 0 &{}\text {if }0<\bar{y}_i^t<1,\\ \le 0 &{}\text {if }\bar{y}_i^t = 0,\end{array}\right. }\end{aligned}$$

(28b)

$$\begin{aligned} \gamma _{ik}^t(\bar{y}_i^t-\phi _t(g_k(\bar{x}^t,\xi _i)))&= 0 \end{aligned}$$

(28c)

are satisfied. Moreover, the sign restrictions $\alpha ^t\ge 0$ and $\gamma _{ik}^t\ge 0$ hold true.

Lemma 2

Assume that $(\bar{x}^t,\bar{y}^t)$ is a stationary point of problem (8). Then the following assertions hold true:

• 1. $g_k(\bar{x}^t,\xi _i)<0 \implies \gamma _{ik}^t=0;$

• 2. $\alpha ^t = 0 \implies \beta _i^t=\gamma _{ik}^t=0$ for all i and k;

Proof

If $g_k(\bar{x}^t,\xi _i)<0$, then from the definition of $\phi _t$ we have $\phi _t(g_k(\bar{x}^t,\xi _i))>1$. Since $\bar{y}_i^t\in [0,1]$, this directly implies $\gamma _{ik}^t=0$ due to (28c). For the second part, consider the case of $\alpha ^t = 0$. Then due to (27b) we have $\beta _i^t + \sum _{k=1}^K \gamma _{ik}^t=0$ for all i. Assume that $\beta _i^t <0$. Then there exists some k such that $\gamma _{ik}^t>0$. From (28b) we further get $\bar{y}_i^t=0$, and thus $\bar{y}_i^t-\phi _t(g_k(\bar{x}^t,\xi _i))<0$. But this is a contradiction with (28c) and thus we have $\beta _i^t\ge 0$. But this together with $\gamma _{ik}^t\ge 0$ and $\beta _i+\gamma _{ik}^t=0$ implies the second statement. $\square$

In the following text, by $\bar{y}_i^t\searrow 0$ we understand that the sequence $\bar{y}_i^t$ is positive and converges monotonically to 0.

Lemma 3

If for all t we have $p^\top \bar{y}^t = 1-\varepsilon$ and for some i and k we have $\phi _t(g_k(\bar{x}^t,\xi _i)) = \bar{y}_i^t\searrow 0$, then there exists a subsequence in t such that $\gamma _{ik}^t=0$ for all t or such that there exists index j such that

$$\begin{aligned} \frac{ -\gamma _{ik}^t \phi _t'(g_k(\bar{x}^t,\xi _i))}{ -\sum _{\tilde{k}=1}^K \gamma _{j\tilde{k}}^t \phi _t'(g_{\tilde{k}}(\bar{x}^t,\xi _j))} \rightarrow 0. \end{aligned}$$

(29)

Proof

Since $\bar{y}_i^t\searrow 0$ and since $p^\top \bar{y}=1-\varepsilon$, there exists index j, and possibly a subsequence in t, such that $\bar{y}_j^t$ is strictly increasing. This implies that $0<\bar{y}_j^t<1$ and $\beta _i^t=\beta _j^t=0$ for all t by (28b). If $\gamma _{ik}^t=0$, then the proof is finished. In the opposite case of $\gamma _{ik}^t>0$, we realize that $\alpha ^t>0$ due to Lemma 2. Since $\gamma _{ik}^t\ge 0$ and $\phi _t'(g_k(\bar{x}^t,\xi _i))<0$, we deduce

$$\begin{aligned} \begin{aligned} 0\le \frac{- \gamma _{ik}^t \phi _t'(g_k(\bar{x}^t,\xi _i))}{- \sum _{\tilde{k}=1}^K \gamma _{j\tilde{k}}^t \phi _t'(g_{\tilde{k}}(\bar{x}^t,\xi _j))}&\le \frac{- \left( \sum _{\tilde{k}=1}^K \gamma _{i\tilde{k}}^t \right) \phi _t'(g_k(\bar{x}^t,\xi _i)) }{- \sum _{\tilde{k}=1}^K \gamma _{j\tilde{k}}^t \phi _t'(g_{\tilde{k}}(\bar{x}^t,\xi _j))}\\&\le \frac{- \sum _{\tilde{k}=1}^K \gamma _{i\tilde{k}}^t }{- \sum _{\tilde{k}=1}^K \gamma _{j\tilde{k}}^t} \frac{ \phi _t'(g_k(\bar{x}^t,\xi _i))}{\phi _t'(g_{\hat{k}^t}(\bar{x}^t,\xi _j))} \\&= \frac{- \alpha ^t p_i}{- \alpha ^t p_j} \frac{ \phi _t'(g_k(\bar{x}^t,\xi _i))}{\phi _t'(g_{\hat{k}^t}(\bar{x}^t,\xi _j))} \rightarrow 0 \end{aligned} \end{aligned}$$

(30)

where $\hat{k}^t := \arg \min _{\tilde{k}} -\phi _t'(g_{\tilde{k}}(\bar{x}^t,\xi _j)$, the last equality follows from (27b) as well as $\beta _i^t=\beta _j^t=0$ and the convergence follows from assumption (9d), for which we realize that $\phi _t(g_{\tilde{k}^t}(\bar{x}^t,\xi _j)) \ge \bar{y}_j^t$, the fact that $\bar{y}_j^t$ is a strictly increasing sequence and the assumed convergence $\phi _t(g_k(\bar{x}^t,\xi _i)) = \bar{y}_i^t\searrow 0$. $\square$

Proof

(Theorem 3) We will show first that $(\bar{x},\bar{y})$ is a feasible point of (4). Due to continuity, it is sufficient to show that $g_k(\bar{x},\xi _i)\bar{y}_i\le 0$. Since this relation is obvious whenever $g_k(\bar{x},\xi _i)\le 0$ for all k, we consider scenario i with $g_k(\bar{x},\xi _i)>0$ for some k. But then $g_k(\bar{x}^t,\xi _i)>0$ for sufficiently large t and thus $0\le \bar{y}_i^t\le \phi _t(g_k(\bar{x}^t,\xi _i))$. But since $g_k(\bar{x}^t,\xi _i)\rightarrow g_k(\bar{x},\xi _i)>0$, assumption (9c) implies that $\bar{y}_i=0$, and thus $(\bar{x},\bar{y})$ is a feasible point of problem (4).

Define now

$$\begin{aligned} \lambda _{ik}^t := - \gamma _{ik}^t \phi _t'(g_k(\bar{x}^t,\xi _i)) \ge 0, \end{aligned}$$

where the nonnegativity follows from the property that $\phi _t$ is decreasing. If $g_k(\bar{x},\xi _i) <0$, then for all sufficiently large t we have $g_k(\bar{x}^t,\xi _i) <0$ and due to Lemma 2 we deduce $\gamma _{ik}^t=0$ and subsequently $\lambda _{ik}^t=0$. Then for sufficiently large t, optimality condition (27a) reads

(31)

Here we can omit pairs of indices (i, k) with $g_k(\bar{x},\xi _i) <0$ due to the discussion above.

We claim now that $\sum _{k=1}^{K}\lambda _{ik}^t$ is uniformly bounded in i and t. If this is not the case, then we have

$$\begin{aligned} \lambda _{max}^t:= \max _{i=1,\ldots ,S} \sum _{k=1}^{K} \lambda _{ik}^t \rightarrow \infty . \end{aligned}$$

(32)

Then dividing equation (31) by $\lambda _{max}^t$ yields

(33)

When taking the limit with respect to $t\rightarrow \infty$, the first term vanishes. Consider now the third term. If $p^\top \bar{y}^t > 1-\varepsilon$, from Lemma 2 we have $\lambda _{ik}^t=0$ for all i and k. Assume thus that $p^\top \bar{y}^t = 1-\varepsilon$ for all t. If $\phi _t(g_k(\bar{x}^t,\xi _i)) > \bar{y}_i^t$, then from (28c) we have $\lambda _{ik}^t=0$. In the opposite case, we may use Lemma 3 to obtain again that $\lambda _{ik}^t=0$ or there exists j such that $\frac{\lambda _{ik}^t}{\sum _{\tilde{k}=1}^{K} \lambda _{j\tilde{k}}^t}\rightarrow 0$. But this implies that the last term in (33) vanishes as well. This means that

$$\begin{aligned} 0 = \sum _{{\left\{ (i,k):g_k(\bar{x},\xi _i)=0,\, \bar{y}_i>0\right\} }}\lim _{t\rightarrow \infty }\frac{\lambda _{ik}^t}{{\lambda _{max}^t}} \nabla _x g_k(\bar{x}^t,\xi _i). \end{aligned}$$

Since $\frac{\lambda _{ik}^t}{\lambda _{max}^t}\in [0,1]$ and the numerators sum to $\lambda _{max}^t$ due to (32) and the discussion above, at least one of these fractions converges to a positive number. However, the existence of such positive limit contradicts Assumption 1 and thus $\sum _{k=1}^{K} \lambda _{ik}^t$ is indeed bounded. Since it is a sum of nonnegative elements, these elements $\lambda _{ik}^t$ are uniformly bounded in i, k and t.

This means that we may pass to a converging subsequence, say $\lambda _{ik}^t\rightarrow \lambda _{ik}$. Since $\lambda _{ik}^t\ge 0$ for all t, the same property holds for $\lambda _{ik}$. In the light of (31), to finish the proof it suffices to show that $\lambda _{ik}=0$ for all pairs (i, k) such that $g_k(\bar{x},\xi _i)\ge 0$ and $\bar{y}_i=0$. But this may be shown as in the previous paragraph via applying Lemmas 2 and 3. Thus $(\bar{x},\bar{y})$ is indeed a stationary point of problem (4). $\square$

C Cut reduction

To propose a method for cut reduction, we start with the following technical lemma.

Lemma 4

Consider two points $\hat{x}^1,\hat{x}^2$ such that for all i we have

$$\begin{aligned} \max _k g_k(\hat{x}^2,\xi _i)> 0 \implies \max _k g_k(\hat{x}^1,\xi _i) > 0,\ \arg \max _k g_k(\hat{x}^1,\xi _i) = \arg \max _k g_k(\hat{x}^2,\xi _i). \end{aligned}$$

(34)

Define now for $j=1,2$ mappings $\mathscr {K}^j:\{1,\ldots ,S\}\rightarrow \{0,1,\ldots ,K\}$ and $v^j:{\mathbb {R}}^n\rightarrow {\mathbb {R}}$ by

$$\begin{aligned} \begin{aligned} \mathscr {K}^j(i)&:= {\left\{ \begin{array}{ll} 0 &{} \text {if } \max _k g_k(\hat{x}^j,\xi _i)\le 0, \\ \arg \max _k g_k(\hat{x}^j,\xi _i) &{} \text {otherwise,}\end{array}\right. } \\ v^j(x)&:= \sum _{\{i|\ \mathscr {K}^j(i) > 0\}} p_i\phi _t(g_{\mathscr {K}^j(i)}(x,\xi _i)) - 1 + \varepsilon + \sum _{\{i|\ \mathscr {K}^j(i) = 0\}} p_i. \end{aligned} \end{aligned}$$

Then for any x we have

$$\begin{aligned} v^1(x) \ge 0 \implies v^2(x) \ge -(\sup \phi _t(\cdot ) - 1). \end{aligned}$$

Proof

Due to (34) we have $\{i|\ \mathscr {K}^2(i)>0\}\subset \{i|\ \mathscr {K}^1(i)>0\}$ and subsequently

$$\begin{aligned} \begin{aligned} v^2(x)&= v^1(x) - \sum _{\{i|\ \mathscr {K}^1(i)>0,\ \mathscr {K}^2(i)=0\}} p_i\phi _t(g_{\mathscr {K}^1(i)}(x,\xi _i)) + \sum _{\{i|\ \mathscr {K}^1(i)>0,\ \mathscr {K}^2(i)=0\}} p_i \\&\ge v^1(x) - \sum _{\{i|\ \mathscr {K}^1(i)>0,\ \mathscr {K}^2(i)=0\}} p_i(\sup \phi _t(\cdot ) - 1) \ge v^1(x) - (\sup \phi _t(\cdot ) - 1), \end{aligned} \end{aligned}$$

which finishes the proof. $\square$

Note that the cuts generated by (17) equal to $\{x|\ v^j(x)\ge 0\}$. This lemma states that if (34) holds true, then the cut generated by $\hat{x}^1$ is tighter than the one generated by $\hat{x}^2$ up to a margin $\sup \phi _t(\cdot ) - 1$. Since $\phi _t(0)=1$, this margin may be made arbitrarily small by a proper choice of $\phi _t$, the best cut in (17) is generated whenever $\max _k g_k(\hat{x},\xi _i)>0$ for all i.

This also gives rise to the following heuristic cut reduction technique. If we add a new cut generated by some $\hat{x}^1$, we remove all the previously included cuts, generated by $\hat{x}^2$, for which (34) is satisfied. Alternatively, we remove all previous cuts, for which (34) is violated only for a small number of scenarios.

Finally, the next lemma states that the added cuts in (17) are optimal as they cannot be linear combination of each other. Recall that a direction d is an extremal direction of a cone C if there do not exist directions $d_1,d_2\in C$ different from d and a scalar $\kappa \in (0,1)$ such that $d=\kappa d_1+(1-\kappa )d_2$.

Lemma 5

The direction defined in (15) is an extremal direction of the feasible set of (13).

Proof

Denote the feasible set of problem (13) by Z. For contradiction assume that $(\hat{u},\hat{v},\hat{w})$ is not an extremal direction of Z. Then there are some $(u^1, v^1, w^1)\in Z$ and $(u^2, v^2, w^2)\in Z$ different from $(\hat{u}, \hat{v}, \hat{w})$ and some $\kappa \in (0,1)$ such that

$$\begin{aligned} (\hat{u}, \hat{v}, \hat{w}) = \kappa (u^1, v^1, w^1) + (1-\kappa )(u^2, v^2, w^2). \end{aligned}$$

(35)

Note now that the role of $\hat{u}_{i\hat{k}_i}$ and $\hat{w}_i$ is symmetric in (15) and thus, it suffices to consider only the first equation in (15). Define

$$\begin{aligned} Z_1 :=\{(v,w_1)|\ vp_1+w_1\le 0,\ v\ge 0,\ w_1\le 0\} \end{aligned}$$

and observe that $(\hat{v},\hat{w}_1), (v^1, w_1^1), (v^2, w_1^2)\in Z_1$. But this due to (35) means that $(\hat{v},\hat{w}_1)$ is not an extremal direction of $Z_1$. But since the extremal directions of $Z_1$ amount to $(0,-p_1)$ and $(1,-p_1)$, this is a contradiction with (15). $\square$