Comb Opt 2: Construction and Improvement Heuristics

Packages Used: No new packages are used in this notebook.

Heuristic procedures for problems in combinatorial optimization look at a small fraction of possible solutions to find a local optimum that may not be the global optimum; the solutions provided are optimal solutions, only

Heuristics are only needed when it is not feasible to otherwise find an optimal solution. In cases where realistic size instances a problem can be formulated as a MILP and easily solved, for example, the Set Covering problem, there is no need to consider the use of heuristics; but in the case of the Bin Packing, even relatively small instances of size 200 were difficult BIP to solve and thus, the use of a heuristic is helpful; also, in some cases like Bin Packing, the heuristic solution can be used as an initial incumbent solution to improve the effectiveness of the BIP.

Heuristics: The computational effort is split between

The degree to which a problem is constrained affects the relative impact of construction versus improvement procedures. The following are two extremes:

Easy but ineffective construction followed by effective improvement:

Hard but effective construction, little improvement possible:

1. Bin Packing

General formulation of the 1-D bin packing problem (where the capacity restriction is assumed to be cubic volume):

$ \begin{eqnarray*} \quad M &=& \bigl\{ 1, \dots, m \bigr\}, \quad \mbox{objects to be packed into bins} \\ v_j &=& \mbox{volume of object } n \\ V &=& \mbox{volume of each bin } B_i \bigl( \max v_j \le V \bigr) \\ B^* &=& \mathrm{arg}\underset{B}{\operatorname{min}} \bigl\{ \lvert B \rvert : \sum_{j \in B_i} v_j \le V, \bigcup_{B_i \in B} = M \bigr\}, \quad \mbox{min cost bin packing of } M. \end{eqnarray*}$

First-Fit construction procedure for bin packing

Returns array where each element is the indices of objects from v in a bin. Each element of v is the weight of the object, and each bin is packed so that its total weight of objects does not exceed V:

Ex 2: 200-Object Bin Packing (Ex 6 in MIP 3)

Since it is not possible to easily improve a constructed bin-packing solution, the only easy way to find alternate solutions is to include randomness by randomly re-ordering the items v used by firstfit multiple times and return the best-constructed solution found:

Since the solution found by multifirstfit equals the LB, it is the global optimal solution. In general, this will not be the case, and the solution found by multifirstfit is just the best-known solution.

Ex 2: 2000-Object Bin Packing

Ex 3: 50 Large Objects and Small Bins

2. Quadratic Assignment Problem (QAP)

In the linear assignment problem (LAP) we looked at in Net Mod 1, the cost of making an assignment of i to k did not depend on any other j being assigned a different l; in the quadratic assignment problem (QAP), the cost of making an assignment of i to k depends on the assignment of all other j's to particular l's. The QAP can be formulated as the following binary integer program (BIP):

$ \begin{eqnarray*} \quad \textbf{QAP:} \quad \mbox{Minimize} \quad \sum_{i \in N} \sum_{j \in N} \sum_{k \in N} \sum_{l \in N} c_{ijkl} x_{ik} x_{jl} \\ \quad \mbox{subject to} \quad \sum_{i \in N} x_{ik} &=& 1 \mbox{,} &\quad k \in N \\ \sum_{k \in N} x_{ik} &=& 1 \mbox{,} &\quad i \in N \\ x_{ik} &\in& \bigl\{ 0, 1 \bigr\}, \end{eqnarray*} $

where

$ \begin{eqnarray*} \quad x_{ik} &=& \begin{cases} 1, & \mbox{if } i \mbox{ is assigned to } k \\ 0, & \mbox{otherwise} \end{cases} \\ c_{ijkl} &=& \mbox{cost of assigning } i \mbox{ to } k \mbox{ if } j \mbox{ is assigned to } l. \end{eqnarray*} $

The objective function is quadratic because of the $x_{ik} x_{jl}$ terms, and, as a result, is an example of a nonlinear programming problem.

In comparison, the following LAP formulation is a linear programming problem because of the single $x_{ij}$ term in the objecive:

$ \begin{eqnarray*} \quad \textbf{LAP:} \quad \mbox{Minimize} \quad \sum_{i \in N} \sum_{j \in N} c_{ij} x_{ij} \\ \quad \mbox{subject to} \quad \sum_{j \in N} x_{ij} &\le& 1\mbox{,} &\quad i \in N \\ \sum_{i \in N} x_{ij} &=& 1\mbox{,} &\quad j \in N \\ x_i &\ge& 0. \end{eqnarray*} $

Layout Problems

The QAP can be used to model layout problems, where a set of resources are assigned to a set of sites. The major cost is the cost of moving an item from site to site as different items visit different resources. The items can be parts being fabricated or patients visiting stations in a clinic. The sequence of resources needs by each item is termed its routing, and each resource can be shared between several items, making the total cost (typically, total distance) dependant on the assignment (or layout) of the resources to the sites.

In a layout problem, the QAP cost term can be separated into two terms:

$ \begin{eqnarray*} \quad c_{ijkl} &=& \mbox{cost of assigning resource } i \mbox{ to site } k \mbox{ if resource } j \mbox{ is assigned to site } l \\ &=& w_{ij} d_{kl} \end{eqnarray*} $

where

$ \begin{eqnarray*} \quad w_{ij} &=& \mbox{total flow from resource } i \mbox{ to resource } k \\ &=& \sum_{p \in P} w_{ijp} \\ w_{ijp} &=& \mbox{total flow from } i \mbox{ to } j \mbox{ for item } p \\ d_{kl} &=& \mbox{distance from site } k \mbox{ to site } l. \end{eqnarray*} $

Ex 4: 3-Item, 4-Machine Layout

In this example, A, B, and C are three different types of items transferred between four machine resources, as shown below. The total number of each item to be produced per shift is 24, 10, and 12, respectively, and their routings from machine to machine are:

$ \quad A: 1–2–3–4; \quad B: 2–4–1–2–3; \quad C: 3–4–1–2–4 $

image.png

There are four possbile sites at which the machines can be located. Each site is represented as a circle and its cooridinate along 1-D is given:

image.png

The site-to-site distance matrix D is:

A machine-to-machine flow matrix W can be created using the number of items produced each shift f and the routing rte[i] for each item i, where each routing is the sequence of machines visited by the item:

The number of machines n can be determined by finding the maximum index in the routings:

The machine-to-machine flow matrix can calculate using zip to iterate from machine j to machine k along each routing:

Given n machines and sites, each element $ w_{ij} $ of W represents some measure of the "weight" between machine i and machine j. The machine-to-site assignment vector $\alpha$ can be used to translate machine-to-machine weights to site-to-site weights so that they can be combined with the site-to-site distances to calculate to total cost TC of an assignement/layout. Given $\alpha(k) = i$ and $\alpha(l) = j$, the weight between site k and site l is equal to the following element of the machine-to-machine weight matrix W: $ w_{\alpha(k) \alpha(l)} = w_{i,j} $. For a given α, W, and D,

$ \begin{eqnarray} \quad TC = \sum_{k \in N} \sum_{l \in N} w_{\alpha(k) \alpha(l)} d_{kl} \end{eqnarray} $

represents the total cost of the assignment.

The assignment vector $\alpha = \big[ 3, 4, 2, 1 \big] $ represents the assignment (or layout) of machines (squares) to sites (circles):

image.png

The total cost of this assignment can be determined by rearranging the rows and columns of the machine-to-machine matrix W so that they correspond to the site-to-site matrix D:

Steepest descent pairwise interchange (SDPI) improvement procedure for QAP

Given an initial machine-to-site assignment array α, a simple heuristic is to determine which pairwise interchange of machines results in the greatest decrease in TC. If no interchange reduces TC, then keep the initial assignment; otherwise, make the pairwise interchange and then continue by looking at pairwise interchanges of the machines in the new assignment array.

The steepest descent pairwise interchange (SDPI) improvement procedure for QAP returns the total cost TC and assignment α of local optimum given initial assignment α, machine-to-machine weight matrix W, and site-to-site distance matrix D:

Pairwise interchange graph

In order to illustrate the operation of the SDPI heuristic, a "pairwise interchange graph" can be constructed for small (i.e., less than five machines/sites) problems. In such a graph, each node represents one of the n! possible assignment arrays, and each arc represents a pairwise interchange that transforms assignment into another via an interchange of the elements at two positions. Each node is connected via arcs representing all of the possible pairwise interchanges from this assignment. The value assigned to each node is the total cost TC. Although the global optimum assignment can be read off the graph for small problems, there would be too many nodes to make this feasible for larger problems.

image.png

In this example, the pairwise interchange graph has 4! = 24 nodes corresponding to all of the possible assignments of 4 machines to 4 sites. The arc from _a_1 to _a_2 represents the interchange of machines 3 and 4 located at sites 3 and 4. The global optimum assignment is _a_18, corresponding to TC = 3,670.

Ex 5: Kitchen Layout

Determining a layout for the kitchen shown below corresponds to assigning each of the seven different appliances (the resources or "machines") to one of the sites in the kitchen. In the figure below left, the red dots in front of each site location correspond to the location in the kitchen at which a person would stand when using the appliance.

image.png

The sequence of appliances that are visited while preparing an estimated number of the meals per week (the routings) are as follows:

image.png

Return weight matrix of flow volumes using routings rte and their frequencies f

Top 10 layouts out of 7! = 5,040 total layouts, where each corresponds to a local optimal solution found by using SDPI:

image.png

Multistart

Finally, a multistart procedure can be used to find the best of n runs of sdpi: