# problem

question:
 Show that the second smallest of $n$ elements can be found with $n + \lceil \lg n \rceil - 2$ comparisons in the worst case. ($\textit{Hint:}$ Also find the smallest element.)

answer:
We can compare the elements in a tournament fashion - we split them into pairs, compare each pair and then proceed to compare the winners in the same fashion. We need to keep track of each "match" the potential winners have participated in.

We select a winner in $n − 1$ matches. At this point, we know that the second smallest element is one of the lgn elements that lost to the smallest - each of them is smaller than the ones it has been compared to, prior to losing. In another $\lceil \lg n \rceil − 1$ comparisons we can find the smallest element out of those.


question:
 Prove the lower bound of $\lceil 3n / 2 \rceil - 2$ comparisons in the worst case to find both the maximum and minimum of $n$ numbers. ($\textit{Hint:}$ Consider how many numbers are potentially either the maximum or minimum, and investigate how a comparison affects these counts.)

answer:
If $n$ is odd, there are

$$
\begin{aligned}
1 + \frac{3(n-3)}{2} + 2
    & = \frac{3n}{2} - \frac{3}{2} \\\\
    & = (\bigg\lceil \frac{3n}{2} \bigg\rceil - \frac{1}{2}) - \frac{3}{2} \\\\
    & = \bigg\lceil \frac{3n}{2} \bigg\rceil - 2
\end{aligned}
$$

comparisons.

If $n$ is even, there are

$$
\begin{aligned}
1 + \frac{3(n - 2)}{2}
    & = \frac{3n}{2} - 2 \\\\
    & = \bigg\lceil \frac{3n}{2} \bigg\rceil - 2
\end{aligned}
$$

comparisons.


question:
 In the algorithm $\text{SELECT}$, the input elements are divided into groups of $5$. Will the algorithm work in linear time if they are divided into groups of $7$? Argue that $\text{SELECT}$ does not run in linear time if groups of $3$ are used.

answer:
It will still work if they are divided into groups of $7$, because we will still know that the median of medians is less than at least $4$ elements from half of the $\lceil n / 7 \rceil$ groups, so, it is greater than roughly $4n / 14$ of the elements.

Similarly, it is less than roughly $4n / 14$ of the elements. So, we are never calling it recursively on more than $10n / 14$ elements. $T(n) \le T(n / 7) + T(10n / 14) + O(n)$. So, we can show by substitution this is linear.

We guess $T(n) < cn$ for $n < k$. Then, for $m \ge k$,

$$
\begin{aligned}
T(m) & \le T(m / 7) + T(10m / 14) + O(m) \\\\
     & \le cm(1 / 7 + 10 / 14) + O(m),
\end{aligned}
$$

therefore, as long as we have that the constant hidden in the big-Oh notation is less than $c / 7$, we have the desired result.

Suppose now that we use groups of size $3$ instead. So, For similar reasons, we have that the recurrence we are able to get is $T(n) = T(\lceil n / 3 \rceil) + T(4n / 6) + O(n) \ge T(n / 3) + T(2n / 3) + O(n)$ So, we will show it is $\ge cn \lg n$.

$$
\begin{aligned}
T(m) & \ge c(m / 3)\lg (m / 3) + c(2m / 3) \lg (2m / 3) + O(m) \\\\
     & \ge cm\lg m + O(m),
\end{aligned}
$$

therefore, we have that it grows more quickly than linear.


question:
 Analyze $\text{SELECT}$ to show that if $n \ge 140$, then at least $\lceil n / 4 \rceil$ elements are greater than the median-of-medians $x$ and at least $\lceil n / 4 \rceil$ elements are less than $x$.

answer:
$$
\begin{aligned}
\frac{3n}{10} - 6 & \ge \lceil \frac{n}{4} \rceil \\\\
\frac{3n}{10} - 6 & \ge \frac{n}{4} + 1 \\\\
        12n - 240 & \ge 10n + 40 \\\\
                n & \ge 140.
\end{aligned}
$$


question:
 Show how quicksort can be made to run in $O(n\lg n)$ time in the worst case, assuming that all elements are distinct.

answer:
We can modify quicksort to run in worst case $n\lg n$ time by choosing our pivot element to be the exact median by using quick select. Then, we are guaranteed that our pivot will be good, and the time taken to find the median is on the same order of the rest of the partitioning.


question:
 Suppose that an algorithm uses only comparisons to find the $i$th smallest element in a set of $n$ elements. Show that it can also find the $i - 1$ smaller elements and $n - i$ larger elements without performing additional comparisons.

answer:
Create a graph with $n$ vertices and draw a directed edge from vertex $i$ to vertex $j$ if the $i$th and $j$th elements of the array are compared in the algorithm and we discover that $A[i] \ge A[j]$. Observe that $A[i]$ is one of the $i - 1$ smaller elements if there exists a path from $x$ to $i$ in the graph, and $A[i]$ is one of the $n - i$ larger elements if there exists a path from $i$ to $x$ in the graph. Every vertex $i$ must either lie on a path to or from $x$ because otherwise the algorithm can't distinguish between $i \le x$ and $i \ge x$. Moreover, if a vertex $i$ lies on both a path to $x$ and a path from $x$, then it must be such that $x \le A[i] \le x$,so $x = A[i]$. In this case, we can break ties arbitrarily.


question:
 Suppose that you have a "black-box" worst-case linear-time median subroutine. Give a simple, linear-time algorithm that solves the selection problem for an arbitrary order statistic.

answer:
To use it, just find the median, partition the array based on that median.

- If $i$ is less than half the length of the original array, recurse on the first half.
- If $i$ is half the length of the array, return the element coming from the median finding black box.
- If $i$ is more than half the length of the array, subtract half the length of the array, and then recurse on the second half of the array.


question:
 The $k$th **_quantiles_** of an $n$-element set are the $k - 1$ order statistics that divide the sorted set into $k$ equal-sized sets (to within $1$). Give an $O(n\lg k)$-time algorithm to list the $k$th quantiles of a set.

answer:
Pre-calculate the positions of the quantiles in $O(k)$, we use the $O(n)$ select algorithm to find the $\lfloor k / 2 \rfloor$th position, after that the elements are divided into two sets by the pivot the $\lfloor k / 2 \rfloor$th position, we do it recursively in the two sets to find other positions. Since the maximum depth is $\lceil \lg k \rceil$, the total running time is $O(n\lg k)$.

```cpp
PARTITION(A, p, r)
    x = A[r]
    i = p
    for k = p to r
        if A[k] < x
            i = i + 1
            swap A[i] with A[k]
    i = i + 1
    swap a[i] with a[r]
    return i
```

```cpp
RANDOMIZED-PARTITION(A, p, r)
    x = RANDOM(p, r)
    swap A[x] with A[r]
    return PARTITION(A, p, r)
```

```cpp
RANDOMIZED-SELECT(A, p, r, i)
    while true
        if p == r
            return p, A[p]
        q = RANDOMIZED-PARTITION(A, p, r)
        k = q - p + 1
        if i == k
            return q, A[q]
        if i < k
            r = q
        else
            p = q + 1
            i = i - k
```

```cpp
k-QUANTITLES-SUB(A, p, r, pos, f, e, quantiles)
    if f + 1 > e
        return
    mid = (f + e) / 2
    q, val = RANDOMIZED-SELECT(A, p, r, pos[mid])
    quantiles[mid] = val
    k = q - p + 1
    for i = mid + 1 to e
        pos[i] = pos[i] - k
    k-QUANTILES-SUB(A, q + 1, r, pos, mid + 1, e, quantiles)
```

```cpp
k-QUANTITLES(A, k)
    num = A.size() / k
    mod = A.size() % k
    pos = num[1..k]
    for i = 1 to mod
        pos[i] = pos[i] + 1
    for i = 1 to k
        pos[i] = pos[i] + pos[i - 1]
    quantiles = [1..k]
    k-QUANTITLES-SUB(A, 0, A.length, pos, 0, pos.size(), quantiles)
    return quantiles
```


question:
 Describe an $O(n)$-time algorithm that, given a set $S$ of $n$ distinct numbers and a positive integer $k \le n$, determines the $k$ numbers in $S$ that are closest to the median of $S$.

answer:
Find the median in $O(n)$; create a new array, each element is the absolute value of the original value subtract the median; find the $k$th smallest number in $O(n)$, then the desired values are the elements whose absolute difference with the median is less than or equal to the $k$th smallest number in the new array.


question:
 Let $X[1..n]$ and $Y[1..n]$ be two arrays, each containing $n$ numbers already in sorted order. Give an $O(\lg n)$-time algorithm to find the median of all $2n$ elements in arrays $X$ and $Y$.

answer:
Without loss of generality, assume $n$ is a power of $2$.

```cpp
MEDIAN(X, Y, n)
    if n == 1
        return min(X[1], Y[1])
    if X[n / 2] < Y[n / 2]
        return MEDIAN(X[n / 2 + 1..n], Y[1..n / 2], n / 2)
    return MEDIAN(X[1..n / 2], Y[n / 2 + 1..n], n / 2)
```


question:
 Professor Olay is consulting for an oil company, which is planning a large pipeline running east to west through an oil field of $n$ wells. The company wants to connect a spur pipeline from each well directly to the main pipeline along a shortest route (either north or south), as shown in Figure 9.2. Given the $x$- and $y$-coordinates of the wells, how should the professor pick the optimal location of the main pipeline, which would be the one that minimizes the total length of the spurs? Show how to determine the optimal location in linear time.

answer:
- If $n$ is odd, we pick the $y$ coordinate of the main pipeline to be equal to the median of all the $y$ coordinates of the wells.

- If $n$ is even, we pick the $y$ coordinate of the pipeline to be anything between the $y$ coordinates of the wells with $y$-coordinates which have order statistics $\lfloor (n + 1) / 2 \rfloor$ and the $\lceil (n + 1) / 2 \rceil$. These can all be found in linear time using the algorithm from this section.


question: Show that $\text{RANDOMIZED-SELECT}$ never makes a recursive call to a $0$-length array.
answer:
Calling a $0$-length array would mean that the second and third arguments are equal. So, if the call is made on line 8, we would need that $p = q - 1$, which means that $q - p + 1 = 0$.

However, $i$ is assumed to be a nonnegative number, and to be executing line 8, we would need that $i < k = q - p + 1 = 0$, a contradiction. The other possibility is that the bad recursive call occurs on line 9. This would mean that $q + 1 = r$. To be executing line 9, we need that $i > k = q - p + 1 = r - p$. This would be a nonsensical original call to the array though because we are asking for the ith element from an array of strictly less size.


question:
 Argue that the indicator random variable $X_k$ and the value $T(\max(k - 1, n - k))$ are independent.

answer:
The probability that $X_k$ is equal to $1$ is unchanged when we know the max of $k - 1$ and $n - k$. In other words, $\Pr\\{X_k = a \mid \max(k - 1, n - k) = m\\} = \Pr\\{X_k = a\\}$ for $a = 0, 1$ and $m = k - 1, n - k$ so $X_k$ and $\max(k - 1, n - k)$ are independent.

By C.3-5, so are $X_k$ and $T(\max(k - 1, n - k))$.


question: Write an iterative version of $\text{RANDOMIZED-SELECT}$.
answer:
```cpp
PARTITION(A, p, r)
    x = A[r]
    i = p
    for k = p - 1 to r
       if A[k] < x
           i = i + 1
           swap A[i] with A[k]
    i = i + 1
    swap A[i] with A[r]
    return i
```

```cpp
RANDOMIZED-PARTITION(A, p, r)
    x = RANDOM(p - 1, r)
    swap A[x] with A[r]
    return PARTITION(A, p, r)
```

```cpp
RANDOMIZED-SELECT(A, p, r, i)
    while true
        if p == r
            return A[p]
        q = RANDOMIZED-PARTITION(A, p, r)
        k = q - p + 1
        if i == k
            return A[q]
        if i < k
            r = q - 1
        else
            p = q + 1
            i = i - k
```


question:
 Suppose we use $\text{RANDOMIZED-SELECT}$ to select the minimum element of the array $A = \langle 3, 2, 9, 0, 7, 5, 4, 8, 6, 1 \rangle$. Describe a sequence of partitions that results in a worst-case performance of $\text{RANDOMIZED-SELECT}$.

answer:
When the partition selected is always the maximum element of the array we get worst-case performance. In the example, the sequence would be $\langle 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 \rangle$.


question:
 Put the single-pair shortest-path linear program from $\text{(29.44)}$–$\text{(29.46)}$ into standard form.

answer:
The objective is already in normal form. However, some of the constraints are equality constraints instead of $\le$ constraints. This means that we need to rewrite them as a pair of inequality constraints, the overlap of whose solutions is just the case where we have equality. we also need to deal with the fact that most of the variables can be negative. To do that, we will introduce variables for the negative part and positive part, each of which need be positive, and we'll just be sure to subtract the negative part. $d_s$ need not be changed in this way since it can never be negative since we are not assuming the existence of negative weight cycles.

$$
\begin{aligned}
d_v^+ - d_v^- - d_u^+ + d_u^- \le w(u, v) \text{ for every edge } (u, v) \\\\
d_s \le 0
\end{aligned}
$$


question:
 Write out explicitly the linear program corresponding to finding the shortest path from node $s$ to node $y$ in Figure 24.2(a).

answer:
$$
\begin{array}{ll}
\text{minimize}   & d_y \\\\
\text{subject to} & \\\\
                  & d_t \le d_s + 3 \\\\
                  & d_x \le d_t + 6 \\\\
                  & d_y \le d_s + 5 \\\\
                  & d_y \le d_t + 2 \\\\
                  & d_z \le d_x + 2 \\\\
                  & d_t \le d_y + 1 \\\\
                  & d_x \le d_y + 4 \\\\
                  & d_z \le d_y + 1 \\\\
                  & d_s \le d_z + 1 \\\\
                  & d_x \le d_z + 7 \\\\
                  & d_2 =   0.
\end{array}
$$


question:
 In the single-source shortest-paths problem, we want to find the shortest-path weights from a source vertex $s$ to all vertices $v \in V$. Given a graph $G$, write a linear program for which the solution has the property that $d_v$ is the shortest-path weight from $s$ to $v$ for each vertex $v \in V$.

answer:
We will follow a similar idea to the way to when we were finding the shortest path between two particular vertices.

$$
\begin{array}{ll}
\text{maximize}   & \sum_{v \in V} d_v \\\\
\text{subject to} & \\\\
                  & d_v \le d_u + w(u, v) \text{ for each edge } (u, v) \\\\
                  & d_s =   0.
\end{array}
$$

The first type of constraint makes sure that we never say that a vertex is further away than it would be if we just took the edge corresponding to that constraint. Also, since we are trying to maximize all of the variables, we will make it so that there is no slack anywhere, and so all the dv values will correspond to lengths of shortest paths to $v$. This is because the only thing holding back the variables is the information about relaxing along the edges, which is what determines shortest paths.


question:
 Write out explicitly the linear program corresponding to finding the maximum flow in Figure 26.1(a).

answer:
$$
\begin{array}{lll}
\text{maximize}   & f_{sv_1} + f_{sv_2} \\\\
\text{subject to} & \\\\
                  & f_{sv_1}                & \le 16 \\\\
                  & f_{sv_2}                & \le 14 \\\\
                  & f_{v_1v_3}              & \le 12 \\\\
                  & f_{v_2v_1}              & \le  4 \\\\
                  & f_{v_2v_4}              & \le 14 \\\\
                  & f_{v_3v_2}              & \le  9 \\\\
                  & f_{v_3t}                & \le 20 \\\\
                  & f_{v_4v_3}              & \le  7 \\\\
                  & f_{v_4t}                & \le  4 \\\\
                  & f_{sv_1} + f_{v_2v_1}   & =   f_{v_1v_3} \\\\
                  & f_{sv_2} + f_{v_3v_2}   & =   f_{v_2v_1} + f_{v_2v_4} \\\\
                  & f_{v_1v_3} + f_{v_4v_3} & =   f_{v_3v_2} + f_{v_3t} \\\\
                  & f_{v_2v_4}              & =   f_{v_4v_3} + f_{v_4t} \\\\
                  & f_{uv}                  & \ge 0 \text{ for } u, v \in \\{s, v_1, v_2, v_3, v_4, t\\}.
\end{array}
$$


question:
 Rewrite the linear program for maximum flow $\text{(29.47)}$–$\text{(29.50)}$ so that it uses only $O(V + E)$ constraints.

answer:
All we need to do to bring the number of constraints down from $O(V^2)$ to $O(V + E)$ is to replace the way we index the flows. Instead of indexing it by a pair of vertices, we will index it by an edge. This won't change anything about the analysis because between pairs of vertices that don't have an edge between them, there definitely won't be any flow. Also, it brings the number of constraints of the first and third time down to $O(E)$ and the number of constraints of the second kind stays at $O(V)$.

$$
\begin{array}{lll}
\text{maximize}   & \sum_{\text{edges $e$ leaving $s$}} f_e - \sum_{\text{edges $e$ entering $s$}} f_s \\\\
\text{subject to} & \\\\
                  & f_{(u, v)} \le c(u, v) \text{ for each edge } (u, v) \\\\
                  & \sum_{\text{edges $e$ leaving $u$}} f_e - \sum_{\text{edges $e$ entering $u$}} f_e \text{ for each edge } u \in V - \\{s, t\\} \\\\
                  & f_e \ge 0 \text{ for each edge } e.
\end{array}
$$


question:
 Write a linear program that, given a bipartite graph $G = (V, E)$ solves the maximum-bipartite-matching problem.

answer:
Recall from section 26.3 that we can solve the maximum-bipartite-matching problem by viewing it as a network flow problem, where we append a source $s$ and sink $t$, each connected to every vertex is $L$ and $R$ respectively by an edge with capacity $1$, and we give every edge already in the bipartite graph capacity $1$. The integral maximum flows are in correspondence with maximum bipartite matchings. In this setup, the linear programming problem to solve is as follows:

$$
\begin{aligned}
\text{maximize}   & \sum_{v \in L} f_{sv} \\\\
\text{subject to} & \\\\
                  & f_{(u, v)} \le 1 \text{ for each } u, v \in \\{s\\} \cup L \cup R \cup \\{t\\} = V \\\\
                  & \sum_{v \in V} f_{vu} = \sum_{v \in V} f_{uv} \text{ for each } u \in L \cup R \\\\
                  & f_{uv} \ge 0 \text{ for each } u, v \in V
\end{aligned}
$$


question:
 In the **_minimum-cost multicommodity-flow problem_**, we are given directed graph $G = (V, E)$ in which each edge $(u, v) \in E$ has a nonnegative capacity $c(u, v) \ge 0$ and a cost $a(u, v)$. As in the multicommodity-flow problem, we are given $k$ different commodities, $K_1, K_2, \ldots, K_k$, where we specify commodity $i$ by the triple $K_i = (s_i, t_i, d_i)$. We define the flow $f_i$ for commodity $i$ and the aggregate flow $f_{uv}$ on edge $(u, v)$ as in the multicommodity-flow problem. A feasible flow is one in which the aggregate flow on each edge $(u, v)$ is no more than the capacity of edge $(u, v)$. The cost of a flow is $\sum_{u, v \in V} a(u, v)f_{uv}$, and the goal is to find the feasible flow of minimum cost. Express this problem as a linear program.

answer:
As in the minimum cost flow problem, we have constraints for the edge capacities, for the conservation of flow, and nonegativity. The difference is that the restraint that before we required exactly $d$ units to flow, now, we require that for each commodity, the right amount of that commodity flows. the conservation equalities will be applied to each different type of commodity independently. If we super script $f$ that will denote the type of commodity the flow is describing, if we do not superscript it, it will denote the aggregate flow

We want to minimize

$$\sum_{u, v \in V} a(u, v) f_{uv}.$$

The capacity constraints are that

$$\sum_{i \in [k]} \sum_{u, v \in V} f_{uv}^i \le c(u, v).$$

The conservation constraints are that for every $i \in [k]$, for every $u \in V \backslash \\{s_i, t_i\\}$.

$$\sum_{v \in V} f_{uv}^i = \sum_{v \in V} f_{vu}^i.$$

Now, the constraints that correspond to requiring a certain amount of flow are that for every $i \in [k]$.

$$\sum_{v \in V} f_{s_i, v}^i - \sum_{v \in V} f_{v, s_i}^i = d.$$

Now, we put in the constraint that makes sure what we called the aggregate flow is actually the aggregate flow, so, for every $u, v \in V$,

$$f_{uv} = \sum_{i \in [k]} f_{uv}^i.$$

Finally, we get to the fact that all flows are nonnegative, for every $u, v \in V$,

$$f_{uv} \ge 0.$$


question: Complete the proof of Lemma 29.4 by showing that it must be the case that $c = c'$ and $v = v'$.
answer:
We subtract equation $\text{(29.81)}$ from equation $\text{(29.79)}$.

> $$z = v + \sum_{j \in N} c_j x_j, \tag{29.79}$$

> $$z = v' + \sum_{j \in N} c_j' x_j. \tag{29.81}$$

Thus we have,

$$
\begin{aligned}
                      0 & = v - v' + \sum_{j \in N} (c_j - c_j') x_j. \\\\
\sum_{j \in N} c_j' x_j & = v - v' + \sum_{j \in N} c_j x_j.
\end{aligned}
$$

By Lemma 29.3, we have $c_j = c_j'$ for every $j$ and $v = v'$ since $v - v' = 0$.


question:
 Show that the call to $\text{PIVOT}$ in line 12 of $\text{SIMPLEX}$ never decreases the value of $v$.

answer:
The only time $v$ is updated in $\text{PIVOT}$ is line 14, so it will suffice to show that $c_e \hat b_e \ge 0$. Prior to making the call to $\text{PIVOT}$, we choose an index $e$ such that $c_e > 0$, and this is unchanged in $\text{PIVOT}$. We set $\hat b_e$ in line 3 to be $b_l / a_{le}$.

The loop invariant proved in Lemma 29.2 tells us that $b_l \ge 0$. The if-condition of line 6 of $\text{SIMPLEX}$ tells us that only the noninfinite $\delta_i$ must have $a_{ie} > 0$, and we choose $l$ to minimize $\delta_l$, so we must have $a_{le} > 0$. Thus, $c_e \hat b_e \ge 0$, which implies $v$ can never decrease.


question:
 Prove that the slack form given to the $\text{PIVOT}$ procedure and the slack form that the procedure returns are equivalent.

answer:
To show that the two slack forms are equivalent, we will show both that they have equal objective functions, and their sets of feasible solutions are equal.

First, we'll check that their sets of feasible solutions are equal. Basically all we do to the constraints when we pivot is take the non-basic variable, $e$, and solve the equation corresponding to the basic variable $l$ for $e$. We are then taking that expression and replacing $e$ in all the constraints with this expression we got by solving the equation corresponding to $l$. Since each of these algebraic operations are valid, the result of the sequence of them is also algebraically equivalent to the original.

Next, we'll see that the objective functions are equal. We decrease each $c_j$ by $c_e \hat a_{ej}$, which is to say that we replace the non-basic variable we are making basic with the expression we got it was equal to once we made it basic.

Since the slack form returned by $\text{PIVOT}$, has the same feasible region and an equal objective function, it is equivalent to the original slack form passed in.


question:
 Suppose we convert a linear program $(A, b, c)$ in standard form to slack form. Show that the basic solution is feasible if and only if $b_i \ge 0$ for $i = 1, 2, \ldots, m$.

answer:
First suppose that the basic solution is feasible. We set each $x_i = 0$ for $1 \le i \le n$, so we have $x_{n + i} = b_i - \sum_{j = 1}^n a_{ij}x_j = b_i$ as a satisfied constraint. Since we also require $x_{n + i} \ge 0$ for all $1 \le i \le m$, this implies $b_i \ge 0$.

Now suppose $b_i \ge 0$ for all $i$. In the basic solution we set $x_i = 0$ for $1 \le i \le n$ which satisfies the nonnegativity constraints. We set $x_{n + i} = b_i$ for $1 \le i \le m$ which satisfies the other constraint equations, and also the nonnegativity constraints on the basic variables since $b_i \ge 0$. Thus, every constraint is satisfied, so the basic solution is feasible.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrl}
> \text{maximize}   & 18x_1 & + & 12.5x_2 \\\\
> \text{subject to} & \\\\
>                   &   x_1 & + &     x_2 & \le & 20 \\\\
>                   &   x_1 &   &         & \le & 12 \\\\
>                   &       &   &     x_2 & \le & 16 \\\\
>                   &       & x_1, x_2 &  & \ge &  0 & .
> \end{array}
> $$

answer:
First, we rewrite the linear program into it's slack form

$$
\begin{array}{lrl}
\text{maximize}   & 18x_1 + 12.5x_2 \\\\
\text{subject to} & \\\\
                  & x_3 & = 20 - x_1 - x_2 \\\\
                  & x_4 & = 12 - x_1 \\\\
                  & x_5 & = 16 - x_2 \\\\
                  & x_1, x_2, x_3, x_4, x_5 & \ge 0.
\end{array}
$$

We now stop since no more non-basic variables appear in the objective with a positive coefficient. Our solution is $(12, 8, 0, 0, 8)$ and has a value of $316$. Going back to the standard form we started with, we just disregard the values of $x_3$ through $x_5$ and have the solution that $x_1 = 12$ and $x_2 = 8$. We can check that this is both feasible and has the objective achieve $316$.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrl}
> \text{maximize}   & 5x_1 & - & 3x_2 \\\\
> \text{subject to} & \\\\
>                   &  x_1 & - &  x_2 &   \le & 1 \\\\
>                   & 2x_1 & + &  x_2 &   \le & 2 \\\\
>                   &      & x_1, x_2 & & \ge & 0 & .
> \end{array}
> $$

answer:
First, we convert the linear program into it's slack form

$$
\begin{array}{rcrcrcrl}
z   & = &   &   & 5x_1 & - & 3x_2 \\\\
x_3 & = & 1 & - &  x_1 & + &  x_2 \\\\
x_4 & = & 2 & - & 2x_1 & - &  x_2 & .
\end{array}
$$

The nonbasic variables are $x_1$ and $x_2$. Of these, only $x_1$ has a positive coefficient in the objective function, so we must choose $x_e = x_1$. Both equations limit $x_1$ by $1$, so we'll choose the first one to rewrite $x_1$ with. Using $x_1 = 1 − x_3 + x_2$ we obtain the new system

$$
\begin{array}{rcrcrcrl}
z   & = & 5 & - & 5x_3 & + & 2x_2 & \\\\
x_1 & = & 1 & - &  x_3 & + &  x_2 & \\\\
x_4 & = &   &   & 2x_3 & - & 3x_2 & .
\end{array}
$$

Now $x_2$ is the only nonbasic variable with positive coefficient in the objective function, so we set $x_e = x_2$. The last equation limits $x_2$ by $0$ which is most restrictive, so we set $x_2 = \frac{2}{3} x_3 − \frac{1}{3} x_4$. Rewriting, our new system becomes

$$
\begin{array}{rcrcrcrl}
z   & = & 5 & - & \frac{11}{3} x_3 & - & \frac{2}{3} x_4 \\\\
x_1 & = & 1 & - & \frac{1}{3}  x_3 & - & \frac{1}{3} x_4 \\\\
x_2 & = &   &   & \frac{2}{3}  x_3 & - & \frac{1}{3} x_4 & .
\end{array}
$$

Every nonbasic variable now has negative coefficient in the objective function, so we take the basic solution $(x_1, x_2, x_3, x_4) = (1, 0, 0, 0)$. The objective value this achieves is $5$.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrcrl}
> \text{minimize}   &   x_1 & + &     x_2 & + &   x_3 \\\\
> \text{subject to} & \\\\
>                   &  2x_1 & + &  7.5x_2 & + &  3x_3 & \ge & 10000 \\\\
>                   & 20x_1 &   &    5x_2 & + & 10x_3 & \ge & 30000 \\\\
>                   &       & x_1, x_2, x_3 & & &     & \ge &     0 & .
> \end{array}
> $$

answer:
First, we convert this equation to the slack form. Doing so doesn't change the objective, but the constraints become

$$
\begin{array}{rcrcrcrcr}
z   & = &        & - &   x_1 & - &    x_2 & - &   x_3 \\\\
x_4 & = & -10000 & + &  2x_1 & + & 7.5x_2 & + &  3x_3 \\\\
x_5 & = & -30000 & + & 20x_1 & + &   5x_2 & + & 10x_3 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Also, since the objective is to minimize a given function, we'll change it over to maximizing the negative of that function. In particular maximize $−x_1 − x_2 − x_3$. Now, we note that the initial basic solution is not feasible, because it would leave $x_4$ and $x_5$ being negative. This means that finding an initial solution requires using the method of section 29.5. The auxiliary linear program in slack form is

$$
\begin{array}{rcrcrcrcrcr}
z   & = &        & - & x_0 \\\\
x_4 & = & -10000 & + & x_0 & + &  2x_1 & + & 7.5x_2 & + &  3x_3 \\\\
x_5 & = & -30000 & + & x_0 & + & 20x_1 & + &   5x_2 & + & 10x_3 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

We choose $x_0$ as the entering variable and $x_5$ as the leaving variable, since it is the basic variable whose value in the basic solution is most negative. After pivoting, we have the slack form

$$
\begin{array}{rcrcrcrcrcr}
z   & = & -30000 & + & 20x_1 & + &   5x_2 & + & 10x_3 & - & x_5 \\\\
x_0 & = &  30000 & - & 20x_1 & - &   5x_2 & - & 10x_3 & + & x_5 \\\\
x_4 & = &  20000 & - & 18x_1 & + & 2.5x_2 & - &  7x_3 & + & x_5 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

The associated basic solution is feasible, so now we just need to repeatedly call $\text{PIVOT}$ until we obtain an optimal solution to $L_{aux}$. We'll choose $x_2$ as our entering variable. This gives

$$
\begin{array}{rcrcrcrcrcr}
z   & = &       & - &    x_0 \\\\
x_2 & = &  6000 & - & 0.2x_0 & - &  4x_1 & - &  2x_3 & + & 0.2x_5   \\\\
x_4 & = & 35000 & - & 0.5x_0 & - & 28x_1 & - & 12x_3 & + & 1.5x_5 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

This slack form is the final solution to the auxiliary problem. Since this solution has $x_0 = 0$, we know that our initial problem was feasible. Furthermore, since $x_0 = 0$, we can just remove it from the set of constraints. We then restore the original objective function, with appropriate substitutions made to include only the nonbasic variables. This yields

$$
\begin{array}{rcrcrcrcr}
z   & = & -6000 & + &  3x_1 & + &   x_3 & - & 0.2x_5 \\\\
x_2 & = &  6000 & - &  4x_1 & - &  2x_3 & + & 0.2x_5 \\\\
x_4 & = & 35000 & - & 28x_1 & - & 12x_3 & + & 1.5x_5 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

This slack form has a feasible basic solution, and we can return it to $\text{SIMPLEX}$. We choose $x_1$ as our entering variable. This gives

$$
\begin{array}{rcrcrcrcr}
z   & = & -2250 & - & \frac{2}{7} x_3 & - & \frac{3}{28} x_4 & - & \frac{11}{280} x_5 \\\\
x_1 & = &  1250 & - & \frac{3}{7} x_3 & - & \frac{1}{28} x_4 & + & \frac{15}{280} x_5 \\\\
x_2 & = &  1000 & - & \frac{2}{7} x_3 & + & \frac{4}{28} x_4 & - &  \frac{4}{280} x_5 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

At this point, all coefficients in the objective function are negative, so the basic solution is an optimal solution. This solution is $(x_1, x_2, x_3) = (1250, 1000, 0)$.


question:
 In the proof of Lemma 29.5, we argued that there are at most $\binom{m + n}{n}$ ways to choose a set $B$ of basic variables. Give an example of a linear program in which there are strictly fewer than $\binom{m + n}{n}$ ways to choose the set $B$.

answer:
Consider the simple program

$$
\begin{array}{rcrcrl}
z   & = &   & - & x_1 \\\\
x_2 & = & 1 & - & x_1 & .
\end{array}
$$

In this case, we have $m = n = 1$, so $\binom{m + n}{n} = \binom{2}{1} = 2$, however, since the only coefficients of the objective function are negative, we can't make any other choices for basic variable. We must immediately terminate with the basic solution $(x_1, x_2) = (0, 1)$, which is optimal.


question:
 If we express the linear program in $\text{(29.24)}$–$\text{(29.28)}$ in the compact notation of $\text{(29.19)}$–$\text{(29.21)}$, what are $n$, $m$, $A$, $b$, and $c$?

answer:
$$
\begin{aligned}
n = m & = 3, \\\\
A & =
\begin{pmatrix}
 1 &  1 & -1 \\\\
-1 & -1 &  1 \\\\
 1 & -2 &  2
\end{pmatrix}
, \\\\
b & =
\begin{pmatrix}
 7 \\\\
-7 \\\\
 4
\end{pmatrix}
, \\\\
c & =
\begin{pmatrix}
 2 \\\\
-3 \\\\
 3
\end{pmatrix}
.
\end{aligned}
$$


question:
 Give three feasible solutions to the linear program in $\text{(29.24)}$–$\text{(29.28)}$. What is the objective value of each one?

answer:
1. $(x_1, x_2, x_3) = (6, 1, 0)$ with objective value $9$.
2. $(x_1, x_2, x_3) = (5, 2, 0)$ with objective value $4$.
3. $(x_1, x_2, x_3) = (4, 3, 0)$ with objective value $-1$.


question:
 For the slack form in $\text{(29.38)}$–$\text{(29.41)}$, what are $N$, $B$, $A$, $b$, $c$, and $v$?

answer:
$$
\begin{aligned}
N & = \\{1, 2, 3\\}, \\\\
B & = \\{4, 5, 6\\}, \\\\
A & =
\begin{pmatrix}
 1 &  1 & -1 \\\\
-1 & -1 &  1 \\\\
 1 & -2 &  2
\end{pmatrix}
, \\\\
b & =
\begin{pmatrix}
 7 \\\\
-7 \\\\
 4
\end{pmatrix}
, \\\\
c & =
\begin{pmatrix}
 2 \\\\
-3 \\\\
 3
\end{pmatrix}
, \\\\
v & = 0.
\end{aligned}
$$


question:
 Convert the following linear program into standard form:
>
> $$
> \begin{array}{lrcrcrcrl}
> \text{minimize}   & 2x_1 & + & 7x_2 & + & x_3 &     & \\\\
> \text{subject to} & \\\\
>                   &  x_1 &   &      & - & x_3 & =   &  7 \\\\
>                   & 3x_1 & + &  x_2 &   &     & \ge & 24 \\\\
>                   &      &   &  x_2 &   &     & \ge &  0 \\\\
>                   &      &   &      &   & x_3 & \le &  0 & .
> \end{array}
> $$

answer:
$$
\begin{array}{lrcrcrcrcrl}
\text{maximize}   & -2x_1 & + & 2x_2 & - & 7x_3 & +  & x_4 &     & \\\\
\text{subject to} & \\\\
                  &  -x_1 & + &  x_2 &   &      & -  & x_4 & \le &  -7 \\\\
                  &   x_1 & - &  x_2 &   &      & +  & x_4 & \le &   7 \\\\
                  & -3x_1 & + & 3x_2 & - &  x_3 &    &     & \le & -24 \\\\
                  &       & x_1, x_2, x_3, x_4  & & & & &  & \ge &   0 & .
\end{array}
$$


question:
 Convert the following linear program into slack form:
>
> $$
> \begin{array}{lrcrcrcrl}
> \text{maximize}   & 2x_1 &   &      & - & 6x_3 \\\\
> \text{subject to} & \\\\
>                   &  x_1 & + &  x_2 & - &  x_3 & \le & 7 \\\\
>                   & 3x_1 & - &  x_2 &   &      & \ge & 8 \\\\
>                   & -x_1 & + & 2x_2 & + & 2x_3 & \ge & 0 \\\\
>                   &      & x_1, x_2, x_3 & & & & \ge & 0 & .
> \end{array}
> $$
>
> What are the basic and nonbasic variables?

answer:
First, we will multiply the second and third inequalities by minus one to make it so that they are all $\le$ inequalities. We will introduce the three new variables $x_4$, $x_5$, $x_6$, and perform the usual procedure for rewriting in slack form

$$
\begin{array}{rcrcrcrcr}
                         x_4 & =   &  7 & - &  x_1 & - &  x_2 & + &  x_3 \\\\
                         x_5 & =   & -8 & + & 3x_1 & - &  x_2 \\\\
                         x_6 & =   &    & - &  x_1 & + & 2x_2 & + & 2x_3 \\\\
x_1, x_2, x_3, x_4, x_5, x_6 & \ge &  0 & ,
\end{array}
$$

where we are still trying to maximize $2x_1 - 6x_3$. The basic variables are $x_4$, $x_5$, $x_6$ and the nonbasic variables are $x_1$, $x_2$, $x_3$.


question:
 Show that the following linear program is infeasible:
>
> $$
> \begin{array}{lrcrcrl}
> \text{minimize}   &  3x_1 & - & 2x_2 \\\\
> \text{subject to} & \\\\
>                   &   x_1 & + &  x_2   & \le &   2 \\\\
>                   & -2x_1 & - & 2x_2   & \le & -10 \\\\
>                   &       & x_1, x_2 & & \ge &   0 & .
> \end{array}
> $$

answer:
By dividing the second constraint by $2$ and adding to the first, we have $0 \le −3$, which is impossible. Therefore there linear program is unfeasible.


question:
 Show that the following linear program is unbounded:
>
> $$
> \begin{array}{lrcrcrl}
> \text{minimize}   &   x_1 & - &  x_2 \\\\
> \text{subject to} & \\\\
>                   & -2x_1 & + &  x_2   & \le & -1 \\\\
>                   &  -x_1 & - & 2x_2   & \le & -2 \\\\
>                   &       & x_1, x_2 & & \ge &  0 & .
> \end{array}
> $$

answer:
For any number $r > 1$, we can set $x_1 = 2r$ and $x_2 = r$. Then, the restaints become

$$
\begin{array}{rcrcrl}
-2(2r) & + &  r = -3r & \le & -1 \\\\
   -2r & - & 2r = -4r & \le & -2 \\\\
       & 2r, r &      & \ge &  0 & .
\end{array}
$$

All of these inequalities are clearly satisfied because of our initial restriction in selecting $r$. Now, we look to the objective function, it is $2r - r = r$. So, since we can select $r$ to be arbitrarily large, and still satisfy all of the constraints, we can achieve an arbitrarily large value of the objective function.


question:
 Suppose that we have a general linear program with $n$ variables and $m$ constraints, and suppose that we convert it into standard form. Give an upper bound on the number of variables and constraints in the resulting linear program.

answer:
In the worst case, we have to introduce 2 variables for every variable to ensure that we have nonnegativity constraints, so the resulting program will have $2n$ variables. If each constraint is an equality, we would have to double the number of constraints to create inequalities, resulting in $2m$ constraints. Changing minimization to maximization and greater-than signs to less-than signs don't affect the number of variables or constraints, so the upper bound is $2n$ on variables and $2m$ on constraints.


question:
 Give an example of a linear program for which the feasible region is not bounded, but the optimal objective value is finite.

answer:
Consider the linear program where we want to maximize $x_1 − x_2$ subject to the constraints $x_1 − x_2 \le 1$ and $x_1$, $x_2 \ge 0$. clearly the objective value can never be greater than one, and it is easy to achieve the optimal value of $1$, by setting $x_1 = 1$ and $x_2 = 0$. Then, this feasible region is unbounded because for any number $r$, we could set $x_1 = x_2 = r$, and that would be feasible because the difference of the two would be zero which is $\le 1$.

If we further wanted it so that there was a single solution that achieved the finite optimal value, we could add the requirements that $x_1 \le 1$.


question: Give detailed pseudocode to implement lines 5 and 14 of $\text{INITIALIZE-SIMPLEX}$.
answer:
For line 5, first let $(N, B, A, b, c, v)$ be the result of calling $\text{PIVOT}$ on $L_{aux}$ using $x_0$ as the entering variable. Then repeatedly call $\text{PIVOT}$ until an optimal solution to $L_{aux}$ is obtained, and return this to $(N, B, A, b, c, v)$. To remove $x_0$ from the constraints, set $a_{i, 0} = 0$ for all $i \in B$, and set $N = N \backslash \\{0\\}$. To restore the original objective function of $L$, for each $j \in N$ and each $i \in B$, set $c_j = c_j − c_ia_{ij}$.


question:
 Show that when the main loop of $\text{SIMPLEX}$ is run by $\text{INITIALIZE-SIMPLEX}$, it can never return "unbounded."

answer:
In order to enter line 10 of $\text{INITIALIZE-SIMPLEX}$ and begin iterating the main loop of $\text{SIMPLEX}$, we must have recovered a basic solution which is feasible for $L_{aux}$. Since $x_0 \ge 0$ and the objective function is $−x_0$, the objective value associated to this solution (or any solution) must be negative. Since the goal is to aximize, we have an upper bound of $0$ on the objective value. By Lemma 29.2, $\text{SIMPLEX}$ correctly determines whether or not the input linear program is unbounded. Since $L_{aux}$ is not unbounded, this can never be returned by $\text{SIMPLEX}$.


question:
 Suppose that we are given a linear program $L$ in standard form, and suppose that for both $L$ and the dual of $L$, the basic solutions associated with the initial slack forms are feasible. Show that the optimal objective value of $L$ is $0$.

answer:
Since it is in standard form, the objective function has no constant term, it is entirely given by $\sum_{i = 1}^n c_ix_i$, which is going to be zero for any basic solution. The same thing goes for its dual. Since there is some solution which has the objective function achieve the same value both for the dual and the primal, by the corollary to the weak duality theorem, that common value must be the optimal value of the objective function.


question:
 Suppose that we allow strict inequalities in a linear program. Show that in this case, the fundamental theorem of linear programming does not hold.

answer:
Consider the linear program in which we wish to maximize $x_1$ subject to the constraint $x_1 < 1$ and $x_1 \ge 0$. This has no optimal solution, but it is clearly bounded and has feasible solutions. Thus, the Fundamental theorem of linear programming does not hold in the case of strict inequalities.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrl}
> \text{maxmize}    &  x_1 & + &   3x_2 \\\\
> \text{subject to} & \\\\
>                   &  x_1 & - &    x_2 & \le &  8 \\\\
>                   & -x_1 & - &    x_2 & \le & -3 \\\\
>                   & -x_1 & + &   4x_2 & \le &  2 \\\\
>                   &      & x_1, x_2 & & \ge &  0 & .
> \end{array}
> $$

answer:
The initial basic solution isn't feasible, so we will need to form the auxiliary linear program,

$$
\begin{array}{lrcrcrcrl}
\text{maxmize}    & -x_0 \\\\
\text{subject to} & \\\\
                  & -x_0 & + & x_1 & - &   x_2 & \le &  8 \\\\
                  & -x_0 & - & x_1 & - &   x_2 & \le & -3 \\\\
                  & -x_0 & - & x_1 & + &  4x_2 & \le &  2 \\\\
                  &      & x_0, x_1, x_2 & & & & \ge &  0 & .
\end{array}
$$

Writing this linear program in slack form,

$$
\begin{array}{rcrcrcrcr}
z   & = &    & - & x_0 \\\\
x_3 & = &  8 & + & x_0 & - & x_1 & + &  x_2 \\\\
x_4 & = & -3 & + & x_0 & + & x_1 & + &  x_2 \\\\
x_5 & = &  2 & + & x_0 & + & x_1 & - & 4x_2 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Next we make one call to $\text{PIVOT}$ where $x_0$ is the entering variable and $x_4$ is the leaving variable.

$$
\begin{array}{rcrcrcrcr}
z   & = & -3 & + &  x_1 & + &  x_2 & - & x_4 \\\\
x_0 & = &  3 & - &  x_1 & - &  x_2 & + & x_4 \\\\
x_3 & = & 11 & - & 2x_1 &   &      & + & x_4 \\\\
x_5 & = &  5 &   &      & - & 5x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

The basic solution is feasible, so we repeatedly call $\text{PIVOT}$ to get the optimal solution to $L_{aux}$. We'll choose $x_1$ to be our entering variable and $x_0$ to be the leaving variable. This gives

$$
\begin{array}{rcrcrcrcr}
z   & = &   &   & -x_0 \\\\
x_1 & = & 3 & - &  x_0 & - &  x_2 & + & x_4 \\\\
x_3 & = & 5 & + & 2x_0 & + & 2x_2 & - & x_4 \\\\
x_5 & = & 5 &   &      & - & 5x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

The basic solution is now optimal for $L_{aux}$, so we return this slack form to $\text{SIMPLEX}$, set $x_0 = 0$, and update the objective function which yields

$$
\begin{array}{rcrcrcr}
z   & = & 3 & + & 2x_2 & + & x_4 \\\\
x_1 & = & 3 & - &  x_2 & + & x_4 \\\\
x_3 & = & 5 & + & 2x_2 & - & x_4 \\\\
x_5 & = & 5 & - & 5x_2 & + & x_4 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

We'll choose $x_2$ as our entering variable, which makes $x_5$ our leaving variable. $\text{PIVOT}$ then gives,

$$
\begin{array}{rcrcrcr}
z   & = & 5 & + & (7 / 5)x_4 & - & (2 / 5)x_5 \\\\
x_1 & = & 2 & + & (1 / 5)x_4 & + & (1 / 5)x_5 \\\\
x_2 & = & 1 & + & (4 / 5)x_4 & - & (1 / 5)x_5 \\\\
x_3 & = & 7 & - & (3 / 5)x_4 & - & (2 / 5)x_5 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

We'll choose $x_4$ as our entering variable, which makes $x_3$ our leaving variable. $\text{PIVOT}$ then gives,

$$
\begin{array}{rcrcrcr}
z   & = & (64 / 3) & - & (7 / 3)x_3 & - & (4 / 3)x_5 \\\\
x_1 & = & (34 / 3) & - & (4 / 3)x_3 & - & (1 / 3)x_5 \\\\
x_2 & = & (10 / 3) & - & (1 / 3)x_3 & - & (1 / 3)x_5 \\\\
x_4 & = & (35 / 3) & - & (5 / 3)x_3 & - & (2 / 3)x_5 \\\\
x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Now all coefficients in the objective function are negative, so the basic solution is the optimal solution. It is $(x_1, x_2) = (34 / 3, 10 / 3)$.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrl}
> \text{maxmize}    &   x_1 & - &   2x_2 \\\\
> \text{subject to} & \\\\
>                   &   x_1 & + &   2x_2 & \le &   4 \\\\
>                   & -2x_1 & - &   6x_2 & \le & -12 \\\\
>                   &       &   &    x_2 & \le &   1 \\\\
>                   &       & x_1, x_2 & & \ge &   0 & .
> \end{array}
> $$

answer:
The initial basic solution isn't feasible, so we will need to form the auxiliary linear program,

$$
\begin{array}{lrcrcrcrl}
\text{maxmize}    & -x_0 \\\\
\text{subject to} & \\\\
                  & -x_0 & + &  x_1 & + & 2x_2 & \le &   4 \\\\
                  & -x_0 & - & 2x_1 & - & 6x_2 & \le & -12 \\\\
                  & -x_0 &   &      & + &  x_2 & \le &   1 \\\\
                  &      & x_0, x_1, x_2 & & & & \ge &   0 & .
\end{array}
$$

Writing this linear program in slack form,

$$
\begin{array}{rcrcrcrcr}
z   & = &     & - & x_0 \\\\
x_3 & = &   4 & + & x_0 & - &  x_1 & - & 2x_2 \\\\
x_4 & = & -12 & + & x_0 & + & 2x_1 & + & 6x_2 \\\\
x_5 & = &   1 & + & x_0 &   &      & - &  x_2 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Next we make one call to $\text{PIVOT}$ where $x_0$ is the entering variable and $x_4$ is the leaving variable.

$$
\begin{array}{rcrcrcrcr}
z   & = & -12 & + & 2x_1 & + & 6x_2 & - & x_4 \\\\
x_0 & = &  12 & - & 2x_1 & - & 6x_2 & + & x_4 \\\\
x_3 & = &  16 & - & 3x_1 & - & 8x_2 & + & x_4 \\\\
x_5 & = &  13 & - & 2x_1 & - & 8x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

The basic solution is $(x_0, x_1, x_2, x_3, x_4, x_5) = (12, 0, 0, 16, 0, 13)$ which is feasible for the auxiliary program. Now we need to run $\text{SIMPLEX}$ to find the optimal objective value to $L_{aux}$. Let $x_1$ be our next entering variable. It is most constrained by $x_3$, which will be our leaving variable. After $\text{PIVOT}$, the new linear program is

$$
\begin{array}{rcrcrcrcr}
z   & = & -(4 / 3) & + & (2 / 3)x_2 & - & (2 / 3)x_3 & + & (1 / 3) x_4 \\\\
x_0 & = &  (4 / 3) & - & (2 / 3)x_2 & + & (2 / 3)x_3 & + & (1 / 3) x_4 \\\\
x_1 & = & (16 / 3) & - & (8 / 3)x_2 & - & (1 / 3)x_3 & + & (1 / 3) x_4 \\\\
x_5 & = &  (7 / 3) & - & (8 / 3)x_2 & + & (2 / 3)x_3 & + & (1 / 3) x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Every coefficient in the objective function is negative, so we take the basic solution $(x_0, x_1, x_2, x_3, x_4, x_5) = (4 / 3, 16 / 3, 0, 0, 0, 7 / 3)$ which is also optimal. Since $x_0 \ne 0$, the original linear program must be unfeasible.


question:
 Solve the following linear program using $\text{SIMPLEX}$:
>
> $$
> \begin{array}{lrcrcrl}
> \text{maxmize}    &  x_1 & + &   3x_2 \\\\
> \text{subject to} & \\\\
>                   & -x_1 & + &    x_2 & \le & -1 \\\\
>                   & -x_1 & - &    x_2 & \le & -3 \\\\
>                   & -x_1 & + &   4x_2 & \le &  2 \\\\
>                   &      & x_1, x_2 & & \ge &  0 & .
> \end{array}
> $$

answer:
The initial basic solution isn't feasible, so we will need to form the auxiliary linear program,

$$
\begin{array}{lrcrcrcrl}
\text{maxmize}    & -x_0 \\\\
\text{subject to} & \\\\
                  & -x_0 & - & x_1 & + &   x_2 & \le & -1 \\\\
                  & -x_0 & - & x_1 & - &   x_2 & \le & -3 \\\\
                  & -x_0 & - & x_1 & + &  4x_2 & \le &  2 \\\\
                  &      & x_0, x_1, x_2 & & & & \ge &  0 & .
\end{array}
$$

Writing this linear program in slack form,

$$
\begin{array}{rcrcrcrcr}
z   & = &    & - & x_0 \\\\
x_3 & = & -1 & + & x_0 & + & x_1 & - &  x_2 \\\\
x_4 & = & -3 & + & x_0 & + & x_1 & + &  x_2 \\\\
x_5 & = &  2 & + & x_0 & + & x_1 & - & 4x_2 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Next we make one call to $\text{PIVOT}$ where $x_0$ is the entering variable and $x_4$ is the leaving variable.

$$
\begin{array}{rcrcrcrcr}
z   & = & -3 & + & x_1 & + &  x_2 & - & x_4 \\\\
x_0 & = &  3 & - & x_1 & - &  x_2 & + & x_4 \\\\
x_3 & = &  2 &   &     & - & 2x_2 & + & x_4 \\\\
x_5 & = &  5 &   &     & - & 5x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Let $x_1$ be our entering variable. Then $x_0$ is our leaving variable, and we have

$$
\begin{array}{rcrcrcrcr}
z   & = &   & - & x_0 \\\\
x_1 & = & 3 & - & x_0 & - &  x_2 & + & x_4 \\\\
x_3 & = & 2 &   &     & - & 2x_2 & + & x_4 \\\\
x_5 & = & 5 &   &     & - & 5x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

The basic solution is feasible, and optimal for $L_{aux}$, so we return this and run $\text{SIMPLEX}$. Updating the objective function and setting $x_0 = 0$ gives

$$
\begin{array}{rcrcrcr}
z   & = & 3 & + & 2x_2 & + & x_4 \\\\
x_1 & = & 3 & - &  x_2 & + & x_4 \\\\
x_3 & = & 2 & - & 2x_2 & + & x_4 \\\\
x_5 & = & 5 & - & 5x_2 & + & x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

We'll choose $x_2$ as our entering variable, which makes $x_3$ our leaving variable. This gives

$$
\begin{array}{rcrcrcr}
z   & = & 5 & - &        x_3 & + &       2x_4 \\\\
x_1 & = & 2 & + & (1 / 2)x_3 & + & (1 / 2)x_4 \\\\
x_2 & = & 1 & - & (1 / 2)x_3 & + & (1 / 2)x_4 \\\\
x_5 & = &   &   & (5 / 2)x_3 & - & (3 / 2)x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Next we use $x_4$ as our entering variable, which makes $x_5$ our leaving variable. This gives

$$
\begin{array}{rcrcrcr}
z   & = & 5 & + & (7 / 3)x_3 & - & (4 / 3)x_5 \\\\
x_1 & = & 2 & + & (4 / 3)x_3 & - & (1 / 3)x_5 \\\\
x_2 & = & 1 & + & (1 / 3)x_3 & - & (1 / 3)x_5 \\\\
x_4 & = &   &   & (5 / 3)x_3 & - & (2 / 3)x_5 \\\\
x_0, x_1, x_2, x_3, x_4, x_5 & \ge & 0 & .
\end{array}
$$

Finally, we would like to choose $x_3$ as our entering variable, but every coefficient on $x_3$ is positive, so $\text{SIMPLEX}$ returns that the linear program is unbounded.


question: Solve the linear program given in $\text{(29.6)}$–$\text{(29.10)}$.
answer:
We first put the linear program in standard form,

$$
\begin{array}{lrcrcrcrcrl}
\text{maxmize}    &   x_1 & + &  x_2 & + &   x_3 & + &   x_4 \\\\
\text{subject to} & \\\\
                  &  2x_1 & - & 8x_2 &   &       & - & 10x_4 & \le &  -50 \\\\
                  & -5x_1 & - & 2x_2 &   &       &   &       & \le & -100 \\\\
                  & -3x_1 & + & 5x_2 & - & 10x_3 & + &  2x_4 & \le &  -25 \\\\
                  &       & x_1, x_2, x_3, x_4 & & & & &     & \ge &    0 & .
\end{array}
$$

The initial basic solution isn't feasible, so we will need to form the auxiliary linear program.

$$
\begin{array}{rcrcrcrcrcrrcl}
z   & = &      & - & x_0 \\\\
x_5 & = &  -50 & + & x_0 & - & 2x_1 & + & 8x_2 &   &       & + & 10x_4 \\\\
x_6 & = & -100 & + & x_0 & + & 5x_1 & + & 2x_2 \\\\
x_7 & = &  -25 & + & x_0 & + & 3x_1 & - & 5x_2 & + & 10x_3 & - &  2x_4 \\\\
x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7 & \ge & 0 & .
\end{array}
$$

The index of the minimum $b_i$ is $2$, so we take $x_0$ to be our entering variable and $x_6$ to be our leaving variable. The call to $\text{PIVOT}$ on line 8 yields

$$
\begin{array}{rcrcrcrcrcrrcl}
z   & = & -100 & + & 5x_1 & + & 2x_2 &   &       &   &       & - & x_6 \\\\
x_0 & = &  100 & - & 5x_1 & - & 2x_2 &   &       &   &       & + & x_6 \\\\
x_5 & = &   50 & - & 7x_1 & + & 8x_2 &   &       & + & 10x_4 & + & x_6 \\\\
x_7 & = &   75 & - & 2x_1 & - & 7x_2 & + & 10x_3 & - &  2x_4 & + & x_6 \\\\
x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7 & \ge & 0 & .
\end{array}
$$

Next we'll take $x_2$ to be our entering variable and $x_5$ to be our leaving variable. The call to $\text{PIVOT}$ yields

$$
\begin{array}{rcrcrcrcrcrcr}
z   & = & -225 / 2 & + & (27 / 4)x_1 &   &       & - & (10 / 4)x_4 & + & (1 / 4)x_5 & - &  (5 / 4)x_6 \\\\
x_0 & = &  225 / 2 & - & (27 / 4)x_1 &   &       & + & (10 / 4)x_4 & - & (1 / 4)x_5 & + &  (5 / 4)x_6 \\\\
x_2 & = &  -50 / 8 & + &  (7 / 8)x_1 &   &       & - & (10 / 8)x_4 & + & (1 / 8)x_5 & - &  (1 / 8)x_6 \\\\
x_7 & = &  475 / 4 & - & (65 / 8)x_1 & + & 10x_3 & + & (54 / 8)x_4 & - & (7 / 8)x_5 & + & (15 / 8)x_6 \\\\
x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7 & \ge & 0 & .
\end{array}
$$

The work gets rather messy, but $\text{INITIALIZE-SIMPLEX}$ does eventually give a feasible solution to the linear program, and after running the simplex method we find that $(x_1, x_2, x_3, x_4) = (175 / 11, 225 / 22, 125 / 44, 0)$ is an optimal solution to the original linear programming problem.


question:
 Consider the following $1$-variable linear program, which we call $P$:
>
> $$
> \begin{array}{lrcrl}
> \text{maximize}   & tx \\\\
> \text{subject to} & rx & \le & s \\\\
>                   & x  & \ge & 0 & ,
> \end{array}
> $$
>
> where $r$, $s$, and $t$ are arbitrary real numbers. Let $D$ be the dual of $P$.
>
> State for which values of $r$, $s$, and $t$ you can assert that
>
> 1. Both $P$ and $D$ have optimal solutions with finite objective values.
> 2. $P$ is feasible, but $D$ is infeasible.
> 3. $D$ is feasible, but $P$ is infeasible.
> 4. Neither $P$ nor $D$ is feasible.

answer:
1. One option is that $r = 0$, $s \ge 0$ and $t \le 0$. Suppose that $r > 0$, then, if we have that $s$ is non-negative and $t$ is non-positive, it will be as we want.
2. We will split into two cases based on $r$. If $r = 0$, then this is exactly when $t$ is non-positive and $s$ is non-negative. The other possible case is that $r$ is negative, and $t$ is positive. In which case, because $r$ is negative, we can always get $rx$ as small as we want so s doesn't matter, however, we can never make $rx$ positive so it can never be $\ge t$.
3. Again, we split into two possible cases for $r$. If $r = 0$, then it is when $t$ is nonnegative and $s$ is non-positive. The other possible case is that $r$ is positive, and $s$ is negative. Since $r$ is positive, $rx$ will always be non-negative, so it cannot be $\le s$. But since $r$ is positive, we have that we can always make $rx$ as big as we want, in particular, greater than $t$.
4. If we have that $r = 0$ and $t$ is positive and $s$ is negative. If $r$ is nonzero, then we can always either make $rx$ really big or really small depending on the sign of $r$, meaning that either the primal or the dual would be feasable.


question: Formulate the dual of the linear program given in Exercise 29.3-5.
answer:
By just transposing $A$, swapping $b$ and $c$, and switching the maximization to a minimization, we want to minimize $20y_1 + 12y_2 + 16y_3$ subject to the constrain

$$
\begin{aligned}
    y_1 + y_2 & \ge 18 \\\\
    y_1 + y_3 & \ge 12.5 \\\\
y_1, y_2, y_3 & \ge 0
\end{aligned}
$$


question:
 Suppose that we have a linear program that is not in standard form. We could produce the dual by first converting it to standard form, and then taking the dual. It would be more convenient, however, to be able to produce the dual directly. Explain how we can directly take the dual of an arbitrary linear program.

answer:
By working through each aspect of putting a general linear program into standard form, as outlined on page 852, we can show how to deal with transforming each into the dual individually. If the problem is a minimization instead of a maximization, replace $c_j$ by $−c_j$ in $\text{(29.84)}$. If there is a lack of nonnegativity constraint on $x_j$ we duplicate the $j$th column of $A$, which corresponds to duplicating the $j$th row of $A^{\text T}$. If there is an equality constraint for $b_i$, we convert it to two inequalities by duplicating then negating the $i$th column of $A^{\text T}$, duplicating then negating the $i$th entry of $b$, and adding an extra $y_i$ variable. We handle the greater-than-or-equal-to sign $\sum_{i = 1}^n a_{ij}x_j \ge b_i$ by negating $i$th column of $A^{\text T}$ and negating $b_i$. Then we solve the dual problem of minimizing $b^{\text T}y$ subject to $A^{\text T}y$ and $y \ge 0$.


question:
 Write down the dual of the maximum-flow linear program, as given in lines $\text{(29.47)}$–$\text{(29.50)}$ on page 860. Explain how to interpret this formulation as a minimum-cut problem.

answer:
First, we'll convert the linear program for maximum flow described in equation $\text{(29.47)}$-$\text{(29.50)}$ into standard form. The objective function says that $c$ is a vector indexed by a pair of vertices, and it is positive one if $s$ is the first index and negative one if $s$ is the second index (zero if it is both). Next, we'll modify the constraints by switching the equalities over into inequalities to get

$$
\begin{array}{rcll}
               f_{uv} & \le & c(u, v)               & \text{ for each } u, v \in V \\\\
\sum_{u \in V} f_{vu} & \le & \sum_{u \in V} f_{uv} & \text{ for each } v \in V - \\{s, t\\} \\\\
\sum_{u \in V} f_{vu} & \ge & \sum_{u \in V} f_{uv} & \text{ for each } v \in V - \\{s, t\\} \\\\
               f_{uv} & \ge & 0                     & \text{ for each } u, v \in V
\end{array}
$$

Then, we'll convert all but the last set of the inequalities to be $\le$ by multiplying the third line by $-1$.

$$
\begin{array}{rcll}
                f_{uv} & \le & c(u, v)                & \text{ for each } u, v \in V \\\\
\sum_{u \in V}  f_{vu} & \le & \sum_{u \in V}  f_{uv} & \text{ for each } v \in V - \\{s, t\\} \\\\
\sum_{u \in V} -f_{vu} & \le & \sum_{u \in V} -f_{uv} & \text{ for each } v \in V - \\{s, t\\} \\\\
                f_{uv} & \ge & 0                      & \text{ for each } u, v \in V
\end{array}
$$

Finally, we'll bring all the variables over to the left to get

$$
\begin{array}{rcll}
                                         f_{uv} & \le & c(u, v) & \text{ for each } u, v \in V \\\\
\sum_{u \in V}  f_{vu} - \sum_{u \in V}  f_{uv} & \le & 0       & \text{ for each } v \in V - \\{s, t\\} \\\\
\sum_{u \in V} -f_{vu} - \sum_{u \in V} -f_{uv} & \le & 0       & \text{ for each } v \in V - \\{s, t\\} \\\\
                f_{uv}                          & \ge & 0       & \text{ for each } u, v \in V
\end{array}
$$

Now, we can finally write down our $A$ and $b$. $A$ will be a $|V|^2 \times |V|^2 + 2|V| − 4$ matrix built from smaller matrices $A_1$ and $A_2$ which correspond to the three types of constraints that we have (of course, not counting the non-negativity constraints). We will let $g(u, v)$ be any bijective mapping from $V \times V$ to $[|V|^2]$. We'll also let $h$ be any bijection from $V - \\{s, t\\}$ to $[|V| - 2]$.

$$
A =
\begin{pmatrix}
 A_1 \\\\
 A_2 \\\\
-A_2
\end{pmatrix},
$$

where $A_1$ is defined as having its row $g(u, v)$ be all zeroes except for having the value $1$ at at the $g(u, v)$th entry. We define $A_2$ to have it's row $h(u)$ be equal to $1$ at all columns $j$ for which $j = g(v, u)$ for some $v$ and equal to $-1$ at all columns $j$ for which $j = g(u, v)$ for some $v$. Lastly, we mention that $b$ is defined as having it's $j$th entry be equal to $c(u, v)$ if $j = g(u, v)$ and zero if $j > |V|^2$.

Now that we have placed the linear program in standard form, we can take its dual. We want to minimize $\sum_{i = 1}^{|V|^2 + 2|V| - 2} b_iy_i$ given the constraints that all the $y$ values are non-negative, and $A^{\text T} y \ge c$.


question:
 Write down the dual of the minimum-cost-flow linear program, as given in lines $\text{(29.51)}$–$\text{(29.52)}$ on page 862. Explain how to interpret this problem in terms of graphs and flows.

answer:
First we need to put the linear programming problem into standard form, as follows:

$$
\begin{array}{lrcrl}
\text{maximize}   & \sum_{(u, v) \in E} -a(u, v) f_{uv} \\\\
\text{subject to} & \\\\
    & f_{uv}                                        & \le & c(u, v) & \text{ for each } u, v \in V \\\\
    & \sum_{v \in V} f_{vu} - \sum_{v \in V} f_{uv} & \le & 0       & \text{ for each } u \in V - \\{s, t\\} \\\\
    & \sum_{v \in V} f_{uv} - \sum_{v \in V} f_{vu} & \le & 0       & \text{ for each } u \in V - \\{s, t\\} \\\\
    & \sum_{v \in V} f_{sv} - \sum_{v \in V} f_{vs} & \le & d \\\\
    & \sum_{v \in V} f_{vs} - \sum_{v \in V} f_{sv} & \le & -d \\\\
    & f_{uv} & \ge & 0 & .
\end{array}
$$

We now formulate the dual problem. Let the vertices be denoted $v_1, v_2, \dots, v_n, s, t$ and the edges be $e_1, e_2, \dots, e_k$. Then we have $b_i = c(e_i)$ for $1 \le i \le k$, $b_i = 0$ for $k + 1 \le i \le k + 2n$, $b_{k + 2n + 1} = d$, and $b_{k + 2n + 2} = −d$. We also have $c_i = −a(e_i)$ for $1 \le i \le k$. For notation, let $j.left$ denote the tail of edge $e_j$ and $j.right$ denote the head. Let $\chi_s(e_j) = 1$ if $e_j$ enters $s$, set it equal to $-1$ if $e_j$ leaves $s$, and set it equal to $0$ if $e_j$ is not incident with $s$. The dual problem is:

$$
\begin{array}{ll}
\text{minimize}   & \sum_{i = 1}^k c(e_j)y_i + dy_{k + 2n + 1} - dy_{k + 2n + 2} \\\\
\text{subject to} & \\\\
    & y_j + y_{k + e_j.right} - y_{k + j.left} - y_{k + n + e_j.right} + y_{k + n + e_j.left} - \chi_s(e_j) y_{k + 2n + 1} + \chi_s(e_j) y_{k + 2n + 2} \ge -a(e_j),
\end{array}
$$

where $j$ runs between $1$ and $k$. There is one constraint equation for each edge $e_j$.


question: Show that the dual of the dual of a linear program is the primal linear program.
answer:
Suppose that our original linear program is in standard form for some $A$, $b$, $c$. Then, the dual of this is to minimize $\sum_{i = 1}^m b_iy_i$ subject to $A^{\text T} y \ge c$ This can be rewritten as wanting to maximize $\sum_{i = 1}^m (−b_i)y_i$ subject to $(−A)^{\text T} y \le −c$. Since this is a standard form, we can take its dual easily, it is minimize $\sum_{j = 1}^n (−c_j)x_j$ subject to $(−A)x \ge −b$. This is the same as minimizing $\sum_{j = 1}^n c_jx_j$ subject to $Ax \le b$, which was the original linear program.


question:
 Which is a more efficient way to determine the optimal number of multiplications in a matrix-chain multiplication problem: enumerating all the ways of parenthesizing the product and computing the number of multiplications for each, or running $\text{RECURSIVE-MATRIX-CHAIN}$? Justify your answer.

answer:
Running $\text{RECURSIVE-MATRIX-CHAIN}$ is asymptotically more efficient than enumerating all the ways of parenthesizing the product and computing the number of multiplications for each.

Consider the treatment of subproblems by each approach:

1. For each possible place to split the matrix chain, the enumeration approach finds all ways to parenthesize the left half, finds all ways to parenthesize the right half, and looks at all possible combinations of the left half with the right half. The amount of work to look at each combination of left and right half subproblem results is thus the product of the number of ways to parenthesize the left half and the number of ways to parenthesize the right half.

2. For each possible place to split the matrix chain, $\text{RECURSIVE-MATRIX-CHAIN}$ finds the best way to parenthesize the left half, finds the best way to parenthesize the right half, and combines just those two results. Thus the amount of work to combine the left and right half subproblem results is $O(1)$.


question:
 Draw the recursion tree for the $\text{MERGE-SORT}$ procedure from Section 2.3.1 on an array of $16$ elements. Explain why memoization fails to speed up a good divide-and-conquer algorithm such as $\text{MERGE-SORT}$.

answer:
Draw a recursion tree. 
![](../img/15.3-2.png)
The $\text{MERGE-SORT}$ procedure performs at most a single call to any pair of indices of the array that is being sorted. In other words, the subproblems do not overlap and therefore memoization will not improve the running time.


question:
 Consider a variant of the matrix-chain multiplication problem in which the goal is to parenthesize the sequence of matrices so as to maximize, rather than minimize, the number of scalar multiplications. Does this problem exhibit optimal substructure?

answer:
Yes, this problem also exhibits optimal substructure. If we know that we need the subproduct $(A_l \cdot A_r)$, then we should still find the most expensive way to compute it — otherwise, we could do better by substituting in the most expensive way.


question:
 As stated, in dynamic programming we first solve the subproblems and then choose which of them to use in an optimal solution to the problem. Professor Capulet claims that we do not always need to solve all the subproblems in order to find an optimal solution. She suggests that we can find an optimal solution to the matrix-chain multiplication problem by always choosing the matrix $A_k$ at which to split the subproduct $A_i A_{i + 1} \cdots A_j$ (by selecting $k$ to minimize the quantity $p_{i - 1} p_k p_j$) _before_ solving the subproblems. Find an instance of the matrix-chain multiplication problem for which this greedy approach yields a suboptimal solution.

answer:
Suppose that we are given matrices $A_1$, $A_2$, $A_3$, and $A_4$ with dimensions such that

$$p_0, p_1, p_2, p_3, p_4 = 1000, 100, 20, 10, 1000.$$

Then $p_0 p_k p_4$ is minimized when $k = 3$, so we need to solve the subproblem of multiplying $A_1 A_2 A_3$, and also $A_4$ which is solved automatically. By her algorithm, this is solved by splitting at $k = 2$. Thus, the full parenthesization is $(((A_1A_2)A_3)A_4)$.

This requires

$$1000 \cdot 100 \cdot 20 + 1000 \cdot 20 \cdot 10 + 1000 \cdot 10 \cdot 1000 = 12200000$$

scalar multiplications.

On the other hand, suppose we had fully parenthesized the matrices to multiply as $((A_1(A_2A_3))A_4)$. Then we would only require

$$100 \cdot 20 \cdot 10 + 1000 \cdot 100 \cdot 10 + 1000 \cdot 10 \cdot 1000 = 11020000$$

scalar multiplications, which is fewer than Professor Capulet's method.

Therefore her greedy approach yields a suboptimal solution.


question:
 Suppose that in the rod-cutting problem of Section 15.1, we also had limit $l_i$ on the number of pieces of length $i$ that we are allowed to produce, for $i = 1, 2, \ldots, n$. Show that the optimal-substructure property described in Section 15.1 no longer holds.

answer:
The optimal substructure property doesn't hold because the number of pieces of length $i$ used on one side of the cut affects the number allowed on the other. That is, there is information about the particular solution on one side of the cut that changes what is allowed on the other.

To make this more concrete, suppose the rod was length $4$, the values were $l_1 = 2, l_2 = l_3 = l_4 = 1$, and each piece has the same worth regardless of length. Then, if we make our first cut in the middle, we have that the optimal solution for the two rods left over is to cut it in the middle, which isn't allowed because it increases the total number of rods of length $1$ to be too large.


question:
 Imagine that you wish to exchange one currency for another. You realize that instead of directly exchanging one currency for another, you might be better off making a series of trades through other currencies, winding up with the currency you want. Suppose that you can trade $n$ different currencies, numbered $1, 2, \ldots, n$, where you start with currency $1$ and wish to wind up with currency $n$. You are given, for each pair of currencies $i$ and $j$ , an exchange rate $r_{ij}$, meaning that if you start with $d$ units of currency $i$ , you can trade for $dr_{ij}$ units of currency $j$. A sequence of trades may entail a commission, which depends on the number of trades you make. Let $c_k$ be the commission that you are charged when you make $k$ trades. Show that, if $c_k = 0$ for all $k = 1, 2, \ldots, n$, then the problem of finding the best sequence of exchanges from currency $1$ to currency $n$ exhibits optimal substructure. Then show that if commissions $c_k$ are arbitrary values, then the problem of finding the best sequence of exchanges from currency $1$ to currency $n$ does not necessarily exhibit optimal substructure.

answer:
First we assume that the commission is always zero. Let $k$ denote a currency which appears in an optimal sequence $s$ of trades to go from currency $1$ to currency $n$. $p_k$ denote the first part of this sequence which changes currencies from $1$ to $k$ and $q_k$ denote the rest of the sequence. Then $p_k$ and $q_k$ are both optimal sequences for changing from $1$ to $k$ and $k$ to $n$ respectively. To see this, suppose that $p_k$ wasn't optimal but that $p_k'$ was. Then by changing currencies according to the sequence $p_k'q_k$ we would have a sequence of changes which is better than $s$, a contradiction since $s$ was optimal. The same argument applies to $q_k$.

Now suppose that the commissions can take on arbitrary values. Suppose we have currencies $1$ through $6$, and $r_{12} = r_{23} = r_{34} = r_{45} = 2$, $r_{13} = r_{35} = 6$, and all other exchanges are such that $r_{ij} = 100$. Let $c_1 = 0$, $c_2 = 1$, and $c_k = 10$ for $k \ge 3$.

The optimal solution in this setup is to change $1$ to $3$, then $3$ to $5$, for a total cost of $13$. An optimal solution for changing $1$ to $3$ involves changing $1$ to $2$ then $2$ to $3$, for a cost of $5$, and an optimal solution for changing $3$ to $5$ is to change $3$ to $4$ then $4$ to $5$, for a total cost of $5$. However, combining these optimal solutions to subproblems means making more exchanges overall, and the total cost of combining them is $18$, which is not optimal.


answer: $\langle 1, 0, 0, 1, 1, 0 \rangle$ or $\langle 1, 0, 1, 0, 1, 0 \rangle$.
question:
 Determine an $\text{LCS}$ of $\langle 1, 0, 0, 1, 0, 1, 0, 1 \rangle$ and $\langle 0, 1, 0, 1, 1, 0, 1, 1, 0 \rangle$.


question:
 Give pseudocode to reconstruct an $\text{LCS}$ from the completed $c$ table and the original sequences $X = \langle x_1, x_2, \ldots, x_m \rangle$ and $Y = \langle y_1, y_2, \ldots, y_n \rangle$ in $O(m + n)$ time, without using the $b$ table.

answer:
```cpp
PRINT-LCS(c, X, Y, i, j)
    if c[i, j] == 0
        return
    if X[i] == Y[j]
        PRINT-LCS(c, X, Y, i - 1, j - 1)
        print X[i]
    else if c[i - 1, j] > c[i, j - 1]
        PRINT-LCS(c, X, Y, i - 1, j)
    else
        PRINT-LCS(c, X, Y, i, j - 1)
```


question: Give a memoized version of $\text{LCS-LENGTH}$ that runs in $O(mn)$ time.
answer:
```cpp
MEMOIZED-LCS-LENGTH(X, Y, i, j)
    if c[i, j] > -1
        return c[i, j]
    if i == 0 or j == 0
        return c[i, j] = 0
    if x[i] == y[j]
        return c[i, j] = LCS-LENGTH(X, Y, i - 1, j - 1) + 1
    return c[i, j] = max(LCS-LENGTH(X, Y, i - 1, j), LCS-LENGTH(X, Y, i, j - 1))
```


question:
 Show how to compute the length of an $\text{LCS}$ using only $2 \cdot \min(m, n)$ entries in the $c$ table plus $O(1)$ additional space. Then show how to do the same thing, but using $\min(m, n)$ entries plus $O(1)$ additional space.

answer:
Since we only use the previous row of the $c$ table to compute the current row, we compute as normal, but when we go to compute row $k$, we free row $k - 2$ since we will never need it again to compute the length. To use even less space, observe that to compute $c[i, j]$, all we need are the entries $c[i − 1, j]$, $c[i − 1, j − 1]$, and $c[i, j − 1]$. Thus, we can free up entry-by-entry those from the previous row which we will never need again, reducing the space requirement to $\min(m, n)$. Computing the next entry from the three that it depends on takes $O(1)$ time and space.


question:
 Give an $O(n^2)$-time algorithm to find the longest monotonically increasing subsequence of a sequence of $n$ numbers.

answer:
Given a list of numbers $L$, make a copy of $L$ called $L'$ and then sort $L'$.

```cpp
PRINT-LCS(c, X, Y)
    n = c[X.length, Y.length]
    let s[1..n] be a new array
    i = X.length
    j = Y.length
    while i > 0 and j > 0
        if x[i] == y[j]
            s[n] = x[i]
            n = n - 1
            i = i - 1
            j = j - 1
        else if c[i - 1, j] ≥ c[i, j - 1]
            i = i - 1
        else j = j - 1
    for i = 1 to s.length
        print s[i]
```

```cpp
MEMO-LCS-LENGTH-AUX(X, Y, c, b)
    m = |X|
    n = |Y|
    if c[m, n] != 0 or m == 0 or n == 0
        return
    if x[m] == y[n]
        b[m, n] = ↖
        c[m, n] = MEMO-LCS-LENGTH-AUX(X[1..m - 1], Y[1..n - 1], c, b) + 1
    else if MEMO-LCS-LENGTH-AUX(X[1..m - 1], Y, c, b) ≥ MEMO-LCS-LENGTH-AUX(X, Y[1..n - 1], c, b)
        b[m, n] = ↑
        c[m, n] = MEMO-LCS-LENGTH-AUX(X[1..m - 1], Y, c, b)
    else
        b[m, n] = ←
        c[m, n] = MEMO-LCS-LENGTH-AUX(X, Y[1..n - 1], c, b)
```

```cpp
MEMO-LCS-LENGTH(X, Y)
    let c[1..|X|, 1..|Y|] and b[1..|X|, 1..|Y|] be new tables
    MEMO-LCS-LENGTH-AUX(X, Y, c, b)
    return c and b
```

Then, just run the $\text{LCS}$ algorithm on these two lists. The longest common subsequence must be monotone increasing because it is a subsequence of $L'$ which is sorted. It is also the longest monotone increasing subsequence because being a subsequence of $L'$ only adds the restriction that the subsequence must be monotone increasing. Since $|L| = |L'| = n$, and sorting $L$ can be done in $o(n^2)$ time, the final running time will be $O(|L||L'|) = O(n^2)$.


question:
 Give an $O(n\lg n)$-time algorithm to find the longest monotonically increasing subsequence of a sequence of $n$ numbers. ($\textit{Hint:}$ Observe that the last element of a candidate subsequence of length $i$ is at least as large as the last element of a candidate subsequence of length $i - 1$. Maintain candidate subsequences by linking them through the input sequence.)

answer:
The algorithm $\text{LONG-MONOTONIC}(A)$ returns the longest monotonically increasing subsequence of $A$, where $A$ has length $n$.

The algorithm works as follows: a new array B will be created such that $B[i]$ contains the last value of a longest monotonically increasing subsequence of length $i$. A new array $C$ will be such that $C[i]$ contains the monotonically increasing subsequence of length $i$ with smallest last element seen so far.

To analyze the runtime, observe that the entries of $B$ are in sorted order, so we can execute line 9 in $O(\lg n)$ time. Since every other line in the for-loop takes constant time, the total run-time is $O(n\lg n)$.

```cpp
LONG-MONOTONIC(A)
    let B[1..n] be a new array where every value = ∞
    let C[1..n] be a new array
    L = 1
    for i = 1 to n
        if A[i] < B[1]
            B[1] = A[i]
            C[1].head.key = A[i]
        else
            let j be the largest index of B such that B[j] < A[i]
            B[j + 1] = A[i]
            C[j + 1] = C[j]
            INSERT(C[j + 1], A[i])
            if j + 1 > L
                L = L + 1
    print C[L]
```


question:
 Show that equation $\text{(15.4)}$ follows from equation $\text{(15.3)}$ and the initial condition $T(0) = 1$.

answer:
- For $n = 0$, this holds since $2^0 = 1$.
- For $n > 0$, substituting into the recurrence, we have

    $$
    \begin{aligned}
    T(n) & = 1 + \sum_{j = 0}^{n - 1} 2^j \\\\
         & = 1 + (2^n - 1) \\\\
         & = 2^n.
    \end{aligned}
    $$


question:
 Show, by means of a counterexample, that the following "greedy" strategy does not always determine an optimal way to cut rods. Define the **_density_** of a rod of length $i$ to be $p_i / i$, that is, its value per inch. The greedy strategy for a rod of length $n$ cuts off a first piece of length $i$, where $1 \le i \le n$, having maximum density. It then continues by applying the greedy strategy to the remaining piece of length $n - i$.

answer:
The counterexample:

$$
\begin{array}{c|cccc}
\text{length $i$}  &  1 &  2 &  3 &  4 \\\\
\hline
\text{price $p_i$} &  1 & 20 & 33 & 36 \\\\
p_i / i            &  1 & 10 & 11 &  9
\end{array}
$$


question:
 Consider a modification of the rod-cutting problem in which, in addition to a price $p_i$ for each rod, each cut incurs a fixed cost of $c$. The revenue associated with a solution is now the sum of the prices of the pieces minus the costs of making the cuts. Give a dynamic-programming algorithm to solve this modified problem.

answer:
We can modify $\text{BOTTOM-UP-CUT-ROD}$ algorithm from section 15.1 as follows:

```cpp
MODIFIED-CUT-ROD(p, n, c)
    let r[0..n] be a new array
    r[0] = 0
    for j = 1 to n
        q = p[j]
        for i = 1 to j - 1
            q = max(q, p[i] + r[j - i] - c)
        r[j] = q
    return r[n]
```

We need to account for cost $c$ on every iteration of the loop in lines 5-6 but the last one, when $i = j$ (no cuts).

We make the loop run to $j - 1$ instead of $j$, make sure $c$ is subtracted from the candidate revenue in line 6, then pick the greater of current best revenue $q$ and $p[j]$ (no cuts) in line 7.


question: Modify $\text{MEMOIZED-CUT-ROD}$ to return not only the value but the actual solution, too.
answer:
```cpp
MEMOIZED-CUT-ROD(p, n)
    let r[0..n] and s[0..n] be new arrays
    for i = 0 to n
        r[i] = -∞
    (val, s) = MEMOIZED-CUT-ROD-AUX(p, n, r, s)
    print "The optimal value is" val "and the cuts are at" s
    j = n
    while j > 0
        print s[j]
        j = j - s[j]
```

```cpp
MEMOIZED-CUT-ROD-AUX(p, n, r, s)
    if r[n] ≥ 0
        return r[n]
    if n == 0
        q = 0
    else q = -∞
        for i = 1 to n
            (val, s) = MEMOIZED-CUT-ROD-AUX(p, n - i, r, s)
            if q < p[i] + val
                q = p[i] + val
                s[n] = i
    r[n] = q
    return (q, s)
```


question:
 The Fibonacci numbers are defined by recurrence $\text{(3.22)}$. Give an $O(n)$-time dynamic-programming algorithm to compute the nth Fibonacci number. Draw the subproblem graph. How many vertices and edges are in the graph?

answer:
```cpp
FIBONACCI(n)
    let fib[0..n] be a new array
    fib[0] = 1
    fib[1] = 1
    for i = 2 to n
        fib[i] = fib[i - 1] + fib[i - 2]
    return fib[n]
```

There are $n + 1$ vertices in the subproblem graph, i.e., $v_0, v_1, \dots, v_n$.

- For $v_0, v_1$, each has $0$ leaving edge.
- For $v_2, v_3, \dots, v_n$, each has $2$ leaving edges.

Thus, there are $2n - 2$ edges in the subproblem graph.


answer: $$((5 \times 10)(10 \times 3))(((3 \times 12)(12 \times 5))((5 \times 50)(50 \times 6))).$$
question:
 Find an optimal parenthesization of a matrix-chain product whose sequence of dimensions is $\langle 5, 10, 3, 12, 5, 50, 6 \rangle$.


question:
 Give a recursive algorithm $\text{MATRIX-CHAIN-MULTIPLY}(A, s, i, j)$ that actually performs the optimal matrix-chain multiplication, given the sequence of matrices $\langle A_1, A_2, \ldots ,A_n \rangle$, the $s$ table computed by $\text{MATRIX-CHAIN-ORDER}$, and the indices $i$ and $j$. (The initial call would be $\text{MATRIX-CHAIN-MULTIPLY}(A, s, 1, n)$.)

answer:
```cpp
MATRIX-CHAIN-MULTIPLY(A, s, i, j)
    if i == j
        return A[i]
    if i + 1 == j
        return A[i] * A[j]
    b = MATRIX-CHAIN-MULTIPLY(A, s, i, s[i, j])
    c = MATRIX-CHAIN-MULTIPLY(A, s, s[i, j] + 1, j)
    return b * c
```


question:
 Use the substitution method to show that the solution to the recurrence $\text{(15.6)}$ is $\Omega(2^n)$.

answer:
Suppose $P(n) \ge c2^n$,

$$
\begin{aligned}
P(n) & \ge   \sum_{k = 1}^{n - 1} c2^k \cdot c2^{n - k} \\\\
     & =     \sum_{k = 1}^{n - 1} c^2 2^n \\\\
     & =     c^2 (n - 1) 2^n \\\\
     & \ge   c^2 2^n & (n > 1) \\\\
     & \ge   c 2^n.  & (c \ge 1)
\end{aligned}
$$


question:
 Describe the subproblem graph for matrix-chain multiplication with an input chain of length $n$. How many vertices does it have? How many edges does it have, and which edges are they?

answer:
The vertices of the subproblem graph are the ordered pair $v_{ij}$, where $i \le j$.

- If $i = j$, the vertex $v_{ij}$ has no output edge.
- If $i < j$, for each $k$, s.t. $i \le k < j$, the subproblem graph contains edges $(v_{ij}, v_{ik})$ and $(v_{ij}, v_{k+1, j})$, and these edges indicate that to solve the subproblem of optimally parenthesizing the product $A_i \cdots A_j$, we need to solve subproblems of optimally parenthesizing the products $A_i \cdots A_k$ and $A_{k + 1} \cdots A_j$.

The number of vertices is

$$\sum_{i = 1}^n \sum_{j = i}^n = \frac{n(n + 1)}{2}.$$

The number of edges is

$$\sum_{i = 1}^n \sum_{j = i}^n (j - i) = \frac{(n - 1)n(n + 1)}{6}.$$


question:
 Let $R(i, j)$ be the number of times that table entry $m[i, j]$ is referenced while computing other table entries in a call of $\text{MATRIX-CHAIN-ORDER}$. Show that the total number of references for the entire table is
>
> $$\sum_{i = 1}^n \sum_{j = i}^n R(i, j) = \frac{n^3 - n}{3}.$$
>
> ($\textit{Hint:}$ You may find equation $\text{(A.3)}$ useful.)

answer:
We count the number of times that we reference a different entry in $m$ than the one we are computing, that is, $2$ times the number of times that line 10 runs.

$$
\begin{aligned}
\sum_{l = 2}^n \sum_{i = 1}^{n - l + 1} \sum_{k = i}^{i + l - 2} 2
    & = \sum_{l = 2}^n \sum_{i = 1}^{n - l + 1} 2(l - 1) \\\\
    & = \sum_{l = 2}^n 2(l - 1)(n - l + 1) \\\\
    & = \sum_{l = 1}^{n - 1} 2l(n - l) \\\\
    & = 2n \sum_{l = 1}^{n - 1} l - 2 \sum_{l = 1}^{n - 1} l^2 \\\\
    & = n^2(n - 1) - 2 \cdot \frac{(n - 1)n(2n - 1)}{6} \\\\
    & = n^3 - n^2 - \frac{2n^3 - 3n^2 + n}{3} \\\\
    & = \frac{n^3 - n}{3}.
\end{aligned}
$$


question:
 Show that a full parenthesization of an $n$-element expression has exactly $n - 1$ pairs of parentheses.

answer:
We proceed by induction on the number of matrices. A single matrix has no pairs of parentheses. Assume that a full parenthesization of an $n$-element expression has exactly $n − 1$ pairs of parentheses. Given a full parenthesization of an $(n + 1)$-element expression, there must exist some $k$ such that we first multiply $B = A_1 \cdots A_k$ in some way, then multiply $C = A_{k + 1} \cdots A_{n + 1}$ in some way, then multiply $B$ and $C$. By our induction hypothesis, we have $k − 1$ pairs of parentheses for the full parenthesization of $B$ and $n + 1 − k − 1$ pairs of parentheses for the full parenthesization of $C$. Adding these together, plus the pair of outer parentheses for the entire expression, yields $k - 1 + n + 1 - k - 1 + 1 = (n + 1) - 1$ parentheses, as desired.


question:
 Write pseudocode for the procedure $\text{CONSTRUCT-OPTIMAL-BST}(root)$ which, given the table $root$, outputs the structure of an optimal binary search tree. For the example in Figure 15.10, your procedure should print out the structure
>
> $$
> \begin{aligned}
> & \text{$k_2$ is the root} \\\\
> & \text{$k_1$ is the left  child of $k_2$} \\\\
> & \text{$d_0$ is the left  child of $k_1$} \\\\
> & \text{$d_1$ is the right child of $k_1$} \\\\
> & \text{$k_5$ is the right child of $k_2$} \\\\
> & \text{$k_4$ is the left  child of $k_5$} \\\\
> & \text{$k_3$ is the left  child of $k_4$} \\\\
> & \text{$d_2$ is the left  child of $k_3$} \\\\
> & \text{$d_3$ is the right child of $k_3$} \\\\
> & \text{$d_4$ is the right child of $k_4$} \\\\
> & \text{$d_5$ is the right child of $k_5$}
> \end{aligned}
> $$
>
> corresponding to the optimal binary search tree shown in Figure 15.9(b).

answer:
```cpp
CONSTRUCT-OPTIMAL-BST(root, i, j, last)
    if i == j
        return
    if last == 0
        print root[i, j] + "is the root"
    else if j < last
        print root[i, j] + "is the left child of" + last
    else
        print root[i, j] + "is the right child of" + last
    CONSTRUCT-OPTIMAL-BST(root, i, root[i, j] - 1, root[i, j])
    CONSTRUCT-OPTIMAL-BST(root, root[i, j] + 1, j, root[i, j])
```


question:
 Determine the cost and structure of an optimal binary search tree for a set of $n = 7$ keys with the following probabilities
>
> $$
> \begin{array}{c|cccccccc}
> i   & 0    & 1    & 2    & 3    & 4    & 5    & 6    & 7 \\\\
> \hline
> p_i &      & 0.04 & 0.06 & 0.08 & 0.02 & 0.10 & 0.12 & 0.14 \\\\
> q_i & 0.06 & 0.06 & 0.06 & 0.06 & 0.05 & 0.05 & 0.05 & 0.05
> \end{array}
> $$

answer:
Cost is $3.12$.

$$
\begin{aligned}
& \text{$k_5$ is the root} \\\\
& \text{$k_2$ is the left  child of $k_5$} \\\\
& \text{$k_1$ is the left  child of $k_2$} \\\\
& \text{$d_0$ is the left  child of $k_1$} \\\\
& \text{$d_1$ is the right child of $k_1$} \\\\
& \text{$k_3$ is the right child of $k_2$} \\\\
& \text{$d_2$ is the left  child of $k_3$} \\\\
& \text{$k_4$ is the right child of $k_3$} \\\\
& \text{$d_3$ is the left  child of $k_4$} \\\\
& \text{$d_4$ is the right child of $k_4$} \\\\
& \text{$k_7$ is the right child of $k_5$} \\\\
& \text{$k_6$ is the left  child of $k_7$} \\\\
& \text{$d_5$ is the left  child of $k_6$} \\\\
& \text{$d_6$ is the right child of $k_6$} \\\\
& \text{$d_7$ is the right child of $k_7$}
\end{aligned}
$$


question:
 Suppose that instead of maintaining the table $w[i, j]$, we computed the value of $w(i, j)$ directly from equation $\text{(15.12)}$ in line 9 of $\text{OPTIMAL-BST}$ and used this computed value in line 11. How would this change affect the asymptotic running time of $\text{OPTIMAL-BST}$?

answer:
Computing $w(i, j)$ from the equation is $\Theta(j - i)$, since the loop below on lines 10-14 is also $\Theta(j - i)$, it wouldn't affect the asymptotic running time of $\text{OPTIMAL-BST}$ which would stay $\Theta(n^3)$.


question:
 Knuth [212] has shown that there are always roots of optimal subtrees such that $root[i, j - 1] \le root[i, j] \le root[i + 1, j]$ for all $1 \le i < j \le n$. Use this fact to modify the $\text{OPTIMAL-BST}$ procedure to run in $\Theta(n^2)$ time.

answer:
Change the **for** loop of line 10 in $\text{OPTIMAL-BST}$ to

```cpp
    for r = r[i, j - 1] to r[i + 1, j]
```

Knuth's result implies that it is sufficient to only check these values because optimal root found in this range is in fact the optimal root of some binary search tree. The time spent within the **for** loop of line 6 is now $\Theta(n)$. This is because the bounds on $r$ in the new **for** loop of line 10 are nonoverlapping.

To see this, suppose we have fixed $l$ and $i$. On one iteration of the **for** loop of line 6, the upper bound on $r$ is

$$r[i + 1, j] = r[i + 1, i + l - 1].$$

When we increment $i$ by $1$ we increase $j$ by $1$. However, the lower bound on $r$ for the next iteration subtracts this, so the lower bound on the next iteration is

$$r[i + 1, j + 1 - 1] = r[i + 1, j].$$

Thus, the total time spent in the **for** loop of line 6 is $\Theta(n)$. Since we iterate the outer **for** loop of line 5 $n$ times, the total runtime is $\Theta(n^2)$.


question:
 Run the Floyd-Warshall algorithm on the weighted, directed graph of Figure 25.2. Show the matrix $D^{(k)}$ that results for each iteration of the outer loop.

answer:
$k = 1$:

$$
\begin{pmatrix}
     0 & \infty & \infty & \infty &     -1 & \infty \\\\
     1 &      0 & \infty &      2 &      0 & \infty \\\\
\infty &      2 &      0 & \infty & \infty &     -8 \\\\
    -4 & \infty & \infty &      0 &     -5 & \infty \\\\
\infty &      7 & \infty & \infty &      0 & \infty \\\\
\infty &      5 &     10 & \infty & \infty & 0
\end{pmatrix}
$$

$k = 2$:

$$
\begin{pmatrix}
 0 & \infty & \infty & \infty & -1 & \infty \\\\
 1 &      0 & \infty &      2 &  0 & \infty \\\\
 3 &      2 &      0 &      4 &  2 & -    8 \\\\
-4 & \infty & \infty &      0 & -5 & \infty \\\\
 8 &      7 & \infty &      9 &  0 & \infty \\\\
 6 &      5 &     10 &      7 &  5 & 0
\end{pmatrix}
$$

$k = 3$:

$$
\begin{pmatrix}
 0 & \infty & \infty & \infty & -1 & \infty \\\\
 1 &      0 & \infty &      2 &  0 & \infty \\\\
 3 &      2 &      0 &      4 &  2 &     -8 \\\\
-4 & \infty & \infty &      0 & -5 & \infty \\\\
 8 &      7 & \infty &      9 &  0 & \infty \\\\
 6 &      5 &     10 &      7 &  5 & 0
\end{pmatrix}
$$

$k = 4$:

$$
\begin{pmatrix}
 0 & \infty & \infty & \infty & -1 & \infty \\\\
-2 &      0 & \infty &      2 & -3 & \infty \\\\
 0 &      2 &      0 &      4 & -1 &     -8 \\\\
-4 & \infty & \infty &      0 & -5 & \infty \\\\
 5 &      7 & \infty &      9 &  0 & \infty \\\\
 3 &      5 &     10 &      7 &  2 & 0
\end{pmatrix}
$$

$k = 5$:

$$
\begin{pmatrix}
 0 & 6 & \infty & 8 & -1 & \infty \\\\
-2 & 0 & \infty & 2 & -3 & \infty \\\\
 0 & 2 &      0 & 4 & -1 &     -8 \\\\
-4 & 2 & \infty & 0 & -5 & \infty \\\\
 5 & 7 & \infty & 9 &  0 & \infty \\\\
 3 & 5 &     10 & 7 &  2 & 0
\end{pmatrix}
$$

$k = 6$:

$$
\begin{pmatrix}
 0 &  6 & \infty &  8 & -1 & \infty \\\\
-2 &  0 & \infty &  2 & -3 & \infty \\\\
-5 & -3 &      0 & -1 & -6 &     -8 \\\\
-4 &  2 & \infty &  0 & -5 & \infty \\\\
 5 &  7 & \infty &  9 &  0 & \infty \\\\
 3 &  5 &     10 &  7 &  2 & 0
\end{pmatrix}
$$


question: Show how to compute the transitive closure using the technique of Section 25.1.
answer:
We set $w_{ij} = 1$ if $(i, j)$ is an edge, and $w_{ij} = 0$ otherwise. Then we replace line 7 of $\text{EXTEND-SHORTEST-PATHS}(L, W)$ by $l''\_{ij} = l''\_{ij} \lor (l_{ik} \land w_{kj})$. Then run the $\text{SLOW-ALL-PAIRS-SHORTEST-PATHS}$ algorithm.


question:
 Modify the $\text{FLOYD-WARSHALL}$ procedure to compute the $\prod^{(k)}$ matrices according to equations $\text{(25.6)}$ and $\text{(25.7)}$. Prove rigorously that for all $i \in V$, the predecessor subgraph $G_{\pi, i}$ is a shortest-paths tree with root $i$. ($\textit{Hint:}$ To show that $G_{\pi, i}$ is acyclic, first show that $\pi_{ij}^{(k)} = l$ implies $d_{ij}^{(k)} \ge d_{il}^{(k)} + w_{lj}$, according to the definition of $\pi_{ij}^{(k)}$. Then, adapt the proof of Lemma 23.16.)

answer:
```cpp
MOD-FLOYD-WARSHALL(W)
    n = W.rows
    D(0) = W
    let π(0) be a new n × n matrix
    for i = 1 to n
        for j = 1 to n
            if i != j and D[i, j](0) < ∞
                π[i, j](0) = i
    for k = 1 to n
        let D(k) be a new n × n matrix
        let π(k) be a new n × n matrix
        for i = 1 to n
            for j = 1 to n
                if d[i, j](k - 1) ≤ d[i, k](k - 1) + d[k, j](k - 1)
                    d[i, j](k) = d[i, j](k - 1)
                    π[i, j](k) = π[i, j](k - 1)
                else
                    d[i, j](k) = d[i, k](k - 1) + d[k, j](k - 1)
                    π[i, j](k) = π[k, j](k - 1)
```

In order to have that $\pi^{(k)}\_{ij} = l$, we need that $d^{(k)}\_{ij} \ge d^{(k)}\_{il} + w_{lj}$. To see this fact, we will note that having $\pi^{(k)}\_{ij} = l$ means that a shortest path from $i$ to $j$ last goes through $l$. A path that last goes through $l$ corresponds to taking a chepest path from $i$ to $l$ and then following the single edge from $l$ to $j$. However, This means that $d_{il} \le d_{ij} - w_{ij}$, which we can rearrange to get the desired inequality. We can just continue following this inequality
around, and if we ever get some cycle, $i_1, i_2, \ldots, i_c$, then we would have that $d_{ii_1} \le d_{ii_1} + w_{i_1i_2} + w_{i_2i_3} + \cdots + w_{i_ci_1}$. So, if we subtract the common term sfrom both sides, we get that $0 \le w_{i_ci_1} + \sum_{q = 1}^{c - 1} w_{i_qi_{q + 1}}$. So, we have that we would only have a cycle in the precedessor graph if we ahvt that there is a zero weight cycle in the original graph. However, we would never have to go around the weight zero cycle since the constructed path of shortest weight favors ones with a fewer number of edges because of the way that we handle the equality case in equation $\text{(25.7)}$.


answer: (Removed)
question:
 As it appears above, the Floyd-Warshall algorithm requires $\Theta(n^3)$ space, since we compute $d_{ij}^{(k)}$ for $i, j, k = 1, 2, \ldots, n$. Show that the following procedure, which simply drops all the superscripts, is correct, and thus only $\Theta(n^2)$ space is required.
>
> ```cpp
> FLOYD-WARSHALL'(W)
>     n = W.rows
>     D = W
>     for k = 1 to n
>         for i = 1 to n
>             for j = 1 to n
>                 d[i, j] = min(d[i, j], d[i, k] + d[k, j])
>     return D
> ```


question:
 Suppose that we modify the way in which equation $\text{(25.7)}$ handles equality:
>
> $$
> \pi_{ij}^{(k)} =
> \begin{cases}
> \pi_{ij}^{(k - 1)} & \text{ if } d_{ij}^{(k - 1)} <   d_{ik}^{(k - 1)} + d_{kj}^{(k - 1)}, \\\\
> \pi_{kj}^{(k - 1)} & \text{ if } d_{ij}^{(k - 1)} \ge d_{ik}^{(k - 1)} + d_{kj}^{(k - 1)}.
> \end{cases}
> $$
>
> Is this alternative definition of the predecessor matrix $\prod$ correct?

answer:
If we change the way that we handle the equality case, we will still be generating a the correct values for the $\pi$ matrix. This is because updating the $\pi$ values to make paths that are longer but still tied for the lowest weight. Making $\pi_{ij} = \pi_{kj}$ means that we are making the shortest path from $i$ to $j$ passes through $k$ at some point. This has the same cost as just going from $i$ to $j$, since $d_{ij} = d_{ik} + d_{kj}$.


answer: (Removed)
question:
 How can we use the output of the Floyd-Warshall algorithm to detect the presence of a negative-weight cycle?


question:
 Another way to reconstruct shortest paths in the Floyd-Warshall algorithm uses values $\phi_{ij}^{(k)}$ for $i, j, k = 1, 2, \ldots, n$, where $\phi_{ij}^{(k)}$ is the highest-numbered intermediate vertex of a shortest path from $i$ to $j$ in which all intermediate vertices are in the set $\\{1, 2, \ldots, k \\}$. Give a recursive formulation for $\phi_{ij}^{(k)}$, modify the $\text{FLOYD-WARSHALL}$ procedure to compute the $\phi_{ij}^{(k)}$ values, and rewrite the $\text{PRINT-ALLPAIRS-SHORTEST-PATH}$ procedure to take the matrix $\Phi = \big(\phi_{ij}^{(n)}\big)$ as an input. How is the matrix $\Phi$ like the $s$ table in the matrix-chain multiplication problem of Section 15.2?

answer:
We can recursively compute the values of $\phi_{ij}^{(k)}$ by, letting it be $\phi_{ij}^{(k - 1)}$ if $d_{ik}^{(k)} + d_{kj}^{(k)} \ge d_{ij}^{(k - 1)}$, and otherwise, let it be $k$. This works correctly because it perfectly captures whether we decided to use vertex $k$ when we were repeatedly allowing ourselves use of each vertex one at a time. To modify Floyd-Warshall to compute this, we would just need to stick within the innermost for loop, something that computes $\phi(k)$ by this recursive rule, this would only be a constant amount of work in this innermost for loop, and so would not cause the asymptotic runtime to increase. It is similar to the s table in matrix-chain multiplication because it is computed by a similar recurrence.

If we already have the $n^3$ values in $\phi_{ij}^{(k)}$ provided, then we can reconstruct the shortest path from $i$ to $j$ because we know that the largest vertex in the path from $i$ to $j$ is $\phi_{ij}^{(n)}$, call it $a_1$. Then, we know that the largest vertex in the path before $a_1$ will be $\phi_{ia_1}^{(a_1 - 1)}$ and the largest after $a_1$ will be $\phi_{a_1j}^{(a_1 - 1)}$. By continuing to recurse until we get that the largest element showing up at some point is $\text{NIL}$, we will be able to continue subdividing the path until it is entirely constructed.


question:
 Give an $O(VE)$-time algorithm for computing the transitive closure of a directed
> graph $G = (V, E)$.

answer:
We can determine the vertices reachable from a particular vertex in $O(V + E)$ time using any basic graph searching algorithm. Thus we can compute the transitive closure in $O(VE + V^2)$ time by searching the graph with each vertex as the source. If $|V| = O(E)$, we're done as $VE$ is now the dominating term in the running time bound. If not, we preprocess the graph and mark all degree-$0$ vertices in $O(V + E)$ time. The rows representing these vertices in the transitive closure are all $0$s, which means that the algorithm remains correct if we ignore these vertices when searching. After preprocessing, $|V| = O(E)$ as $|E| \geq |V|/2$. Therefore searching can be done in $O(VE)$ time.


question:
 Suppose that we can compute the transitive closure of a directed acyclic graph in $f(|V|, |E|)$ time, where $f$ is a monotonically increasing function of $|V|$ and $|E|$. Show that the time to compute the transitive closure $G^\* = (V, E^\*)$ of a general directed graph $G = (V, E)$ is then $f(|V|, |E|) + O(V + E^\*)$.

answer:
First, compute the strongly connected components of the directed graph, and look at it's component graph. This component graph is going to be acyclic and have at most as many vertices and at most as many edges as the original graph. Since it is acyclic, we can run our transitive closure algorithm on it. Then, for every edge $(S_1, S_2)$ that shows up in the transitive closure of the component graph, we add an edge from each vertex in $S_1$ to a vertex in $S_2$. This takes time equal to $O(V + E')$. So, the total time required is $\le f(|V|, |E|) + O(V + E)$.


question:
 Use Johnson's algorithm to find the shortest paths between all pairs of vertices in the graph of Figure 25.2. Show the values of $h$ and $\hat w$ computed by the algorithm.

answer:
$$
\begin{array}{c|c}
v & h(v) \\\\
\hline
1 & -5 \\\\
2 & -3 \\\\
3 &  0 \\\\
4 & -1 \\\\
5 & -6 \\\\
6 & -8
\end{array}
$$

$$
\begin{array}{ccc|ccc}
u & v & \hat w(u, v) & u & v & \hat w(u, v) \\\\
\hline
1 & 2 & \text{NIL} & 4 & 1 & 0          \\\\
1 & 3 & \text{NIL} & 4 & 2 & \text{NIL} \\\\
1 & 4 & \text{NIL} & 4 & 3 & \text{NIL} \\\\
1 & 5 & 0          & 4 & 5 & 8          \\\\
1 & 6 & \text{NIL} & 4 & 6 & \text{NIL} \\\\
2 & 1 & 3          & 5 & 1 & \text{NIL} \\\\
2 & 3 & \text{NIL} & 5 & 2 & 4          \\\\
2 & 4 & 0          & 5 & 3 & \text{NIL} \\\\
2 & 5 & \text{NIL} & 5 & 4 & \text{NIL} \\\\
2 & 6 & \text{NIL} & 5 & 6 & \text{NIL} \\\\
3 & 1 & \text{NIL} & 6 & 1 & \text{NIL} \\\\
3 & 2 & 5          & 6 & 2 & 0          \\\\
3 & 4 & \text{NIL} & 6 & 3 & 2         \\\\
3 & 5 & \text{NIL} & 6 & 4 & \text{NIL} \\\\
3 & 6 & 0          & 6 & 5 & \text{NIL} \\\\
\end{array}
$$

So, the $d_{ij}$ values that we get are

$$
\begin{pmatrix}
 0 &  6 & \infty &  8 & -1 & \infty \\\\
-2 &  0 & \infty &  2 & -3 & \infty \\\\
-5 & -3 &      0 & -1 & -6 &     -8 \\\\
-4 &  2 & \infty &  0 & -5 & \infty \\\\
 5 &  7 & \infty &  9 &  0 & \infty \\\\
 3 &  5 &     10 &  7 &  2 &      0
\end{pmatrix}
.
$$


question: What is the purpose of adding the new vertex $s$ to $V'$, yielding $V'$?
answer:
This is only important when there are negative-weight cycles in the graph. Using a dummy vertex gets us around the problem of trying to compute $-\infty + \infty$ to find $\hat w$. Moreover, if we had instead used a vertex $v$ in the graph instead of the new vertex $s$, then we run into trouble if a vertex fails to be reachable from $v$.


question:
 Suppose that $w(u, v) \ge 0$ for all edges $(u, v) \in E$. What is the relationship between the weight functions $w$ and $\hat w$?

answer:
If all the edge weights are nonnegative, then the values computed as the shortest distances when running Bellman-Ford will be all zero. This is because when constructing $G'$ on the first line of Johnson's algorithm, we place an edge of weight zero from s to every other vertex. Since any path within the graph has no negative edges, its cost cannot be negative, and so, cannot beat the trivial path that goes straight from $s$ to any given vertex. Since we have that $h(u) = h(v)$ for every $u$ and $v$, the reweighting that occurs only adds and subtracts $0$, and so we have that $w(u, v) = \hat w(u, v)$


question:
 Professor Greenstreet claims that there is a simpler way to reweight edges than the method used in Johnson's algorithm. Letting $w^\* = \min_{(u, v) \in E} \\{w(u, v)\\}$, just define $\hat w(u, v) = w(u, v) - w^\*$ for all edges $(u, v) \in E$. What is wrong with the professor's method of reweighting?

answer:
Consider a graph with four vertices labeled $A$, $B$, $C$, and $D$. We want to find the shortest path from vertex $A$ to vertex $C$. Initially, there are two potential paths: $A \to B \to C$ (with 2 edges) and $A \to C$ (with a direct edge).

Introducing a reweighting factor, denoted as $w^\*$, may lead to incorrect results. Reweighting could potentially make the path with more edges ($A \to B \to C$) longer than the direct edge ($A \to C$), which contradicts our goal of finding the shortest path.

![](../img/25.3-4.png)


question:
 Suppose that we run Johnson's algorithm on a directed graph $G$ with weight function $w$. Show that if $G$ contains a $0$-weight cycle $c$, then $\hat w(u, v) = 0$ for every edge $(u, v)$ in $c$.

answer:
If $\delta(s, v) - \delta(s, u) \le w(u, v)$, we have

$$\delta(s, u) \le \delta(s, v) + (0 - w(u, v)) < \delta(s, u) + w(u, v) - w(u, v) = \delta(s, u),$$

which is impossible, thus $\delta(s, v) - \delta(s, u) = w(u, v)$, $\hat w(u, v) = w(u, v) + \delta(s, u) - \delta(s, v) = 0$.


answer: (Removed)
question:
 Professor Michener claims that there is no need to create a new source vertex in line 1 of $\text{JOHNSON}$. He claims that instead we can just use $G' = G$ and let $s$ be any vertex. Give an example of a weighted, directed graph $G$ for which incorporating the professor's idea into $\text{JOHNSON}$ causes incorrect answers. Then show that if $G$ is strongly connected (every vertex is reachable from every other vertex), the results returned by $\text{JOHNSON}$ with the professor's modification are correct.


question:
 Run $\text{SLOW-ALL-PAIRS-SHORTEST-PATHS}$ on the weighted, directed graph of Figure 25.2, showing the matrices that result for each iteration of the loop. Then do the same for $\text{FASTER-ALL-PAIRS-SHORTEST-PATHS}$.

answer:
- Initial:

    $$
    \begin{pmatrix}
         0 & \infty & \infty & \infty &     -1 & \infty \\\\
         1 &      0 & \infty &      2 & \infty & \infty \\\\
    \infty &      2 &      0 & \infty & \infty &     -8 \\\\
        -4 & \infty & \infty &      0 &      3 & \infty \\\\
    \infty &      7 & \infty & \infty &      0 & \infty \\\\
    \infty &      5 &     10 & \infty & \infty & 0
    \end{pmatrix}
    $$

- Slow:

    $m = 2$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty & \infty &     -1 & \infty \\\\
    -2 &  0 & \infty &      2 &      0 & \infty \\\\
     3 & -3 &      0 &      4 & \infty &     -8 \\\\
    -4 & 10 & \infty &      0 &     -5 & \infty \\\\
     8 &  7 & \infty &      9 &      0 & \infty \\\\
     6 &  5 &     10 &      7 & \infty & 0
    \end{pmatrix}
    $$

    $m = 3$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty &  8 & -1 & \infty \\\\
    -2 &  0 & \infty &  2 & -3 & \infty \\\\
    -2 & -3 &      0 & -1 &  2 &     -8 \\\\
    -4 &  2 & \infty &  0 & -5 & \infty \\\\
     5 &  7 & \infty &  9 &  0 & \infty \\\\
     3 &  5 &     10 &  7 &  5 & 0
    \end{pmatrix}
    $$

    $m = 4$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty &  8 & -1 & \infty \\\\
    -2 &  0 & \infty &  2 & -3 & \infty \\\\
    -5 & -3 &      0 & -1 & -3 &     -8 \\\\
    -4 &  2 & \infty &  0 & -5 & \infty \\\\
     5 &  7 & \infty &  9 &  0 & \infty \\\\
     3 &  5 &     10 &  7 &  2 & 0
    \end{pmatrix}
    $$

    $m = 5$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty &  8 & -1 & \infty \\\\
    -2 &  0 & \infty &  2 & -3 & \infty \\\\
    -5 & -3 &      0 & -1 & -6 &     -8 \\\\
    -4 &  2 & \infty &  0 & -5 & \infty \\\\
     5 &  7 & \infty &  9 & 0  & \infty \\\\
     3 &  5 &     10 &  7 & 2  & 0
    \end{pmatrix}
    $$

- Fast:

    $m = 2$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty & \infty &     -1 & \infty \\\\
    -2 &  0 & \infty &      2 &      0 & \infty \\\\
     3 & -3 &      0 &      4 & \infty &     -8 \\\\
    -4 & 10 & \infty &      0 &     -5 & \infty \\\\
     8 &  7 & \infty &      9 &      0 & \infty \\\\
     6 &  5 &     10 &      7 & \infty & 0
    \end{pmatrix}
    $$

    $m = 4$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty &  8 & -1 & \infty \\\\
    -2 &  0 & \infty &  2 & -3 & \infty \\\\
    -5 & -3 &      0 & -1 & -3 &     -8 \\\\
    -4 &  2 & \infty &  0 & -5 & \infty \\\\
     5 &  7 & \infty &  9 &  0 & \infty \\\\
     3 &  5 &     10 &  7 &  2 & 0
    \end{pmatrix}
    $$

    $m = 8$:

    $$
    \begin{pmatrix}
     0 &  6 & \infty &  8 & -1 & \infty \\\\
    -2 &  0 & \infty &  2 & -3 & \infty \\\\
    -5 & -3 &      0 & -1 & -6 &     -8 \\\\
    -4 &  2 & \infty &  0 & -5 & \infty \\\\
     5 &  7 & \infty &  9 &  0 & \infty \\\\
     3 &  5 &     10 &  7 &  2 & 0
    \end{pmatrix}
    $$


question: Why do we require that $w_{ii} = 0$ for all $1 \le i \le n$?
answer:
This is consistent with the fact that the shortest path from a vertex to itself is the empty path of weight $0$. If there were another path of weight less than $0$ then it must be a negative-weight cycle, since it starts and ends at $v_i$.

If $w_{ii} \ne 0$, then $L^{(1)}$ produced after the first run of $\text{EXTEND-SHORTEST-PATHS}$ would not contain the minimum weight of any path from $i$ to its neighbours. If $w_{ii} = 0$, then in line 7 of $\text{EXTEND-SHORTEST-PATHS}$, the second argument to $min$ would not equal the weight of the edge going from $i$ to its neighbours.


answer: The identity matrix.
question:
 What does the matrix
>
> $$
> L^{(0)} =
> \begin{pmatrix}
>      0 & \infty & \infty & \cdots & \infty \\\\
> \infty &      0 & \infty & \cdots & \infty \\\\
> \infty & \infty &      0 & \cdots & \infty \\\\
> \vdots & \vdots & \vdots & \ddots & \vdots \\\\
> \infty & \infty & \infty & \cdots & 0
> \end{pmatrix}
> $$
>
> used in the shortest-paths algorithms correspond to in regular matrix multiplication?


question: Show that matrix multiplication defined by $\text{EXTEND-SHORTEST-PATHS}$ is associative.
answer:
To verify associativity, we need to check that $(W^iW^j)W^p = W^i(W^jW^p)$ for all $i$, $j$ and $p$, where we use the matrix multiplication defined by the $\text{EXTEND-SHORTEST-PATHS}$ procedure. Consider entry $(a, b)$ of the left hand side. This is:

$$
\begin{aligned}
\min_{1 \le k \le n} [W^iW^j]\_{a, k} + W_{k, b}^p
    & = \min_{1 \le k \le n} \min_{1 \le q \le n} W_{a, q}^i + W_{q, k}^j + W_{k, b}^p \\\\
    & = \min_{1 \le q \le n} W_{a, q}^i + \min_{1 \le k \le n} W_{q, k}^j + W_{k, b}^p \\\\
    & = \min_{1 \le q \le n} W_{a, q}^i + [W^jW^p]\_{q, b},
\end{aligned}
$$

which is precisely entry $(a, b)$ of the right hand side.


answer: (Removed)
question:
 Show how to express the single-source shortest-paths problem as a product of matrices and a vector. Describe how evaluating this product corresponds to a Bellman-Ford-like algorithm (see Section 24.1).


question:
 Suppose we also wish to compute the vertices on shortest paths in the algorithms of this section. Show how to compute the predecessor matrix $\prod$ from the completed matrix $L$ of shortest-path weights in $O(n^3)$ time.

answer:
For each source vertex $v_i$ we need to compute the shortest-paths tree for $v_i$. To do this, we need to compute the predecessor for each $j \ne i$. For fixed $i$ and $j$, this is the value of $k$ such that $L_{i, k} + w(k, j) = L[i, j]$. Since there are $n$ vertices whose trees need computing, $n$ vertices for each such tree whose predecessors need computing, and it takes $O(n)$ to compute this for each one (checking each possible $k$), the total time is $O(n^3)$.


question:
 We can also compute the vertices on shortest paths as we compute the shortestpath weights. Define $\pi_{ij}^{(m)}$ as the predecessor of vertex $j$ on any minimum-weight path from $i$ to $j$ that contains at most $m$ edges. Modify the $\text{EXTEND-SHORTESTPATHS}$ and $\text{SLOW-ALL-PAIRS-SHORTEST-PATHS}$ procedures to compute the matrices$\prod^{(1)}, \prod^{(2)}, \ldots, \prod^{(n - 1)}$ as the matrices $L^{(1)}, L^{(2)}, \ldots, L^{(n - 1)}$ are computed.

answer:
To have the procedure compute the predecessor along the shortest path, see the modified procedures, $\text{EXTEND-SHORTEST-PATH-MOD}$ and $\text{SLOW-ALL-PAIRS-SHORTEST-PATHS-MOD}$

```cpp
EXTEND-SHORTEST-PATH-MOD(∏, L, W)
    n = L.row
    let L' = l'[i, j] be a new n × n matirx
    ∏' = π'[i, j] is a new n × n matrix
    for i = 1 to n
        for j = 1 to n
            l'[i, j] = ∞
            π'[i, j] = NIL
            for k = 1 to n
                if l[i, k] + l[k, j] < l[i, j]
                    l[i, j] = l[i, k] + l[k, j]
                    if k != j
                        π'[i, j] = k
                    else
                        π'[i, j] = π[i, j]
    return (∏', L')
```

```cpp
SLOW-ALL-PAIRS-SHORTEST-PATHS-MOD(W)
    n = W.rows
    L(1) = W
    ∏(1) = π[i, j](1) where π[i, j](1) = i if there is an edge from i to j, and NIL otherwise
    for m = 2 to n - 1
        ∏(m), L(m) = EXTEND-SHORTEST-PATH-MOD(∏(m - 1), L(m - 1), W)
    return (∏(n - 1), L(n - 1))
```


question:
 The $\text{FASTER-ALL-PAIRS-SHORTEST-PATHS}$ procedure, as written, requires us to store $\lceil \lg(n - 1) \rceil$ matrices, each with $n^2$ elements, for a total space requirement of $\Theta(n^2\lg n)$. Modify the procedure to require only $\Theta(n^2)$ space by using only two $n \times n$ matrices.

answer:
We can overwrite matrices as we go. Let $A \star B$ denote multiplication defined by the $\text{EXTEND-SHORTEST-PATHS}$ procedure. Then we modify $\text{FASTER-ALL-EXTEND-SHORTEST-PATHS}(W)$. We initially create an $n$ by $n$ matrix $L$. Delete line 5 of the algorithm, and change line 6 to $L = W \star W$, followed by $W = L$.


question:
 Modify $\text{FASTER-ALL-PAIRS-SHORTEST-PATHS}$ so that it can determine whether the graph contains a negative-weight cycle.

answer:
For the modification, keep computing for one step more than the original, that is, we compute all the way up to $L^{(2k + 1)}$ where $2^k > n - 1$. Then, if there aren't any negative weight cycles, then, we will have that the two matrices should be equal since having no negative weight cycles means that between any two vertices, there is a path that is tied for shortest and contains at most $n - 1$ edges.

However, if there is a cycle of negative total weight, we know that it's length is at most $n$, so, since we are allowing paths to be larger by $2k \ge n$ between these two matrices, we have that we would need to have all of the vertices on the cycle have their distance reduce by at least the negative weight of the cycle. Since we can detect exactly when there is a negative cycle, based on when these two matrices are different. This algorithm works. It also only takes time equal to a single matrix multiplication which is littlee oh of the unmodified algorithm.


answer: (Removed)
question:
 Give an efficient algorithm to find the length (number of edges) of a minimum-length negative-weight cycle in a graph.


question:
 For the set of $\\{ 1, 4, 5, 10, 16, 17, 21 \\}$ of keys, draw binary search trees of heights $2$, $3$, $4$, $5$, and $6$.

answer:
- $height = 2$:

  ![](../img/12.1-1-1.png)

- $height = 3$:

  ![](../img/12.1-1-2.png)

- $height = 4$:

  ![](../img/12.1-1-3.png)

- $height = 5$:

  ![](../img/12.1-1-4.png)

- $height = 6$:

  ![](../img/12.1-1-5.png)


question:
 What is the difference between the binary-search-tree property and the min-heap property (see page 153)? Can the min-heap property be used to print out the keys of an $n$-node tree in sorted order in $O(n)$ time? Show how, or explain why not.

answer:
- The binary-search-tree property guarantees that all nodes in the left subtree are smaller, and all nodes in the right subtree are larger.
- The min-heap property only guarantees the general child-larger-than-parent relation, but doesn't distinguish between left and right children. For this reason, the min-heap property can't be used to print out the keys in sorted order in linear time because we have no way of knowing which subtree contains the next smallest element.


question:
 Give a nonrecursive algorithm that performs an inorder tree walk. ($\textit{Hint:}$ An easy solution uses a stack as an auxiliary data structure. A more complicated, but elegant, solution uses no stack but assumes that we can test two pointers for equality.)

answer:
```cpp
INORDER-TREE-WALK(T)
    let S be an empty stack
    current = T.root
    done = 0
    while !done
        if current != NIL
            PUSH(S, current)
            current = current.left
        else
            if !S.EMPTY()
                current = POP(S)
                print current
                current = current.right
            else done = 1
```


question:
 Give recursive algorithms that perform preorder and postorder tree walks in $\Theta(n)$ time on a tree of $n$ nodes.

answer:
```cpp
PREORDER-TREE-WALK(x)
    if x != NIL
        print x.key
        PREORDER-TREE-WALK(x.left)
        PREORDER-TREE-WALK(x.right)
```

```cpp
POSTORDER-TREE-WALK(x)
    if x != NIL
        POSTORDER-TREE-WALK(x.left)
        POSTORDER-TREE-WALK(x.right)
        print x.key
```


question:
 Argue that since sorting $n$ elements takes $\Omega(n\lg n)$ time in the worst case in the comparison model, any comparison-based algorithm for constructing a binary search tree from an arbitrary list of $n$ elements takes $\Omega(n\lg n)$ time in the worst case.

answer:
Assume, for the sake of contradiction, that we can construct the binary search tree by comparison-based algorithm using less than $\Omega(n\lg n)$ time, since the inorder tree walk is $\Theta(n)$, then we can get the sorted elements in less than $\Omega(n\lg n)$ time, which contradicts the fact that sorting $n$ elements takes $\Omega(n\lg n)$ time in the worst case.


question:
 Suppose that we have numbers between $1$ and $1000$ in a binary search tree, and we want to search for the number $363$. Which of the following sequences could _not_ be the sequence of nodes examined?
>
> **a.** $2, 252, 401, 398, 330, 344, 397, 363$.
>
> **b.** $924, 220, 911, 244, 898, 258, 362, 363$.
>
> **c.** $925, 202, 911, 240, 912, 245, 363$.
>
> **d.** $2, 399, 387, 219, 266, 382, 381, 278, 363$.
>
> **e.** $935, 278, 347, 621, 299, 392, 358, 363$.

answer:
- **c.** could not be the sequence of nodes explored because we take the left child from the $911$ node, and yet somehow manage to get to the $912$ node which cannot belong the left subtree of $911$ because it is greater.
- **e.** is also impossible because we take the right subtree on the $347$ node and yet later come across the $299$ node.


question: Write recursive versions of $\text{TREE-MINIMUM}$ and $\text{TREE-MAXIMUM}$.
answer:
```cpp
TREE-MINIMUM(x)
    if x.left != NIL
        return TREE-MINIMUM(x.left)
    else return x
```

```cpp
TREE-MAXIMUM(x)
    if x.right != NIL
        return TREE-MAXIMUM(x.right)
    else return x
```


question: Write the $\text{TREE-PREDECESSOR}$ procedure.
answer:
```cpp
TREE-PREDECESSOR(x)
    if x.left != NIL
        return TREE-MAXIMUM(x.left)
    y = x.p
    while y != NIL and x == y.left
        x = y
        y = y.p
    return y
```


question:
 Professor Bunyan thinks he has discovered a remarkable property of binary search trees. Suppose that the search for key $k$ in a binary search tree ends up in a leaf. Consider three sets: $A$, the keys to the left of the search path; $B$, the keys on the search path; and $C$, the keys to the right of the search path. Professor Bunyan claims that any three keys $a \in A$, $b \in B$, and $c \in C$ must satisfy $a \le b \le c$. Give a smallest possible counterexample to the professor's claim.

answer:
Search for $9$ in this tree. Then $A = \\{7\\}$, $B = \\{5, 8, 9\\}$ and $C = \\{\\}$. So, since $7 > 5$ it breaks professor's claim.


question:
 Show that if a node in a binary search tree has two children, then its successor has no left child and its predecessor has no right child.

answer:
Suppose the node $x$ has two children. Then it's successor is the minimum element of the BST rooted at $x.right$. If it had a left child then it wouldn't be
the minimum element. So, it must not have a left child. Similarly, the predecessor must be the maximum element of the left subtree, so cannot have a right
child.


question:
 Consider a binary search tree $T$ whose keys are distinct. Show that if the right subtree of a node $x$ in $T$ is empty and $x$ has a successor $y$, then $y$ is the lowest ancestor of $x$ whose left child is also an ancestor of $x$. (Recall that every node is its own ancestor.)

answer:
First we establish that $y$ must be an ancestor of $x$. If $y$ weren't an ancestor of $x$, then let $z$ denote the first common ancestor of $x$ and $y$. By the binary-search-tree property, $x < z < y$, so $y$ cannot be the successor of $x$.

Next observe that $y.left$ must be an ancestor of $x$ because if it weren't, then $y.right$ would be an ancestor of $x$, implying that $x > y$. Finally, suppose that $y$ is not the lowest ancestor of $x$ whose left child is also an ancestor of $x$. Let $z$ denote this lowest ancestor. Then $z$ must be in the left subtree of $y$, which implies $z < y$, contradicting the fact that $y$ is the successor of $x$.


question:
 An alternative method of performing an inorder tree walk of an $n$-node binary search tree finds the minimum element in the tree by calling $\text{TREE-MINIMUM}$ and then making $n - 1$ calls to $\text{TREE-SUCCESSOR}$. Prove that this algorithm runs in $\Theta(n)$ time.

answer:
To show this bound on the runtime, we will show that using this procedure, we traverse each edge twice. This will suffice because the number of edges in a tree is one less than the number of vertices.

Consider a vertex of a BST, say $x$. Then, we have that the edge between $x.p$ and $x$ gets used when successor is called on $x.p$ and gets used again when it is called on the largest element in the subtree rooted at $x$. Since these are the only two times that that edge can be used, apart from the initial finding of tree minimum. We have that the runtime is $O(n)$. We trivially get the runtime is $\Omega(n)$ because that is the size of the output.


question:
 Prove that no matter what node we start at in a height-$h$ binary search tree, $k$ successive calls to $\text{TREE-SUCCESSOR}$ take $O(k + h)$ time.

answer:
Suppose $x$ is the starting node and $y$ is the ending node. The distance between $x$ and $y$ is at most $2h$, and all the edges connecting the $k$ nodes are visited twice, therefore it takes $O(k + h)$ time.


question:
 Let $T$ be a binary search tree whose keys are distinct, let $x$ be a leaf node, and let $y$ be its parent. Show that $y.key$ is either the smallest key in $T$ larger than $x.key$ or the largest key in $T$ smaller than $x.key$.

answer:
- If $x = y.left$, then calling successor on $x$ will result in no iterations of the while loop, and so will return $y$.
- If $x = y.right$, the while loop for calling predecessor (see exercise 3) will be run no times, and so $y$ will be returned.


question: Prove equation $\text{(12.3)}$.
answer:
Consider all the possible positions of the largest element of the subset of $n + 3$ of size $4$. Suppose it were in position $i + 4$ for some $i \le n − 1$. Then, we have that there are $i + 3$ positions from which we can select the remaining three elements of the subset. Since every subset with different largest element is different, we get the total by just adding them all up (inclusion exclusion principle).


question:
 Describe a binary search tree on n nodes such that the average depth of a node in the tree is $\Theta(\lg n)$ but the height of the tree is $\omega(\lg n)$. Give an asymptotic upper bound on the height of an $n$-node binary search tree in which the average depth of a node is $\Theta(\lg n)$.

answer:
To keep the average depth low but maximize height, the desired tree will be a complete binary search tree, but with a chain of length $c(n)$ hanging down from one of the leaf nodes. Let $k = \lg(n − c(n))$ be the height of the complete binary search tree. Then the average height is approximately given by

$$\frac{1}{n} \left[\sum_{i = 1}^{n - c(n)} \lg i + (k + 1) + (k + 2) + \cdots + (k + c(n))\right] \approx \lg(n - c(n)) + \frac{c(n)^2}{2n}.$$

The upper bound is given by the largest $c(n)$ such that $\lg(n − c(n))+ \frac{c(n)^2}{2n} = \Theta(\lg n)$ and $c(n) = \omega(\lg n)$. One function which works is $\sqrt n$.


question:
 Show that the notion of a randomly chosen binary search tree on $n$ keys, where each binary search tree of $n$ keys is equally likely to be chosen, is different from the notion of a randomly built binary search tree given in this section. ($\textit{Hint:}$ List the possibilities when $n = 3$.)

answer:
Suppose we have the elements $\\{1, 2, 3\\}$. Then, if we construct a tree by a random ordering, then, we get trees which appear with probabilities some multiple of $\frac{1}{6}$. However, if we consider all the valid binary search trees on the key set of $\\{1, 2, 3\\}$. Then, we will have only five different possibilities. So, each will occur with probability $\frac{1}{5}$, which is a different probability distribution.


question:
 Consider $\text{RANDOMIZED-QUICKSORT}$ operating on a sequence of $n$ distinct input numbers. Prove that for any constant $k > 0$, all but $O(1 / n^k)$ of the $n!$ input permutations yield an $O(n\lg n)$ running time.

answer:
Let $A(n)$ denote the probability that when quicksorting a list of length $n$, some pivot is selected to not be in the middle $n^{1 - k / 2}$ of the numberes. This doesn't happen with probability $\frac{1}{n^{k / 2}}$. Then, we have that the two subproblems are of size $n_1, n_2$ with $n_1 + n_2 = n - 1$, then

$$A(n) \le \frac{1}{n^{k / 2}} + T(n_1)+T(n_2).$$

Since we bounded the depth by $O(1 / \lg n)$ let $\\{a_{i, j}\\}_i$ be all the subproblem sizes left at depth $j$,

$$A(n) \le \frac{1}{n^{k / 2}} \sum_j\sum_i \frac{1}{a}.$$


question: Give a recursive version of the $\text{TREE-INSERT}$ procedure.
answer:
```cpp
RECURSIVE-TREE-INSERT(T, z)
    if T.root == NIL
        T.root = z
    else INSERT(NIL, T.root, z)
```

```cpp
INSERT(p, x, z)
    if x == NIL
        z.p = p
        if z.key < p.key
            p.left = z
        else p.right = z
    else if z.key < x.key
        INSERT(x, x.left, z)
    else INSERT(x, x.right, z)
```


question:
 Suppose that we construct a binary search tree by repeatedly inserting distinct values into the tree. Argue that the number of nodes examined in searching for a value in the tree is one plus the number of nodes examined when the value was first inserted into the tree.

answer:
Number of nodes examined while searching also includes the node which is searched for, which isn't the case when we inserted it.


question:
 We can sort a given set of $n$ numbers by first building a binary search tree containing these numbers (using $\text{TREE-INSERT}$ repeatedly to insert the numbers one by one) and then printing the numbers by an inorder tree walk. What are the worst-case and best-case running times for this sorting algorithm?

answer:
- The worst-case is that the tree formed has height $n$ because we were inserting them in already sorted order. This will result in a runtime of $\Theta(n^2)$.
- The best-case is that the tree formed is approximately balanced. This will mean that the height doesn't exceed $O(\lg n)$. Note that it can't have a smaller height, because a complete binary tree of height $h$ only has $\Theta(2^h)$ elements. This will result in a rutime of $O(n\lg n)$. We showed $\Omega(n\lg n)$ in exercise [12.1-5](../12.1/#121-5).


question:
 Is the operation of deletion "commutative" in the sense that deleting $x$ and then $y$ from a binary search tree leaves the same tree as deleting $y$ and then $x$? Argue why it is or give a counterexample.

answer:
No, giving the following courterexample.

- Delete $A$ first, then delete $B$:

    ```
      A        C        C
     / \      / \        \
    B   D    B   D        D
       /
      C
    ```

- Delete $B$ first, then delete $A$:

    ```
      A        A        D
     / \        \      /
    B   D        D    C
       /        /
      C        C
    ```


question:
 Suppose that instead of each node $x$ keeping the attribute $x.p$, pointing to $x$'s parent, it keeps $x.succ$, pointing to $x$'s successor. Give pseudocode for $\text{SEARCH}$, $\text{INSERT}$, and $\text{DELETE}$ on a binary search tree $T$ using this representation. These procedures should operate in time $O(h)$, where $h$ is the height of the tree $T$. ($\textit{Hint:}$ You may wish to implement a subroutine that returns the parent of a node.)

answer:
We don't need to change $\text{SEARCH}$.

We have to implement $\text{PARENT}$, which facilitates us a lot.

```cpp
PARENT(T, x)
    if x == T.root
        return NIL
    y = TREE-MAXIMUM(x).succ
    if y == NIL
        y = T.root
    else
        if y.left == x
            return y
        y = y.left
    while y.right != x
        y = y.right
    return y
```

```cpp
INSERT(T, z)
    y = NIL
    x = T.root
    pred = NIL
    while x != NIL
        y = x
        if z.key < x.key
            x = x.left
        else
            pred = x
            x = x.right
    if y == NIL
        T.root = z
        z.succ = NIL
    else if z.key < y.key
        y.left = z
        z.succ = y
        if pred != NIL
            pred.succ = z
    else
        y.right = z
        z.succ = y.succ
        y.succ = z
```

We modify $\text{TRANSPLANT}$ a bit since we no longer have to keep the pointer of $p$.

```cpp
TRANSPLANT(T, u, v)
    p = PARENT(T, u)
    if p == NIL
        T.root = v
    else if u == p.left
        p.left = v
    else
        p.right = v
```

Also, we have to implement $\text{TREE-PREDECESSOR}$, which helps us easily find the predecessor in line 2 of $\text{DELETE}$.

```cpp
TREE-PREDECESSOR(T, x)
    if x.left != NIL
        return TREE-MAXIMUM(x.left)
    y = T.root
    pred = NIL
    while y != NIL
        if y.key == x.key
            break
        if y.key < x.key
            pred = y
            y = y.right
        else
            y = y.left
    return pred
```

```cpp
DELETE(T, z)
    pred = TREE-PREDECESSOR(T, z)
    pred.succ = z.succ
    if z.left == NIL
        TRANSPLANT(T, z, z.right)
    else if z.right == NIL
        TRANSPLANT(T, z, z.left)
    else
        y = TREE-MIMIMUM(z.right)
        if PARENT(T, y) != z
            TRANSPLANT(T, y, y.right)
            y.right = z.right
        TRANSPLANT(T, z, y)
        y.left = z.left
```

Therefore, all these five algorithms are still $O(h)$ despite the increase in the hidden constant factor.


question:
 When node $z$ in $\text{TREE-DELETE}$ has two children, we could choose node $y$ as its predecessor rather than its successor. What other changes to $\text{TREE-DELETE}$ would be necessary if we did so? Some have argued that a fair strategy, giving equal priority to predecessor and successor, yields better empirical performance. How might $\text{TREE-DELETE}$ be changed to implement such a fair strategy?

answer:
Update line 5 so that $y$ is set equal to $\text{TREE-MAXIMUM}(z.left)$ and lines 6-12 so that every $y.left$ and $z.left$ is replaced with $y.right$ and $z.right$ and vice versa.

To implement the fair strategy, we could randomly decide each time $\text{TREE-DELETE}$ is called whether or not to use the predecessor or successor.


question: Show the results of deleting $C$, $P$, and $V$, in order, from the tree of Figure 18.8(f).
answer:
- Figure 18.8(f)

    ![](../img/18.3-1-1.png)

- delete $C$

    ![](../img/18.3-1-2.png)

- delete $P$

    ![](../img/18.3-1-3.png)

- delete $V$

    ![](../img/18.3-1-4.png)


question: Write pseudocode for $\text{B-TREE-DELETE}$.
answer:
The algorithm $\text{B-TREE-DELETE}(x, k)$ is a recursive procedure which deletes key $k$ from the B-tree rooted at node $x$.

The functions $\text{PREDECESSOR}(k, x)$ and $\text{SUCCESSOR}(k, x)$ return the predecessor and successor of $k$ in the B-tree rooted at $x$ respectively.

The cases where $k$ is the last key in a node have been omitted because the pseudocode is already unwieldy. For these, we simply use the left sibling as opposed to the right sibling, making the appropriate modifications to the indexing in the **for** loops.


question: Why don't we allow a minimum degree of $t = 1$?
answer:
According to the definition, minimum degree $t$ means every node other than the root must have at least $t - 1$ keys, and every internal node other than the root thus has at least $t$ children. So, when $t = 1$, it means every node other than the root must have at least $t - 1 = 0$ key, and every internal node other than the root thus has at least $t = 1$ child.

Thus, we can see that the minimum case doesn't exist, because no node exists with $0$ key, and no node exists with only $1$ child in a B-tree.


question: For what values of $t$ is the tree of Figure 18.1 a legal B-tree?
answer:
According to property 5 of B-tree, every node other than the root must have at least $t - 1$ keys and may contain at most $2t - 1$ keys. In Figure 18.1, the number of keys of each node (except the root) is either $2$ or $3$. So to make it a legal B-tree, we need to guarantee that $t - 1 \le 2 \text{ and } 2 t - 1 \ge 3$, which yields $2 \le t \le 3$. So $t$ can be $2$ or $3$.


question: Show all legal B-trees of minimum degree $2$ that represent $\\{1, 2, 3, 4, 5\\}$.
answer:
We know that every node except the root must have at least $t - 1 = 1$ keys, and at most $2t - 1 = 3$ keys. Also remember that the leaves stay in the same depth. Thus, there are $2$ possible legal B-trees:

- $$| 1, 2, 3, 4, 5 |$$

- $$| 3 |$$

    $$\swarrow \quad \searrow$$

    $$| 1, 2 | \qquad\qquad | 4, 5 |$$


question:
 As a function of the minimum degree $t$, what is the maximum number of keys that can be stored in a B-tree of height $h$?

answer:
$$
\begin{aligned}
n & = (1 + 2t + (2t) ^ 2 + \cdots + (2t) ^ {h}) \cdot (2t - 1) \\\\
  & = (2t)^{h + 1} - 1.
\end{aligned}
$$


question:
 Describe the data structure that would result if each black node in a red-black tree were to absorb its red children, incorporating their children with its own.

answer:
After absorbing each red node into its black parent, each black node may contain $1, 2$ ($1$ red child), or $3$ ($2$ red children) keys, and all leaves of the resulting tree have the same depth, according to property 5 of red-black tree (For each node, all paths from the node to descendant leaves contain the same number of black nodes). Therefore, a red-black tree will become a Btree with minimum degree $t = 2$, i.e., a 2-3-4 tree.


answer: (Omit!)
question:
 Show the results of inserting the keys
>
> $$F, S, Q, K, C, L, H, T, V, W, M, R, N, P, A, B, X, Y, D, Z, E$$
>
> in order into an empty B-tree with minimum degree $2$. Draw only the configurations of the tree just before some node must split, and also draw the final configuration.


question:
 Explain under what circumstances, if any, redundant $\text{DISK-READ}$ or $\text{DISK-WRITE}$ operations occur during the course of executing a call to $\text{B-TREE-INSERT}$. (A redundant $\text{DISK-READ}$ is a $\text{DISK-READ}$ for a page that is already in memory. A redundant $\text{DISK-WRITE}$ writes to disk a page of information that is identical to what is already stored there.)

answer:
In order to insert the key into a full child node but without its parent being full, we need the following operations:

- $\text{DISK-READ}$: Key placement
- $\text{DISK-WRITE}$: Split nodes
- $\text{DISK-READ}$: Get to the parent
- $\text{DISK-WRITE}$: Fill parent

If both were full, we'd have to do the same, but instead of the final step, repeat the above to split the parent node and write into the child nodes. With both considerations in mind, there should never be a redundant $\text{DISK-READ}$ or $\text{DISK-WRITE}$ on a $\text{B-TREE-INSERT}$.


question:
 Explain how to find the minimum key stored in a B-tree and how to find the predecessor of a given key stored in a B-tree.

answer:
- Finding the minimum in a B-tree is quite similar to finding a minimum in a binary search tree. We need to find the left most leaf for the given root, and return the first key.

    - **PRE:** $x$ is a node on the B-tree $T$. The top level call is $\text{B-TREE-FIND-MIN}(T.root)$.
    - **POST:** $\text{FCTVAL}$ is the minimum key stored in the subtree rooted at $x$.

    ```cpp
    B-TREE-FIND-MIN(x)
        if x == NIL           // T is empty
            return NIL
        else if x.leaf        // x is leaf
            return x.key[1]   // return the minimum key of x
        else
            DISK-READ(x.c[1])
            return B-TREE-FIND-MIN(x.c[1])
    ```

- Finding the predecessor of a given key $x.key_i$ is according to the following rules:

    1. If $x$ is not a leaf, return the maximum key in the $i$-th child of $x$, which is also the maximum key of the subtree rooted at $x.c_i$.
    2. If $x$ is a leaf and $i > 1$, return the $(i - 1)$st key of $x$, i.e., $x.key_{i - 1}$.
    3. Otherwise, look for the last node y (from the bottom up) and $j > 0$, such that $x.key_i$ is the leftmost key in $y.c_j$; if $j = 1$, return $\text{NIL}$ since $x.key_i$ is the minimum key in the tree; otherwise we return $y.key_{j - 1}$.

        - **PRE:** $x$ is a node on the B-tree $T$. $i$ is the index of the key.
        - **POST:** $\text{FCTVAL}$ is the predecessor of $x.key_i$.

        ```cpp
        B-TREE-FIND-PREDECESSOR(x, i)
            if !x.leaf
                DISK-READ(x.c[i])
                return B-TREE-FIND-MAX(x.c[i])
            else if i > 1      // x is a leaf and i > 1
                return x.key[i - 1]
            else
                z = x
                while true
                    if z.p == NIL    // z is root
                        return NIL   // z.key[i] is the minimum key in T; no predecessor
                      y = z.p
                      j = 1
                      DISK-READ(y.c[1])
                      while y.c[j] != x
                          j = j + 1
                          DISK-READ(y.c[j])
                      if j == 1
                          z = y
                      else
                          return y.key[j - 1]
        ```

        - **PRE:** $x$ is a node on the B-tree $T$. The top level call is $\text{B-TREE-FIND-MAX}(T.root)$.
        - **POST:** $\text{FCTVAL}$ is the maximum key stored in the subtree rooted at $x$.

        ```cpp
        B-TREE-FIND-MAX(x)
            if x == NIL           // T is empty
                return NIL
            else if x.leaf        // x is leaf
                return x.[x.n]    // return the maximum key of x
            else
                DISK-READ(x.c[x.n + 1])
                return B-TREE-FIND-MIN(x.c[x.n + 1])
        ```


question:
 Suppose that we insert the keys $\\{1, 2, \ldots, n\\}$ into an empty B-tree with minimum degree 2. How many nodes does the final B-tree have?

answer:
The final tree can have as many as $n - 1$ nodes. Unless $n = 1$ there cannot ever be $n$ nodes since we only ever insert a key into a non-empty node, so there will always be at least one node with $2$ keys.

Next observe that we will never have more than one key in a node which is not a right spine of our B-tree. This is because every key we insert is larger than all keys stored in the tree, so it will be inserted into the right spine of the tree. Nodes not in the right spine are a result of splits, and since $t = 2$, every split results in child nodes with one key each. The fewest possible number of nodes occurs when every node in the right spine has $3$ keys. In this case, $n = 2h + 2^{h + 1} - 1$ where $h$ is the height of the B-tree, and the number of nodes is $2^{h + 1} - 1$. Asymptotically these are the same, so the number of nodes is $\Theta(n)$.


question:
 Since leaf nodes require no pointers to children, they could conceivably use a different (larger) $t$ value than internal nodes for the same disk page size. Show how to modify the procedures for creating and inserting into a B-tree to handle this variation.

answer:
You would modify the insertion procedure by, in $\text{B-TREE-INSERT}$, check if the node is a leaf, and if it is, only split it if there twice as many keys stored as expected. Also, if an element needs to be inserted into a full leaf, we would split the leaf into two separate leaves, each of which doesn't have too many keys stored in it.


question:
 Suppose that we were to implement $\text{B-TREE-SEARCH}$ to use binary search rather than linear search within each node. Show that this change makes the CPU time required $O(\lg n)$, independently of how $t$ might be chosen as a function of $n$.

answer:
As in the $\text{TREE-SEARCH}$ procedure for binary search trees, the nodes encountered during the recursion form a simple path downward from the root of the tree. Thus, the $\text{B-TREE-SEARCH}$ procedure needs $O(h) = O(\log_t n)$ CPU time to search along the path, where $h$ is the height of the B-tree and $n$ is the number of keys in the B-tree, and we know that $h \le \log_t \frac{n + 1}{2}$. Since the number of keys in each nodeis less than $2t - 1$, a binary search within each node is $O(\lg t)$. So the total time is:

$$
\begin{aligned}
O(\lg t \cdot \log_t n) & = O(\lg t \cdot \frac{\lg n}{\lg t}) & \text{by changing the base of the logarithm.} \\\\
                        & = O(\lg n).
\end{aligned}
$$

Thus, the CPU time required is $O(\lg n)$.


question:
 Suppose that disk hardware allows us to choose the size of a disk page arbitrarily, but that the time it takes to read the disk page is $a + bt$, where $a$ and $b$ are specified constants and $t$ is the minimum degree for a B-tree using pages of the selected size. Describe how to choose $t$ so as to minimize (approximately) the B-tree search time. Suggest an optimal value of $t$ for the case in which $a = 5$ milliseconds and $b = 10$ microseconds.

answer:
$$\min \log_t n \cdot (a + bt) = \min \frac{a + bt}{\ln t}$$

$$\frac{\partial}{\partial t} (\frac{a + bt}{\ln t}) = - \frac{a + bt - bt \ln t}{t \ln^2 t}$$

$$
\begin{aligned}
 a + bt & = bt \ln t \\\\
5 + 10t & = 10t \ln t \\\\
      t & = e^{W \left(\frac{1}{2e} \right) + 1}, \\\\
\end{aligned}
$$

where $W$ is the LambertW function, and we should choose $t = 3$.


question:
 Using Figure 10.1 as a model, illustrate the result of each operation in the sequence $\text{PUSH}(S, 4)$, $\text{PUSH}(S, 1)$, $\text{PUSH}(S, 3)$, $\text{POP}(S)$, $\text{PUSH}(S, 8)$, and $\text{POP}(S)$ on an initially empty stack $S$ stored in array $S[1..6]$.

answer:
$$
\begin{array}{l|ccc}
\text{PUSH($S, 4$)} & 4 &   &   \\\\
\text{PUSH($S, 1$)} & 4 & 1 &   \\\\
\text{PUSH($S, 3$)} & 4 & 1 & 3 \\\\
\text{POP($S$)}     & 4 & 1 &   \\\\
\text{PUSH($S, 8$)} & 4 & 1 & 8 \\\\
\text{POP($S$)}     & 4 & 1 &
\end{array}
$$


question:
 Explain how to implement two stacks in one array $A[1..n]$ in such a way that neither stack overflows unless the total number of elements in both stacks together is $n$. The $\text{PUSH}$ and $\text{POP}$ operations should run in $O(1)$ time.

answer:
The first stack starts at $1$ and grows up towards n, while the second starts form $n$ and grows down towards $1$. Stack overflow happens when an element is pushed when the two stack pointers are adjacent.


question:
 Using Figure 10.2 as a model, illustrate the result of each operation in the sequence $\text{ENQUEUE}(Q, 4)$, $\text{ENQUEUE}(Q ,1)$, $\text{ENQUEUE}(Q, 3)$, $\text{DEQUEUE}(Q)$, $\text{ENQUEUE}(Q, 8)$, and $\text{DEQUEUE}(Q)$ on an initially empty queue $Q$ stored in array $Q[1..6]$.

answer:
$$
\begin{array}{l|cccc}
\text{ENQUEUE($Q, 4$)} & 4 &   &   &   \\\\
\text{ENQUEUE($Q, 1$)} & 4 & 1 &   &   \\\\
\text{ENQUEUE($Q, 3$)} & 4 & 1 & 3 &   \\\\
\text{DEQUEUE($Q$)}    &   & 1 & 3 &   \\\\
\text{ENQUEUE($Q, 8$)} &   & 1 & 3 & 8 \\\\
\text{DEQUEUE($Q$)}    &   &   & 3 & 8
\end{array}
$$


question: Rewrite $\text{ENQUEUE}$ and $\text{DEQUEUE}$ to detect underflow and overflow of a queue.
answer:
To detect underflow and overflow of a queue, we can implement $\text{QUEUE-EMPTY}$ and $\text{QUEUE-FULL}$ first.

```cpp
QUEUE-EMPTY(Q)
    if Q.head == Q.tail
        return true
    else return false
```

```cpp
QUEUE-FULL(Q)
    if Q.head == Q.tail + 1 or (Q.head == 1 and Q.tail == Q.length)
        return true
    else return false
```

```cpp
ENQUEUE(Q, x)
    if QUEUE-FULL(Q)
        error "overflow"
    else
        Q[Q.tail] = x
        if Q.tail == Q.length
            Q.tail = 1
        else Q.tail = Q.tail + 1
```

```cpp
DEQUEUE(Q)
    if QUEUE-EMPTY(Q)
        error "underflow"
    else
        x = Q[Q.head]
        if Q.head == Q.length
            Q.head = 1
        else Q.head = Q.head + 1
        return x
```


question:
 Whereas a stack allows insertion and deletion of elements at only one end, and a queue allows insertion at one end and deletion at the other end, a **_deque_** (double-ended queue) allows insertion and deletion at both ends. Write four $O(1)$-time procedures to insert elements into and delete elements from both ends of a deque implemented by an array.

answer:
The procedures $\text{QUEUE-EMPTY}$ and $\text{QUEUE-FULL}$ are implemented in Exercise 10.1-4.

```cpp
HEAD-ENQUEUE(Q, x)
    if QUEUE-FULL(Q)
        error "overflow"
    else
        if Q.head == 1
            Q.head = Q.length
        else Q.head = Q.head - 1
        Q[Q.head] = x
```

```cpp
TAIL-ENQUEUE(Q, x)
    if QUEUE-FULL(Q)
        error "overflow"
    else
        Q[Q.tail] = x
        if Q.tail == Q.length
            Q.tail = 1
        else Q.tail = Q.tail + 1
```

```cpp
HEAD-DEQUEUE(Q)
    if QUEUE-EMPTY(Q)
        error "underflow"
    else
        x = Q[Q.head]
        if Q.head == Q.length
            Q.head = 1
        else Q.head = Q.head + 1
        return x
```

```cpp
TAIL-DEQUEUE(Q)
    if QUEUE-EMPTY(Q)
        error "underflow"
    else
        if Q.tail == 1
            Q.tail = Q.length
        else Q.tail = Q.tail - 1
        x = Q[Q.tail]
        return x
```


question: Show how to implement a queue using two stacks. Analyze the running time of the queue operations.
answer:
- $\text{ENQUEUE}$: $\Theta(1)$.
- $\text{DEQUEUE}$: worst $O(n)$, amortized $\Theta(1)$.

Let the two stacks be $A$ and $B$.

$\text{ENQUEUE}$ pushes elements on $B$. $\text{DEQUEUE}$ pops elements from $A$. If $A$ is empty, the contents of $B$ are transfered to $A$ by popping them out of $B$ and pushing them to $A$. That way they appear in reverse order and are popped in the original.

A $\text{DEQUEUE}$ operation can perform in $\Theta(n)$ time, but that will happen only when $A$ is empty. If many $\text{ENQUEUE}$s and $\text{DEQUEUE}$s are performed, the total time will be linear to the number of elements, not to the largest length of the queue.


question: Show how to implement a stack using two queues. Analyze the running time of the stack operations.
answer:
- $\text{PUSH}$: $\Theta(1)$.
- $\text{POP}$: $\Theta(n)$.

We have two queues and mark one of them as active. $\text{PUSH}$ queues an element on the active queue. $\text{POP}$ should dequeue all but one element of the active queue and queue them on the inactive. The roles of the queues are then reversed, and the final element left in the (now) inactive queue is returned.

The $\text{PUSH}$ operation is $\Theta(1)$, but the $\text{POP}$ operation is $\Theta(n)$ where $n$ is the number of elements in the stack.


question:
 Draw a picture of the sequence $\langle 13, 4, 8, 19, 5, 11 \rangle$ stored as a doubly linked list using the multiple-array representation. Do the same for the single-array representation.

answer:
- A multiple-array representation with $L = 2$,

    $$
    \begin{array}{|r|c|c|c|c|c|c|c|}
    \hline
    index & 1 &  2      & 3 & 4 &  5 & 6 & 7 \\\\
    \hline
    next  &   &  3      & 4 & 5 &  6 & 7 & \diagup \\\\
    \hline
     key  &   & 13      & 4 & 8 & 19 & 5 & 11 \\\\
    \hline
    prev  &   & \diagup & 2 & 3 &  4 & 5 &  6 \\\\
    \hline
    \end{array}
    $$

- A single-array version with $L = 1$,

    $$
    \begin{array}{|r|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|}
    \hline
    index &  1 & 2 & 3       & 4 & 5 & 6 & 7 &  8 & 9 & 10 & 11 & 12 & 13 & 14 & 15 & 16 & 17       & 18 \\\\
    \hline
        key & 13 & 4 & \diagup & 4 & 7 & 1 & 8 & 10 & 4 & 19 & 13 &  7 &  5 & 16 & 10 & 11 & \diagup  & 13 \\\\
    \hline
    \end{array}
    $$


question:
 Write the procedures $\text{ALLOCATE-OBJECT}$ and $\text{FREE-OBJECT}$ for a homogeneous collection of objects implemented by the single-array representation.

answer:
```cpp
ALLOCATE-OBJECT()
    if free == NIL
        error "out of space"
    else x = free
        free = A[x + 1]
        return x
```

```cpp
FREE-OBJECT(x)
    A[x + 1] = free
    free = x
```


question:
 Why don't we need to set or reset the $prev$ attributes of objects in the implementation of the $\text{ALLOCATE-OBJECT}$ and $\text{FREE-OBJECT}$ procedures?

answer:
We implement $\text{ALLOCATE-OBJECT}$ and $\text{FREE-OBJECT}$ in the hope of managing the storage of currently non-used object in the free list so that one can be allocated for reusing. As the free list acts like a stack, to maintain this stack-like collection, we merely remember its first pointer and set the $next$ attribute of objects. There is no need to worry the $prev$ attribute, for it hardly has any impact on the resulting free list.


question:
 It is often desirable to keep all elements of a doubly linked list compact in storage, using, for example, the first $m$ index locations in the multiple-array representation. (This is the case in a paged, virtual-memory computing environment.) Explain how to implement the procedures $\text{ALLOCATE-OBJECT}$ and $\text{FREE-OBJECT}$ so that the representation is compact. Assume that there are no pointers to elements of the linked list outside the list itself. ($\textit{Hint:}$ Use the array implementation of a stack.)

answer:
```cpp
ALLOCATE-OBJECT()
    if STACK-EMPTY(F)
        error "out of space"
    else x = POP(F)
        return x
```

```cpp
FREE-OBJECT(x)
    p = F.top - 1
    p.prev.next = x
    p.next.prev = x
    x.key = p.key
    x.prev = p.prev
    x.next = p.next
    PUSH(F, p)
```


question:
 Let $L$ be a doubly linked list of length $n$ stored in arrays $key$, $prev$, and $next$ of length $m$. Suppose that these arrays are managed by $\text{ALLOCATE-OBJECT}$ and $\text{FREE-OBJECT}$ procedures that keep a doubly linked free list $F$. Suppose further that of the $m$ items, exactly $n$ are on list $L$ and $m - n$ are on the free list. Write a procedure $\text{COMPACTIFY-LIST}(L, F)$ that, given the list $L$ and the free list $F$, moves the items in $L$ so that they occupy array positions $1, 2, \ldots, n$ and adjusts the free list $F$ so that it remains correct, occupying array positions $n + 1, n + 2, \ldots, m$. The running time of your procedure should be $\Theta(n)$, and it should use only a constant amount of extra space. Argue that your procedure is correct.

answer:
We represent the combination of arrays $key$, $prev$, and $next$ by a multible-array $A$. Each object of $A$'s is either in list $L$ or in the free list $F$, but not in both. The procedure $\text{COMPACTIFY-LIST}$ transposes the first object in $L$ with the first object in $A$, the second objects until the list $L$ is exhausted.

```cpp
COMPACTIFY-LIST(L, F)
    TRANSPOSE(A[L.head], A[1])
    if F.head == 1
        F.head = L.head
    L.head = 1
    l = A[L.head].next
    i = 2
    while l != NIL
        TRANSPOSE(A[l], A[i])
        if F == i
            F = l
        l = A[l].next
        i = i + 1
```

```cpp
TRANSPOSE(a, b)
    SWAP(a.prev.next, b.prev.next)
    SWAP(a.prev, b.prev)
    SWAP(a.next.prev, b.next.prev)
    SWAP(a.next, b.next)
```


answer: ![](../img/10.4-1.png)
question:
 Draw the binary tree rooted at index $6$ that is represented by the following attributes:
>
> $$
> \begin{array}{cccc}
> \text{index} & key & left & right \\\\
> \hline
>  1 & 12 &  7         &  3         \\\\
>  2 & 15 &  8         & \text{NIL} \\\\
>  3 &  4 & 10         & \text{NIL} \\\\
>  4 & 10 &  5         &  9         \\\\
>  5 &  2 & \text{NIL} & \text{NIL} \\\\
>  6 & 18 &  1         &  4         \\\\
>  7 &  7 & \text{NIL} & \text{NIL} \\\\
>  8 & 14 &  6         &  2         \\\\
>  9 & 21 & \text{NIL} & \text{NIL} \\\\
> 10 &  5 & \text{NIL} & \text{NIL}
> \end{array}
> $$


question:
 Write an $O(n)$-time recursive procedure that, given an $n$-node binary tree, prints out the key of each node in the tree.

answer:
```cpp
PRINT-BINARY-TREE(T)
    x = T.root
    if x != NIL
        PRINT-BINARY-TREE(x.left)
        print x.key
        PRINT-BINARY-TREE(x.right)
```


question:
 Write an O$(n)$-time nonrecursive procedure that, given an $n$-node binary tree, prints out the key of each node in the tree. Use a stack as an auxiliary data structure.

answer:
```cpp
PRINT-BINARY-TREE(T, S)
    PUSH(S, T.root)
    while !STACK-EMPTY(S)
        x = S[S.top]
        while x != NIL      // store all nodes on the path towards the leftmost leaf
            PUSH(S, x.left)
            x = S[S.top]
        POP(S)              // S has NIL on its top, so pop it
        if !STACK-EMPTY(S)  // print this nodes, leap to its in-order successor
            x = POP(S)
            print x.key
            PUSH(S, x.right)
```


question:
 Write an $O(n)$-time procedure that prints all the keys of an arbitrary rooted tree with $n$ nodes, where the tree is stored using the left-child, right-sibling representation.

answer:
```cpp
PRINT-LCRS-TREE(T)
    x = T.root
    if x != NIL
        print x.key
        lc = x.left-child
        if lc != NIL
            PRINT-LCRS-TREE(lc)
            rs = lc.right-sibling
            while rs != NIL
                PRINT-LCRS-TREE(rs)
                rs = rs.right-sibling
```


question:
 Write an $O(n)$-time nonrecursive procedure that, given an $n$-node binary tree, prints out the key of each node. Use no more than constant extra space outside of the tree itself and do not modify the tree, even temporarily, during the procedure.

answer:
```cpp
PRINT-KEY(T)
    prev = NIL
    x = T.root
    while x != NIL
        if prev = x.parent
            print x.key
            prev = x
            if x.left
                x = x.left
            else
                if x.right
                    x = x.right
                else 
                    x = x.parent
        else if prev == x.left and x.right != NIL
                prev = x
                x = x.right
        else
                prev = x
                x = x.parent
```


question:
 The left-child, right-sibling representation of an arbitrary rooted tree uses three pointers in each node: _left-child_, _right-sibling_, and _parent_. From any node, its parent can be reached and identified in constant time and all its children can be reached and identified in time linear in the number of children. Show how to use only two pointers and one boolean value in each node so that the parent of a node or all of its children can be reached and identified in time linear in the number of children.

answer:
Use boolean to identify the last sibling, and the last sibling's right-sibling points to the parent.


question:
 Can you implement the dynamic-set operation $\text{INSERT}$ on a singly linked list in $O(1)$ time? How about $\text{DELETE}$?

answer:
- $\text{INSERT}$: can be implemented in constant time by prepending it to the list.

    ```cpp
    LIST-INSERT(L, x)
        x.next = L.head
        L.head = x
    ```

- $\text{DELETE}$: you can copy the value from the successor to element you want to delete, and then you can delete the successor in $O(1)$ time. This solution is not good in situations when you have a large object, in that case copying the whole object will be a bad idea.


question:
 Implement a stack using a singly linked list $L$. The operations $\text{PUSH}$ and $\text{POP}$ should still take $O(1)$ time.

answer:
```cpp
STACK-EMPTY(L)
    if L.head == NIL
        return true
    else return false
```

- $\text{PUSH}$: adds an element in the beginning of the list.

    ```cpp
    PUSH(L, x)
        x.next = L.head
        L.head = x
    ```

- $\text{POP}$: removes the first element from the list.

    ```cpp
    POP(L)
        if STACK-EMPTY(L)
            error "underflow"
        else
            x = L.head
            L.head = L.head.next
            return x
    ```


question:
 Implement a queue by a singly linked list $L$. The operations $\text{ENQUEUE}$ and $\text{DEQUEUE}$ should still take $O(1)$ time.

answer:
```cpp
QUEUE-EMPTY(L)
    if L.head == NIL
        return true
    else return false
```

- $\text{ENQUEUE}$: inserts an element at the end of the list. In this case we need to keep track of the last element of the list. We can do that with a sentinel.

    ```cpp
    ENQUEUE(L, x)
        if QUEUE-EMPTY(L)
            L.head = x
        else L.tail.next = x
        L.tail = x
        x.next = NIL
    ```

- $\text{DEQUEUE}$: removes an element from the beginning of the list.

    ```cpp
    DEQUEUE(L)
        if QUEUE-EMPTY(L)
            error "underflow"
        else
            x = L.head
            if L.head == L.tail
                L.tail = NIL
            L.head = L.head.next
            return x
    ```


question:
 As written, each loop iteration in the $\text{LIST-SEARCH}'$ procedure requires two tests: one for $x \ne L.nil$ and one for $x.key \ne k$. Show how to eliminate the test for $x \ne L.nil$ in each iteration.

answer:
```cpp
LIST-SEARCH'(L, k)
    x = L.nil.next
    L.nil.key = k
    while x.key != k
        x = x.next
    return x
```


question:
 Implement the dictionary operations $\text{INSERT}$, $\text{DELETE}$, and $\text{SEARCH}$ using singly linked, circular lists. What are the running times of your procedures?

answer:
- $\text{INSERT}$: $O(1)$.

    ```cpp
    LIST-INSERT''(L, x)
        x.next = L.nil.next
        L.nil.next = x
    ```

- $\text{DELETE}$: $O(n)$.

    ```cpp
    LIST-DELETE''(L, x)
        prev = L.nil
        while prev.next != x
            if prev.next == L.nil
                error "element not exist"
            prev = prev.next
        prev.next = x.next
    ```

- $\text{SEARCH}$: $O(n)$.

    ```cpp
    LIST-SEARCH''(L, k)
        x = L.nil.next
        while x != L.nil and x.key != k
            x = x.next
        return x
    ```


question:
 The dynamic-set operation $\text{UNION}$ takes two disjoint sets $S_1$ and $S_2$ as input, and it returns a set $S = S_1 \cup S_2$ consisting of all the elements of $S_1$ and $S_2$. The sets $S_1$ and $S_2$ are usually destroyed by the operation. Show how to support $\text{UNION}$ in $O(1)$ time using a suitable list data structure.

answer:
If both sets are a doubly linked lists, we just point link the last element of the first list to the first element in the second. If the implementation uses sentinels, we need to destroy one of them.

```cpp
LIST-UNION(L[1], L[2])
    L[2].nil.next.prev = L[1].nil.prev
    L[1].nil.prev.next = L[2].nil.next
    L[2].nil.prev.next = L[1].nil
    L[1].nil.prev = L[2].nil.prev
```


question:
 Give a $\Theta(n)$-time nonrecursive procedure that reverses a singly linked list of $n$ elements. The procedure should use no more than constant storage beyond that needed for the list itself.

answer:
```cpp
LIST-REVERSE(L)
    p[1] = NIL
    p[2] = L.head
    while p[2] != NIL
        p[3] = p[2].next
        p[2].next = p[1]
        p[1] = p[2]
        p[2] = p[3]
    L.head = p[1]
```


question:
 Explain how to implement doubly linked lists using only one pointer value $x.np$ per item instead of the usual two ($next$ and $prev$). Assume all pointer values can be interpreted as $k$-bit integers, and define $x.np$ to be $x.np = x.next \text{ XOR } x.prev$, the $k$-bit "exclusive-or" of $x.next$ and $x.prev$. (The value $\text{NIL}$ is represented by $0$.) Be sure to describe what information you need to access the head of the list. Show how to implement the $\text{SEARCH}$, $\text{INSERT}$, and $\text{DELETE}$ operations on such a list. Also show how to reverse such a list in $O(1)$ time.

answer:
```cpp
LIST-SEARCH(L, k)
    prev = NIL
    x = L.head
    while x != NIL and x.key != k
        next = prev XOR x.np
        prev = x
        x = next
    return x
```

```cpp
LIST-INSERT(L, x)
    x.np = NIL XOR L.tail
    if L.tail != NIL
        L.tail.np = (L.tail.np XOR NIL) XOR x   // tail.prev XOR x
    if L.head == NIL
        L.head = x
    L.tail = x
```

```cpp
LIST-DELETE(L, x)
    y = L.head
    prev = NIL
    while y != NIL
        next = prev XOR y.np
        if y != x
            prev = y
            y = next
        else
            if prev != NIL
                prev.np = (prev.np XOR y) XOR next  // prev.prev XOR next
            else L.head = next
            if next != NIL
                next.np = prev XOR (y XOR next.np)  // prev XOR next.next
            else L.tail = prev
```

```cpp
LIST-REVERSE(L)
    tmp = L.head
    L.head = L.tail
    L.tail = tmp
```


question:
 Using Figure 8.3 as a model, illustrate the operation of $\text{RADIX-SORT}$ on the following list of English words: COW, DOG, SEA, RUG, ROW, MOB, BOX, TAB, BAR, EAR, TAR, DIG, BIG, TEA, NOW, FOX.

answer:
$$
\begin{array}{cccc}
     0     &           1           &           2           &           3           \\\\
\hline
\text{COW} & \text{SE$\textbf{A}$} & \text{T$\textbf{A}$B} & \text{$\textbf{B}$AR} \\\\
\text{DOG} & \text{TE$\textbf{A}$} & \text{B$\textbf{A}$R} & \text{$\textbf{B}$IG} \\\\
\text{SEA} & \text{MO$\textbf{B}$} & \text{E$\textbf{A}$R} & \text{$\textbf{B}$OX} \\\\
\text{RUG} & \text{TA$\textbf{B}$} & \text{T$\textbf{A}$R} & \text{$\textbf{C}$OW} \\\\
\text{ROW} & \text{DO$\textbf{G}$} & \text{S$\textbf{E}$A} & \text{$\textbf{D}$IG} \\\\
\text{MOB} & \text{RU$\textbf{G}$} & \text{T$\textbf{E}$A} & \text{$\textbf{D}$OG} \\\\
\text{BOX} & \text{DI$\textbf{G}$} & \text{D$\textbf{I}$G} & \text{$\textbf{E}$AR} \\\\
\text{TAB} & \text{BI$\textbf{G}$} & \text{B$\textbf{I}$G} & \text{$\textbf{F}$OX} \\\\
\text{BAR} & \text{BA$\textbf{R}$} & \text{M$\textbf{O}$B} & \text{$\textbf{M}$OB} \\\\
\text{EAR} & \text{EA$\textbf{R}$} & \text{D$\textbf{O}$G} & \text{$\textbf{N}$OW} \\\\
\text{TAR} & \text{TA$\textbf{R}$} & \text{C$\textbf{O}$W} & \text{$\textbf{R}$OW} \\\\
\text{DIG} & \text{CO$\textbf{W}$} & \text{R$\textbf{O}$W} & \text{$\textbf{R}$UG} \\\\
\text{BIG} & \text{RO$\textbf{W}$} & \text{N$\textbf{O}$W} & \text{$\textbf{S}$EA} \\\\
\text{TEA} & \text{NO$\textbf{W}$} & \text{B$\textbf{O}$X} & \text{$\textbf{T}$AB} \\\\
\text{NOW} & \text{BO$\textbf{X}$} & \text{F$\textbf{O}$X} & \text{$\textbf{T}$AR} \\\\
\text{FOX} & \text{FO$\textbf{X}$} & \text{R$\textbf{U}$G} & \text{$\textbf{T}$EA} \\\\
\end{array}
$$


question:
 Which of the following sorting algorithms are stable: insertion sort, merge sort, heapsort, and quicksort? Give a simple scheme that makes any sorting algorithm stable. How much additional time and space does your scheme entail?

answer:
Insertion sort and merge sort are stable. Heapsort and quicksort are not.

To make any sorting algorithm stable we can preprocess, replacing each element of an array with an ordered pair. The first entry will be the value of the element, and the second value will be the index of the element.

For example, the array $[2, 1, 1, 3, 4, 4, 4]$ would become $[(2, 1), (1, 2), (1, 3), (3, 4), (4, 5), (4, 6), (4, 7)]$. We now interpret $(i, j) < (k, m)$ if $i < k$ or $i = k$ and $j < m$. Under this definition of less-than, the algorithm is guaranteed to be stable because each of our new elements is distinct and the index comparison ensures that if a repeat element appeared later in the original array, it must appear later in the sorted array. This doubles the space requirement, but the running time will be asymptotically unchanged.


question:
 Use induction to prove that radix sort works. Where does your proof need the assumption that the intermediate sort is stable?

answer:
**Loop invariant:** At the beginning of the **for** loop, the array is sorted on the last $i − 1$ digits.

**Initialization:** The array is trivially sorted on the last $0$ digits.

**Maintenance:** Let's assume that the array is sorted on the last $i − 1$ digits. After we sort on the $i$th digit, the array will be sorted on the last $i$ digits. It is obvious that elements with different digit in the $i$th position are ordered accordingly; in the case of the same $i$th digit, we still get a correct order, because we're using a stable sort and the elements were already sorted on the last $i − 1$ digits.

**Termination:** The loop terminates when $i = d + 1$. Since the invariant holds, we have the numbers sorted on $d$ digits.


question: Show how to sort $n$ integers in the range $0$ to $n^3 - 1$ in $O(n)$ time.
answer:
First run through the list of integers and convert each one to base $n$, then radix sort them. Each number will have at most $\log_n n^3 = 3$ digits so there will only need to be $3$ passes. For each pass, there are $n$ possible values which can be taken on, so we can use counting sort to sort each digit in $O(n)$ time.


question:
 In the first card-sorting algorithm in this section, exactly how many sorting passes are needed to sort $d$-digit decimal numbers in the worst case? How many piles of cards would an operator need to keep track of in the worst case?

answer:
Given $n$ $d$-digit numbers in which each digit can take on up to $k$ possible values, we'll perform $\Theta(k^d)$ passes and keep track of $\Theta(nk)$ piles in the worst case.


question:
 Using Figure 8.4 as a model, illustrate the operation of $\text{BUCKET-SORT}$ on the array $A = \langle .79, .13, .16, .64, .39, .20, .89, .53, .71, .42 \rangle$.

answer:
$$
\begin{array}{cl}
R &         \\\\
\hline
0 &         \\\\
1 & .13 .16 \\\\
2 & .20     \\\\
3 & .39     \\\\
4 & .42     \\\\
5 & .53     \\\\
6 & .64     \\\\
7 & .79 .71 \\\\
8 & .89     \\\\
9 &         \\\\
\end{array}
$$

$$A = \langle.13, .16, .20, .39, .42, .53, .64, .71, .79, .89 \rangle.$$


question:
 Explain why the worst-case running time for bucket sort is $\Theta(n^2)$. What simple change to the algorithm preserves its linear average-case running time and makes its worst-case running time $O(n\lg n)$?

answer:
If all the keys fall in the same bucket and they happen to be in reverse order, we have to sort a single bucket with $n$ items in reversed order with insertion sort. This is $\Theta(n^2)$.

We can use merge sort or heapsort to improve the worst-case running time. Insertion sort was chosen because it operates well on linked lists, which has optimal time and requires only constant extra space for short linked lists. If we use another sorting algorithm, we have to convert each list to an array, which might slow down the algorithm in practice.


question:
 Let $X$ be a random variable that is equal to the number of heads in two flips of a fair coin. What is $\text E[X^2]$? What is $\text E^2[X]$?

answer:
$$
\begin{aligned}
  \text E[X] & = 2 \cdot \frac{1}{4} + 1 \cdot \frac{1}{2} + 0 \cdot \frac{1}{4} = 1 \\\\
\text E[X^2] & = 4 \cdot \frac{1}{4} + 1 \cdot \frac{1}{2} + 0 \cdot \frac{1}{4} = 1.5 \\\\
\text E^2[X] & = \text E[X] \cdot \text E[X] = 1 \cdot 1 = 1.
\end{aligned}
$$


question:
 We are given $n$ points in the unit circle, $p_i = (x_i, y_i)$, such that $0 < x_i^2 + y_i^2 \le 1$ for $i = 1, 2, \ldots, n$. Suppose that the points are uniformly distributed; that is, the probability of finding a point in any region of the circle is proportional to the area of that region. Design an algorithm with an average-case running time of $\Theta(n)$ to sort the $n$ points by their distances $d_i = \sqrt{x_i^2 + y_i^2}$ from the origin. ($\textit{Hint:}$ Design the bucket sizes in $\text{BUCKET-SORT}$ to reﬂect the uniform distribution of the points in the unit circle.)

answer:
Bucket sort by radius,

$$
\begin{aligned}
\pi r_i^2 & = \frac{i}{n} \cdot \pi 1^2 \\\\
r_i & = \sqrt{\frac{i}{n}}.
\end{aligned}
$$


question:
 A **_probability distribution function_** $P(x)$ for a random variable $X$ is defined by $P(x) = \Pr\\{X \le x\\}$. Suppose that we draw a list of $n$ random variables $X_1, X_2, \ldots, X_n$ from a continuous probability distribution function $P$ that is computable in $O(1)$ time. Give an algorithm that sorts these numbers in linear average-case time.

answer:
Bucket sort by $p_i$, so we have $n$ buckets: $[p_0, p_1), [p_1, p_2), \dots, [p_{n - 1}, p_n)$. Note that not all buckets are the same size, which is ok as to ensure linear run time, the inputs should on average be uniformly distributed amongst all buckets, of which the intervals defined with $p_i$ will do so.

$p_i$ is defined as follows:

$$P(p_i) = \frac{i}{n}.$$


question:
 Using Figure 8.2 as a model, illustrate the operation of $\text{COUNTING-SORT}$ on the array $A = \langle 6, 0, 2, 0, 1, 3, 4, 6, 1, 3, 2 \rangle$.

answer:
We have that $C = \langle 2, 4, 6, 8, 9, 9, 11 \rangle$. Then, after successive iterations of the loop on lines 10-12, we have

$$
\begin{aligned}
B & = \langle, , , , , 2, , , , , \rangle, \\\\
B & = \langle, , , , , 2, 3, , , \rangle, \\\\
B & = \langle, , , 1, , 2, 3, , , \rangle
\end{aligned}
$$

and at the end,

$$B = \langle 0, 0, 1, 1, 2, 2, 3, 3, 4, 6, 6 \rangle.$$


question: Prove that $\text{COUNTING-SORT}$ is stable.
answer:
Suppose positions $i$ and $j$ with $i < j$ both contain some element $k$. We consider lines 10 through 12 of $\text{COUNTING-SORT}$, where we construct the output array. Since $j > i$, the loop will examine $A[j]$ before examining $A[i]$. When it does so, the algorithm correctly places $A[j]$ in position $m = C[k]$ of $B$. Since $C[k]$ is decremented in line 12, and is never again incremented, we are guaranteed that when the **for** loop examines $A[i]$ we will have $C[k] < m$. Therefore $A[i]$ will be placed in an earlier position of the output array, proving stability.


question:
 Suppose that we were to rewrite the **for** loop header in line 10 of the $\text{COUNTING-SORT}$ as
>
> ```cpp
> 10  for j = 1 to A.length
> ```
>
> Show that the algorithm still works properly. Is the modified algorithm stable?

answer:
The algorithm still works correctly. The order that elements are taken out of $C$ and put into $B$ doesn't affect the placement of elements with the same key. It will still fill the interval $(C[k − 1], C[k]]$ with elements of key $k$. The question of whether it is stable or not is not well phrased. In order for stability to make sense, we would need to be sorting items which have information other than their key, and the sort as written is just for integers, which don't. We could think of extending this algorithm by placing the elements of $A$ into a collection of elements for each cell in array $C$. Then, if we use a FIFO collection, the modification of line 10 will make it stable, if we use LILO, it will be anti-stable.


question:
 Describe an algorithm that, given n integers in the range $0$ to $k$, preprocesses its input and then answers any query about how many of the $n$ integers fall into a range $[a..b]$ in $O(1)$ time. Your algorithm should use $\Theta(n + k)$ preprocessing time.

answer:
The algorithm will begin by preprocessing exactly as $\text{COUNTING-SORT}$ does in lines 1 through 9, so that $C[i]$ contains the number of elements less than or equal to $i$ in the array. When queried about how many integers fall into a range $[a..b]$, simply compute $C[b] − C[a − 1]$. This takes $O(1)$ times and yields the desired output.


question: What is the smallest possible depth of a leaf in a decision tree for a comparison sort?
answer:
For a permutation $a_1 \le a_2 \le \ldots \le a_n$, there are $n - 1$ pairs of relative ordering, thus the smallest possible depth is $n - 1$.


question:
 Obtain asymptotically tight bounds on $\lg(n!)$ without using Stirling's approximation. Instead, evaluate the summation $\sum_{k = 1}^n\lg k$ using techniques from Section A.2.

answer:
$$
\begin{aligned}
\sum_{k = 1}^n\lg k
    & \le \sum_{k = 1}^n\lg n \\\\
    & =   n\lg n.
\end{aligned}
$$

$$
\begin{aligned}
\sum_{k = 1}^n\lg k
    & =   \sum_{k = 2}^{n / 2} \lg k + \sum_{k = n / 2}^n\lg k \\\\
    & \ge \sum_{k = 1}^{n / 2} 1 + \sum_{k = n / 2}^n\lg n / 2 \\\\
    & =   \frac{n}{2} + \frac{n}{2}(\lg n - 1) \\\\
    & =   \frac{n}{2}\lg n.
\end{aligned}
$$


question:
 Show that there is no comparison sort whose running time is linear for at least half of the $n!$ inputs of length $n$. What about a fraction of $1 / n$ of the inputs of length $n$? What about a fraction $1 / 2^n$?

answer:
Consider a decision tree of height $h$ with $r$ reachable leaves corresponding to a comparison sort on $n$ elements. From **Theorem 8.1**, We have $n! / 2 \le n! \le r \le 2^h$. By taking logarithms, we have

$$h \ge \lg (n! / 2) = \lg (n!) - 1 = \Theta (n\lg n) - 1 = \Theta (n\lg n).$$

From the equation above, there is no comparison sort whose running time is linear for at least half of the $n!$ inputs of length $n$.

Consider the $1/n$ of inputs of length $n$ condition. we have $(1/n)n! \le n! \le r \le 2^h$. By taking logarithms, we have

$$h \ge \lg (n! / n) = \lg (n!) - \lg n = \Theta (n\lg n) - \lg n = \Theta (n\lg n).$$

From the equation above, there is no comparison sort whose running time is linear for $1/n$ of the $n!$ inputs of length $n$.

Consider the $1 / 2^n$ of inputs of length $n$ condition. we have $(1/2^n)n! \le n! \le r \le 2^h$. By taking logarithms, we have

$$h \ge \lg (n! / 2^n) = \lg (n!) - n = \Theta (n\lg n) - n = \Theta (n\lg n).$$

From the equation above, there is no comparison sort whose running time is linear for $1/2^n$ of the $n!$ inputs of length $n$.


question:
 Suppose that you are given a sequence of $n$ elements to sort. The input sequence consists of $n / k$ subsequences, each containing $k$ elements. The elements in a given subsequence are all smaller than the elements in the succeeding subsequence and larger than the elements in the preceding subsequence. Thus, all that is needed to sort the whole sequence of length $n$ is to sort the $k$ elements in each of the $n / k$ subsequences. Show an $\Omega(n\lg k)$ lower bound on the number of comparisons needed to solve this variant of the sorting problem. ($\textit{Hint:}$ It is not rigorous to simply combine the lower bounds for the individual subsequences.)

answer:
Assume that we need to construct a binary decision tree to represent comparisons. Since length of each subsequece is $k$, there are $(k!)^{n / k}$ possible output permutations. To compute the height $h$ of the decision tree, we must have $(k!)^{n / k} \le 2^h$. Taking logs on both sides, we know that

$$h \ge \frac{n}{k} \times \lg (k!) \ge \frac{n}{k} \times \left( \frac{k\ln k - k}{\ln 2} \right) = \frac{n\ln k - n}{\ln 2} = \Omega (n\lg k).$$


question:
 Show the $d$ and $\pi$ values that result from running breadth-first search on the directed graph of Figure 22.2(a), using vertex $3$ as the source.

answer:
$$
\begin{array}{c|cccccc}
\text{vertex} & 1 & 2 & 3 & 4 & 5 & 6 \\\\
\hline
d & \infty & 3 & 0 & 2 & 1 & 1 \\\\
\pi & \text{NIL} & 4 & \text{NIL} & 5 & 3 & 3
\end{array}
$$


question:
 Show the $d$ and $\pi$ values that result from running breadth-first search on the undirected graph of Figure 22.3, using vertex $u$ as the source.

answer:
$$
\begin{array}{c|cccccc}
\text{vertex} & r & s & t & u & v & w & x & y \\\\
\hline
d & 4 & 3 & 1 & 0 & 5 & 2 & 1 & 1 \\\\
\pi & s & w & u & \text{NIL} & r & t & u & u
\end{array}
$$


question:
 Show that using a single bit to store each vertex color suffices by arguing that the $\text{BFS}$ procedure would produce the same result if lines 5 and 14 were removed.

answer:
The textbook introduces the $\text{GRAY}$ color for the pedagogical purpose to distinguish between the $\text{GRAY}$ nodes (which are enqueued) and the $\text{BLACK}$ nodes (which are dequeued).

Therefore, it suffices to use a single bit to store each vertex color.


question:
 What is the running time of $\text{BFS}$ if we represent its input graph by an adjacency matrix and modify the algorithm to handle this form of input?

answer:
The time of iterating all edges becomes $O(V^2)$ from $O(E)$. Therefore, the running time is $O(V + V^2) = O(V^2)$.


question:
 Argue that in a breadth-first search, the value $u.d$ assigned to a vertex $u$ is independent of the order in which the vertices appear in each adjacency list. Using Figure 22.3 as an example, show that the breadth-first tree computed by $\text{BFS}$ can depend on the ordering within adjacency lists.

answer:
First, we will show that the value $d$ assigned to a vertex is independent of the order that entries appear in adjacency lists. To show this, we rely on theorem 22.5, which proves correctness of $\text{BFS}$. In particular, the theorem states that $v.d = \delta(s, v)$ at the termination of  $\text{BFS}$. Since $\delta(s, v)$ is a property of the underlying graph, for any adjacency list representation of the graph (including any reordering of the adjacency lists), $\delta(s, v)$ will not change. Since the $d$ values are equal to $\delta(s, v)$ and $\delta(s, v)$ is invariant for any ordering of the adjacency list, $d$ is also not dependent of the ordering of the adjacency list.

Now, to show that $\pi$ does depend on the ordering of the adjacency lists, we will be using Figure 22.3 as a guide.

First, we note that in the given worked out procedure, we have that in the adjacency list for $w$, $t$ precedes $x$. Also, in the worked out procedure, we have that $u.\pi = t$.

Now, suppose instead that we had $x$ preceding $t$ in the adjacency list of $w$. Then, it would get added to the queue before $t$, which means that it would $u$ as it's child before we have a chance to process the children of $t$. This will mean that $u.\pi = x$ in this different ordering of the adjacency list for $w$.


question:
 Give an example of a directed graph $G = (V, E)$, a source vertex $s \in V$, and a set of tree edges $E_\pi \subseteq E$ such that for each vertex $v \in V$, the unique simple path in the graph $(V, E_\pi)$ from $s$ to $v$ is a shortest path in $G$, yet the set of edges $E_\pi$ cannot be produced by running $\text{BFS}$ on $G$, no matter how the vertices are ordered in each adjacency list.

answer:
Let $G$ be the graph shown in the first picture, $G_\pi = (V, E_\pi)$ be the graph shown in the second picture, and $s$ be the source vertex.

We could see that $E_\pi$ will never be produced by running BFS on $G$.

<center>
![](../img/22.2-6-2.png)
![](../img/22.2-6-1.png)
</center>

- If $y$ precedes $v$ in the $Adj[s]$. We'll dequeue $y$ before $v$, so $u.\pi$ and $x.\pi$ are both $y$. However, this is not the case.
- If $v$ preceded $y$ in the $Adj[s]$. We'll dequeue $v$ before $y$, so $u.\pi$ and $x.\pi$ are both $v$, which again isn't true.

Nonetheless, the unique simple path in $G_\pi$ from $s$ to any vertex is a shortest path in $G$.


question:
 There are two types of professional wrestlers: "babyfaces" ("good guys") and "heels" ("bad guys"). Between any pair of professional wrestlers, there may or may not be a rivalry. Suppose we have $n$ professional wrestlers and we have a list of $r$ pairs of wrestlers for which there are rivalries. Give an $O(n + r)$-time algorithm that determines whether it is possible to designate some of the wrestlers as babyfaces and the remainder as heels such that each rivalry is between a babyface and a heel. If it is possible to perform such a designation, your algorithm should produce it.

answer:
This problem is basically just a obfuscated version of two coloring. We will try to color the vertices of this graph of rivalries by two colors, "babyface" and "heel". To have that no two babyfaces and no two heels have a rivalry is the same as saying that the coloring is proper. To two color, we perform a breadth first search of each connected component to get the $d$ values for each vertex. Then, we give all the odd ones one color say "heel", and all the even d values a different color. We know that no other coloring will succeed where this one fails since if we gave any other coloring, we would have that a vertex $v$ has the same color as $v.\pi$ since $v$ and $v.\pi$ must have different parities for their $d$ values. Since we know that there is no better coloring, we just need to check each edge to see if this coloring is valid. If each edge works, it is possible to find a designation, if a single edge fails, then it is not possible. Since the BFS took time $O(n + r)$ and the checking took time $O(r)$, the total runtime is $O(n + r)$.


question:
 The **_diameter_** of a tree $T = (V, E)$ is defined as $\max_{u,v \in V} \delta(u, v)$, that is, the largest of all shortest-path distances in the tree. Give an efficient algorithm to compute the diameter of a tree, and analyze the running time of your algorithm.

answer:
Suppose that a and b are the endpoints of the path in the tree which achieve the diameter, and without loss of generality assume that $a$ and $b$ are the unique pair which do so. Let $s$ be any vertex in $T$. We claim that the result of a single $\text{BFS}$ will return either $a$ or $b$ (or both) as the vertex whose distance from $s$ is greatest.

To see this, suppose to the contrary that some other vertex $x$ is shown to be furthest from $s$. (Note that $x$ cannot be on the path from $a$ to $b$, otherwise we could extend). Then we have

$$d(s, a) < d(s, x)$$

and

$$d(s, b) < d(s, x).$$

Let $c$ denote the vertex on the path from $a$ to $b$ which minimizes $d(s, c)$. Since the graph is in fact a tree, we must have

$$d(s, a) = d(s, c) + d(c, a)$$

and

$$d(s, b) = d(s, c) + d(c, b).$$

(If there were another path, we could form a cycle). Using the triangle inequality and inequalities and equalities mentioned above we must have

$$
\begin{aligned}
d(a, b) + 2d(s, c) & = d(s, c) + d(c, b) + d(s, c) + d(c, a) \\\\
                   & < d(s, x) + d(s, c) + d(c, b).
\end{aligned}
$$

I claim that $d(x, b) = d(s, x) + d(s, b)$. If not, then by the triangle inequality we must have a strict less-than. In other words, there is some path from $x$ to $b$ which does not go through $c$. This gives the contradiction, because it implies there is a cycle formed by concatenating these paths. Then we have

$$d(a, b) < d(a, b) + 2d(s, c) < d(x, b).$$

Since it is assumed that $d(a, b)$ is maximal among all pairs, we have a contradiction. Therefore, since trees have $|V| - 1$ edges, we can run $\text{BFS}$ a single time in $O(V)$ to obtain one of the vertices which is the endpoint of the longest simple path contained in the graph. Running $\text{BFS}$ again will show us where the other one is, so we can solve the diameter problem for trees in $O(V)$.


question:
 Let $G = (V, E)$ be a connected, undirected graph. Give an $O(V + E)$-time algorithm to compute a path in $G$ that traverses each edge in $E$ exactly once in each direction. Describe how you can find your way out of a maze if you are given a large supply of pennies.

answer:
First, the algorithm computes a minimum spanning tree of the graph. Note that this can be done using the procedures of Chapter 23. It can also be done by performing a breadth first search, and restricting to the edges between $v$ and $v.\pi$ for every $v$. To aide in not double counting edges, fix any ordering $\le$ on the vertices before hand. Then, we will construct the sequence of steps by calling $\text{MAKE-PATH}(s)$, where $s$ was the root used for the $\text{BFS}$.

```cpp
MAKE-PATH(u)
    for each v ∈ Adj[u] but not in the tree such that u ≤ v
        go to v and back to u
    for each v ∈ Adj[u] but not equal to u.π
        go to v
        perform the path proscribed by MAKE-PATH(v)
    go to u.π
```


question:
 Show the ordering of vertices produced by $\text{TOPOLOGICAL-SORT}$ when it is run on the dag of Figure 22.8, under the assumption of Exercise 22.3-2.

answer:
Our start and finish times from performing the $\text{DFS}$ are

$$
\begin{array}{ccc}
\text{label} & d & f \\\\
\hline
m & 1  & 20 \\\\
q & 2  & 5  \\\\
t & 3  & 4  \\\\
r & 6  & 19 \\\\
u & 7  & 8  \\\\
y & 9  & 18 \\\\
v & 10 & 17 \\\\
w & 11 & 14 \\\\
z & 12 & 13 \\\\
x & 15 & 16 \\\\
n & 21 & 26 \\\\
o & 22 & 25 \\\\
s & 23 & 24 \\\\
p & 27 & 28
\end{array}
$$

And so, by reading off the entries in decreasing order of finish time, we have the sequence $p, n, o, s, m, r, y, v, x, w, z, u, q, t$.


question:
 Give a linear-time algorithm that takes as input a directed acyclic graph $G = (V, E)$ and two vertices $s$ and $t$, and returns the number of simple paths from $s$ to $t$ in $G$. For example, the directed acyclic graph of Figure 22.8 contains exactly four simple paths from vertex $p$ to vertex $v: pov$, $poryv$, $posryv$, and $psryv$. (Your algorithm needs only to count the simple paths, not list them.)

answer:
The algorithm works as follows. The attribute $u.paths$ of node $u$ tells the number of simple paths from $u$ to $v$, where we assume that $v$ is fixed throughout the entire process. First of all, a topo sort should be conducted and list the vertex between $u$, $v$ as $\\{v[1], v[2], \dots, v[k - 1]\\}$. To count the number of paths, we should construct a solution from $v$ to $u$. Let's call $u$ as $v[0]$ and $v$ as $v[k]$, to avoid overlapping subproblem, the number of paths between $v_k$ and $u$ should be remembered and used as $k$ decrease to $0$. Only in this way can we solve the problem in $\Theta(V + E)$.

An bottom-up iterative version is possible only if the graph uses adjacency matrix so whether $v$ is adjacency to $u$ can be determined in $O(1)$ time. But building a adjacency matrix would cost $\Theta(|V|^2)$, so never mind.

```cpp
SIMPLE-PATHS(G, u, v)
    TOPO-SORT(G)
    let {v[1], v[2]..v[k - 1]} be the vertex between u and v
    v[0] = u
    v[k] = v
    for j = 0 to k - 1
        DP[j] = ∞
    DP[k] = 1
    return SIMPLE-PATHS-AID(G, DP, 0)
```

```cpp
SIMPLE-PATHS-AID(G, DP, i)
    if i > k
        return 0
    else if DP[i] != ∞
        return DP[i]
    else
       DP[i] = 0
       for v[m] in G.adj[v[i]] and 0 < m ≤ k
            DP[i] += SIMPLE-PATHS-AID(G, DP, m)
       return DP[i]
```


answer: (Removed)
question:
 Give an algorithm that determines whether or not a given undirected graph $G = (V, E)$ contains a cycle. Your algorithm should run in $O(V)$ time, independent of $|E|$.


question:
 Prove or disprove: If a directed graph $G$ contains cycles, then $\text{TOPOLOGICAL-SORT}(G)$ produces a vertex ordering that minimizes the number of "bad" edges that are inconsistent with the ordering produced.

answer:
This is not true. Consider the graph $G$ consisting of vertices $a, b, c$, and $d$. Let the edges be $(a, b)$, $(b, c)$, $(a, d)$, $(d, c)$, and $(c, a)$. Suppose that we start the $\text{DFS}$ of $\text{TOPOLOGICAL-SORT}$ at vertex $c$. Assuming that $b$ appears before $d$ in the adjacency list of $a$, the order, from latest to earliest, of finish times is $c, a, d, b$.

The "bad" edges in this case are $(b, c)$ and $(d, c)$. However, if we had instead ordered them by $a, b, d, c$ then the only bad edges would be $(c, a)$. Thus $\text{TOPOLOGICAL-SORT}$ doesn't always minimizes the number of "bad" edges


answer: (Removed)
question:
 Another way to perform topological sorting on a directed acyclic graph $G = (V, E)$ is to repeatedly find a vertex of $\text{in-degree}$ $0$, output it, and remove it and all of its outgoing edges from the graph. Explain how to implement this idea so that it runs in time $O(V + E)$. What happens to this algorithm if $G$ has cycles?


question: How can the number of strongly connected components of a graph change if a new edge is added?
answer:
It can either stay the same or decrease. To see that it is possible to stay the same, just suppose you add some edge to a cycle. To see that it is possible to decrease, suppose that your original graph is on three vertices, and is just a path passing through all of them, and the edge added completes this path to a cycle. To see that it cannot increase, notice that adding an edge cannot remove any path that existed before.

So, if $u$ and $v$ are in the same connected component in the original graph, then there are a path from one to the other, in both directions. Adding an edge wont disturb these two paths, so we know that $u$ and $v$ will still be in the same $\text{SCC}$ in the graph after adding the edge. Since no components can be split apart, this means that the number of them cannot increase since they form a partition of the set of vertices.


question:
 Show how the procedure $\text{STRONGLY-CONNECTED-COMPONENTS}$ works on the graph of Figure 22.6. Specifically, show the finishing times computed in line 1 and the forest produced in line 3. Assume that the loop of lines 5–7 of $\text{DFS}$ considers vertices in alphabetical order and that the adjacency lists are in alphabetical order.

answer:
The finishing times of each vertex were computed in exercise 22.3-2. The forest consists of 5 trees, each of which is a chain. We'll list the vertices of each tree in order from root to leaf: $r$, $u$, $q - y - t$, $x - z$, and $s - w - v$.


question:
 Professor Bacon claims that the algorithm for strongly connected components would be simpler if it used the original (instead of the transpose) graph in the second depth-first search and scanned the vertices in order of _increasing_ finishing times. Does this simpler algorithm always produce correct results?

answer:
Professor Bacon's suggestion doesn't work out. As an example, suppose that our graph is on the three vertices $\\{1, 2, 3\\}$ and consists of the edges $(2, 1), (2, 3), (3, 2)$. Then, we should end up with $\\{2, 3\\}$ and $\\{1\\}$ as our $\text{SCC}$'s. However, a possible $\text{DFS}$ starting at $2$ could explore $3$ before $1$, this would mean that the finish time of $3$ is lower than of $1$ and $2$. This means that when we first perform the $\text{DFS}$ starting at $3$. However, a $\text{DFS}$ starting at $3$ will be able to reach all other vertices. This means that the algorithm would return that the entire graph is a single $\text{SCC}$, even though this is clearly not the case since there is neither a path from $1$ to $2$ of from $1$ to $3$.


question:
 Prove that for any directed graph $G$, we have $((G^\text T)^{\text{SCC}})^\text T = G^{\text{SCC}}$. That is, the transpose of the component graph of $G^\text T$ is the same as the component graph of $G$.

answer:
First observe that $C$ is a strongly connected component of $G$ if and only if it is a strongly connected component of $G^\text T$. Thus the vertex sets of $G^{\text{SCC}}$ and $(G^\text T)^{\text{SCC}}$ are the same, which implies the vertex sets of $((G^\text T)^\text{SCC})^\text T$ and $G^{\text{SCC}}$ are the same. It suffices to show that their edge sets are the same. Suppose $(v_i, v_j)$ is an edge in $((G^\text T)^{\text{SCC}})^\text T$. Then $(v_j, v_i)$ is an edge in $(G^\text T)^{\text{SCC}}$. Thus there exist $x \in C_j$ and $y \in C_i$ such that $(x, y)$ is an edge of $G^\text T$, which implies $(y, x)$ is an edge of $G$. Since components are preserved, this means that $(v_i, v_j)$ is an edge in $G^{\text{SCC}}$. For the opposite implication we simply note that for any graph $G$ we have $(G^\text T)^{\text T} = G$.


answer: (Removed)
question:
 Give an $O(V + E)$-time algorithm to compute the component graph of a directed graph $G = (V, E)$. Make sure that there is at most one edge between two vertices in the component graph your algorithm produces.


answer: (Removed)
question:
 Given a directed graph $G = (V, E)$, explain how to create another graph $G' = (V, E')$ such that (a) $G'$ has the same strongly connected components as $G$, (b) $G'$ has the same component graph as $G$, and \(c\) $E'$ is as small as possible. Describe a fast algorithm to compute $G'$.


question:
 A directed graph $G = (V, E)$ is **_semiconnected_** if, for all pairs of vertices $u, v \in V$, we have $u \leadsto v$ or $v \leadsto u$. Give an efficient algorithm to determine whether or not $G$ is semiconnected. Prove that your algorithm is correct, and analyze its running time.

answer:
Algorithm:

1. Run $\text{STRONG-CONNECTED-COMPONENTS}(G)}$.
2. Take each strong connected component as a virtual vertex and create a new virtual graph $G'$.
3. Run $\text{TOPOLOGICAL-SORT}(G')$.
4. Check if for all consecutive vertices $(v\_i, v\_{i + 1})$ in a topological sort of $G'$, there is an edge $(v_i, v_{i + 1})$ in graph $G'$. if so, the original graph is semiconnected. Otherwise, it isn't.

Proof:

It is easy to show that $G'$ is a DAG. Consider consecutive vertices $v_i$ and $v_{i + 1}$ in $G'$. If there is no edge from $v_i$ to $v_{i + 1}$, we also conclude that there is no path from $v_{i + 1}$ to $v_i$ since $v_i$ finished after $v_{i + 1}$. From the definition of $G'$, we conclude that, there is no path from any vertices in $G$ who is represented as $v_i$ in $G'$ to those represented as $v_{i + 1}$. Thus, $G$ is not semi-connected. If there is an edge between all consecutive vertices, we claim that there is an edge between any two vertices. Therefore, $G$ is semi-connected.

Running-time: $O(V + E)$.


question:
 Make a $3$-by-$3$ chart with row and column labels $\text{WHITE}$, $\text{GRAY}$, and $\text{BLACK}$. In each cell $(i, j)$, indicate whether, at any point during a depth-first search of a directed graph, there can be an edge from a vertex of color $i$ to a vertex of color $j$. For each possible edge, indicate what edge types it can be. Make a second such chart for depth-first search of an undirected graph.

answer:
According to Theorem 22.7 (Parenthesis theorem), there are 3 cases of relationship between intervals of vertex $u$ and $v$:

- $[u.d, u.f]$ and $[v.d, v.f]$ are entirely disjointed,
- $[u.d, u.f] \subset [v.d, v.f]$, and
- $[v.d, v.f] \subset [u.d, u.f]$.

We judge the possibility according to this Theorem.

- For **directed graph**, we can use the edge classification given by exercise 22.3-5 to simplify the problem.

    $$
    \begin{array}{c|ccc}
    from \diagdown to & \text{WHITE}         & \text{GRAY}                & \text{BLACK} \\\\
    \hline
    \text{WHITE}      & \text{All kinds}     & \text{Cross, Back}         & \text{Cross} \\\\
    \text{GRAY}       & \text{Tree, Forward} & \text{Tree, Forward, Back} & \text{Tree, Forward, Cross} \\\\
    \text{BLACK}      & -                    & \text{Back}                & \text{All kinds}
    \end{array}
    $$

- For **undirected graph**, starting from directed chart, we remove the forward edge and the cross edge, and

    - when a back edge exist, we add a tree edge;
    - when a tree edge exist, we add a back edge.

    This is correct for the following reasons:

    1. Theorem 22.10: In a depth-first search of an undirected graph $G$, every edge of $G$ is either a tree or back edge. So tree and back edge only.
    2. If $(u, v)$ is a tree edge from $u$'s perspective, $(u, v)$ is also a back edge from $v$'s perspective.

    $$
    \begin{array}{c|ccc}
    from \diagdown to & \text{WHITE}      & \text{GRAY}       & \text{BLACK} \\\\
    \hline
    \text{WHITE}      & -                 & \text{Tree, Back} & \text{Tree, Back} \\\\
    \text{GRAY}       & \text{Tree, Back} & \text{Tree, Back} & \text{Tree, Back} \\\\
    \text{BLACK}      & \text{Tree, Back} & \text{Tree, Back} & -
    \end{array}
    $$


question:
 Show how depth-first search works on the graph of Figure 22.6. Assume that the **for** loop of lines 5–7 of the $\text{DFS}$ procedure considers the vertices in alphabetical order, and assume that each adjacency list is ordered alphabetically. Show the discovery and finishing times for each vertex, and show the classification of each edge.

answer:
The following table gives the discovery time and finish time for each vetex in the graph.

See the [C++ demo](https://github.com/walkccc/CLRS-cpp/blob/master/Chap22/22.3.cpp).

$$
\begin{array}{ccc}
\text{Vertex} & \text{Discovered} & \text{Finished} \\\\
\hline
q &  1 & 16 \\\\
r & 17 & 20 \\\\
s &  2 &  7 \\\\
t &  8 & 15 \\\\
u & 18 & 19 \\\\
v &  3 &  6 \\\\
w &  4 &  5 \\\\
x &  9 & 12 \\\\
y & 13 & 14 \\\\
z & 10 & 11
\end{array}
$$

- **Tree edges:** $(q, s)$, $(s, v)$, $(v, w)$, $(q, t)$, $(t, x)$, $(x, z)$, $(t, y)$, $(r, u)$.
- **Back edges:** $(w, s)$, $(z, x)$, $(y, q)$.
- **Forward edges:** $(q, w)$.
- **Cross edges:** $(r, y)$, $(u, y)$.


question:
 Show that using a single bit to store each vertex color suffices by arguing that the $\text{DFS}$ procedure would produce the same result if line 3 of $\text{DFS-VISIT}$ was removed.

answer:
Change line 3 to `color = BLACK` and remove line 8. Then, the algorithm would produce the same result.


question:
 Show that edge $(u, v)$ is
>
> **a.** a tree edge or forward edge if and only if $u.d < v.d < v.f < u.f$,
>
> **b.** a back edge if and only if $v.d \le u.d < u.f \le v.f$, and
>
> **c.** a cross edge if and only if $v.d < v.f < u.d < u.f$.

answer:
**a.** $u$ is an ancestor of $v$.

**b.** $u$ is a descendant of $v$.

**c.** $v$ is visited before $u$.


question:
 Show that in an undirected graph, classifying an edge $(u, v)$ as a tree edge or a back edge according to whether $(u, v)$ or $(v, u)$ is encountered first during the depth-first search is equivalent to classifying it according to the ordering of the four types in the classification scheme.

answer:
By Theorem 22.10, every edge of an undirected graph is either a tree edge or a back edge. First suppose that $v$ is first discovered by exploring edge $(u, v)$. Then by definition, $(u, v)$ is a tree edge. Moreover, $(u, v)$ must have been discovered before $(v, u)$ because once $(v, u)$ is explored, $v$ is necessarily discovered. Now suppose that $v$ isn't first discovered by $(u, v)$. Then it must be discovered by $(r, v)$ for some $r\ne u$. If $u$ hasn't yet been discovered then if $(u, v)$ is explored first, it must be a back edge since $v$ is an ancestor of $u$. If $u$ has been discovered then $u$ is an ancestor of $v$, so $(v, u)$ is a back edge.


question: Rewrite the procedure $\text{DFS}$, using a stack to eliminate recursion.
answer:
See the [C++ demo](https://github.com/walkccc/CLRS-cpp/blob/master/Chap22/22.3-7/22.3-7.cpp).

Also, see this [issue](https://github.com/walkccc/CLRS/issues/329) for [@i-to](https://github.com/i-to)'s discussion.

```cpp
DFS-STACK(G)
    for each vertex u ∈ G.V
        u.color = WHITE
        u.π = NIL
    time = 0
    for each vertex u ∈ G.V
        if u.color == WHITE
            DFS-VISIT-STACK(G, u)
```

```cpp
DFS-VISIT-STACK(G, u)
    S = Ø
    PUSH(S, u)
    time = time + 1             // white vertex u has just been discovered
    u.d = time
    u.color = GRAY
    while !STACK-EMPTY(S)
        u = TOP(S)
        v = FIRST-WHITE-NEIGHBOR(G, u)
        if v == NIL
            // u's adjacency list has been fully explored
            POP(S)
            time = time + 1
            u.f = time
            u.color = BLACK     // blackend u; it is finished
        else
            // u's adjacency list hasn't been fully explored
            v.π = u
            time = time + 1
            v.d = time
            v.color = GRAY
            PUSH(S, v)
```

```cpp
FIRST-WHITE-NEIGHBOR(G, u)
    for each vertex v ∈ G.Adj[u]
        if v.color == WHITE
            return v
    return NIL
```


question:
 Give a counterexample to the conjecture that if a directed graph $G$ contains a path from $u$ to $v$, and if $u.d < v.d$ in a depth-first search of $G$, then $v$ is a descendant of $u$ in the depth-first forest produced.

answer:
Consider a graph with $3$ vertices $u$, $v$, and $w$, and with edges $(w, u)$, $(u, w)$, and $(w, v)$. Suppose that $\text{DFS}$ first explores $w$, and that $w$'s adjacency list has $u$ before $v$. We next discover $u$. The only adjacent vertex is $w$, but $w$ is already grey, so $u$ finishes. Since $v$ is not yet a descendant of $u$ and $u$ is finished, $v$ can never be a descendant of $u$.


question:
 Give a counterexample to the conjecture that if a directed graph $G$ contains a path from $u$ to $v$, then any depth-first search must result in $v.d \le u.f$.

answer:
Consider the directed graph on the vertices $\\{1, 2, 3\\}$, and having the edges $(1, 2)$, $(1, 3)$, $(2, 1)$ then there is a path from $2$ to $3$. However, if we start a $\text{DFS}$ at $1$ and process $2$ before $3$, we will have $2.f = 3 < 4 = 3.d$ which provides a counterexample to the given conjecture.


question:
 Modify the pseudocode for depth-first search so that it prints out every edge in the directed graph $G$, together with its type. Show what modifications, if any, you need to make if $G$ is undirected.

answer:
If $G$ is undirected we don't need to make any modifications.

See the [C++ demo](https://github.com/walkccc/CLRS-cpp/blob/master/Chap22/22.3-10/22.3-10.cpp).

```cpp
DFS-VISIT-PRINT(G, u)
    time = time + 1
    u.d = time
    u.color = GRAY
    for each vertex v ∈ G.Adj[u]
        if v.color == WHITE
            print "(u, v) is a tree edge."
            v.π = u
            DFS-VISIT-PRINT(G, v)
        else if v.color == GRAY
            print "(u, v) is a back edge."
        else if v.d > u.d
            print "(u, v) is a forward edge."
        else
            print "(u, v) is a cross edge."
    u.color = BLACK
    time = time + 1
    u.f = time
```


question:
 Explain how a vertex $u$ of a directed graph can end up in a depth-first tree containing only $u$, even though $u$ has both incoming and outgoing edges in $G$.

answer:
Suppose that we have a directed graph on the vertices $\\{1, 2, 3\\}$ and having edges $(1, 2)$ and $(2, 3)$. Then, $2$ has both incoming and outgoing edges.

If we pick our first root to be $3$, that will be in its own $\text{DFS}$ tree. Then, we pick our second root to be $2$, since the only thing it points to has already been marked $\text{BLACK}$, we won't be exploring it. Then, picking the last root to be $1$, we don't screw up the fact that $2$ is along in a $\text{DFS}$ tree even though it has both an incoming and outgoing edge in $G$.


question:
 Show that we can use a depth-first search of an undirected graph $G$ to identify the connected components of $G$, and that the depth-first forest contains as many trees as $G$ has connected components. More precisely, show how to modify depth-first search so that it assigns to each vertex $v$ an integer label $v.cc$ between $1$ and $k$, where $k$ is the number of connected components of $G$, such that $u.cc = v.cc$ if and only if $u$ and $v$ are in the same connected component.

answer:
The modifications work as follows: each time the **if**-condition of line 8 is satisfied in $\text{DFS-CC}$, we have a new root of a tree in the forest, so we update its $cc$ label to be a new value of $k$. In the recursive calls to $\text{DFS-VISIT-CC}$, we always update a descendant's connected component to agree with its ancestor's.

See the [C++ demo](https://github.com/walkccc/CLRS-cpp/blob/master/Chap22/22.3-12/22.3-12.cpp).

```cpp
DFS-CC(G)
    for each vertex u ∈ G.V
        u.color = WHITE
        u.π = NIL
    time = 0
    cc = 1
    for each vertex u ∈ G.V
        if u.color == WHITE
            u.cc = cc
            cc = cc + 1
            DFS-VISIT-CC(G, u)
```

```cpp
DFS-VISIT-CC(G, u)
    time = time + 1
    u.d = time
    u.color = GRAY
    for each vertex v ∈ G.Adj[u]
        if v.color == WHITE
            v.cc = u.cc
            v.π = u
            DFS-VISIT-CC(G, v)
    u.color = BLACK
    time = time + 1
    u.f = time
```


question:
 A directed graph $G = (V, E)$ is **_singly connected_** if $u \leadsto v$ implies that $G$ contains at most one simple path from $u$ to $v$ for all vertices $u, v \in V$. Give an efficient algorithm to determine whether or not a directed graph is singly connected.

answer:
This can be done in time $O(|V||E|)$. To do this, first perform a topological sort of the vertices. Then, we will contain for each vertex a list of it's ancestors with $in\text-degree$ $0$. We compute these lists for each vertex in the order starting from the earlier ones topologically.

Then, if we ever have a vertex that has the same degree $0$ vertex appearing in the lists of two of its immediate parents, we know that the graph is not singly connected. however, if at each step we have that at each step all of the parents have disjoint sets of degree $0$ vertices as ancestors, the graph is singly connected. Since, for each vertex, the amount of time required is bounded by the number of vertices times the $in\text-degree$ of the particular vertex, the total runtime is bounded by $O(|V||E|)$.


question:
 Given an adjacency-list representation of a directed graph, how long does it take to compute the $\text{out-degree}$ of every vertex? How long does it take to compute the $\text{in-degree}$s?

answer:
- The time to compute the $\text{out-degree}$ of every vertex is

    $$\sum_{v \in V}O(\text{out-degree}(v)) = O(|E| + |V|),$$

    which is straightforward.

- As for the $\text{in-degree}$, we have to scan through all adjacency lists and keep counters for how many times each vertex has been pointed to. Thus, the time complexity is also $O(|E| + |V|)$ because we'll visit all nodes and edges.


question:
 Give an adjacency-list representation for a complete binary tree on $7$ vertices. Give an equivalent adjacency-matrix representation. Assume that vertices are numbered from $1$ to $7$ as in a binary heap.

answer:
- **Adjacency-list representation**

    $$
    \begin{aligned}
    1 & \to 2 \to 3 \\\\
    2 & \to 1 \to 4 \to 5 \\\\
    3 & \to 1 \to 6 \to 7 \\\\
    4 & \to 2 \\\\
    5 & \to 2 \\\\
    6 & \to 3 \\\\
    7 & \to 3
    \end{aligned}
    $$

- **Adjacency-matrix representation**

    $$
    \begin{array}{c|ccccccc|}
      & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\\\
    \hline
    1 & 0 & 1 & 1 & 0 & 0 & 0 & 0 \\\\
    2 & 1 & 0 & 0 & 1 & 1 & 0 & 0 \\\\
    3 & 1 & 0 & 0 & 0 & 0 & 1 & 1 \\\\
    4 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\\\
    5 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\\\
    6 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\\\
    7 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\\\
    \hline
    \end{array}
    $$


question:
 The **_transpose_** of a directed graph $G = (V, E)$ is the graph $G^\text T = (V, E^\text T)$, where $E^\text T = \\{(v, u) \in V \times V: (u, v) \in E \\}$. Thus, $G^\text T$ is $G$ with all its edges reversed. Describe efficient algorithms for computing $G^\text T$ from $G$, for both the adjacency-list and adjacency-matrix representations of $G$. Analyze the running times of your algorithms.

answer:
- **Adjacency-list representation**

    Assume the original adjacency list is $Adj$.

    ```cpp
    let Adj'[1..|V|] be a new adjacency list of the transposed G^T
    for each vertex u ∈ G.V
        for each vertex v ∈ Adj[u]
            INSERT(Adj'[v], u)
    ```

    Time complexity: $O(|E| + |V|)$.

- **Adjacency-matrix representation**

    Transpose the original matrix by looking along every entry above the diagonal, and swapping it with the entry that occurs below the diagonal.

    Time complexity: $O(|V|^2)$.


question:
 Given an adjacency-list representation of a multigraph $G = (V, E)$, describe an $O(V + E)$-time algorithm to compute the adjacency-list representation of the "equivalent" undirected graph $G' = (V, E')$, where $E'$ consists of the edges in $E$ with all multiple edges between two vertices replaced by a single edge and with all self-loops removed.

answer:
```cpp
EQUIVALENT-UNDIRECTED-GRAPH
    let Adj'[1..|V|] be a new adjacency list
    let A be a 0-initialized array of size |V|
    for each vertex u ∈ G.V
        for each v ∈ Adj[u]
            if v != u && A[v] != u
                A[v] = u
                INSERT(Adj'[u], v)
```

Note that $A$ does not contain any element with value $u$ before each iteration of the inner for-loop. That's why we use $A[v] = u$ to mark the existence of an edge $(u, v)$ in the inner for-loop.
Since we lookup in the adjacency-list $Adj$ for $|V| + |E|$ times, the time complexity is $O(|V| + |E|)$.


question:
 The **_square_** of a directed graph $G = (V, E)$ is the graph $G^2 = (V, E^2)$ such that $(u, v) \in E^2$ if and only if $G$ contains a path with at most two edges between $u$ and $v$. Describe efficient algorithms for computing $G^2$ from $G$ for both the adjacency-list and adjacency-matrix representations of $G$. Analyze the running times of your algorithms.

answer:
- **Adjacency-list representation**

    To compute $G^2$ from the adjacency-list representation $Adj$ of $G$, we perform the following for each $Adj[u]$:

    ```cpp
    for each v ∈ Adj[u]
        INSERT(Adj2[u], v)
        for each w ∈ Adj[v]
            // edge(u, w) ∈ E^2
            INSERT(Adj2[u], w)
    ```

    where $Adj2$ is the adjacency-list representation of $G^2$. For every edge in $Adj$ we scan at most $|V|$ vertices, we compute $Adj2$ in time $O(|V||E|)$.

    After we have computed $Adj2$, we have to remove duplicate edges from the lists. Removing duplicate edges is done in $O(V + E')$ where $E' = O(VE)$ is the number of edges in $Adj2$ as shown in exercise 22.1-4. Thus the total running time is

    $$O(VE) + O(V + VE) = O(VE).$$

    However, if the original graph $G$ contains self-loops, we should modify the algorithm so that self-loops are not removed.

- **Adjacency-matrix representation**

    Let $A$ denote the adjacency-matrix representation of $G$. The adjacency-matrix representation of $G^2$ is the square of $A$. Computing $A^2$ can be done in time $O(V^3)$ (and even faster, theoretically; Strassen's algorithm for example will compute $A^2$ in $O(V^{\lg 7})$).


question:
 Most graph algorithms that take an adjacency-matrix representation as input require time $\Omega(V^2)$, but there are some exceptions. Show how to determine whether a directed graph $G$ contains a **_universal sink_** $-$ a vertex with $\text{in-degree}$ $|V| - 1$ and $\text{out-degree}$ $0$ $-$ in time $O(V)$, given an adjacency matrix for $G$.

answer:
Start by examining position $(1, 1)$ in the adjacency matrix. When examining position $(i, j)$,

- if a $1$ is encountered, examine position $(i + 1, j)$, and
- if a $0$ is encountered, examine position $(i, j + 1)$.

Once either $i$ or $j$ is equal to $|V|$, terminate.

```cpp
IS-CONTAIN-UNIVERSAL-SINK(M)
    i = j = 1
    while i < |V| and j < |V|
        // There's an out-going edge, so examine the next row
        if M[i, j] == 1
            i = i + 1
        // There's no out-going edge, so see if we could reach the last column of current row
        else if M[i, j] == 0
            j = j + 1
    check if vertex i is a universal sink
```

If a graph contains a universal sink, then it must be at vertex $i$.

To see this, suppose that vertex $k$ is a universal sink. Since $k$ is a universal sink, row $k$ will be filled with $0$'s, and column $k$ will be filled with $1$'s except for $M[k, k]$, which is filled with a $0$. Eventually, once row $k$ is hit, the algorithm will continue to increment column $j$ until $j = |V|$.

To be sure that row $k$ is eventually hit, note that once column $k$ is reached, the algorithm will continue to increment $i$ until it reaches $k$.

This algorithm runs in $O(V)$ and checking if vertex $i$ is a universal sink is done in $O(V)$. Therefore, the total running time is $O(V) + O(V) = O(V)$.


question:
 The **_incidence matrix_** of a directed graph $G = (V, E)$ with no self-loops is a $|V| \times |E|$ matrix $B = (b_{ij})$ such that
>
> $$
> b_{ij} =
> \begin{cases}
> -1 & \text{if edge $j$ leaves vertex $i$}, \\\\
>  1 & \text{if edge $j$ enters vertex $i$}, \\\\
>  0 & \text{otherwise}.
> \end{cases}
> $$
>
> Describe what the entries of the matrix product $BB^\text T$ represent, where $B^\text T$ is the transpose of $B$.

answer:
$$BB^\text T(i, j) = \sum\limits_{e \in E}b_{ie} b_{ej}^\text T = \sum\limits_{e \in E} b_{ie}b_{je}.$$

- If $i = j$, then $b_{ie} b_{je} = 1$ (it is $1 \cdot 1$ or $(-1) \cdot (-1)$) whenever $e$ enters or leaves vertex $i$, and $0$ otherwise.
- If $i \ne j$, then $b_{ie} b_{je} = -1$ when $e = (i, j)$ or $e = (j, i)$, and $0$ otherwise.

Thus,

$$
BB^\text T(i, j) =
\begin{cases}
\text{degree of $i$ = in-degree + out-degree}   & \text{if $i = j$}, \\\\
\text{$-$(\\# of edges connecting $i$ and $j$)} & \text{if $i \ne j$}.
\end{cases}
$$


question:
 Suppose that instead of a linked list, each array entry $Adj[u]$ is a hash table containing the vertices $v$ for which $(u, v) \in E$. If all edge lookups are equally likely, what is the expected time to determine whether an edge is in the graph? What disadvantages does this scheme have? Suggest an alternate data structure for each edge list that solves these problems. Does your alternative have disadvantages compared to the hash table?

answer:
The expected lookup time is $O(1)$, but in the worst case it could take $O(|V|)$.

If we first sorted vertices in each adjacency list then we could perform a binary search so that the worst case lookup time is $O(\lg |V|)$, but this has the disadvantage of having a much worse expected lookup time.


question:
 Write pseudocode for $\text{PROTO-vEB-DELETE}$. It should update the appropriate summary bit by scanning the related bits within the cluster. What is the worst-case running time of your procedure?

answer:
```cpp
PROTO-vEB-DELETE(V, x)
    if V.u == 2
        V.A[x] = 0
    else
        PROTO-vEB-DELETE(V.cluster[high(x)], low(x))
        inCluster = false
        for i = 0 to sqrt(u) - 1
            if PROTO-vEB-MEMBER(V.cluster[high(x)], low(i))
                inCluster = true
                break
        if inCluster == false
            PROTO-vEB-DELETE(V.summary, high(x))
```

When we delete a key, we need to check membership of all keys of that cluster to know how to update the summary structure. There are $\sqrt u$ of these, and each membership takes $O(\lg\lg u)$ time to check. With the recursive calls, recurrence for running time is

$$T(u) = T(\sqrt u) + O(\sqrt u\lg\lg u).$$

We make the substitution $m = \lg u$ and $S(m) = T(2^m)$. Then we apply the Master Theorem, using case 3, to solve the recurrence. Substituting back, we find that the runtime is $T(u) = O(\sqrt u\lg\lg u)$.


question:
 Add the attribute $n$ to each $\text{proto-vEB}$ structure, giving the number of elements currently in the set it represents, and write pseudocode for $\text{PROTO-vEB-DELETE}$ that uses the attribute $n$ to decide when to reset summary bits to $0$. What is the worst-case running time of your procedure? What other procedures need to change because of the new attribute? Do these changes affect their running times?

answer:
We would keep the same as before, but insert immediately after the else, a check of whether $n = 1$. If it doesn't continue as usual, but if it does, then we can just immediately set the summary bit to $0$, null out the pointer in the table, and be done immediately. This has the upside that it can sometimes save up to $\lg\lg u$. The procedure has the big downside that the number of elements that are in the set could be as high as $\lg(\lg u)$, in which case $\lg u$ many bits are needed to store $n$.


question: Modify the $\text{proto-vEB}$ structure to support duplicate keys.
answer:
The array $A$ found in a proto van Emde Boas structure of size $2$ should now support integers, instead of just bits. All other pats of the structure will remain the same. The integer will store the number of duplicates at that position. The modifications to insert, delete, minimum, successor, etc will be minor. Only the base cases will need to be updated.


question: Modify the $\text{proto-vEB}$ structure to support keys that have associated satellite data.
answer:
The only modification necessary would be for the $u = 2$ trees. They would need to also include a length two array that had pointers to the corresponding satellite data which would be populated in case the corresponding entry in $A$ were $1$.


question: Write pseudocode for a procedure that creates a $\text{proto-vEB}(u)$ structure.
answer:
This algorithm recursively allocates proper space and appropriately initializes attributes for a proto van Emde Boas structure of size $u$.

```cpp
MAKE-PROTO-vEB(u)
    allocate a new vEB tree V
    V.u = u
    if u == 2
        let A be an array of size 2
        V.A[1] = V.A[0] = 0
    else
        V.summary = MAKE-PROTO-vEB(sqrt(u))
        for i = 0 to sqrt(u) - 1
            V.cluster[i] = MAKE-PROTO-vEB(sqrt(u))
```


question:
 Argue that if line 9 of $\text{PROTO-vEB-MINIMUM}$ is executed, then the $\text{proto-vEB}$ structure is empty.

answer:
For line 9 to be executed, we would need that in the summary data, we also had a $\text{NIL}$ returned. This could of either happened through line 9, or 6. Eventually though, it would need to happen in line 6, so, there must be some number of summarizations that happened of $V$ that caused us to get an empty $u = 2$ $\text{vEB}$. However, a summarization has an entry of one if any of the corresponding entries in the data structure are one. This means that there are no entries in $V$, and so, we have that $V$ is empty.


question:
 Suppose that we designed a $\text{proto-vEB}$ structure in which each _cluster_ array had only $u^{1 / 4}$ elements. What would the running times of each operation be?

answer:
There are $u^{3 / 4}$ clusters in each $\text{proto-vEB}$.

- **MEMBER/INSERT:**

    $$T(u) = T(u^{1 / 4}) + O(1) = \Theta(\lg\log_4 u) = \Theta(\lg\lg u).$$

- **MINIMUM/MAXIMUM:**

    $$T(u) = T(u^{1 / 4}) + T(u^{3 / 4}) + O(1) = \Theta(\lg u).$$

- **SUCCESSOR/PREDECESSOR/DELETE:**

    $$T(u) = T(u^{1 / 4}) + T(u^{3 / 4}) + \Theta(\lg u^{1 / 4}) = \Theta(\lg u \lg\lg u).$$


question: Modify vEB trees to support duplicate keys.
answer:
To support duplicate keys, for each $u = 2$ vEB tree, instead of storing just a bit in each of the entries of its array, it should store an integer representing how many elements of that value the vEB contains.


question: Modify vEB trees to support keys that have associated satellite data.
answer:
For any key which is a minimum on some vEB, we'll need to store its satellite data with the min value since the key doesn't appear in the subtree. The rest of the satellite data will be stored alongside the keys of the vEB trees of size $2$. Explicitly, for each non-summary vEB tree, store a pointer in addition to min. If min is $\text{NIL}$, the pointer should also point to $\text{NIL}$. Otherwise, the pointer should point to the satellite data associated with that minimum. In a size $2$ vEB tree, we'll have two additional pointers, which will each point to the minimum's and maximum's satellite data, or $\text{NIL}$ if these don't exist. In the case where $\min = \max$, the pointers will point to the same data.


question: Write pseudocode for a procedure that creates an empty van Emde Boas tree.
answer:
We define the procedure for any $u$ that is a power of $2$. If $u = 2$, then, just slap that fact together with an array of length $2$ that contains $0$ in both entries.

If $u = 2k > 2$, then, we create an empty vEB tree called Summary with $u = 2^{\lceil k / 2 \rceil}$. We also make an array called cluster of length $2^{\lceil k / 2 \rceil}$ with each entry initialized to an empty vEB tree with $u = 2^{\lfloor k / 2 \rfloor}$. Lastly, we create a min and max element, both initialized to $\text{NIL}$.


question:
 What happens if you call $\text{VEB-TREE-INSERT}$ with an element that is already in the vEB tree? What happens if you call $\text{VEB-TREE-DELETE}$ with an element that is not in the vEB tree? Explain why the procedures exhibit the behavior that they do. Show how to modify vEB trees and their operations so that we can check in constant time whether an element is present.

answer:
Suppose that $x$ is already in $V$ and we call $\text{INSERT}$. Then we can't satisfy lines 1, 3, 6, or 10, so we will enter the else case on line 9 every time until we reach the base case. If $x$ is already in the base-case tree, then we won't change anything. If $x$ is stored in a min attribute of a vEB tree that isn't base-case, however, we will insert a duplicate of it in some base-case tree. Now suppose we call $\text{DELETE}$ when $x$ isn't in $V$ . If there is only a single element in $V$, lines 1 through 3 will delete it, regardless of what element it is. To enter the elseif of line 4, $x$ can't be equal to $0$ or $1$ and the vEB tree must be of size $2$. In this case, we delete the max element, regardless of what it is. Since the recursive call always puts us in this case, we always delete an element we shouldn't. To avoid these issue, keep and updated auxiliary array $A$ with $u$ elements. Set $A[i] = 0$ if $i$ is not in the tree, and $1$ if it is. Since we can perform constant time updates to this array, it won't affect the runtime of any of our operations. When inserting $x$, check first to be sure $A[x] = 0$. If it's not, simply return. If it is, set $A[x] = 1$ and proceed with insert as usual. When deleting $x$, check if $A[x] = 1$. If it isn't, simply return. If it is, set $A[x] = 0$ and proceed with delete as usual.


question:
 Suppose that instead of $\sqrt[\uparrow]u$ clusters, each with universe size $\sqrt[\downarrow]u$, we constructed vEB trees to have $u^{1 / k}$ clusters, each with universe size $u^{1 - 1 / k}$, where $k > 1$ is a constant. If we were to modify the operations appropriately, what would be their running times? For the purpose of analysis, assume that $u^{1 / k}$ and $u^{1 - 1 / k}$ are always integers.

answer:
Similar to the analysis of $\text{(20.4)}$, we will analyze

$$T(u) \le T(u^{1 - 1 / k}) + T(u^{1 / k}) + O(1).$$

This is a good choice for analysis because for many operations we first check the summary vEB tree, which will have size $u^{1 / k}$ (the second term). And then possible have to check a vEB tree somewhere in cluster, which will have size $u^{1 - 1/k}$ (the first term). We let $T(2^m) = S(m)$, so the equation becomes

$$S(m) \le S(m(1 - 1/k)) + S(m/k) + O(1).$$

If $k > 2$ the first term dominates, so by master theorem, we'll have that $S(m)$ is $O(\lg m)$, this means that T will be $O(\lg(\lg u))$ just as in the original case where we took squareroots.


question:
 Creating a vEB tree with universe size $u$ requires $O(u)$ time. Suppose we wish to explicitly account for that time. What is the smallest number of operations $n$ for which the amortized time of each operation in a vEB tree is $O(\lg\lg u)$?

answer:
Set $n = u / \lg\lg u$. Then performing $n$ operations takes $c(u + n\lg\lg u)$ time for some constant $c$. Using the aggregate amortized analysis, we divide by $n$ to see that the amortized cost of each operations is $c(\lg\lg u + \lg\lg u) = O(\lg\lg u)$ per operation. Thus we need $n \ge u/ \lg \lg u$.


question: Modify the data structures in this section to support duplicate keys.
answer:
To modify these structure to allow for multiple elements, instead of just storing a bit in each of the entries, we can store the head of a linked list representing how many elements of that value that are contained in the structure, with a $\text{NIL}$ value to represent having no elements of that value.


question: Modify the data structures in this section to support keys that have associated satellite data.
answer:
All operations will remain the same, except instead of the leaves of the tree being an array of integers, they will be an array of nodes, each of which stores $x.key$ in addition to whatever additional satellite data you wish.


question:
 Observe that, using the structures in this section, the way we find the successor and predecessor of a value $x$ does not depend on whether $x$ is in the set at the time. Show how to find the successor of $x$ in a binary search tree when $x$ is not stored in the tree.

answer:
To find the successor of a given key $k$ from a binary tree, call the procedure $\text{SUCC}(x, T.root)$. Note that this will return $\text{NIL}$ if there is no entry in the tree with a larger key.


question:
 Suppose that instead of superimposing a tree of degree $\sqrt u$, we were to superimpose a tree of degree $u^{1 / k}$, where $k > 1$ is a constant. What would be the height of such a tree, and how long would each of the operations take?

answer:
The new tree would have height $k$. $\text{INSERT}$ would take $O(k)$, $\text{MINIMUM}$, $\text{MAXIMUM}$, $\text{SUCCESSOR}$, $\text{PREDECESSOR}$, and $\text{DELETE}$ would take $O(ku^{1 / k})$.


answer: $|\pi^\*[q]| < q$.
question:
 Give an upper bound on the size of $\pi^\*[q]$ as a function of $q$. Give an example to show that your bound is tight.


answer: $\\{ q \mid \pi[q] = m \text{ and } q \ge 2m \\}$.
question:
 Explain how to determine the occurrences of pattern $P$ in the text $T$ by examining the $\pi$ function for the string $PT$ (the string of length $m + n$ that is the concatenation of $P$ and $T$).


question:
 Show how to improve $\text{KMP-MATCHER}$ by replacing the occurrence of $\pi$ in line 7 (but not line 12) by $\pi'$, where $\pi'$ is defined recursively for $q = 1, 2, \ldots, m - 1$ by the equation
>
> $$
> \pi'[q] =
> \begin{cases}
>            0 & \text{ if } \pi[q] = 0, \\\\
> \pi'[\pi[q]] & \text{ if } \pi[q] \ne 0 \text{ and } P[\pi[q] + 1] = P[q + 1] \\\\
>       \pi[q] & \text{ if } \pi[q] \ne 0 \text{ and } P[\pi[q] + 1] \ne P[q + 1].
> \end{cases}
> $$
>
> Explain why the modified algorithm is correct, and explain in what sense this change constitutes an improvement.

answer:
If $P[q + 1] \ne T[i]$, then if $P[\pi[q] + q] = P[q + 1] \ne T[i]$, there is no need to compare $P[\pi[q] + q]$ with $T[i]$.


answer: Find $T'$ in $TT$.
question:
 Give a linear-time algorithm to determine whether a text $T$ is a cyclic rotation of another string $T'$. For example, $\text{arc}$ and $\text{car}$ are cyclic rotations of each other.


answer: Compute the prefix function $m$ times.
question:
 Give an $O(m|\Sigma|)$-time algorithm for computing the transition function $\delta$ for the string-matching automaton corresponding to a given pattern $P$. (Hint: Prove that $\delta(q, a) = \delta(\pi[q], a)$ if $q = m$ or $P[q + 1] \ne a$.)


question:
 Construct the string-matching automaton for the pattern $P = aabab$ and illustrate its operation on the text string $T = \text{aaababaabaababaab}$.

answer:
$$0 \rightarrow 1 \rightarrow 2 \rightarrow 2 \rightarrow 3 \rightarrow 4 \rightarrow 5 \rightarrow 1 \rightarrow 2 \rightarrow 3 \rightarrow 4 \rightarrow 2 \rightarrow 3 \rightarrow 4 \rightarrow 5 \rightarrow 1 \rightarrow 2 \rightarrow 3.$$


question:
 Draw a state-transition diagram for a string-matching automaton for the pattern
> $ababbabbababbababbabb$ over the alphabet $\sigma = \\{a, b\\}$.

answer:
$$
\begin{array}{c|c|c}
 0 &  1 &  0 \\\\
 1 &  1 &  2 \\\\
 2 &  3 &  0 \\\\
 3 &  1 &  4 \\\\
 4 &  3 &  5 \\\\
 5 &  6 &  0 \\\\
 6 &  1 &  7 \\\\
 7 &  3 &  8 \\\\
 8 &  9 &  0 \\\\
 9 &  1 & 10 \\\\
10 & 11 &  0 \\\\
11 &  1 & 12 \\\\
12 &  3 & 13 \\\\
13 & 14 &  0 \\\\
14 &  1 & 15 \\\\
15 & 16 &  8 \\\\
16 &  1 & 17 \\\\
17 &  3 & 18 \\\\
18 & 19 &  0 \\\\
19 &  1 & 20 \\\\
20 &  3 & 21 \\\\
21 &  9 &  0
\end{array}
$$


answer: $\delta(q, a) \in \\{0, 1, q + 1\\}$.
question:
 We call a pattern $P$ **_nonoverlappable_** if $P_k \sqsupset P_q$ implies $k = 0$ or $k = q$. Describe the state-transition diagram of the string-matching automaton for a nonoverlappable pattern.


answer: Combine the common prefix and suffix.
question:
 Given two patterns $P$ and $P'$, describe how to construct a finite automaton that determines all occurrences of either pattern. Try to minimize the number of states in your automaton.


question:
 Given a pattern $P$ containing gap characters (see Exercise 32.1-4), show how to build a finite automaton that can find an occurrence of $P$ in a text $T$ in $O(n)$ matching time, where $n = |T|$.

answer:
Split the string with the gap characters, build finite automatons for each substring. When a substring is matched, moved to the next finite automaton.


question:
 Show the comparisons the naive string matcher makes for the pattern $P = 0001$ in the text $T = 000010001010001$.

answer:
```cpp
STRING-MATCHER(P, T, i)
    for j = i to i + P.length
        if P[j - i + 1] != T[j]
            return false
    return true
```


question:
 Suppose that all characters in the pattern $P$ are different. Show how to accelerate $\text{NAIVE-STRING-MATCHER}$ to run in time $O(n)$ on an $n$-character text $T$.

answer:
Suppose $T[i] \ne P[j]$, then for $k \in [1, j)$, $T[i - k] = P[j - k] \ne P[0]$, the $[i - k, i)$ are all invalid shifts which could be skipped, therefore we can compare $T[i]$ with $P[0]$ in the next iteration.


question:
 Suppose that pattern $P$ and text $T$ are randomly chosen strings of length $m$ and $n$, respectively, from the $d$-ary alphabet $\Sigma_d = \\{ 0, 1, \ldots, d - 1 \\}$, where $d \ge 2$. Show that the expected number of character-to-character comparisons made by the implicit loop in line 4 of the naive algorithm is
>
> $$(n - m + 1) \frac{1 - d^{-m}}{1 - d^{-1}} \le 2(n - m + 1)$$
>
> over all executions of this loop. (Assume that the naive algorithm stops comparing characters for a given shift once it finds a mismatch or matches the entire pattern.) Thus, for randomly chosen strings, the naive algorithm is quite efficient.

answer:
Suppose for each shift, the number of compared characters is $L$, then:

$$
\begin{aligned}
\text E[L] & = 1 \cdot \frac{d - 1}{d} + 2 \cdot (\frac{1}{d})^1 \frac{d - 1}{d} + \cdots + m \cdot (\frac{1}{d})^{m - 1} \frac{d - 1}{d} + m \cdot (\frac{1}{d})^{m} \\\\
           & = (1 + 2 \cdot (\frac{1}{d})^1 + \cdots + m \cdot (\frac{1}{d})^{m}) \frac{d - 1}{d} + m \cdot (\frac{1}{d})^{m}.
\end{aligned}
$$

$$
\begin{aligned}
                S & = 1 + 2 \cdot (\frac{1}{d})^1 + \cdots + m \cdot (\frac{1}{d})^{m - 1} \\\\
     \frac{1}{d}S & = 1 \cdot (\frac{1}{d})^1 + \cdots + (m - 1) \cdot (\frac{1}{d})^{m - 1} + m \cdot (\frac{1}{d})^{m} \\\\
 \frac{d - 1}{d}S & = 1 + (\frac{1}{d})^1 + \cdots + \cdot (\frac{1}{d})^{m - 1} - m \cdot (\frac{1}{d})^{m} \\\\
 \frac{d - 1}{d}S & = \frac{1 - d^{-m}}{1 - d^{-1}} - m \cdot (\frac{1}{d})^{m}.
\end{aligned}
$$

$$
\begin{aligned}
\text E[L] & = (1 + 2 \cdot (\frac{1}{d})^1 + \cdots + m \cdot (\frac{1}{d})^{m}) \frac{d - 1}{d} + m \cdot (\frac{1}{d})^{m} \\\\
           & = \frac{1 - d^{-m}}{1 - d^{-1}} - m \cdot (\frac{1}{d})^{m} + m \cdot (\frac{1}{d})^{m} \\\\
           & = \frac{1 - d^{-m}}{1 - d^{-1}}.
\end{aligned}
$$

There are $n - m + 1$ shifts, therefore the expected number of comparisons is:

$$(n - m + 1) \cdot \text E[L] = (n - m + 1) \frac{1 - d^{-m}}{1 - d^{-1}}$$

Since $d \ge 2$, $1 - d^{-1} \ge 0.5$, $1 - d^{-m} < 1$, and $ \frac{1 - d^{-m}}{1 - d^{-1}} \le 2$, therefore

$$(n - m + 1) \frac{1 - d^{-m}}{1 - d^{-1}} \le 2 (n - m + 1).$$


question:
 Suppose we allow the pattern $P$ to contain occurrences of a **_gap character_** $\diamond$ that can match an _arbitrary_ string of characters (even one of zero length). For example, the pattern $ab\diamond ba\diamond c$ occurs in the text $cabccbacbacab$ as
>
> $$c \underbrace{ab}\_{ab} \underbrace{cc}\_{\diamond} \underbrace{ba}\_{ba} \underbrace{cba}\_{\diamond} \underbrace{c}\_{c} ab$$
>
> and as
>
> $$c \underbrace{ab}\_{ab} \underbrace{ccbac}\_{\diamond} \underbrace{ba}\_{ba} \underbrace{\text{ }}\_{\diamond} \underbrace{c}\_{c} ab$$
>
> Note that the gap character may occur an arbitrary number of times in the pattern but not at all in the text. Give a polynomial-time algorithm to determine whether such a pattern $P$ occurs in a given text $T$, and analyze the running time of your algorithm.

answer:
By using dynamic programming, the time complexity is $O(mn)$ where $m$ is the length of the text $T$ and $n$ is the length of the pattern $P$; the space complexity is $O(mn)$, too.

This problem is similar to LeetCode [44. WildCard Matching](https://leetcode.com/problems/wildcard-matching/), except that it has no question mark (`?`) requirement. You can see my naive DP implementation [here](https://github.com/walkccc/LeetCode/blob/master/solutions/cpp/0044.cpp).


answer: $|\\{15, 59, 92\\}| = 3$.
question:
 Working modulo $q = 11$, how many spurious hits does the Rabin-Karp matcher encounter in the text $T = 3141592653589793$ when looking for the pattern $P = 26$?


answer: Truncation.
question:
 How would you extend the Rabin-Karp method to the problem of searching a text string for an occurrence of any one of a given set of $k$ patterns? Start by assuming that all $k$ patterns have the same length. Then generalize your solution to allow the patterns to have different lengths.


question:
 Show how to extend the Rabin-Karp method to handle the problem of looking for a given $m \times m$ pattern in an $n \times n$ array of characters. (The pattern may be shifted vertically and horizontally, but it may not be rotated.)

answer:
Calculate the hashes in each column just like the Rabin-Karp in one-dimension, then treat the hashes in each row as the characters and hashing again.


answer: (Omit!)
question:
 Alice has a copy of a long $n$-bit file $A = \langle a_{n - 1}, a_{n - 2}, \ldots, a_0 \rangle$, and Bob similarly has an $n$-bit file $B = \langle b_{n - 1}, b_{n - 2}, \ldots, b_0 \rangle$. Alice and Bob wish to know if their files are identical. To avoid transmitting all of $A$ or $B$, they use the following fast probabilistic check. Together, they select a prime $q > 1000n$ and randomly select an integer $x$ from $\\{ 0, 1, \ldots, q - 1 \\}$. Then, Alice evaluates
>
> $$A(x) = (\sum_{i = 0}^{n - 1} a_i x^i) \mod q$$
>
> and Bob similarly evaluates $B(x)$. Prove that if $A \ne B$, there is at most one chance in $1000$ that $A(x) = B(x)$, whereas if the two files are the same, $A(x)$ is necessarily the same as $B(x)$. ($\textit{Hint:}$ See Exercise 31.4-4.)


question: Prove that every diagonal element of a symmetric positive-definite matrix is positive.
answer:
To see this, let $e_i$ be the vector that is $0$s except for a $1$ in the $i$th
position. Then, we consider the quantity $e_i^{\text T}Ae_i$ for every $i$. $Ae_i$ takes each row of $A$ and pulls out the $i$th column of it, and puts those values into a column vector. Then, we multiply that on the left by $e_i^{\text T}$, pulls out the $i$th row of this quantity, which means that the quantity $e_i^{\text T}Ae_i$ exactly the value of $A_{i, i}$.

So, we have that by positive definiteness, since $e_i$ is nonzero, that quantity must be positive. Since we do this for every $i$, we have that every entry along the diagonal must be positive.


question:
 Let
>
> $$
> A =
> \begin{pmatrix}
> a & b \\\\
> b & c
> \end{pmatrix}
> $$
>
> be a $2 \times 2$ symmetrix positive-definite matrix. Prove that its determinant $ac - b^2$ is positive by "completing the square" in a manner similar to that used in the proof of Lemma 28.5.

answer:
Let $x = -by / a$. Since $A$ is positive-definite, we have

$$
\begin{aligned}
0 & <
\begin{pmatrix} x & y \end{pmatrix}^{\text T} A
\begin{pmatrix} x \\\\ y \end{pmatrix} \\\\
  & =
\begin{pmatrix} x & y \end{pmatrix}^{\text T}
\begin{pmatrix} ax + by \\\\ bx + cy \end{pmatrix} \\\\
  & = ax^2 + 2bxy + cy^2 \\\\
  & = cy^2 - \frac{b^2y^2}{a} \\\\
  & = (c - b^2 / a)y^2.
\end{aligned}
$$

Thus, $c - b^2 / a > 0$, which implies $ac - b^2 > 0$, since $a > 0$.


question: Prove that the maximum element in a symmetric positive-definite matrix lies on the diagonal.
answer:
Suppose to a contradiction that there were some element $a_{ij}$ with $i \ne j$ so that $a_{ij}$ were a largest element. We will use $e_i$ to denote the vector that is all zeroes except for having a $1$ at position $i$. Then, we consider the value $(e_i − e_j)^{\text T} A(e_i − e_j)$. When we compute $A(e_i - e_j)$ this will return a vector which is column $i$ minus column $j$. Then, when we do the last multiplication, we will get the quantity which is the $i$th row minus the $j$th row. So,

$$
\begin{aligned}
(e_i - e_j)^{\text T} A(e_i - e_j)
    & = a_{ii} - a_{ij} - a_{ji} + a_{jj} \\\\
    & = a_{ii} + a_{jj} - 2a_{ij} \le 0
\end{aligned}
$$

Where we used symmetry to get that $a_{ij} = a_{ji}$. This result contradicts the fact that $A$ was positive definite. So, our assumption that there was a element tied for largest off the diagonal must of been false.


question:
 Prove that the determinant of each leading submatrix of a symmetrix positive-definite matrix is positive.

answer:
The claim clearly holds for matrices of size $1$ because the single entry in the matrix is positive the only leading submatrix is the matrix itself. Now suppose
the claim holds for matrices of size $n$, and let $A$ be an $(n + 1) \times (n + 1)$ symmetric positive-definite matrix. We can write $A$ as

$$
A =
\left[
\begin{array}{ccc|c}
 &    & & \\\\
 & A' & & w \\\\
 &    & & \\\\
\hline
 & v  & & c
\end{array}
\right]
.
$$

Then $A'$ is clearly symmetric, and for any $x$ we have $x^{\text T} A'x = \begin{pmatrix} x & 0 \end{pmatrix} A \begin{pmatrix} x \\\\ 0 \end{pmatrix} > 0$, so $A'$ is positive-definite. By our induction hypothesis, every leading submatrix of $A'$ has positive determinant, so we are left only to show that $A$ has positive determinant. By Theorem D.4, the determinant of $A$ is equal to the determinant of the matrix

$$
B =
\left[
\begin{array}{c|ccc}
c & & v  & \\\\
\hline
  & &    & \\\\
w & & A' & \\\\
  & &    &
\end{array}
\right]
.
$$

Theorem D.4 also tells us that the determinant is unchanged if we add a multiple of one column of a matrix to another. Since $0 < e_{n + 1}^{\text T} Ae_{n + 1} = c$, we can use multiples of the first column to zero out every entry in the first row other than $c$. Specifically, the determinant of $B$ is the same as the determinant of the matrix obtained in this way, which looks like

$$
C =
\left[
\begin{array}{c|ccc}
c & & 0   & \\\\
\hline
  & &     & \\\\
w & & A'' & \\\\
  & &     &
\end{array}
\right]
.
$$

By definition, $\det(A) = c\det(A'')$. By our induction hypothesis, $\det(A'') > 0$. Since $c > 0$ as well, we conclude that $\det(A) > 0$, which completes the proof.


question:
 Let $A_k$ denote the $k$th leading submatrix of a symmetric positive-definite matrix $A$. Prove that $\text{det}(A_k) / \text{det}(A_{k - 1})$ is the $k$th pivot during $\text{LU}$ decomposition, where, by convention, $\text{det}(A_0) = 1$.

answer:
When we do an LU decomposition of a positive definite symmetric matrix, we never need to permute the rows. This means that the pivot value being used from the first operation is the entry in the upper left corner. This gets us that for the case $k = 1$, it holds because we were told to define $\det(A_0) = 1$, getting us, $a_{11} = \det(A_1) / \det(A_0)$. When Diagonalizing a matrix, the product of the pivot values used gives the determinant of the matrix. So, we have that the determinant of $A_k$ is a product of the $k$th pivot value with all the previous values. By induction, the product of all the previous values is $\det(A_{k − 1})$. So, we have that if $x$ is the $k$th pivot value, $\det(A_k) = x\det(A_{k − 1})$, giving us the desired result that the $k$th pivot value is $\det(A_k) / \det(A_{k − 1})$.


question:
 Find the function of the form
>
> $$F(x) = c_1 + c_2x\lg x + c_3 e^x$$
>
> that is the best least-squares fit to the data points
>
> $$(1, 1), (2, 1), (3, 3), (4, 8).$$

answer:
First we form the $A$ matrix

$$
A =
\begin{pmatrix}
1 &      0 & e   \\\\
1 &      2 & e^2 \\\\
1 & 3\lg 3 & e^3 \\\\
1 &      8 & e^4
\end{pmatrix}
.
$$

We compute the pseudoinverse, then multiply it by $y$, to obtain the coefficient vector

$$
c =
\begin{pmatrix}
 0.411741 \\\\
-0.20487  \\\\
 0.16546
\end{pmatrix}
.
$$


question:
 Show that the pseudoinverse $A^+$ satisfies the following four equations:
>
> $$
> \begin{aligned}
>            AA^+A & = A, \\\\
>          A^+AA^+ & = A^+, \\\\
> (AA^+)^{\text T} & = AA^+, \\\\
> (A^+A)^{\text T} & = A^+A.
> \end{aligned}
> $$

answer:
$$
\begin{aligned}
AA^+A & = A((A^{\text T}A)^{-1}A^{\text T})A \\\\
      & = A(A^{\text T}A)^{-1}(A^{\text T}A) \\\\
      & = A,
\end{aligned}
$$

$$
\begin{aligned}
A^+AA^+ & = ((A^{\text T}A)^{-1}A^{\text T})A((A^{\text T}A)^{-1}A^{\text T}) \\\\
        & = (A^{\text T}A)^{-1}(A^{\text T}A)(A^{\text T}A)^{-1}A^{\text T} \\\\
        & = (A^{\text T}A)^{-1}A^{\text T} \\\\
        & = A^+,
\end{aligned}
$$

$$
\begin{aligned}
(AA^+)^{\text T}
    & = (A(A^{\text T}A)^{-1}A^{\text T})^{\text T} \\\\
    & = A((A^{\text T}A)^{-1})^{\text T}A^{\text T} \\\\
    & = A((A^{\text T}A)^{\text T})^{-1}A^{\text T} \\\\
    & = A(A^{\text T}A)^{-1}A^{\text T} \\\\
    & = AA^+,
\end{aligned}
$$

$$
\begin{aligned}
(A^+A)^{\text T}
    & = ((A^{\text T}A)^{-1}A^{\text T}A)^{\text T} \\\\
    & = ((A^{\text T}A)^{-1}(A^{\text T}A))^{\text T} \\\\
    & = I^{\text T} \\\\
    & = I \\\\
    & = (A^{\text T}A)^{-1}(A^{\text T}A) \\\\
    & = A^+A.
\end{aligned}
$$


question:
 Let $M(n)$ be the time to multiply two $n \times n$ matrices, and let $S(n)$ denote the time required to square an $n \times n$ matrix. Show that multiplying and squaring matrices have essentially the same difficulty: an $M(n)$-time matrix-multiplication algorithm implies an $O(M(n))$-time squaring algorithm, and an $S(n)$-time squaring algorithm implies an $O(S(n))$-time matrix-multiplication algorithm.

answer:
Showing that being able to multiply matrices in time $M(n)$ implies being able to square matrices in time $M(n)$ is trivial because squaring a matrix is just multiplying it by itself.

The more tricky direction is showing that being able to square matrices in time $S(n)$ implies being able to multiply matrices in time $O(S(n))$.

As we do this, we apply the same regularity condition that $S(2n) \in O(S(n))$. Suppose that we are trying to multiply the matrices, $A$ and $B$, that is, find $AB$. Then, define the matrix

$$
C =
\begin{pmatrix}
I & A \\\\
0 & B
\end{pmatrix}
$$

Then, we can find $C^2$ in time $S(2n) \in O(S(n))$. Since

$$
C^2 =
\begin{pmatrix}
I & A + AB \\\\
0 & B
\end{pmatrix}
$$

Then we can just take the upper right quarter of $C^2$ and subtract $A$ from it to obtain the desired result. Apart from the squaring, we've only done work that is $O(n^2)$. Since $S(n)$ is $\Omega(n^2)$ anyways, we have that the total amount of work we've done is $O(n^2)$.


question:
 Let $M(n)$ be the time to multiply two $n \times n$ matrices, and let $L(n)$ be the time to compute the LUP decomposition of an $n \times n$ matrix. Show that multiplying matrices and computing LUP decompositions of matrices have essentially the same difficulty: an $M(n)$-time matrix-multiplication algorithm implies an $O(M(n))$-time LUP-decomposition algorithm, and an $L(n)$-time LUP-decomposition algorithm implies an $O(L(n))$-time matrix-multiplication algorithm.

answer:
Let $A$ be an $n \times n$ matrix. Without loss of generality we'll assume $n = 2^k$, and impose the regularity condition that $L(n / 2) \le cL(n)$ where $c < 1 / 2$ and $L(n)$ is the time it takes to find an LUP decomposition of an $n \times n$ matrix. First, decompose $A$ as

$$
A =
\begin{pmatrix}
A_1 \\\\
A_2
\end{pmatrix}
$$

where $A_1$ is $n / 2$ by $n$. Let $A_1 = L_1U_1P_1$ be an LUP decomposition of $A_1$, where $L_1$ is $ / 2$ by $n / 2$, $U_1$ is $n / 2$ by $n$, and $P_1$ is $n$ by $n$. Perform a block decomposition of $U_1$ and $A_2P_1^{-1}$ as $U_1 = [\overline U_1|B]$ and $A_2P_1^{-1} = [C|D]$ where $\overline U_1$ and $C$ are $n / 2$ by $n / 2$ matrices. Since we assume that $A$ is nonsingular, $\overline U_1$ must also be nonsingular.

Set $F = D - C\overline U_1^{-1}B$. Then we have

$$
A =
\begin{pmatrix}
                L_1 & 0 \\\\
C\overline U_1^{-1} & I_{n / 2}
\end{pmatrix}
\begin{pmatrix}
\overline U_1 & B \\\\
            0 & F
\end{pmatrix}
P_1.
$$

Now let $F = L_2U_2P_2$ be an LUP decomposition of $F$, and let $\overline P = \begin{pmatrix} I_{n / 2} & 0 \\\\ 0 & P_2 \end{pmatrix}$. Then we may write

$$
A =
\begin{pmatrix}
                L_1 & 0 \\\\
C\overline U_1^{-1} & L_2
\end{pmatrix}
\begin{pmatrix}
\overline U_1 & BP_2^{-1} \\\\
            0 & U_2
\end{pmatrix}
\overline P P_1.
$$

This is an LUP decomposition of $A$. To achieve it, we computed two LUP decompositions of half size, a constant number of matrix multiplications, and a constant number of matrix inversions. Since matrix inversion and multiplication are computationally equivalent, we conclude that the runtime is $O(M(n))$.


answer: (Omit!)
question:
 Let $M(n)$ be the time to multiply two $n \times n$ matrices, and let $D(n)$ denote the time required to find the determinant of an $n \times n$ matrix. Show that multiplying matrices and computing the determinant have essentially the same difficulty: an $M(n)$-time matrix-multiplication algorithm implies an $O(M(n))$-time determinant algorithm, and a $D(n)$-time determinant algorithm implies an $O(D(n))$-time matrix-multiplication algorithm.


question:
 Let $M(n)$ be the time to multiply two $n \times n$ boolean matrices, and let $T(n)$ be the time to find the transitive closure of an $n \times n$ boolean matrix. (See Section 25.2.) Show that an $M(n)$-time boolean matrix-multiplication algorithm implies an $O(M(n)\lg n)$-time transitive-closure algorithm, and a $T(n)$-time transitive-closure algorithm implies an $O(T(n))$-time boolean matrix-multiplication algorithm.

answer:
Suppose we can multiply boolean matrices in $M(n)$ time, where we assume this means that if we're multiplying boolean matrices $A$ and $B$, then $(AB)\_{ij} = (a\_{i1} \wedge b_{1j}) \vee \dots \vee (a_{in} \wedge b_{nj})$. To find the transitive closure of a boolean matrix $A$ we just need to find the $n^{\text{th}}$ power of $A$. We can do this by computing $A^2$, then $(A^2)^2$, then $((A^2)^2)^2$ and so on. This requires only $\lg n$ multiplications, so the transitive closure can be computed in $O(M(n)\lg n)$.

For the other direction, first view $A$ and $B$ as adjacency matrices, and impose the regularity condition $T(3n) = O(T(n))$, where $T(n)$ is the time to compute the transitive closure of a graph on $n$ vertices. We will define a new graph whose transitive closure matrix contains the boolean product of $A$ and $B$. Start by placing $3n$ vertices down, labeling them $1, 2, \dots, n, 1', 2', \dots, n', 1'', 2'', \dots, n''$.

Connect vertex $i$ to vertex $j'$ if and only if $A_{ij} = 1$. Connect vertex $j'$ to vertex $k''$ if and only if $B_{jk} = 1$. In the resulting graph, the only way to get from the first set of $n$ vertices to the third set is to first take an edge which "looks like" an edge in $A$, then take an edge which "looks like" an edge in $B$. In particular, the transitive closure of this graph is:

$$
\begin{pmatrix}
I & A & AB \\\\
0 & I & B \\\\
0 & 0 & I
\end{pmatrix}
.
$$

Since the graph is only of size $3n$, computing its transitive closure can be done in $O(T(3n)) = O(T(n))$ by the regularity condition. Therefore multiplying matrices and finding transitive closure are are equally hard.


question:
 Does the matrix-inversion algorithm based on Theorem 28.2 work when matrix elements are drawn from the field of integers modulo $2$? Explain.

answer:
It does not work necessarily over the field of two elements. The problem comes in in applying theorem D.6 to conclude that $A^{\text T}A$ is positive definite. In the proof of that theorem they obtain that $||Ax||^2 \ge 0$ and only zero if every entry of $Ax$ is zero. This second part is not true over the field with two elements, all that would be required is that there is an even number of ones in $Ax$. This means that we can only say that $A^{\text T}A$ is positive semi-definite instead of the positive definiteness that the algorithm requires.


question:
 Generalize the matrix-inversion algorithm of Theorem 28.2 to handle matrices of complex numbers, and prove that your generalization works correctly. ($\textit{Hint:}$ Instead of the transpose of $A$, use the **_conjugate transpose_** $A^\*$, which you obtain from the transpose of $A$ by replacing every entry with its complex conjugate. Instead of symmetric matrices, consider **_Hermitian_** matrices, which are matrices $A$ such that $A = A^\*$.)

answer:
We may again assume that our matrix is a power of $2$, this time with complex entries. For the moment we assume our matrix $A$ is Hermitian and positivedefinite. The proof goes through exactly as before, with matrix transposes replaced by conjugate transposes, and using the fact that Hermitian positivedefinite matrices are invertible. Finally, we need to justify that we can obtain the same asymptotic running time for matrix multiplication as for matrix inversion when $A$ is invertible, but not Hermitian positive-definite.

For any nonsingular matrix $A$, the matrix $A^\*A$ is Hermitian and positive definite, since for any $x$ we have $x^\*A^\*Ax = \langle Ax, Ax \rangle > 0$ by the definition of inner product. To invert $A$, we first compute $(A^\*A)^{-1} = A^{-1}(A^\*)^{-1}$. Then we need only multiply this result on the right by $A^\*$. Each of these steps takes $O(M(n))$ time, so we can invert any nonsingluar matrix with complex entries in $O(M(n))$ time.


question:
 Solve the equation
>
> $$
> \begin{pmatrix}
>  1 & 0 & 0 \\\\
>  4 & 1 & 0 \\\\
> -6 & 5 & 1
> \end{pmatrix}
> \begin{pmatrix}
> x_1 \\\\
> x_2 \\\\
> x_3
> \end{pmatrix}
> =
> \begin{pmatrix}
>  3 \\\\
> 14 \\\\
> -7
> \end{pmatrix}
> $$
>
> by using forward substitution.

answer:
$$
\begin{pmatrix}
3 \\\\
14 - 4 \cdot 3 \\\\
-7 - 5 \cdot (14 - 4 \cdot 3) - (-6) \cdot 3
\end{pmatrix}
=
\begin{pmatrix}
3 \\\\
2 \\\\
1
\end{pmatrix}
.
$$


question:
 Find an $\text{LU}$ decomposition of the matrix
>
> $$
> \begin{pmatrix}
>  4 & -5 &  6 \\\\
>  8 & -6 &  7 \\\\
> 12 & -7 & 12
> \end{pmatrix}
> .
> $$

answer:
$$
\begin{pmatrix}
 4 & -5 &  6 \\\\
 8 & -6 &  7 \\\\
12 & -7 & 12
\end{pmatrix}
=
\begin{pmatrix}
 1 &  0 &  0 \\\\
 2 &  1 &  0 \\\\
 3 &  2 &  1
\end{pmatrix}
\begin{pmatrix}
 4 & -5 &  6 \\\\
 0 &  4 & -5 \\\\
 0 &  0 &  4
\end{pmatrix}
.
$$


question:
 Solve the equation
>
> $$
> \begin{pmatrix}
> 1 & 5 & 4 \\\\
> 2 & 0 & 3 \\\\
> 5 & 8 & 2
> \end{pmatrix}
> \begin{pmatrix}
> x_1 \\\\
> x_2 \\\\
> x_3
> \end{pmatrix}
> =
> \begin{pmatrix}
> 12 \\\\
>  9 \\\\
>  5
> \end{pmatrix}
> $$
>
> by using forward substitution.

answer:
We have

$$
\begin{aligned}
A & =
\begin{pmatrix}
1 & 5 & 4 \\\\
2 & 0 & 3 \\\\
5 & 8 & 2
\end{pmatrix}
, \\\\
b & =
\begin{pmatrix}
12 \\\\
 9 \\\\
 5
\end{pmatrix}
,
\end{aligned}
$$

and we with to solve for the unknown $x$. The $\text{LUP}$ decomposition is

$$
\begin{aligned}
L & =
\begin{pmatrix}
1   &                0 & 0 \\\\
0.2 &                1 & 0 \\\\
0.4 & -\frac{3.2}{3.4} & 1
\end{pmatrix}
, \\\\
U & =
\begin{pmatrix}
5 & 8   & 2 \\\\
0 & 3.4 & 3.6 \\\\
0 & 0   & 2.2 + \frac{11.52}{3.4}
\end{pmatrix}
, \\\\
P & =
\begin{pmatrix}
0 & 0 & 1 \\\\
1 & 0 & 0 \\\\
0 & 1 & 0
\end{pmatrix}
.
\end{aligned}
$$

Using forward substitution, we solve $Ly = Pb$ for $y$:

$$
\begin{pmatrix}
1   &                0 & 0 \\\\
0.2 &                1 & 0 \\\\
0.4 & -\frac{3.2}{3.4} & 1
\end{pmatrix}
\begin{pmatrix}
y_1 \\\\
y_2 \\\\
y_3
\end{pmatrix}
=
\begin{pmatrix}
 5 \\\\
12 \\\\
 9
\end{pmatrix}
,
$$

obtaining

$$
y =
\begin{pmatrix}
 5 \\\\
11 \\\\
7 + \frac{35.2}{3.4}
\end{pmatrix}
$$

by computing first $y_1$, then $y_2$, and finally $y_3$. Using back substitution, we solve $Ux = y$ for $x$:

$$
\begin{pmatrix}
5 & 8   & 2 \\\\
0 & 3.4 & 3.6 \\\\
0 & 0   & 2.2 + \frac{11.52}{3.4}
\end{pmatrix}
\begin{pmatrix}
x_1 \\\\
x_2 \\\\
x_3
\end{pmatrix}
=
\begin{pmatrix}
 5 \\\\
11 \\\\
7 + \frac{35.2}{3.4}
\end{pmatrix}
,
$$

thereby obtaining the desired answer

$$
x =
\begin{pmatrix}
-\frac{3}{19} \\\\
-\frac{1}{19} \\\\
\frac{49}{19}
\end{pmatrix}
$$

by computing first $x_3$, then $x_2$, and finally $x_1$.


question: Describe the $\text{LUP}$ decomposition of a diagonal matrix.
answer:
The $\text{LUP}$ decomposition of a diagonal matrix $D$ is $D = IDI$ where $I$ is the identity matrix.


question:
 Show that for all $n \ge 1$, there exists a singular $n \times n$ matrix that has an $\text{LU}$ decomposition.

answer:
The zero matrix always has an $\text{LU}$ decomposition by taking $L$ to be any unit lower-triangular matrix and $U$ to be the zero matrix, which is upper triangular.


question:
 In $\text{LU-DECOMPOSITION}$, is it necessary to perform the outermost **for** loop iteration when $k = n$? How about in $\text{LUP-DECOMPOSITION}$?

answer:
For $\text{LU-DECOMPOSITION}$, it is indeed necessary. If we didn't run the line 6 of the outermost **for** loop, $u_{nn}$ would be left its initial value of $0$ instead of being set equal to $a_{nn}$. This can clearly produce incorrect results, because the $\text{LU-DECOMPOSITION}$ of any non-singular matrix must have both $L$ and $U$ having positive determinant. However, if $u_{nn} = 0$, the determinant of $U$ will be $0$ by problem D.2-2.

For $\text{LUP-DECOMPOSITION}$, the iteration of the outermost **for** loop that occurs with $k = n$ will not change the final answer. Since $\pi$ would have to be a permutation on a single element, it cannot be non-trivial. and the **for** loop on line 16 will not run at all.


question:
 Suppose we perform a sequence of stack operations on a stack whose size never exceeds $k$. After every $k$ operations, we make a copy of the entire stack for backup purposes. Show that the cost of $n$ stack operations, including copying the stack, is $O(n)$ by assigning suitable amortized costs to the various stack operations.

answer:
For every stack operation, we charge twice.

- First, we charge the actual cost of the stack operation.
- Second, we charge the cost of copying an element later on.

Since we have the size of the stack never exceed $k$, and there are always $k$ operations between backups, we always overpay by at least enough. Therefore, the amortized cost of the operation is constant, and the cost of the $n$ operation is $O(n)$.


question: Redo Exercise 17.1-3 using an accounting method of analysis.
answer:
Let $c_i =$ cost of $i$th operation.

$$
c_i =
\begin{cases}
i & \text{if $i$ is an exact power of $2$}, \\\\
1 & \text{otherwise}.
\end{cases}
$$

Charge $3$ (amortized cost $\hat c_i$) for each operation.

- If $i$ is not an exact power of $2$, pay $\$1$, and store $\$2$ as credit.
- If $i$ is an exact power of $2$, pay $\$i$, using stored credit.

$$
\begin{array}{cccc}
\text{Operation} & \text{Cost} & \text{Actual cost} & \text{Credit remaining} \\\\
\hline
 1 & 3 & 1 &  2 \\\\
 2 & 3 & 2 &  3 \\\\
 3 & 3 & 1 &  5 \\\\
 4 & 3 & 4 &  4 \\\\
 5 & 3 & 1 &  6 \\\\
 6 & 3 & 1 &  8 \\\\
 7 & 3 & 1 & 10 \\\\
 8 & 3 & 8 &  5 \\\\
 9 & 3 & 1 &  7 \\\\
10 & 3 & 1 &  9 \\\\
\vdots & \vdots & \vdots & \vdots
\end{array}
$$

Since the amortized cost is $\$3$ per operation, $\sum\limits_{i = 1}^n \hat c_i = 3n$.

We know from Exercise 17.1-3 that $\sum\limits_{i = 1}^n \hat c_i < 3n$.

Then we have

$$\sum_{i = 1}^n \hat c_i \ge \sum_{i = 1}^n c_i \Rightarrow \text{credit} = \text{amortized cose} - \text{actual cost} \ge 0.$$

Since the amortized cost of each operation is $O(1)$, and the amount of credit never goes negative, the total cost of $n$ operations is $O(n)$.


answer: (Removed)
question:
 Suppose we wish not only to increment a counter but also to reset it to zero (i.e., make all bits in it $0$). Counting the time to examine or modify a bit as $\Theta(1)$, show how to implement a counter as an array of bits so that any sequence of $n$ $\text{INCREMENT}$ and $\text{RESET}$ operations takes time $O(n)$ on an initially zero counter. ($\textit{Hint:}$ Keep a pointer to the high-order $1$.)


question:
 Suppose we have a potential function $\Phi$ such that $\Phi(D_i) \ge \Phi(D_0)$ for all $i$, but $\Phi(D_0) \ne 0$. Show that there exists a potential fuction $\Phi'$ such that $\Phi'(D_0) = 0$, $\Phi'(D_i) \ge 0$ for all $i \ge 1$, and the amortized costs using $\Phi'$ are the same as the amortized costs using $\Phi$.

answer:
Define the potential function $\Phi'(D_i) = \Phi(D_i) - \Phi(D_0)$ for all $i \ge 1$.

Then

$$\Phi'(D_0) = \Phi(D_0) - \Phi(D_0) = 0,$$

and

$$\Phi'(D_i) = \Phi(D_i) - \Phi(D_0) \ge 0.$$

The amortized cost is

$$
\begin{aligned}
\hat c_i' & = c_i + \Phi'(D_i) - \Phi'(D_{i - 1}) \\\\
          & = c_i + (\Phi(D_i) - \Phi(D_0)) - (\Phi(D_{i - 1}) - \Phi(D_0)) \\\\
          & = c_i + \Phi(D_i) - \Phi(D_{i - 1}) \\\\
          & = \hat c_i.
\end{aligned}
$$


question: Redo Exercise 17.1-3 using a potential method of analysis.
answer:
Define the potential function $\Phi(D_0) = 0$, and $\Phi(D_i) = 2i - 2^{1 + \lfloor \lg i \rfloor}$ for $i > 0$. For operation 1,

$$\hat c_i = c_i + \Phi(D_i) - \Phi(D_{i - 1}) = 1 + 2i - 2^{1+ \lfloor \lg i \rfloor} - 0 = 1.$$

For operation $i(i > 1)$, if $i$ is not a power of $2$, then

$$\hat c_i = c_i + \Phi(D_i) - \Phi(D_{i - 1}) = 1 + 2i - 2^{1 + \lfloor \lg 1 \rfloor} - (2(i - 1) - 2^{1 + \lfloor \lg(i - 1) \rfloor}) = 3.$$

If $i = 2^j$ for some $j \in \mathbb N$, then

$$\hat c_i = c_i + \Phi(D_i) - \Phi(D_{i - 1}) = i + 2i - 2^{1 + j}-(2(i - 1) - 2^{1 + j - 1}) = i + 2i - 2i - 2i + 2 + i = 2.$$

Thus, the amortized cost is $3$ per operation.


question:
 Consider an ordinary binary min-heap data structure with $n$ elements supporting the instructions $\text{INSERT}$ and $\text{EXTRACT-MIN}$ in $O(\lg n)$ worst-case time. Give a potential function $\Phi$ such that the amortized cost of $\text{INSERT}$ is $O(\lg n)$ and the amortized cost of $\text{EXTRACT-MIN}$ is $O(1)$, and show that it works.

answer:
Make the potential function be equal to $\sum_{i = 1}^n \lg i$ where $n$ is the size of the min-heap. Then, there is still a cost of $O(\lg n)$ to $\text{INSERT}$, since only an amount of amortization that is $\lg n$ was spent to increase the size of the heap by $1$.

However, the amortized cost of $\text{EXTRACT-MIN}$ is $0$, as all its actual cost is compensated.


question:
 What is the total cost of executing $n$ of the stack operations $\text{PUSH}$, $\text{POP}$, and $\text{MULTIPOP}$, assuming that the stack begins with $s_0$ objects and finishes with $s_n$ objects?

answer:
Let $\Phi$ be the potential function that returns the number of elements in the stack. We know that for this potential function, we have amortized cost $2$ for $\text{PUSH}$ operation and amortized cost $0$ for $\text{POP}$ and $\text{MULTIPOP}$ operations.

The total amortized cost is

$$\sum_{i = 1}^n \hat c_i = \sum_{i = 1}^n c_i + \Phi(D_n) - \Phi(D_0).$$

Using the potential function and the known amortized costs, we can rewrite the equation as

$$
\begin{aligned}
\sum_{i = 1}^n c_i
    & = \sum_{i = 1}^n \hat c_i + \Phi(D_0) - \Phi(D_n) \\\\
    & = \sum_{i = 1}^n \hat c_i + s_0 - s_n \\\\
    & \le 2n + s_0 - s_n,
\end{aligned}
$$

which gives us the total cost of $O(n + (s_0 - s_n))$. If $s_n \ge s_0$, then this equals to $O(n)$, that is, if the stack grows, then the work done is limited by the number of operations.

(Note that it does not matter here that the potential may go below the starting potential. The condition $\Phi(D_n) \ge \Phi(D_0)$ for all $n$ is only required to have $\sum_{i = 1}^n \hat c_i \ge \sum_{i = 1}^n c_i$, but we do not need for that to hold in this application.)


question:
 Suppose that a counter begins at a number with $b$ $1$s in its binary representation, rather than at $0$. Show that the cost of performing $n$ $\text{INCREMENT}$ operations is $O(n)$ if $n = \Omega(b)$. (Do not assume that $b$ is constant.)

answer:
$$
\begin{aligned}
\sum_{i = 1}^n c_i
    & =   \sum_{i = 1}^n \hat c_i - \Phi(D_n) + \Phi(D_0) \\\\
    & =   n - x + b \\\\
    & \le n - x + n \\\\
    & =   O(n).
\end{aligned}
$$


question:
 Show how to implement a queue with two ordinary stacks (Exercise 10.1-6) so that the amortized cost of each $\text{ENQUEUE}$ and each $\text{DEQUEUE}$ operation is $O(1)$.

answer:
We'll use the accounting method for the analysis. Assign cost $3$ to the $\text{ENQUEUE}$ operation and $0$ to the $\text{DEQUEUE}$ operation. Recall the implementation of 10.1-6 where we enqueue by pushing on to the top of stack 1, and dequeue by popping from stack 2.

If stack 2 is empty, then we must pop every element from stack 1 and push it onto stack 2 before popping the top element from stack 2. For each item that we enqueue we accumulate 2 credits. Before we can dequeue an element, it must be moved to stack 2. Note: this might happen prior to the time at which we wish to dequeue it, but it will happen only once overall. One of the 2 credits will be used for this move. Once an item is on stack 2 its pop only costs $1$ credit, which is exactly the remaining credit associated to the element. Since each operation's cost is $O(1)$, the amortized cost per operation is $O(1)$.


question:
 Design a data structure to support the following two operations for a dynamic multiset $S$ of integers, which allows duplicate values:
>
> $\text{INSERT}(S, x)$ inserts $x$ into $S$.
>
> $\text{DELETE-LARGER-HALF}(S)$ deletes the largest $\lceil |S| / 2 \rceil$ elements from $S$.
>
> Explain how to implement this data structure so that any sequence of $m$ $\text{INSERT}$ and $\text{DELETE-LARGER-HALF}$ operations runs in $O(m)$ time. Your implementation should also include a way to output the elements of $S$ in $O(|S|)$ time.

answer:
We'll store all our elements in an array, and if ever it is too large, we will copy all the elements out into an array of twice the length.

To delete the larger half, we first find the element $m$ with order statistic $\lceil |S| / 2 \rceil$ by the algorithm presented in section 9.3. Then, scan through the array and copy out the elements that are smaller or equal to $m$ into an array of half the size.

Since the delete half operation takes time $O(|S|)$ and reduces the number of elements by $\lfloor |S| / 2 \rfloor \in \Omega(|S|)$, we can make these operations take ammortized constant time by selecting our potential function to be linear in $|S|$.

Since the insert operation only increases $|S|$ by one, we have that there is only a constant amount of work going towards satisfying the potential, so the total ammortized cost of an insertion is still constant. To output all the elements just iterate through the array and output each.


question:
 If the set of stack operations included a $\text{MULTIPUSH}$ operation, which pushses $k$ items onto the stack, would the $O(1)$ bound on the amortized cost of stack operations continue to hold?

answer:
No. The time complexity of such a series of operations depends on the number of pushes (pops vice versa) could be made. Since one $\text{MULTIPUSH}$ needs $\Theta(k)$ time, performing $n$ $\text{MULTIPUSH}$ operations, each with $k$ elements, would take $\Theta(kn)$ time, leading to amortized cost of $\Theta(k)$.


question:
 Show that if a $\text{DECREMENT}$ operation were included in the $k$-bit counter example, $n$ operations could cost as much as $\Theta(nk)$ time.

answer:
The logarithmic bit flipping predicate does not hold, and indeed a sequence of events could consist of the incrementation of all $1$s and decrementation of all $0$s; yielding $\Theta(nk)$.


question:
 Suppose we perform a sequence of $n$ operations on a data structure in which the $i$th operation costs $i$ if $i$ is an exact power of $2$, and $1$ otherwise. Use aggregate analysis to determine the amortized cost per operation.

answer:
Let $n$ be arbitrary, and have the cost of operation $i$ be $c(i)$. Then we have,

$$
\begin{aligned}
\sum_{i = 1}^n c(i)
  & =   \sum_{i = 1}^{\left\lceil\lg n\right\rceil} 2^i + \sum_{i \le n \text{ is not a power of } 2} 1 \\\\
  & \le \sum_{i = 1}^{\left\lceil\lg n\right\rceil} 2^i + n \\\\
  & =   2^{1 + \left\lceil\lg n\right\rceil} - 1 + n \\\\
  & \le 2n - 1 + n \\\\
  & \le 3n \in O(n).
\end{aligned}
$$

To find the average, we divide by $n$, and the amortized cost per operation is $O(1)$.


question:
 Suppose that we wish to implement a dynamic, open-address hash table. Why might we consider the table to be full when its load factor reaches some value $\alpha$ that is strictly less than $1$? Describe briefly how to make insertion into a dynamic, open-address hash table run in such a way that the expected value of the amortized cost per insertion is $O(1)$. Why is the expected value of the actual cost per insertion not necessarily $O(1)$ for all insertions?

answer:
By theorems 11.6-11.8, the expected cost of performing insertions and searches in an open address hash table approaches infinity as the load factor approaches one, for any load factor fixed away from $1$, the expected time is bounded by a constant though. The expected value of the actual cost my not be $O(1)$ for every insertion because the actual cost may include copying out the current values from the current table into a larger table because it became too full. This would take time that is linear in the number of elements stored.


question:
 Show that if $\alpha_{i - 1} \ge 1 / 2$ and the $i$th operation on a dynamic table is $\text{TABLE-DELETE}$, then the amortized cost of the operation with respect to the potential function $\text{(17.6)}$ is bounded above by a constant.

answer:
If $\alpha_{i - 1} \ge 1 / 2$, $\text{TABLE-DELETE}$ cannot **_contract_**, so $c_i = 1$ and $size_i = size_{i - 1}$.

- **Case 1:** if $\alpha_i \ge 1 / 2$,

    $$
    \begin{aligned}
    \hat c_i & = c_i + \Phi_i - \Phi_{i - 1} \\\\
             & = 1 + (2 \cdot num_i - size_i) - (2 \cdot num_{i - 1} - size_{i - 1}) \\\\
             & = 1 + (2 \cdot (num_{i - 1} - 1) - size_{i - 1}) - (2 \cdot num_{i - 1} - size_{i - 1}) \\\\
             & = -1.
    \end{aligned}
    $$

- **Case 2:** if $\alpha_i < 1 / 2$,

    $$
    \begin{aligned}
    \hat c_i & =   c_i + \Phi_i - \Phi_{i - 1} \\\\
             & =   1 + (size_i / 2 - num_i) - (2 \cdot num_{i - 1} - size_{i - 1}) \\\\
             & =   1 + (size_{i - 1} / 2 - (num_{i - 1} - 1)) - (2 \cdot num_{i - 1} - size_{i - 1}) \\\\
             & =   2 + \frac{3}{2} size_{i - 1} - 3 \cdot num_{i - 1} \\\\
             & =   2 + \frac{3}{2} size_{i - 1} - 3\alpha_{i - 1} size_{i - 1} \\\\
             & \le 2 + \frac{3}{2} size_{i - 1} - \frac{3}{2} size_{i - 1} \\\\
             & =   2.
    \end{aligned}
    $$


question:
 Suppose that instead of contracting a table by halving its size when its load factor drops below $1 / 4$, we contract it by multiplying its size by $2 / 3$ when its load factor drops below $1 / 3$. Using the potential function
>
> $$\Phi(T) = | 2 \cdot T.num - T.size |,$$
>
> show that the amortized cost of a $\text{TABLE-DELETE}$ that uses this strategy is bounded above by a constant.

answer:
If $1 / 3 < \alpha_i \le 1 / 2$,

$$
\begin{aligned}
\hat c_i & = c_i + \Phi_i - \Phi_{i - 1} \\\\
         & = 1 + (size_i - 2 \cdot num_i) - (size_i - 2 \cdot (num_i + 1)) \\\\
         & = 3.
\end{aligned}
$$

If the $i$th operation does trigger a contraction,

$$
\begin{aligned}
\frac{1}{3} size_{i - 1} & = num_i + 1 \\\\
            size_{i - 1} & = 3 (num_i + 1) \\\\
            size_{i}     & = \frac{2}{3} size_{i - 1} = 2 (num_i + 1).
\end{aligned}
$$

$$
\begin{aligned}
\hat c_i & = c_i + \Phi_i - \Phi_{i - 1} \\\\
         & = (num_i + 1) + [2 \cdot (num_i + 1) - 2 \cdot num_i] - [3 \cdot (num_i + 1) - 2 \cdot (num_i + 1)] \\\\
         & = 2.
\end{aligned}
$$


question:
 Suppose that a root $x$ in a Fibonacci heap is marked. Explain how $x$ came to be a marked root. Argue that it doesn't matter to the analysis that $x$ is marked, even though it is not a root that was first linked to another node and then lost one child.

answer:
$x$ came to be a marked root because at some point it had been a marked child of $H.min$ which had been removed in $\text{FIB-HEAP-EXTRACT-MIN}$ operation. See figure 19.4 for an example, where the node with key $18$ became a marked root. It doesn't add the potential for having to do any more actual work for it to be marked. This is because the only time that markedness is checked is in line 3 of cascading cut. This however is only ever run on nodes whose parent is non $\text{NIL}$. Since every root has $\text{NIL}$ as it parent, line 3 of cascading cut will never be run on this marked root. It will still cause the potential function to be larger than needed, but that extra computation that was paid in to get the potential function higher will never be used up later.


question:
 Justify the $O(1)$ amortized time of $\text{FIB-HEAP-DECREASE-KEY}$ as an average cost per operation by using aggregate analysis.

answer:
Recall that the actual cost of $\text{FIB-HEAP-DECREASE-KEY}$ is $O\(c\)$, where $c$ is the number of calls made to $\text{CASCADING-CUT}$. If $c_i$ is the number of calls made on the $i$th key decrease, then the total time of $n$ calls to $\text{FIB-HEAPDECREASE-KEY}$ is $\sum_{i = 1}^n O(c_i)$.

Next observe that every call to $\text{CASCADING-CUT}$ moves a node to the root, and every call to a root node takes $O(1)$. Since no roots ever become children during the course of these calls, we must have that $\sum_{i = 1}^n c_i = O(n)$. Therefore the aggregate cost is $O(n)$, so the average, or amortized, cost is $O(1)$.


question:
 Professor Pinocchio claims that the height of an $n$-node Fibonacci heap is $O(\lg n)$. Show that the professor is mistaken by exhibiting, for any positive integer $n$, a sequence of Fibonacci-heap operations that creates a Fibonacci heap consisting of just one tree that is a linear chain of $n$ nodes.

answer:
- **Initialize:** insert $3$ numbers then extract-min.
- **Iteration:** insert $3$ numbers, in which at least two numbers are less than the root of chain, then extract-min. The smallest newly inserted number will be extracted and the remaining two numbers will form a heap whose degree of root is $1$, and since the root of the heap is less than the old chain, the chain will be merged into the newly created heap. Finally we should delete the node which contains the largest number of the 3 inserted numbers.


question:
 Suppose we generalize the cascading-cut rule to cut a node $x$ from its parent as soon as it loses its $k$th child, for some integer constant $k$. (The rule in Section 19.3 uses $k = 2$.) For what values of $k$ is $D(n) = O(\lg n)$?

answer:
Following the proof of lemma 19.1, if $x$ is any node if a Fibonacci heap, $x.degree = m$, and $x$ has children $y_1, y_2, \ldots, y_m$, then $y_1.degree \ge 0$ and $y_i.degree \ge i - k$. Thus, if $s_m$ denotes the fewest nodes possible in a node of degree $m$, then we have $s_0 = 1, s_1 = 2, \ldots, s_{k - 1} = k$ and in general, $s_m = k + \sum_{i = 0}^{m - k} s_i$.

Let

$$
f_m =
\begin{cases}
0                     & m = 0, \\\\
1                     & 0 < m < k, \\\\
f_{m - 1} + f_{m - k} & m \ge k.
\end{cases}
$$

Let $\alpha$ be a root of $x^k - x^{k - 1} = 1$, i.e. $x^{k - 1}(x - 1)=1$. We'll show by induction that $f_{m + k} \ge \alpha^m$.

For the base cases:

$$
\begin{aligned}
      f_k & = f_{k - 1} + f_0 = 1 + 0 = 1 = \alpha^0 \\\\
f_{k + 1} & = f_k + f_1 = 1 + 1 \ge \alpha^1 \\\\
          & \vdots \\\\
f_{k + k} & = f_{2k - 1} + f_k = f_{2k - 1} + 1 \ge 1 + \alpha^{k - 1} = \alpha^k.
\end{aligned}
$$

In general, we have

$$f_{m + k} = f_{m + k - 1} + f_m \ge \alpha^{m - 1} + \alpha^{m - k}  = \alpha^{m - k}(\alpha^{k - 1} + 1) = \alpha^m.$$

Next we show that $f_{m + k} = 1 + \sum_{i = 0}^m f_i$. The base case is clear, since $f_k = f_0 + 1 = 0 + 1$. For the induction step, we have

$$f_{m + k} = f_{m - 1 + k} + f_m = 1 + \sum_{i  0}^{m - 1} f_i + f_m = 1 + \sum_{i = 0}^m f_i.$$

Observe that $s_i \ge f_{i + k}$ for $0 \le i < k$. Again, by induction, for $m \ge k$ we have

$$s_m = k + \sum_{i = 0}^{m - k} s_i \ge k + \sum_{i = 0}^{m - k} f_{i + k} \ge k + \sum_{i = 0}^m f_i = f_{m + k}.$$

So in general, $s_m \ge f_{m + k}$. Putting it all together, we have

$$
\begin{aligned}
size(x) & \ge s_m \\\\
        & \ge k + \sum_{i = k}^m s_{i - k} \\\\
        & \ge k + \sum_{i = k}^m f_i \\\\
        & \ge k + \sum_{i = 0}^m f_i - \sum_{i = 0}^{k - 1} f_i \\\\
        & = k + \sum_{i = 0}^m f_i - (k - 1) \\\\
        & = 1 + \sum_{i = 0}^m f_i \\\\
        & = f_{m + k} \\\\
        & \ge \alpha^m.
\end{aligned}
$$

Taking logs on both sides, we have

$$\log_\alpha n \ge m.$$

In other words, provided that $\alpha$ is a constant, we have a logarithmic bound on the maximum degree.

And I think $k$ should only satisfy $\alpha > 1$ in $\alpha^{k - 1}(\alpha - 1) = 1$.


answer: (Omit!)
question:
 Show the Fibonacci heap that results from calling $\text{FIB-HEAP-EXTRACT-MIN}$ on the Fibonacci heap shown in Figure 19.4(m).


question:
 Show that if $f(n)$ and $g(n)$ are monotonically increasing functions, then so are the functions $f(n) + g(n)$ and $f(g(n))$, and if $f(n)$ and $g(n)$ are in addition nonnegative, then $f(n) \cdot g(n)$ is monotonically increasing.

answer:
$$
\begin{aligned}
           f(m) & \le f(n) \quad \text{ for } m \le n \\\\
           g(m) & \le g(n) \quad \text{ for } m \le n, \\\\
\to f(m) + g(m) & \le f(n) + g(n),
\end{aligned}
$$

which proves the first function.

Then

$$f(g(m)) \le f(g(n)) \text{ for } m \le n.$$

This is true, since $g(m) \le g(n)$ and $f(n)$ is monotonically increasing.

If both functions are nonnegative, then we can multiply the two equalities and we get

$$f(m) \cdot g(m) \le f(n) \cdot g(n).$$


question: Prove equation $\text{(3.16)}$.
answer:
$$
\begin{aligned}
a^{\log_b c} = a^\frac{\log_a c}{\log_a b} = (a^{\log_a c})^{\frac{1}{\log_a b}} = c^{\log_b a}
\end{aligned}
$$


question:
 Prove equation $\text{(3.19)}$. Also prove that $n! \ne \omega(2^n)$ and $n! \ne o(n^n)$.
>
> $$\lg(n!) = \Theta(n\lg n) \tag{3.19}$$

answer:
We can use **Stirling's approximation** to prove these three equations.

For equation $\text{(3.19)}$,

$$
\begin{aligned}
\lg(n!)
  & = \lg\Bigg(\sqrt{2\pi n}\Big(\frac{n}{e}\Big)^n\Big(1 + \Theta(\frac{1}{n})\Big)\Bigg) \\\\
  & = \lg\sqrt{2\pi n } + \lg\Big(\frac{n}{e}\Big)^n + \lg\Big(1+\Theta(\frac{1}{n})\Big) \\\\
  & = \Theta(\sqrt n) + n\lg{\frac{n}{e}} + \lg\Big(\Theta(1) + \Theta(\frac{1}{n})\Big) \\\\
  & = \Theta(\sqrt n) + \Theta(n\lg n) + \Theta(\frac{1}{n}) \\\\
  & = \Theta(n\lg n).
\end{aligned}
$$

For $n! \ne \omega(2^n)$,

$$
\begin{aligned}
\lim_{n \to \infty} \frac{2^n}{n!}
  & =   \lim_{n \to \infty} \frac{2^n}{\sqrt{2\pi n} \left(\frac{n}{e}\right)^n \left(1 + \Theta\left(\frac{1}{n}\right)\right)} \\\\
  & =   \lim_{n \to \infty} \frac{1}{\sqrt{2\pi n} \left(1 + \Theta\left(\frac{1}{n}\right)\right)} \left(\frac{2e}{n}\right)^n \\\\
  & \le \lim_{n \to \infty} \left(\frac{2e}{n}\right)^n \\\\
  & \le \lim_{n \to \infty} \frac{1}{2^n} = 0,
\end{aligned}
$$

where the last step holds for $n > 4e$.

For $n! \ne o(n^n)$,

$$
\begin{aligned}
\lim_{n \to \infty} \frac{n^n}{n!}
  & =   \lim_{n \to \infty} \frac{n^n}{\sqrt{2\pi n} \left(\frac{n}{e}\right)^n \left(1 + \Theta\left(\frac{1}{n}\right)\right)} \\\\
  & =   \lim_{n \to \infty} \frac{e^n}{\sqrt{2\pi n} \left(1 + \Theta\left(\frac{1}{n}\right)\right)} \\\\
  & =   \lim_{n \to \infty} O(\frac{1}{\sqrt n})e^n \\\\
  & \ge \lim_{n \to \infty} \frac{e^n}{c\sqrt n} & \text{(for some constant $c > 0$)}\\\\
  & \ge \lim_{n \to \infty} \frac{e^n}{cn} \\\\
  & =   \lim_{n \to \infty} \frac{e^n}{c} = \infty.
\end{aligned}
$$


question:
 Is the function $\lceil \lg n \rceil!$ polynomially bounded? Is the function $\lceil \lg\lg n \rceil!$ polynomially bounded?

answer:
Proving that a function $f(n)$ is polynomially bounded is equivalent to proving that $\lg(f(n)) = O(\lg n)$ for the following reasons.

- If $f$ is polynomially bounded, then there exist constants $c$, $k$, $n_0$ such that for all $n \ge n_0$, $f(n) \le cn^k$. Hence, $\lg(f(n)) \le kc\lg n$, which means that $\lg(f(n)) = O(\lg n)$.
- If $\lg(f(n)) = O(\lg n)$, then $f$ is polynomially bounded.

In the following proofs, we will make use of the following two facts:

1. $\lg(n!) = \Theta(n\lg n)$
2. $\lceil \lg n \rceil = \Theta(\lg n)$

$\lceil \lg n \rceil!$ is not polynomially bounded because

$$
\begin{aligned}
\lg(\lceil \lg n \rceil!)
    & =   \Theta(\lceil \lg n \rceil \lg \lceil \lg n \rceil) \\\\
    & =   \Theta(\lg n\lg\lg n) \\\\
    & =   \omega(\lg n) \\\\
    & \ne O(\lg n).
\end{aligned}
$$

$\lceil \lg\lg n \rceil!$ is polynomially bounded because

$$
\begin{aligned}
\lg(\lceil \lg\lg n \rceil!)
    & = \Theta(\lceil \lg\lg n \rceil \lg \lceil \lg\lg n \rceil) \\\\
    & = \Theta(\lg\lg n\lg\lg\lg n) \\\\
    & = o((\lg\lg n)^2) \\\\
    & = o(\lg^2(\lg n)) \\\\
    & = o(\lg n) \\\\
    & = O(\lg n).
\end{aligned}
$$

The last step above follows from the property that any polylogarithmic function grows more slowly than any positive polynomial function, i.e., that for constants $a, b > 0$, we have $\lg^b n = o(n^a)$. Substitute $\lg n$ for $n$, $2$ for $b$, and $1$ for $a$, giving $\lg^2(\lg n) = o(\lg n)$.

Therefore, $\lg(\lceil \lg\lg n \rceil!) = O(\lg n)$, and so $\lceil \lg\lg n \rceil!$ is polynomially bounded.


question: Which is asymptotically larger: $\lg(\lg^\*n)$ or $\lg^\*(\lg n)$?
answer:
We have $\lg^\* 2^n = 1 + \lg^\* n$,

$$
\begin{aligned}
\lim_{n \to \infty} \frac{\lg(\lg^\*n)}{\lg^\*(\lg n)}
    & = \lim_{n \to \infty} \frac{\lg(\lg^\* 2^n)}{\lg^\*(\lg 2^n)} \\\\
    & = \lim_{n \to \infty} \frac{\lg(1 + \lg^\* n)}{\lg^\* n} \\\\
    & = \lim_{n \to \infty} \frac{\lg(1 + n)}{n} \\\\
    & = \lim_{n \to \infty} \frac{1}{1 + n} \\\\
    & = 0.
\end{aligned}
$$

Therefore, we have that $\lg^*(\lg n)$ is asymptotically larger.


question:
 Show that the golden ratio $\phi$ and its conjugate $\hat \phi$ both satisfy the equation $x^2 = x + 1$.

answer:
$$
\begin{aligned}
\phi^2 & = \Bigg(\frac{1 + \sqrt 5}{2}\Bigg)^2 = \frac{6 + 2\sqrt 5}{4} = 1 + \frac{1 + \sqrt 5}{2} = 1 + \phi \\\\
\hat\phi^2 & = \Bigg(\frac{1 - \sqrt 5}{2}\Bigg)^2 = \frac{6 - 2\sqrt 5}{4} = 1 + \frac{1 - \sqrt 5}{2} = 1 + \hat\phi.
\end{aligned}
$$


question:
 Prove by induction that the $i$th Fibonacci number satisfies the equality
>
> $$F_i = \frac{\phi^i - \hat\phi^i}{\sqrt 5},$$
>
> where $\phi$ is the golden ratio and $\hat\phi$ is its conjugate.

answer:
- Base case

    For $i = 0$,

    $$
    \begin{aligned}
    \frac{\phi^0 - \hat\phi^0}{\sqrt 5}
        & = \frac{1 - 1}{\sqrt 5} \\\\
        & = 0 \\\\
        & = F_0.
    \end{aligned}
    $$

    For $i = 1$,

    $$
    \begin{aligned}
    \frac{\phi^1 - \hat\phi^1}{\sqrt 5}
        & = \frac{(1 + \sqrt 5) - (1 - \sqrt 5)}{2 \sqrt 5} \\\\
        & = 1 \\\\
        & = F_1.
    \end{aligned}
    $$

- Assume

    - $F_{i - 1} = (\phi^{i - 1} - \hat\phi^{i - 1}) / \sqrt 5$ and
    - $F_{i - 2} = (\phi^{i - 2} - \hat\phi^{i - 2}) / \sqrt 5$,

    $$
    \begin{aligned}
    F_i & = F_{i - 1} + F_{i - 2} \\\\
        & = \frac{\phi^{i - 1} - \hat\phi^{i - 1}}{\sqrt 5} + \frac{\phi^{i - 2} - \hat\phi^{i - 2}}{\sqrt 5} \\\\
        & = \frac{\phi^{i - 2}(\phi + 1) - \hat\phi^{i - 2}(\hat\phi + 1)}{\sqrt 5}  \\\\
        & = \frac{\phi^{i - 2}\phi^2 - \hat\phi^{i - 2}\hat\phi^2}{\sqrt 5} \\\\
        & = \frac{\phi^i - \hat\phi^i}{\sqrt 5}.
    \end{aligned}
    $$


question: Show that $k\ln k = \Theta(n)$ implies $k = \Theta(n / \lg n)$.
answer:
From the symmetry of $\Theta$,

$$k\ln k = \Theta(n) \Rightarrow n = \Theta(k\ln k).$$

Let's find $\ln n$,

$$\ln n = \Theta(\ln(k\ln k)) = \Theta(\ln k + \ln\ln k) = \Theta(\ln k).$$

Let's divide the two,

$$\frac{n}{\ln n} = \frac{\Theta(k\ln k)}{\Theta(\ln k)} = \Theta\Big({\frac{k\ln k}{\ln k}}\Big) = \Theta(k).$$


question:
 Let $f(n) + g(n)$ be asymptotically nonnegative functions. Using the basic definition of $\Theta$-notation, prove that $\max(f(n), g(n)) = \Theta(f(n) + g(n))$.

answer:
For asymptotically nonnegative functions $f(n)$ and $g(n)$, we know that

$$
\begin{aligned}
\exists n_1, n_2: & f(n) \ge 0 & \text{for} \, n > n_1 \\\\
                  & g(n) \ge 0 & \text{for} \, n > n_2.
\end{aligned}
$$

Let $n_0 = \max(n_1, n_2)$ and we know the equations below would be true for $n > n_0$:

$$
\begin{aligned}
f(n)             & \le \max(f(n), g(n)) \\\\
g(n)             & \le \max(f(n), g(n)) \\\\
(f(n) + g(n))/2  & \le \max(f(n), g(n)) \\\\
\max(f(n), g(n)) & \le \(f(n) + g(n)).
\end{aligned}
$$

Then we can combine last two inequalities:

$$0 \le \frac{f(n) + g(n)}{2} \le \max{(f(n), g(n))} \le f(n) + g(n).$$

Which is the definition of $\Theta{(f(n) + g(n))}$ with $c_1 = \frac{1}{2}$ and $c_2 = 1$


question:
 Show that for any real constants $a$ and $b$, where $b > 0$,
>
> $$(n + a)^b = \Theta(n^b). \tag{3.2}$$

answer:
Expand $(n + a)^b$ by the Binomial Expansion, we have

$$(n + a)^b = C_0^b n^b a^0 + C_1^b n^{b - 1} a^1 + \cdots + C_b^b n^0 a^b.$$

Besides, we know below is true for any polynomial when $x \ge 1$.

$$a_0 x^0 + a_1 x^1 + \cdots + a_n x^n \le (a_0 + a_1 + \cdots + a_n) x^n.$$

Thus,

$$C_0^b n^b \le C_0^b n^b a^0 + C_1^b n^{b - 1} a^1 + \cdots + C_b^b n^0 a^b \le (C_0^b + C_1^b + \cdots + C_b^b) n^b = 2^b n^b.$$

$$\implies (n + a)^b = \Theta(n^b).$$


question:
 Explain why the statement, "The running time of algorithm $A$ is at least $O(n^2)$," is meaningless.

answer:
$T(n)$: running time of algorithm $A$. We just care about the upper bound and the lower bound of $T(n)$.

The statement: $T(n)$ is at least $O(n^2)$.

- Upper bound: Because "$T(n)$ is at least $O(n^2)$", there's no information about the upper bound of $T(n)$.
- Lower bound: Assume $f(n) = O(n^2)$, then the statement: $T(n) \ge f(n)$, but $f(n)$ could be any fuction that is "smaller" than $n^2$. For example, constant, $n$, etc, so there's no conclusion about the lower bound of $T(n)$, too.

Therefore, the statement, "The running time of algorithm $A$ is at least $O(n^2)$," is meaningless.


question: Is $2^{n + 1} = O(2^n)$? Is $2^{2n} = O(2^n)$?
answer:
- True. Note that $2^{n + 1} = 2 \times 2^n$. We can choose $c \ge 2$ and $n_0 = 0$, such that $0 \le 2^{n + 1} \le c \times 2^n$ for all $n \ge n_0$. By definition, $2^{n + 1} = O(2^n)$.

- False. Note that $2^{2n} = 2^n \times 2^n = 4^n$. We can't find any $c$ and $n_0$, such that $0 \le 2^{2n} = 4^n \le c \times 2^n$ for all $n \ge n_0$.


question: Prove Theorem 3.1.
answer:
The theorem states:

> For any two functions $f(n)$ and $g(n)$, we have $f(n) = \Theta(g(n))$ if and only if $f(n) = O(g(n))$ and $f(n) = \Omega(g(n))$.

From $f = \Theta(g(n))$, we have that

$$0 \le c_1 g(n) \le f(n) \le c_2g(n) \text{ for } n > n_0.$$

We can pick the constants from here and use them in the definitions of $O$ and $\Omega$ to show that both hold.

From $f(n) = \Omega(g(n))$ and $f(n) = O(g(n))$, we have that

$$
\begin{aligned}
            & 0 \le c_3g(n) \le f(n) & \text{ for all } n \ge n_1 \\\\
\text{and } & 0 \le f(n) \le c_4g(n) & \text{ for all } n \ge n_2.
\end{aligned}
$$

If we let $n_3 = \max(n_1, n_2)$ and merge the inequalities, we get

$$0 \le c_3g(n) \le f(n) \le c_4g(n) \text{ for all } n > n_3.$$

Which is the definition of $\Theta$.


question:
 Prove that the running time of an algorithm is $\Theta(g(n))$ if and only if its worst-case running time is $O(g(n))$ and its best-case running time is $\Omega(g(n))$.

answer:
If $T_w$ is the worst-case running time and $T_b$ is the best-case running time, we know that

$$
\begin{aligned}
            & 0 \le c_1g(n) \le T_b(n) & \text{ for } n > n_b \\\\
\text{and } & 0 \le T_w(n) \le c_2g(n) & \text{ for } n > n_w.
\end{aligned}
$$

Combining them we get

$$0 \le c_1g(n) \le T_b(n) \le T_w(n) \le c_2g(n) \text{ for } n > \max(n_b, n_w).$$

Since the running time is bound between $T_b$ and $T_w$ and the above is the definition of the $\Theta$-notation, proved.


question: Prove $o(g(n)) \cap w(g(n))$ is the empty set.
answer:
Let $f(n) = o(g(n)) \cap w(g(n))$.
We know that for any $c_1 > 0$, $c_2 > 0$,

$$
\begin{aligned}
            & \exists n_1 > 0: 0 \le f(n) < c_1g(n) \\\\
\text{and } & \exists n_2 > 0: 0 \le c_2g(n) < f(n).
\end{aligned}
$$

If we pick $n_0 = \max(n_1, n_2)$, and let $c_1 = c_2$, from the problem definition we get

$$c_1g(n) < f(n) < c_1g(n).$$

There is no solutions, which means that the intersection is the empty set.


question:
 We can extend our notation to the case of two parameters $n$ and $m$ that can go to infinity independently at different rates. For a given function $g(n, m)$ we denote $O(g(n, m))$ the set of functions:
>
> $$
> \begin{aligned}
> O(g(n, m)) = \\{f(n, m):
>   & \text{ there exist positive constants } c, n_0, \text{ and } m_0 \\\\
>   & \text{ such that } 0 \le f(n, m) \le cg(n, m) \\\\
>   & \text{ for all } n \ge n_0 \text{ or } m \ge m_0.\\}
> \end{aligned}
> $$
>
> Give corresponding definitions for $\Omega(g(n, m))$ and $\Theta(g(n, m))$.

answer:
$$
\begin{aligned}
\Omega(g(n, m)) = \\{ f(n, m):
  & \text{ there exist positive constants $c$, $n_0$, and $m_0$ such that } \\\\
  & \text{ $0 \le cg(n, m) \le f(n, m)$ for all $n \ge n_0$ and $m \ge m_0$}.\\}
\end{aligned}
$$

$$
\begin{aligned}
\Theta(g(n, m)) = \\{ f(n, m):
  & \text{ there exist positive constants $c_1$, $c_2$, $n_0$, and $m_0$ such that } \\\\
  & \text{ $0 \le c_1 g(n, m) \le f(n, m) \le c_2 g(n, m)$ for all $n \ge n_0$ and $m \ge m_0$}.\\}
\end{aligned}
$$


answer: $x = 84$, $y = 814$.
question:
 Referring to the execution history shown in Figure 31.7(a), when does \text{POLLARDRHO} print the factor $73$ of $1387$?


answer: (Omit!)
question:
 Suppose that we are given a function $f : \mathbb Z_n \rightarrow \mathbb Z_n$ and an initial value $x_0 \in \mathbb Z_n$. Define $x_i = f(x_{i - 1})$ for $i = 1, 2, \ldots$. Let $t$ and $u > 0$ be the smallest values such that $x_{t + i} = x_{t + u + i}$ for $i = 0, 1, \ldots$. In the terminology of Pollard's rho algorithm, $t$ is the length of the tail and $u$ is the length of the cycle of the rho. Give an efficient algorithm to determine $t$ and $u$ exactly, and analyze its running time.


answer: $\Theta(\sqrt p)$.
question:
 How many steps would you expect $\text{POLLARD-RHO}$ to require to discover a factor of the form $p^e$, where $p$ is prime and $e > 1$?


answer: (Omit!)
question:
 One disadvantage of $\text{POLLARD-RHO}$ as written is that it requires one gcd computation for each step of the recurrence. Instead, we could batch the gcd computations by accumulating the product of several $x_i$ values in a row and then using this product instead of $x_i$ in the gcd computation. Describe carefully how you would implement this idea, why it works, and what batch size you would pick as the most effective when working on a $\beta$-bit number $n$.


question: Find all solutions to the equations $x \equiv 4 \pmod 5$ and $x \equiv 5 \pmod{11}$.
answer:
$$
\begin{aligned}
     m_1 & = 11, m_2 = 5. \\\\
m_1^{-1} & = 1, m_2^{-1} = 9. \\\\
     c_1 & = 11, c_2 = 45. \\\\
       a & = (c_1 \cdot a_1 + c_2 \cdot a_2) \mod (n_1 \cdot n_2) \\\\
         & = (11 \cdot 4 + 45 \cdot 5) \mod 55 = 49.
\end{aligned}
$$


answer: $10 + 504i$, $i \in \mathbb Z$.
question:
 Find all integers $x$ that leave remainders $1$, $2$, $3$ when divided by $9$, $8$, $7$ respectively.


answer: $$\gcd(a, n) = 1 \rightarrow \gcd(a, n_i) = 1.$$
question:
 Argue that, under the definitions of Theorem 31.27, if $\gcd(a, n) = 1$, then
>
> $$(a^{-1} \mod n) \leftrightarrow ((a_1^{-1} \mod n_1), (a_2^{-1} \mod n_2), \ldots, (a_k^{-1} \mod n_k)).$$


answer: Based on $\text{31.28}$–$\text{31.30}$.
question:
 Under the definitions of Theorem 31.27, prove that for any polynomial $f$, the number of roots of the equation $f(x) \equiv 0 (\mod n)$ equals the product of the number of roots of each of the equations
>
> $$f(x) \equiv 0 \pmod{n_1}, f(x) \equiv 0 \pmod{n_2}, \ldots, f(x) \equiv 0 \pmod{n_k}.$$


question:
 Prove that the equation $ax \equiv ay \pmod n$ implies $x \equiv y \pmod n$ whenever $\gcd(a, n) = 1$. Show that the condition $\gcd(a, n) = 1$ is necessary by supplying a counterexample with $\gcd(a, n) > 1$.

answer:
$$d = \gcd(ax, n) = \gcd(x, n)$$

Since $ax \cdot x' + n \cdot y' = d$,

we have

$$x \cdot (ax') + n \cdot y' = d.$$

$$
\begin{aligned}
 x_0 & = x'(ay / d), \\\\
x_0' & = (ax')(y / d) = x'(ay / d) = x_0.
\end{aligned}
$$


question:
 Consider the following change to line 3 of the procedure $\text{MODULAR-LINEAR-EQUATION-SOLVER}$:
>
> ```cpp
>  3  x0 = x'(b / d) mod (n / d)
> ```
>
> Will this work? Explain why or why not.

answer:
Assume that $x_0 \ge n / d$, then the largest solution is $x_0 + (d - 1) \cdot (n / d) \ge d \cdot n / d \ge n$, which is impossible, therefore $x_0 < n / d$.


answer: (Omit!)
question:
 Let $p$ be prime and $f(x) \equiv f_0 + f_1 x + \cdots + f_tx^t \pmod p$ be a polynomial of degree $t$, with coefficients $f_i$ drawn from $\mathbb Z_p$. We say that $a \in \mathbb Z_p$ is a **_zero_** of $f$ if $f(a) \equiv 0 \pmod p$. Prove that if $a$ is a zero of $f$, then $f(x) \equiv (x - a) g(x) \pmod p$ for some polynomial $g(x)$ of degree $t - 1$. Prove by induction on $t$ that if $p$ is prime, then a polynomial $f(x)$ of degree $t$ can have at most $t$ distinct zeros modulo $p$.


answer: $g = 2$, $\\{1, 2, 4, 8, 5, 10, 9, 7, 3, 6\\}$.
question:
 Draw a table showing the order of every element in $\mathbb Z_{11}^*$. Pick the smallest primitive root $g$ and compute a table giving $\text{ind}\_{11, g}(x)$ for all $x \in \mathbb Z_{11}^\*$.


question:
 Give a modular exponentiation algorithm that examines the bits of $b$ from right to left instead of left to right.

answer:
```cpp
MODULAR-EXPONENTIATION(a, b, n)
    i = 0
    d = 1
    while (1 << i) ≤ b
        if (b & (1 << i)) > 0
            d = (d * a) % n
        a = (a * a) % n
        i = i + 1
    return d
```


question:
 Assuming that you know $\phi(n)$, explain how to compute $a^{-1} \mod n$ for any $a \in \mathbb Z_n^\*$ using the procedure $\text{MODULAR-EXPONENTIATION}$.

answer:
$$
\begin{array}{rlll}
           a^{\phi(n)} & \equiv & 1             & \pmod n, \\\\
a\cdot a^{\phi(n) - 1} & \equiv & 1             & \pmod n, \\\\
                a^{-1} & \equiv & a^{\phi(n)-1} & \pmod n.
\end{array}
$$


answer: (Omit!)
question:
 Prove that if an odd integer $n > 1$ is not a prime or a prime power, then there exists a nontrivial square root of $1$ modulo $n$.


question:
 It is possible to strengthen Euler's theorem slightly to the form
>
> $a^{\lambda(n)} \equiv 1 \pmod n$ for all $a \in \mathbb Z_n^\*$,
>
> where $n = p_1^{e_1} \cdots p_r^{e_r}$ and $\lambda(n)$ is defined by
>
> $$\lambda(n) = \text{lcm}(\phi(p_1^{e_1}), \ldots, \phi(p_r^{e_r})). \tag{31.42}$$
>
> Prove that $\lambda(n) \mid \phi(n)$. A composite number $n$ is a Carmichael number if $\lambda(n) \mid n - 1$. The smallest Carmichael number is $561 = 3 \cdot 11 \cdot 17$; here, $\lambda(n) = \text{lcm}(2, 10, 16) = 80$, which divides $560$. Prove that Carmichael numbers must be both "square-free" (not divisible by the square of any prime) and the product of at least three primes. (For this reason, they are not very common.)

answer:
1. Prove that $\lambda(n) \mid \phi(n)$.

    We have

    $$
    \begin{aligned}
          n & = p_1^{e_1} \cdots p_r^{e_r} \\\\
    \phi(n) & = \phi(p_1^{e_1}) \times \dots \times \phi(p_r^{e_r}).
    \end{aligned}
    $$

    Thus,

    $$
    \begin{aligned}
    \lambda(n) & \mid \phi(n) \\\\
    \Rightarrow \text{lcm}(\phi(p_1^{e_1}, \ldots, \phi(p_r^{e_r})) & \mid (\phi(p_1^{e_1}) \times \dots \times \phi(p_r^{e_r}))
    \end{aligned}
    $$

2. Prove that Carmichael numbers must be "square-free" (not divisible by the square of any prime).

    Assume $n = p^\alpha m$ is a Carmichael number, where $\alpha \ge 2$ and $p \nmid m$.

    By the Chinese Remainder Theorem, the system of congruences

    $$
    \begin{aligned}
    x & \equiv 1 + p \pmod p^\alpha, \\\\
    x & \equiv 1 \pmod m
    \end{aligned}
    $$

    has a solution $a$. Note that $\gcd(a, n) = 1$.

    Since $n$ is a Carmichael number, we have $a^{n - 1} \equiv 1 \pmod n$. In particular, $a^{n - 1} \equiv 1 \pmod {p^\alpha}$; therefore, $a^n \equiv a \pmod {p^2}$.

    So, $(1 + p)^n \equiv 1 + p \pmod {p^2}$. Expand $(1 + p)^n$ modulo $p^2$ using the binomial theorem. We have

    $$(1 + p)^n \equiv 1 \pmod {p^2},$$

    since the first two terms of the expansion are $1$ and $np$, and the rest of the terms are divisible by $p^2$.

    Thus, $1 \equiv 1 + p \pmod {p^2}$. This is impossible.

    [Stack Exchange Reference](https://math.stackexchange.com/questions/1764812/carmichael-number-square-free)

3. Prove that Carmichael numbers must be the product of at least three primes.

    Assume that $n = pq$ is a Carmichael number, where $p$ and $q$ are distant primes and $p < q$.

    Then we have

    $$
    \begin{aligned}
    & q \equiv 1 \pmod{q - 1} \\\\
    \Rightarrow & n \equiv pq \equiv p \pmod{q - 1} \\\\
    \Rightarrow & n - 1 \equiv p - 1 \pmod{q - 1},
    \end{aligned}
    $$

    Here, $0 < p − 1 < q − 1$ , so $n − 1$ is not divisible by $q − 1$.

    [Stack Exchange Reference](https://math.stackexchange.com/questions/432162/carmichael-proof-of-at-least-3-factors)


question:
 Prove that if $x$ is a nontrivial square root of $1$, modulo $n$, then $\gcd(x - 1, n)$ and $\gcd(x + 1, n)$ are both nontrivial divisors of $n$.

answer:
$$
\begin{array}{rlll}
           x^2 & \equiv & 1 & \pmod n, \\\\
       x^2 - 1 & \equiv & 0 & \pmod n, \\\\
(x + 1)(x - 1) & \equiv & 0 & \pmod n.
\end{array}
$$

$n \mid (x + 1)(x - 1)$, suppose $\gcd(x - 1, n) = 1$, then $n \mid (x + 1)$, then $x \equiv -1 \pmod n$ which is trivial, it contradicts the fact that $x$ is nontrivial, therefore $\gcd(x - 1, n) \ne 1$, $\gcd(x + 1, n) \ne 1$.


question:
 Draw the group operation tables for the groups $(\mathbb Z_4, +_4)$ and $(\mathbb Z_5^\*, \cdot_5)$. Show that these groups are isomorphic by exhibiting a one-to-one correspondence $\alpha$ between their elements such that $a + b \equiv c \pmod 4$ if and only if $\alpha(a) \cdot \alpha(b) \equiv \alpha(c) \pmod 5$.

answer:
- $(\mathbb Z_4, +_4)$: $\\{ 0, 1, 2, 3 \\}$.
- $(\mathbb Z_5^\*, \cdot_5)$: $\\{ 1, 2, 3, 4 \\}$.

$\alpha(x) = 2^{x-1}$.


question: List all subgroups of $\mathbb Z_9$ and of $\mathbb Z_{13}^\*$.
answer:
- $\mathbb Z_9$:

    - $\langle 0 \rangle = \\{ 0 \\}$,
    - $\langle 1 \rangle = \\{ 0, 1, 2, 3, 4, 5, 6, 7, 8 \\}$,
    - $\langle 2 \rangle = \\{ 0, 2, 4, 6, 8 \\}$.

- $\mathbb Z_{13}^\*$:

    - $\langle 1 \rangle = \\{ 1 \\}$,
    - $\langle 2 \rangle = \\{ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 \\}$.


question: Prove Theorem 31.14.
answer:
A nonempty closed subset of a finite group is a subgroup.

- Closure: the subset is closed.
- Identity: suppose $a \in S'$, then $a^{(k)} \in S'$. Since the subset is finite, there must be a period such that $a^{(m)} = a^{(n)}$, hence $a^{(m)}a^{(-n)} = a^{(m - n)} = 1$, therefore the subset must contain the identity.
- Associativity: inherit from the origin group.
- Inverses: suppose $a^{(k)} = 1$, the inverse of element $a$ is $a^{(k - 1)}$ since $aa^{(k - 1)} = a^{(k)} = 1$.


question:
 Show that for any integer $n > 1$ and for any $a \in \mathbb Z_n^\*$, the function $f_a : \mathbb Z_n^\* \rightarrow \mathbb Z_n^\*$ defined by $f_a(x) = ax \mod n$ is a permutation of $\mathbb Z_n^\*$.

answer:
To prove it is a permutation, we need to prove that

- for each element $x \in \mathbb Z_n^\*$, $f_a(x) \in \mathbb Z_n^\*$,
- the numbers generated by $f_a$ are distinct.

Since $a \in \mathbb Z_{n}^\*$ and $x \in \mathbb Z_n^\*$, then $f_a(x) = ax \mod n \in \mathbb Z_n^\*$ by the closure property.

Suppose there are two distinct numbers $x \in \mathbb Z_n^\*$ and $y \in \mathbb Z_n^\*$ that $f_a(x) = f_a(y)$,

$$
\begin{aligned}
              f_a(x) & = f_a(y) \\\\
           ax \mod n & = ay \mod n \\\\
(a \mod n)(x \mod n) & = (a \mod n)(y \mod n) \\\\
          (x \mod n) & = y \mod n \\\\
                   x & \equiv y \mod n,
\end{aligned}
$$

which contradicts the assumption, therefore the numbers generated by $f_a$ are distinct.


question: Prove that for all integers $a$, $k$, and $n$, > > $\gcd(a, n) = \gcd(a + kn, n)$.
answer:
- $\gcd(a, n) \mid \gcd(a + kn, n)$.

    Let $d = \gcd(a, n)$, then $d \mid a$ and $d \mid n$.

    Since

    $$(a + kn) \mod d = a \mod d + k \cdot (n \mod d) = 0$$

    and $d \mid n$, we have

    $$d \mid \gcd(a + kn, n)$$

    and

    $$\gcd(a, n) \mid \gcd(a + kn, n).$$

- $\gcd(a + kn, n) \mid \gcd(a, n)$.

    Suppose $d = \gcd(a + kn, n)$, we have $d \mid n$ and $d \mid (a + kn)$.

    Since

    $$(a + kn) \mod d = a \mod d + k \cdot (n \mod d) = a \mod d = 0,$$

    we have $d \mid a$.

    Since $d \mid a$ and $d \mid n$, we have

    $$d \mid \gcd(a, n)$$

    and

    $$\gcd(a + kn, n) \mid \gcd(a, n).$$

    Since

    $$\gcd(a, n) \mid \gcd(a + kn, n)$$

    and

    $$\gcd(a + kn, n) \mid \gcd(a, n),$$

    we have

    $$\gcd(a, n) = \gcd(a + kn, n).$$


answer: ```cpp EUCLID(a, b) while b != 0 t = a a = b b = t % b return a ```
question:
 Rewrite $\text{EUCLID}$ in an iterative form that uses only a constant amount of memory (that is, stores only a constant number of integer values).


question:
 If $a > b \ge 0$, show that the call EUCLID$(a, b)$ makes at most $1 + \log_\phi b$ recursive calls. Improve this bound to $1 + \log_\phi(b / \gcd(a, b))$.

answer:
$$b \ge F_{k + 1} \approx \phi^{k + 1} / \sqrt{5}$$

$$k + 1 < \log_\phi \sqrt{5} + \log_\phi b \approx 1.67 + \log_\phi b$$

$$k < 0.67 + \log_\phi b < 1 + \log_\phi b.$$

Since $d \cdot a \mod d \cdot b = d \cdot (a \mod b)$, $\gcd(d \cdot a, d \cdot b)$ has the same number of recursive calls with $\gcd(a, b)$, therefore we could let $b' = b / \gcd(a, b)$, the inequality $k < 1 + \log_\phi(b') = 1 + \log_\phi(b / \gcd(a, b))$. will holds.

# 31.2-6

> What does $\text{EXTENDED-EUCLID}(F_{k + 1}, F_k)$ return? Prove your answer correct.

- If $k$ is odd, then $(1, -F_{k-2}, F_{k - 1})$.
- If $k$ is even, then $(1, F_{k-2}, -F_{k - 1})$.


question:
 Define the $\gcd$ function for more than two arguments by the recursive equation $\gcd(a_0, a_1, \cdots, a_n) = \gcd(a_0, \gcd(a_1, a_2, \cdots, a_n))$. Show that the $\gcd$ function returns the same answer independent of the order in which its arguments are specified. Also show how to find integers $x_0, x_1, \cdots, x_n$ such that $\gcd(a_0, a_1, \ldots, a_n) = a_0 x_0 + a_1 x_1 + \cdots + a_n x_n$. Show that the number of divisions performed by your algorithm is $O(n + \lg (max \\{a_0, a_1, \cdots, a_n \\}))$.

answer:
Suppose

$$\gcd(a_0, \gcd(a_1, a_2, \cdots, a_n)) = a_0 \cdot x + \gcd(a_1, a_2, \cdots, a_n) \cdot y$$

and

$$\gcd(a_1, \gcd(a_2, a_3, \cdots, a_n)) = a_1 \cdot x' + \gcd(a_2, a_3, \cdots, a_n) \cdot y',$$

then the coefficient of $a_1$ is $y \cdot x'$.

```cpp
EXTENDED-EUCLID(a, b)
    if b == 0
        return (a, 1, 0)
    (d, x, y) = EXTENDED-EUCLID(b, a % b)
    return (d, y, x - (a / b) * y)
```

```cpp
EXTENDED-EUCLID-MULTIPLE(a)
    if a.length == 1
        return (a[0], 1)
    g = a[a.length - 1]
    xs = [1] * a.length
    ys = [0] * a.length
    for i = a.length - 2 downto 0
        (g, xs[i], ys[i + 1]) = EXTENDED-EUCLID(a[i], g)
    m = 1
    for i = 1 to a.length
        m *= ys[i]
        xs[i] *= m
    return (g, xs)
```


question:
 Define $\text{lcm}(a_1, a_2, \ldots, a_n)$ to be the **_least common multiple_** of the $n$ integers $a_1, a_2, \ldots, a_n$, that is, the smallest nonnegative integer that is a multiple of each $a_i$. Show how to compute $\text{lcm}(a_1, a_2, \ldots, a_n)$ efficiently using the (two-argument) $\gcd$ operation as a subroutine.

answer:
```cpp
GCD(a, b)
    if b == 0
        return a
    return GCD(b, a % b)
```

```cpp
LCM(a, b)
    return a / GCD(a, b) * b
```

```cpp
LCM-MULTIPLE(a)
    l = a[0]
    for i = 1 to a.length
        l = LCM(l, a[i])
    return l
```


question:
 Prove that $n_1$, $n_2$, $n_3$, and $n_4$ are pairwise relatively prime if and only if
>
> $\gcd(n_1n_2,n_3n_4) = \gcd(n_1n_3, n_2n_4) = 1.$
>
> More generally, show that $n_1, n_2, \ldots, n_k$ are pairwise relatively prime if and only if a set of $\lceil \lg k \rceil$ pairs of numbers derived from the $n_i$ are relatively prime.

answer:
Suppose $n_1n_2 x + n_3n_4 y = 1$, then $n_1(n_2 x) + n_3(n_4 y) = 1$, thus $n_1$ and $n_3$ are relatively prime, $n_1$ and $n_4$, $n_2$ and $n_3$, $n_2$ and $n_4$ are the all relatively prime. And since $\gcd(n_1n_3, n_2n_4) = 1$, all the pairs are relatively prime.

General: in the first round, divide the elements into two sets with equal number of elements, calculate the products of the two set separately, if the two products are relatively prime, then the element in one set is pairwise relatively prime with the element in the other set. In the next iterations, for each set, we divide the elements into two subsets, suppose we have subsets $\\{ (s_1, s_2), (s_3, s_4), \ldots \\}$, then we calculate the products of $\\{ s_1, s_3, \ldots \\}$ and $\\{ s_2, s_4, \ldots \\}$, if the two products are relatively prime, then all the pairs of subset are pairwise relatively prime similar to the first round. In each iteration, the number of elements in a subset is half of the original set, thus there are $\lceil \lg k \rceil$ pairs of products.

To choose the subsets efficiently, in the $k$th iteration, we could divide the numbers based on the value of the index's $k$th bit.


question: Prove that if $a > b > 0$ and $c = a + b$, then $c \mod a = b$.
answer:
$$
\begin{aligned}
c \mod a & = (a + b) \mod a \\\\
         & = (a \mod a) + (b \mod a) \\\\
         & = 0 + b \\\\
         & = b.
\end{aligned}
$$


question: Prove that there are infinitely many primes.
answer:
$$
\begin{aligned}
    ((p_1 p_2 \cdots p_k) + 1) \mod p_i
& = (p_1 p_2 \cdots p_k) \mod p_i + (1 \mod p_i) \\\\
& = 0 + 1 \\\\
& = 1.
\end{aligned}
$$

if $p_1 p_2 \cdots p_k$ is all prime numbers, then $(p_1 p_2 \cdots p_k) + 1$ is a new prime number.


question: Prove that if $a \mid b$ and $b \mid c$, then $a \mid c$.
answer:
- If $a \mid b$, then $b = a \cdot k_1$.
- If $b \mid c$, then $c = b \cdot k_2 = a \cdot (k_1 \cdot k_2) = a \cdot k_3$, then $a \mid c$.


question:
 Prove Corollary 31.5.
>
> For all positive integers $n$, $a$, and $b$, if $n \mid ab$ and $\gcd(a, n) = 1$, then $n \mid b$.

answer:
Assume for the purpose of contradiction that $n \mid ab$ and $\gcd(a, n) = 1$, but $n \nmid b$, so $gcd(n, b)=1$, for theorem 31.6, $gcd(n, ab)=1$ which is contradicting our assumption.


question:
 Prove that if $p$ is prime and $0 < k < p$, then $p \mid \binom{p}{k}$. Conclude that for all integers $a$ and $b$ and all primes $p$,
>
> $(a + b)^p \equiv a^p + b^p \pmod p$.

answer:
$$
\begin{array}{rlll}
(a + b) ^ p & \equiv & a^p + \binom{p}{1} a^{p - 1}b^{1} + \cdots + \binom{p}{p - 1} a^{1}b^{p - 1} + b^p & \pmod p \\\\
            & \equiv & a^p + 0 + \cdots + 0 + b^p                                                         & \pmod p \\\\
            & \equiv & a^p + b^p                                                                          & \pmod p
\end{array}
$$


question:
 Prove that if $a$ and $b$ are any positive integers such that $a \mid b$, then
>
> $$(x \mod b) \mod a = x \mod a$$
>
> for any $x$. Prove, under the same assumptions, that
>
> $x \equiv y \pmod b$ implies $x \equiv y \pmod a$
>
> for any integers $x$ and $y$.

answer:
Suppose $x = kb + c$, we have

$$(x \mod b) \mod a = c \mod a,$$

and

$$x \mod a = (kb + c) \mod a = (kb \mod a) + (c \mod a) = c \mod a.$$


question:
 For any integer $k > 0$, an integer $n$ is a **_$k$th power_** if there exists an integer $a$ such that $a^k = n$. Furthermore, $n > 1$ is a **_nontrivial power_** if it is a $k$th power for some integer $k > 1$. Show how to determine whether a given $\beta$-bit integer $n$ is a nontrivial power in time polynomial in $\beta$.

answer:
Because $2^\beta > n$, we only need to test values of $k$ that satisfy $2 \le k < \beta$, therefore the testing procedure remains $O(\beta)$.

For any nontrivial power $k$, where $2 \le k < \beta$, do a binary search on $a$ that costs

$$O(\log \sqrt n) = O(\log \sqrt{2^\beta}) = O(\frac 1 2\log 2^\beta) = O(\beta).$$

Thus, the total time complexity is

$$O(\beta) \times O(\beta) = O(\beta^2).$$


question:
 Show that the gcd operator is associative. That is, prove that for all integers $a$, $b$, and $c$,
>
> $\gcd(a, \gcd(b, c)) = \gcd(\gcd(a, b), c)$.

answer:
_[The following proof is provided by my friend, Tony Xiao.]_

Let $d = \gcd(a, b, c)$, $a = dp$, $b = dq$ and $c = dr$.

**_Claim_** $\gcd(a, \gcd(b, c)) = d.$

Let $e = \gcd(b, c)$, thus

$$
\begin{aligned}
b = es, \\\\
c = et.
\end{aligned}
$$

Since $d \mid b$ and $d \mid c$, thus $d \mid e$.

Let $e = dm$, thus

$$
\begin{aligned}
b = (dm)s & = dq, \\\\
c = (dm)t & = dr.
\end{aligned}
$$

Suppose $k = \gcd(p, m)$,

$$
\begin{aligned}
            & k \mid p, k \mid m, \\\\
\Rightarrow & dk \mid dp, dk \mid dm, \\\\
\Rightarrow & dk \mid dp, dk \mid (dm)s, dk \mid (dm)t, \\\\
\Rightarrow & dk \mid a, dk \mid b, dk \mid c.
\end{aligned}
$$

Since $d = \gcd(a, b, c)$, thus $k = 1$.

$$
\begin{aligned}
\gcd(a, \gcd(b, c))
    & = \gcd(a, e) \\\\
    & = \gcd(dp, dm) \\\\
    & = d \cdot \gcd(p, m) \\\\
    & = d \cdot k \\\\
    & = d.
\end{aligned}
$$

By the claim,

$$\gcd(a, \gcd(b, c)) = d = \gcd(\gcd(a, b), c).$$


question:
 Give efficient algorithms for the operations of dividing a $\beta$-bit integer by a shorter integer and of taking the remainder of a $\beta$-bit integer when divided by a shorter integer. Your algorithms should run in time $\Theta(\beta^2)$.

answer:
Shift left until the two numbers have the same length, then repeatedly subtract with proper multiplier and shift right.


answer: (Omit!)
question:
 Give an efficient algorithm to convert a given $\beta$-bit (binary) integer to a decimal representation. Argue that if multiplication or division of integers whose length is at most $\beta$ takes time $M(\beta)$, then we can convert binary to decimal in time $\Theta(M(\beta) \lg\beta)$.


question:
 Consider an RSA key set with $p = 11$, $q = 29$, $n = 319$, and $e = 3$. What value of $d$ should be used in the secret key? What is the encryption of the message $M = 100$?

answer:
$\phi(n) = (p - 1) \cdot (q - 1) = 280$.

$d = e^{-1} \mod \phi(n) = 187$.

$P(M) = M^e \mod n = 254$.

$S(C) = C^d \mod n = 254^{187} \mod n =  100$.


question:
 Prove that if Alice's public exponent $e$ is $3$ and an adversary obtains Alice's secret exponent $d$, where $0 < d < \phi(n)$, then the adversary can factor Alice's modulus $n$ in time polynomial in the number of bits in $n$. (Although you are not asked to prove it, you may be interested to know that this result remains true even if the condition $e = 3$ is removed. See Miller [255].)

answer:
$$ed \equiv 1 \mod \phi(n)$$

$$ed - 1 = 3d - 1 = k \phi(n)$$

If $p, q < n / 4$, then

$$\phi(n) = n - (p + q) + 1 > n - n / 2 + 1 = n / 2 + 1 > n / 2.$$

$kn / 2 < 3d - 1 < 3d < 3n$, then $k < 6$, then we can solve $3d - 1 = n - p - n / p + 1$.


question:
 Prove that RSA is multiplicative in the sense that
>
> $P_A(M_1) P_A(M_2) \equiv P_A(M_1, M_2) \pmod n$.
>
> Use this fact to prove that if an adversary had a procedure that could efficiently decrypt $1$ percent of messages from $\mathbb Z_n$ encrypted with $P_A$, then he could employ a probabilistic algorithm to decrypt every message encrypted with $P_A$ with high probability.

answer:
Multiplicative: $e$ is relatively prime to $n$.

Decrypt: In each iteration randomly choose a prime number $m$ that $m$ is relatively prime to $n$, if we can decrypt $m \cdot M$, then we can return $m^{-1}M$ since $m^{-1} = m^{n - 2}$.


answer: (Omit!)
question:
 Give an example of a graph for which $\text{APPROX-VERTEX-COVER}$ always yields a suboptimal solution.


answer: (Omit!)
question:
 Prove that the set of edges picked in line 4 of $\text{APPROX-VERTEX-COVER}$ forms a maximal matching in the graph $G$.


question:
 Professor Bündchen proposes the following heuristic to solve the vertex-cover problem. Repeatedly select a vertex of highest degree, and remove all of its incident edges. Give an example to show that the professor's heuristic does not have an approximation ratio of $2$. ($\textit{Hint:}$ Try a bipartite graph with vertices of uniform degree on the left and vertices of varying degree on the right.)

answer:
Consider a bibartite graph with left part $L$ and right part $R$ such that $L$ has $5$ vertices of degrees $(5, 5, 5, 5, 5)$ and $R$ has $11$ vertices of degrees $(5, 4, 4, 3, 2, 2, 1, 1, 1, 1, 1)$ (the graph is easy to draw and the figure is omitted here).

Clearly there exists a vertex-cover of size $5$ (the left vertices). The idea is to show that the proposed algorithm chooses all the vertices on the right part, resulting in the approximation ratio of $11 / 5 > 2$.

1. After choosing the first vertex in $R$, the degrees on $L$ decrease to $(4, 4, 4, 4, 4)$.
2. After choosing the second vertex in $R$, the degrees on $L$ decrease to $(4, 3, 3, 3, 3)$.
3. After choosing the third vertex in $R$, the degrees on $L$ decrease to $(3, 3, 2, 2, 2)$.
4. After choosing the fourth vertex in $R$, the degrees on $L$ decrease to $(2, 2, 2, 2, 1)$.
5. After choosing the fifth vertex in $R$, the degrees on $L$ decrease to $(2, 2, 1, 1, 1)$.
6. After choosing the sixth vertex in $R$, the degrees on $L$ decrease to $(1, 1, 1, 1, 1)$.

Now the algorithm still has to choose $5$ more vertices.


answer: (Omit!)
question:
 From the proof of Theorem 34.12, we know that the vertex-cover problem and the $\text{NP-complete}$ clique problem are complementary in the sense that an optimal vertex cover is the complement of a maximum-size clique in the complement graph. Does this relationship imply that there is a polynomial-time approximation algorithm with a constant approximation ratio for the clique problem? Justify your answer.


answer: (Omit!)
question:
 Consider each of the following words as a set of letters: $\\{\text{arid}$, $\text{dash}$, $\text{drain}$, $\text{heard}$, $\text{lost}$, $\text{nose}$, $\text{shun}$, $\text{slate}$, $\text{snare}$, $\text{thread}\\}$. Show which set cover $\text{GREEDY-SET-COVER}$ produces when we break ties in favor of the word that appears ﬁrst in the dictionary.


answer: (Omit!)
question:
 Show that the decision version of the set-covering problem is $\text{NP-complete}$ by reducing it from the vertex-cover problem.


answer: (Omit!)
question:
 Show how to implement $\text{GREEDY-SET-COVER}$ in such a way that it runs in time $O\Big(\sum_{S \in \mathcal F} |S|\Big)$.


answer: (Omit!)
question:
 Show that the following weaker form of Theorem 35.4 is trivially true:
>
> $$|\mathcal C| \le |\mathcal C^\*| \max\\{|S|: S \in \mathcal F\\}.$$


answer: (Omit!)
question:
 $\text{GREEDY-SET-COVER}$ can return a number of different solutions, depending on how we break ties in line 4. Give a procedure $\text{BAD-SET-COVER-INSTANCE}(n)$ that returns an $n$-element instance of the set-covering problem for which, depending on how we break ties in line 4, $\text{GREEDY-SET-COVER}$ can return a number of different solutions that is exponential in $n$.


answer: (Omit!)
question:
 Show that even if we allow a clause to contain both a variable and its negation, randomly setting each variable to 1 with probability $1 / 2$ and to $0$ with probability $1 / 2$ still yields a randomized $8 / 7$-approximation algorithm.


answer: (Omit!)
question:
 The **_MAX-CNF satisfiability problem_** is like the $\text{MAX-3-CNF}$ satisfiability problem, except that it does not restrict each clause to have exactly $3$ literals. Give a randomized $2$-approximation algorithm for the $\text{MAX-CNF}$ satisfiability problem.


question:
 In the $\text{MAX-CUT}$ problem, we are given an unweighted undirected graph $G = (V, E)$. We define a cut $(S, V - S)$ as in Chapter 23 and the **_weight_** of a cut as the number of edges crossing the cut. The goal is to find a cut of maximum weight. Suppose that for each vertex $v$, we randomly and independently place $v$ in $S$ with probability $1 / 2$ and in $V - S$ with probability $1 / 2$. Show that this algorithm is a randomized $2$-approximation algorithm.

answer:
$\gdef\Ex{\mathbf{E}}$
$\gdef\Prob{\mathbf{P}}$
$\gdef\opt{\texttt{opt}}$

We first rewrite the algorithm for clarity.

```cpp
APPROX-MAX-CUT(G)
    for each v in V
        flip a fair coin
        if heads
            add v to S
        else
            add v to V - S
```

This algorithm clearly runs in linear time. For each edge $(u, v) \in E$, define the event $A_{uv}$ to be the event where edge
$(i, j)$ crosses the cut $(S, V - S)$, and let $1_{A_{uv}}$ be the indicator random variable for $A_{uv}$.

The event $A_{ij}$ occurs if and only if the vertices $u$ and $v$ are placed in different sets during the main loop in $\text{APPROX-MAX-CUT}$. Hence,

$$
\begin{aligned}
\Prob \\{A_{uv}\\}
    &= \Prob \\{u \in S \wedge v \in V - S\\} + P\\{u \in V - S \wedge v\in S\\} \\\\
    &= \frac{1}{2}\cdot\frac{1}{2} + \frac{1}{2}\cdot\frac{1}{2} \\\\
    &= \frac{1}{2}.
\end{aligned}
$$

Let $\opt$ denote the cost of a maximum cut in $G$, and let $c = |(S, V - S)|$, that is, the size of the cut produced by $\text{APPROX-MAX-CUT}$. Clearly $c = \sum_{(u, v) \in E} 1_{A_{uv}}$. Also, note that $\texttt{opt} \leq |E|$ (this is tight iff $G$ is bipartite). Hence,

$$
\begin{aligned}
\Ex[c]
    &= \Ex\left[\sum_{(u, v) \in E} 1_{A_{uv}}\right]\\\\
    &= \sum_{(u, v)\in E}\Ex[1_{A_{uv}}]\\\\
    &= \sum_{(u, v)\in E}\Prob\\{A_{uv}\\}\\\\
    &= \frac{1}{2}|E|\\\\
    &\ge \frac{1}{2}\opt
\end{aligned}
$$

Hence, $\Ex\left[\frac{\opt}{c}\right] \le \frac{|E|}{1 / 2|E|} = 2$, and so $\text{APPROX-MAX-CUT}$ is a randomized $2$-approximation algorithm.


answer: (Omit!)
question:
 Show that the constraints in line $\text{(35.19)}$ are redundant in the sense that if we remove them from the linear program in lines $\text{(35.17)}–\text{(35.20)}$, any optimal solution to the resulting linear program must satisfy $x(v) \le 1$ for each $v \in V$.


answer: (Omit!)
question:
 Prove equation $\text{(35.23)}$. Then show that after executing line 5 of $\text{EXACT-SUBSET-SUM}$, $L_i$ is a sorted list containing every element of $P_i$ whose value is not more than $t$.


answer: (Omit!)
question:
 How would you modify the approximation scheme presented in this section to ﬁnd a good approximation to the smallest value not less than $t$ that is a sum of some subset of the given input list?


answer: (Omit!)
question:
 Modify the $\text{APPROX-SUBSET-SUM}$ procedure to also return the subset of $S$ that sums to the value $z^\*$.


question:
 Suppose that a complete undirected graph $G = (V, E)$ with at least $3$ vertices has a cost function $c$ that satisfies the triangle inequality. Prove that $c(u, v) \ge 0$ for all $u, v \in V$.

answer:
Let $u, v \in V$. Let $w$ be any other vertex in $V$. Then by the triangle inequality we have

$$
\begin{aligned}
c(w, u) + c(u, v) &\ge c(w, v) \\\\
c(w, v) + c(v, u) &\ge c(w, u).
\end{aligned}
$$

Adding the above inequalities together gives

$$
\cancel{c(w, u)} + \cancel{c(w, v)} + c(u, v) + c(v, u) \ge \cancel{c(w, v)} + \cancel{c(w, u)}.
$$

Since $G$ is undirected we have $c(u, v) = c(v, u)$, and so the above inequality gives $2c(u, v) \ge 0$ from which the result follows.


answer: (Omit!)
question:
 Show how in polynomial time we can transform one instance of the traveling-salesman problem into another instance whose cost function satisfies the triangle inequality. The two instances must have the same set of optimal tours. Explain why such a polynomial-time transformation does not contradict Theorem 35.3, assuming that $\text P \ne \text{NP}$.


answer: (Omit!)
question:
 Consider the following **_closest-point heuristic_** for building an approximate traveling-salesman tour whose cost function satisfies the triangle inequality. Begin with a trivial cycle consisting of a single arbitrarily chosen vertex. At each step, identify the vertex $u$ that is not on the cycle but whose distance to any vertex on the cycle is minimum. Suppose that the vertex on the cycle that is nearest $u$ is vertex $v$. Extend the cycle to include $u$ by inserting $u$ just after $v$. Repeat until all vertices are on the cycle. Prove that this heuristic returns a tour whose total cost is not more than twice the cost of an optimal tour.


answer: (Omit!)
question:
 In the **_bottleneck traveling-salesman problem_**, we wish to find the hamiltonian cycle that minimizes the cost of the most costly edge in the cycle. Assuming that the cost function satisfies the triangle inequality, show that there exists a polynomial-time approximation algorithm with approximation ratio $3$ for this problem. ($\textit{Hint:}$ Show recursively that we can visit all the nodes in a bottleneck spanning tree, as discussed in Problem 23-3, exactly once by taking a full walk of the tree and skipping nodes, but without skipping more than two consecutive intermediate nodes. Show that the costliest edge in a bottleneck spanning tree has a cost that is at most the cost of the costliest edge in a bottleneck hamiltonian cycle.)


answer: (Omit!)
question:
 Suppose that the vertices for an instance of the traveling-salesman problem are points in the plane and that the cost $c(u, v)$ is the euclidean distance between points $u$ and $v$. Show that an optimal tour never crosses itself.


question:
 Using figure 7.1 as a model, illustrate the operation of $\text{PARTITION}$ on the array $A = \langle 13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11 \rangle$.

answer:
$$
\begin{aligned}
\langle 13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 19, 13, 5, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 13, 19, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 13, 19, 12, 8, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 8, 19, 12, 13, 7, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 8, 7, 12, 13, 19, 4, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 8, 7, 4, 13, 19, 12, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 8, 7, 4, 13, 19, 12, 21, 2, 6, 11 \rangle \\\\
\langle 9, 5, 8, 7, 4, 2, 19, 12, 21, 13, 6, 11 \rangle \\\\
\langle 9, 5, 8, 7, 4, 2, 6, 12, 21, 13, 19, 11 \rangle \\\\
\langle 9, 5, 8, 7, 4, 2, 6, 11, 21, 13, 19, 12 \rangle
\end{aligned}
$$


question:
 What value of $q$ does $\text{PARTITION}$ return when all elements in the array $A[p..r]$ have the same value? Modify $\text{PARTITION}$ so that $q = \lfloor (p + r) / 2 \rfloor$ when all elements in the array $A[p..r]$ have the same value.

answer:
It returns $r$.

We can modify $\text{PARTITION}$ by counting the number of comparisons in which $A[j] = A[r]$ and then subtracting half that number from the pivot index.


question:
 Give a brief argument that the running time of $\text{PARTITION}$ on a subarray of size $n$ is $\Theta(n)$.

answer:
There is a for statement whose body executes $r - 1 - p = \Theta(n)$ times. In the worst case every time the body of the if is executed, but it takes constant time and so does the code outside of the loop. Thus the running time is $\Theta(n)$.


question:
 Use the substitution method to prove that the recurrence $T(n) = T(n - 1) + \Theta(n)$ has the solution $T(n) = \Theta(n^2)$, as claimed at the beginning of section 7.2.

answer:
We represent $\Theta(n)$ as $c_2n$ and we guess that $T(n) \le c_1n^2$,

$$
\begin{aligned}
T(n) & =   T(n - 1) + c_2n \\\\
     & \le c_1(n - 1)^2 + c_2n \\\\
     & =   c_1n^2 - 2c_1n + c_1 + c_2n & (2c_1 > c_2, n \ge c_1 / (2c_1 - c_2)) \\\\
     & \le c_1n^2.
\end{aligned}
$$


answer: It is $\Theta(n^2)$, since one of the partitions is always empty (see exercise 7.1-2.)
question:
 What is the running time of $\text{QUICKSORT}$ when all elements of the array $A$ have the same value?


question:
 Show that the running time of $\text{QUICKSORT}$ is $\Theta(n^2)$ when the array $A$ contains distinct elements and is sorted in decreasing order.

answer:
If the array is already sorted in decreasing order, then, the pivot element is less than all the other elements. The partition step takes $\Theta(n)$ time, and then leaves you with a subproblem of size $n − 1$ and a subproblem of size $0$. This gives us the recurrence considered in 7.2-1. Which we showed has a solution that is $\Theta(n^2)$.


question:
 Banks often record transactions on an account in order of the times of the transactions, but many people like to receive their bank statements with checks listed in order by check numbers. People usually write checks in order by check number, and merchants usually cash the with reasonable dispatch. The problem of converting time-of-transaction ordering to check-number ordering is therefore the problem of sorting almost-sorted input. Argue that the procedure $\text{INSERTION-SORT}$ would tend to beat the procedure $\text{QUICKSORT}$ on this problem.

answer:
The more sorted the array is, the less work insertion sort will do. Namely, $\text{INSERTION-SORT}$ is $\Theta(n + d)$, where $d$ is the number of inversions in the array. In the example above the number of inversions tends to be small so insertion sort will be close to linear.

On the other hand, if $\text{PARTITION}$ does pick a pivot that does not participate in an inversion, it will produce an empty partition. Since there is a small number of inversions, $\text{QUICKSORT}$ is very likely to produce empty partitions.


question:
 Suppose that the splits at every level of quicksort are in proportion $1 - \alpha$ to $\alpha$, where $0 < \alpha \le 1 / 2$ is a constant. Show that the minimum depth of a leaf in the recursion tree is approximately $-\lg n / \lg\alpha$ and the maximum depth is approximately $-\lg n / \lg(1 - \alpha)$. (Don't worry about integer round-off.)

answer:
The minimum depth corresponds to repeatedly taking the smaller subproblem, that is, the branch whose size is proportional to $\alpha$. Then, this will fall to $1$ in $k$ steps where $1 \approx \alpha^kn$. Therefore, $k \approx \log_\alpha 1 / n = -\frac{\lg n}{\lg\alpha}$. The longest depth corresponds to always taking the larger subproblem. we then have an identical expression, replacing $\alpha$ with $1 − \alpha$.


question:
 Argue that for any constant $0 < \alpha \le 1 / 2$, the probability is approximately $1 - 2\alpha$ that on a random input array, $\text{PARTITION}$ produces a split more balanced than $1 - \alpha$ to $\alpha$.

answer:
In order to produce a worse split than $1 - \alpha$ to $\alpha$, $\text{PARTITION}$ must pick a pivot that will be either within the smallest $\alpha n$ elements or the largest $\alpha n$ elements. The probability of either is (approximately) $\alpha n / n = \alpha$ and the probability of both is $2\alpha$. Thus, the probability of having a better partition is the complement, $1 - 2\alpha$.


question:
 Why do we analyze the expected running time of a randomized algorithm and not its worst-case running time?

answer:
We analyze the expected run time because it represents the more typical time cost. Also, we are doing the expected run time over the possible randomness used during computation because it can't be produced adversarially, unlike when doing expected run time over all possible inputs to the algorithm.


question:
 When $\text{RANDOMIZED-QUICKSORT}$ runs, how many calls are made to the random number generator $\text{RANDOM}$ in the worst case? How about in the best case? Give your answer in terms of $\Theta$-notation.

answer:
In the worst case, the number of calls to $\text{RANDOM}$ is

$$T(n) = T(n - 1) + 1 = n = \Theta(n).$$

As for the best case,

$$T(n) = 2T(n / 2) + 1 = \Theta(n).$$

This is not too surprising, because each third element (at least) gets picked as pivot.


question:
 Show that in the recurrence
>
> $$
> \begin{aligned}
> T(n) & = \max\limits_{0 \le q \le n - 1} (T(q) + T(n - q - 1)) + \Theta(n), \\\\
> T(n) & = \Omega(n^2).
> \end{aligned}
> $$

answer:
We guess $T(n) \ge cn^2 - 2n$,

$$
\begin{aligned}
T(n) & =   \max_{0 \le q \le n - 1} (T(q) + T(n - q - 1)) + \Theta(n) \\\\
     & \ge \max_{0 \le q \le n - 1} (cq^2 - 2q + c(n - q - 1)^2 - 2n - 2q -1) + \Theta(n) \\\\
     & \ge c\max_{0 \le q \le n - 1} (q^2 + (n - q - 1)^2 - (2n + 4q + 1) / c) + \Theta(n) \\\\
     & \ge cn^2 - c(2n - 1) + \Theta(n) \\\\
     & \ge cn^2 - 2cn + 2c & (c \le 1) \\\\
     & \ge cn^2 - 2n.
\end{aligned}
$$


question: Show that quicksort's best-case running time is $\Omega(n\lg n)$.
answer:
We'll use the substitution method to show that the best-case running time is $\Omega(n\lg n)$. Let $T(n)$ be the best-case time for the procedure $\text{QUICKSORT}$ on an input of size $n$. We have

$$T(n) = \min _{1 \le q \le n - 1} (T(q) + T(n - q - 1)) + \Theta(n).$$

Suppose that $T(n) \ge c(n\lg n + 2n)$ for some constant $c$. Substituting this guess into the recurrence gives

$$
\begin{aligned} 
T(n) & \ge \min _{1 \le q \le n - 1}(cq\lg q + 2cq + c(n - q - 1) \lg(n - q - 1) + 2c(n - q - 1)) + \Theta(n) \\\\ 
     & =   (cn / 2)\lg(n / 2) + cn + c(n / 2 - 1)\lg(n / 2 - 1) + cn - 2c + \Theta(n) \\\\
     & \ge (cn / 2)\lg n - cn / 2 + c(n / 2 - 1)(\lg n - 2) + 2cn - 2c\Theta(n) \\\\
     & =   (cn / 2)\lg n - cn / 2 + (cn / 2) \lg n - cn - c\lg n + 2c + 2cn - 2c\Theta(n) \\\\
     & =   cn\lg n + cn / 2 - c\lg n + 2c - 2c\Theta(n).
\end{aligned}
$$

Taking a derivative with respect to $q$ shows that the minimum is obtained when $q = n / 2$. Taking $c$ large enough to dominate the $−\lg n + 2 − 2c + \Theta(n)$ term makes this greater than $cn\lg n$, proving the bound.


question:
 Show that the expression $q^2 + (n - q - 1)^2$ achieves a maximum over $q = 0, 1, \ldots, n - 1$ when $q = 0$ and $q = n - 1$.

answer:
$$
\begin{aligned}
  f(q) & = q^2 + (n - q - 1)^2 \\\\
 f'(q) & = 2q - 2(n - q - 1) = 4q - 2n + 2 \\\\
f''(q) & = 4. \\\\
\end{aligned}
$$

$f'(q) = 0$ when $q = \frac{1}{2}n - \frac{1}{2}$. $f'(q)$ is also continious. $\forall q: f''(q) > 0$, which means that $f'(q)$ is negative left of $f'(q) = 0$ and positive right of it, which means that this is a local minima. In this case, $f(q)$ is decreasing in the beginning of the interval and increasing in the end, which means that those two points are the only candidates for a maximum in the interval.

$$
\begin{aligned}
    f(0) & = (n - 1)^2 \\\\
f(n - 1) & = (n - 1)^2 + 0^2.
\end{aligned}
$$


question: Show that $\text{RANDOMIZED-QUICKSORT}$'s expected running time is $\Omega(n\lg n)$.
answer:
We use the same reasoning for the expected number of comparisons, we just take in a different direction.

$$
\begin{aligned}
\text E[X]
    & =   \sum_{i = 1}^{n - 1} \sum_{j = i + 1}^n \frac{2}{j - i + 1} \\\\
    & =   \sum_{i = 1}^{n - 1} \sum_{k = 1}^{n - i} \frac{2}{k + 1} & (k \ge 1) \\\\
    & \ge \sum_{i = 1}^{n - 1} \sum_{k = 1}^{n - i} \frac{2}{2k} \\\\
    & \ge \sum_{i = 1}^{n - 1} \Omega(\lg n) \\\\
    & =   \Omega(n\lg n).
\end{aligned}
$$

Using the master method, we get the solution $\Theta(n\lg n)$.


question:
 We can improve the running time of quicksort in practice by taking advantage of the fast running time of insertion sort when its input is "nearly" sorted. Upon calling quicksort on a subarray with fewer than $k$ elements, let it simply return without sorting the subarray. After the top-level call to quicksort returns, run insertion sort on the entire array to finish the sorting process. Argue that this sorting algorithm runs in $O(nk + n\lg(n / k))$ expected time. How should we pick $k$, both in theory and practice?

answer:
In the quicksort part of the proposed algorithm, the recursion stops at level $\lg(n / k)$, which makes the expected running time $O(n\lg(n / k))$. However, this leaves $n / k$ non-sorted, non - intersecting subarrays of (maximum) length $k$.

Because of the nature of the insertion sort algorithm, it will first sort fully one such subarray before consider the next one. Thus, it has the same complexity as sorting each of those arrays, that is $\frac{n}{k}O(k^2) = O(nk)$.

In theory, if we ignore the constant factors, we need to solve

$$
\begin{aligned}
            & n\lg n \ge nk + n\lg{n / k} \\\\
\Rightarrow & \lg n \ge k + \lg n - \lg k \\\\
\Rightarrow & \lg k \ge k.
\end{aligned}
$$

Which is not possible.

If we add the constant factors, we get

$$
\begin{aligned}
            & c_qn\lg n \ge c_ink + c_qn\lg(n / k) \\\\
\Rightarrow & c_q\lg n \ge c_ik + c_q\lg n - c_q\lg k \\\\
\Rightarrow & \lg k \ge \frac{c_i}{c_q}k.
\end{aligned}
$$

Which indicates that there might be a good candidate. Furthermore, the lower-order terms should be taken into consideration too.

In practice, $k$ should be chosed by experiment.


question:
 Consider modifying the $\text{PARTITION}$ procedure by randomly picking three elements from array $A$ and partitioning about their median (the middle value of the three elements). Approximate the probability of getting at worst an $\alpha$-to-$(1 - \alpha)$ split, as a function of $\alpha$ in the range $0 < \alpha < 1$.

answer:
First, for simplicity's sake, let's assume that we can pick the same element twice. Let's also assume that $0 < \alpha \le 1 / 2$.

In order to get such a split, two out of three elements need to be in the smallest $\alpha n$ elements. The probability of having one is $\alpha n / n = \alpha$. The probability of having exactly two is $\alpha^2 - \alpha^3$. There are three ways in which two elements can be in the smallest $\alpha n$ and one way in which all three can be in the smallest $\alpha n$ so the probability of getting such a median is $3\alpha^2 - 2\alpha^3$. We will get the same split if the median is in the largest $\alpha n$. Since the two events are mutually exclusive, the probability is

$$\Pr\\{\text{OK split}\\} = 6\alpha^2 - 4\alpha^3 = 2\alpha^2(3 - 2\alpha).$$


question:
 Explain why, in the proof of Lemma 16.2, if $x.freq = b.freq$, then we must have $a.freq = b.freq = x.freq = y.freq$.

answer:
If we have that $x.freq = b.freq$, then we know that $b$ is tied for lowest frequency. In particular, it means that there are at least two things with lowest frequency, so $y.freq = x.freq$. Also, since $x.freq \le a.freq \le b.freq = x.freq$, we must have $a.freq = x.freq$.


question: Prove that a binary tree that is not full cannot correspond to an optimal prefix code.
answer:
Let $T$ be a binary tree that is not full. $T$ represents a binary prefix code for a file composed of characters from alphabet $C$, where $c \in C$, $f\(c\)$ is th number of occurrences of $c$ in the file. The cost of tree $T$, or the number of bits in the encoding, is $\sum_{c \in C} d_T\(c\) \cdot f\(c\)$, where $d_T\(c\)$ is the depth of character $c$ in tree $T$.

Let $N$ be a node of greatest depth that has exactly one child. If $N$ is the root of $T$, $N$ can be removed and the deepth of each node reduced by one, yielding a tree representing the same alphabet with a lower cost. This mean the original code was not optimal.

Otherwise, let $M$ be the parent of $N$, let $T_1$ be the (possibly non-existent) sibling of $N$, and let $T_2$ be the subtree rooted at the child of $N$. Replace $M$ by $N$, making $T_1$ be the children of $N$. If $T_1$ is empty, repeat the process. We have a new prefix code of lower cost, so the original was not optimal.


question:
 What is an optimal Huffman code for the following set of frequencies, based on  
> the first $8$ Fibonacci numbers?
>
> $$a:1 \quad b:1 \quad c:2 \quad d:3 \quad e:5 \quad f:8 \quad g:13 \quad h:21$$
>
> Can you generalize your answer to find the optimal code when the frequencies are the first $n$ Fibonacci numbers?

answer:
$$
\begin{array}{c|l}
a & 1111111 \\\\
b & 1111110 \\\\
c & 111110 \\\\
d & 11110 \\\\
e & 1110 \\\\
f & 110 \\\\
g & 10 \\\\
h & 0
\end{array}
$$

**GENERALIZATION**

In what follows we use $a_i$ to denote $i$-th Fibonacci number. To avoid any confusiion we stress that we consider Fibonacci's sequence beginning $1$, $1$, i.e. $a_1 = a_2 = 1$.

Let us consider a set of $n$ symbols $\Sigma = \\{c_i ~|~ 1 \le i \le n \\}$ such that for each $i$ we have $c_i.freq = a_i$. We shall prove that the Huffman code for this set of symbols given by the run of algorithm HUFFMAN from CLRS is the following code:

- $code(c_n) = 0$
- $code(c_{i - 1}) = 1code(c_i)$ for $2 \le i \le n - 1$ (i.e. we take a code for symbol $c_i$ and add $1$ to the beginning)
- $code(c_1) = 1^{n - 1}$

By $code(c)$ we mean the codeword assigned to the symbol $c_i$ by the run of HUFFMAN($\Sigma$) for any $c \in \Sigma$.

First we state two technical claims which can be easily proven using the proper induction. Following good manners of our field we leave the proofs to the reader :-)

- (HELPFUL CLAIM 1) $ (\forall k \in \mathbb{N}) ~ \sum\limits_{i = 1}^{k} a_i = a_{k + 2} - 1$
- (HELPFUL CLAIM 2) Let $z$ be an inner node of tree $T$ constructed by the algorithm HUFFMAN. Then $z.freq$ is sum of frequencies of all leafs of the subtree of $T$ rooted in $z$.

Consider tree $T_n$ inductively defined by

- $T_2.left = c_2$, $T_2.right = c_1$ and $T_2.freq = c_1.freq + c_2.freq = 2$
- $(\forall i; 3 \le i \le n) ~ T_i.left = c_i$, $T_i.right = T_{i - 1}$ and $T_i.freq = c_i.freq + T_{i - 1}.freq$

We shall prove that $T_n$ is the tree produced by the run of HUFFMAN($\Sigma$).

**KEY CLAIM:** $T_{i + 1}$ is exactly the node $z$ constructed in $i$-th run of the for-cycle of HUFFMAN($\Sigma$) and the content of the priority queue $Q$ just after $i$-th run of the for-cycle is exactly $Q = (a_{i + 2}, T_{i + 1}, a_{i + 3}, \dots, a_n)$ with $a_{i + 2}$ being the minimal element for each $1 \le i < n$. (Since we prefer not to overload our formal notation we just note that for $i = n - 1$ we claim that $Q = (T_n)$ and our notation grasp this fact in a sense.)

**PROOF OF KEY CLAIM** by induction on $i$.

- for $i = 1$ we see that the characters with lowest frequencies are exactly $c_1$ and $c_2$, thus obviously the algorithm HUFFMAN($\Sigma$) constructs $T_2$ in the first run of its for-cycle. Also it is obvious that just after this run of the for-cycle we have $Q = (a_3, T_{2}, a_4, \dots, a_n)$.
- for $2 \le i < n$ we suppose that our claim is true for all $j < i$ and prove the claim for $i$. Since the claim is true for $i - 1$, we know that just before $i-th$ execution of the for-cycle we have the following content of the priority queue $Q=(a_{i + 1}, T_i, a_{i + 2}, \dots, a_n)$. Thus line 5 of HUFFMAN extracts $a_{i + 1}$ and sets $z.left = a_{i + 1}$ and line 6 of HUFFMAN extracts $T_i$ and sets $z.right = T_i$. Now we can see that indeed $z$ is exactly $T_{i + 1}$. Using (CLAIM 2) and observing the way $T_{i + 1}$ is defined we get that $z.freq = T_{i + 1}.freq = \sum\limits_{i=1}^{i + 1} a_i$. Thus using (CLAIM 1) one can see that $a_{i + 2} < T_{i + 1}.freq < a_{i + 3}$. Therefore for the content of the priority queue $Q$ just after the $i$-th execution of the for-cycle we have $Q=(a_{i + 2}, T_{i + 2}, a_{i + 3}, \dots, a_n)$.

**KEY CLAIM** tells us that just after the last execution of the for-cycle we have $Q = (T_n)$ and therefore the line 9 of HUFFMAN returns $T_n$ as the result. One can easily see that the code given in the beginning is exactly the code which corresponds to the code-tree $T_n$.


question:
 Prove that we can also express the total cost of a tree for a code as the sum, over all internal nodes, of the combined frequencies of the two children of the node.

answer:
Let tree be a full binary tree with $n$ leaves. Apply induction hypothesis on the number of leaves in $T$. When $n = 2$ (the case $n = 1$ is trivially true), there are two leaves $x$ and $y$ with the same parent $z$, then the cost of $T$ is

$$
\begin{aligned}
B(T) & = f(x) d_T(x) + f(y) d_T(y) \\\\
     & = f(x) + f(y) & \text{since $d_T(x) = d_T(y) = 1$}  \\\\
     & = f(\text{child}_1\text{ of }z) + f(\text{child}_2\text{ of }z).
\end{aligned}
$$

Thus, the statement of theorem is true. Now suppose $n > 2$ and also suppose that theorem is true for trees on $n - 1$ leaves.
Let $c_1$ and $c_2$ are two sibling leaves in $T$ such that they have the same parent $p$. Letting $T'$ be the tree obtained by deleting $c_1$ and $c_2$, by induction we know that

$$
\begin{aligned}
B(T) & = \sum_{\text{leaves } l'\in T'} f(l')d_T(l') \\\\
     & = \sum_{\text{internal nodes } i'\in T'} f(\text{child}_1\text{ of }i') + f(\text{child}_2\text{ of }i').
\end{aligned}
$$

Using this information, calculates the cost of $T$.

$$
\begin{aligned}
B(T) & = \sum_{\text{leaves }l \in T} f(l)d_T(l) \\\\
     & = \sum_{l \ne c_1, c_2} f(l)d_T(l) + f(c_1)d_T(c_1) - 1 + f(c_2)d_T(c_2) - 1 + f(c_1) + f(c_2) \\\\
     & = \sum_{\text{internal nodes }i'\in T'} f(\text{child}_1\text{ of }i') + f(\text{child}_2\text{ of }i') + f(c_1) + f(c_2) \\\\
     & = \sum\_{\text{internal nodes }i\in T} f(\text{child}_1\text{ of }i) + f(\text{child}_1\text{ of }i).
\end{aligned}
$$

Thus the statement is true.


question:
 Prove that if we order the characters in an alphabet so that their frequencies are monotonically decreasing, then there exists an optimal code whose codeword lengths are monotonically increasing.

answer:
**Little formal-mathematical note here:** We are required to prove existence of an optimal code with some property. Therefore we are required also to show, that some optimal code exists. It is trivial in this case, since we know that the code produced by a run of Huffman's algorithm produce one such code for us. However, it is good to be aware of this. Proving just the implication "if a code is optimal then it has the desired property" doesn't suffice.

OK, now we are ready to prove the already mentioned implication "if a code is optimal then it has the desired property". Main idea of our proof is that if the code violates desired property, then we find two symbols which violate the property and 'fix the code'. For the formal proof we go as follows.

Suppose that we have an alphabet $C = \{a_1, \ldots, a_n\}$ where the characters are written in monotonically decreasing order, i.e. $a_1.freq \ge a_2.freq \ge \ldots \ge a_n$. Let us consider an optimal code $B$ for $C$. Let us denote the codeword for the character $c \in C$ in the code $B$ by $cw_B(c)$. W.l.o.g. we can assume that for any $i$ such that $a_i.freq = a_{i + 1}.freq$ it holds that $|cw(a_i)| \le |cw(a_{i + 1})|$. This assumption can be made since for any $a_i.freq = a_{i + 1}.freq$ for which $|cw(a_i)| > |cw(a_{i + 1})|$ we can simply swap codewords for $a_i$ and $a_{i + 1}$ and obtain a code with desired property and the same cost as is the cost of $B$. We prove that $B$ has the desired property, i.e., its codeword lengths are monotonically increasing.

We proceed by contradiction. If lengths of the codewords are not monotonically increasing, then there exist an index $i$ such that $|cw_B(a_i)| > |cw_B(a_{i + 1})| $. Using our assumptions on $C$ and $B$ we get that $a_i.freq > a_{i + 1}.freq$. Define new code $B'$ for $C$ such that for $a_j$ such that $j \ne i$ and $j \ne i + 1$ we keep $cw_{B'}(a_j) = cw_B(a_j)$ and we swap codewords for $a_i$ and $a_{j + 1}$, i.e. we set $cw_{B'}(a_i) = cw_{B}(a_{i + 1})$ and $cw_{B'}(a_{i + 1}) = cw_{B}(a_{i})$. Now compare costs of the codes $B$ and $B'$. It holds that

$$
\begin{aligned}
cost(B')
     &= cost(B) - (|cw_B(a_i)|(a_i.freq) + |cw_B(a_{i + 1})|(a_{i + 1}.freq)) \\\\
     &+ (|cw_B(a_i)|(a_{i + 1}.freq) + |cw_B(a_{i + 1})|(a_{i}.freq)) \\\\
     &= cost(B) + |cw_B(a_i)|(a_{i + 1}.freq - a_i.freq) + |cw_B(a_{i + 1})|(a_i.freq - a_{i + 1}.freq)
\end{aligned}
$$

For better readability now denote $a_i.freq - a_{i + 1}.freq = \phi$. Since $a_i.freq > a_{i + 1}.freq$, we get $\phi > 0$ and we can write

$$ cost(B') = cost(B) - \phi|cw_B(a_i)| + \phi|cw_B(a_{i + 1})| = cost(B) - \phi(|cw_B(a_i)| - |cw_B(a_{i + 1})|) $$

Since $|cw_B(a_i)| > |cw_B(a_{i + 1})| $, we get $|cw_B(a_i)| - |cw_B(a_{i + 1})| > 0$. Thus $\phi(|cw_B(a_i)| - |cw_B(a_{i + 1})|) > 0$ which imply $cost(B') < cost(B)$. Therefore the code $B$ is not optimal, a contradiction.

Therefore, we conclude that codeword lengths of $B$ are monotonically increasing and the proof is complete.

**Note:** For those not familiar with mathematical parlance, w.l.o.g means without loss of generality.


question:
 Suppose we have an optimal prefix code on a set $C = \\{0, 1, \ldots, n - 1 \\}$ of characters and we wish to transmit this code using as few bits as possible. Show how to represent any optimal prefix code on $C$ using only $2n - 1 + n \lceil \lg n \rceil$ bits. ($\textit{Hint:}$ Use $2n - 1$ bits to specify the structure of the tree, as discovered by a walk of the tree.)

answer:
First observe that any full binary tree has exactly $2n - 1$ nodes. We can encode the structure of our full binary tree by performing a preorder traversal of $T$.
For each node that we record in the traversal, write a $0$ if it is an internal node and a $1$ if it is a leaf node. Since we know the tree to be full, this uniquely determines its structure.

Next, note that we can encode any character of $C$ in $\lceil \lg n \rceil$ bits. Since there are $n$ characters, we can encode them in order of appearance in our preorder traversal using $n\left\lceil \lg n \right\rceil$ bits.


question:
 Generalize Huffman's algorithm to ternary codewords (i.e., codewords using the symbols $0$, $1$, and $2$), and prove that it yields optimal ternary codes.

answer:
Instead of grouping together the two with lowest frequency into pairs that have the smallest total frequency, we will group together the three with lowest frequency in order to have a final result that is a ternary tree. The analysis of optimality is almost identical to the binary case. We are placing the symbols of lowest frequency lower down in the final tree and so they will have longer codewords than the more frequently occurring symbols.


question:
 Suppose that a data file contains a sequence of $8$-bit characters such that all $256$ characters are about equally common: the maximum character frequency is less than twice the minimum character frequency. Prove that Huffman coding in this case is no more efficient than using an ordinary $8$-bit fixed-length code.

answer:
For any $2$ characters, the sum of their frequencies exceeds the frequency of any other character, so initially Huffman coding makes $128$ small trees with $2$ leaves each. At the next stage, no internal node has a label which is more than twice that of any other, so we are in the same setup as before. Continuing in this fashion, Huffman coding builds a complete binary tree of height $\lg 256 = 8$, which is no more efficient than ordinary $8$-bit length codes.


question:
 Show that no compression scheme can expect to compress a file of randomly chosen $8$-bit characters by even a single bit. ($\textit{Hint:}$ Compare the number of possible files with the number of possible encoded files.)

answer:
If every possible character is equally likely, then, when constructing the Huffman code, we will end up with a complete binary tree of depth $7$. This means that every character, regardless of what it is will be represented using $7$ bits.

This is exactly as many bits as was originally used to represent those characters,
so the total length of the file will not decrease at all.


question:
 Show that $(S, \mathcal I_k)$ is a matroid, where $S$ is any finite set and $\mathcal I_k$ is the set of all subsets of $S$ of size at most $k$, where $k \le |S|$.

answer:
The first condition that $S$ is a finite set is a given. To prove the second condition we assume that $k \ge 0$, this gets us that $\mathcal I_k$ is nonempty. Also, to prove the hereditary property, suppose $A \in \mathcal I_k$ this means that $|A| \le k$.
Then, if $B \subseteq A$, this means that $|B| \le |A| \le k$, so $B \in \mathcal I_k$. Lastly, we prove the exchange property by letting $A, B \in \mathcal I_k$ be such that $|A| < |B|$. Then, we can pick any element $x \in B \backslash A$, then,

$$|A \cup {x}| = |A| + 1 \le |B| \le k,$$

so, we can extend $A$ to $A \cup \\{x\\} \in \mathcal I_k$.


question:
 Given an $m \times n$ matrix $T$ over some field (such as the reals), show that $(S, \mathcal I)$ is a matroid, where $S$ is the set of columns of $T$ and $A \in \mathcal I$ if and only if the columns in $A$ are linearly independent.

answer:
Let $c_1, \dots, c_m$ be the columns of $T$. Suppose $C = \\{c_{i1}, \dots, c_{ik}\\}$ is dependent. Then there exist scalars $d_1, \dots, d_k$ not all zero such that $\sum_{j = 1}^k d_jc_{ij} = 0$. By adding columns to $C$ and assigning them to have coefficient $0$ in the sum, we see that any superset of $C$ is also dependent. By contrapositive, any subset of an independent set must be independent.

Now suppose that $A$ and $B$ are two independent sets of columns with $|A| > |B|$. If we couldn't add any column of $A$ to be whilst preserving independence then it must be the case that every element of $A$ is a linear combination of elements of $B$. But this implies that $B$ spans a $|A|$-dimensional space, which is impossible. Therefore, our independence system must satisfy the exchange property, so it is in fact a matroid.


question:
 Show that if $(S, \mathcal I)$ is a matroid, then $(S, \mathcal I')$ is a matroid, where
>
> $\mathcal I' = \\{A': S - A'$ contains some maximal $A \in \mathcal I\\}$.
>
> That is, the maximal independent sets of $(S, \mathcal I')$ are just the complements of the maximal independent sets of $(S, \mathcal I)$.

answer:
Condition one of being a matroid is still satisfied because the base set hasn't changed. Next we show that $\mathcal I'$ is nonempty. Let $A$ be any maximal element of $\mathcal I$, then we have that $S - A \in \mathcal I'$ because $S - (S - A) = A \subseteq A$ which is maximal in $\mathcal I$.

Next we show the hereditary property, suppose that $B \subseteq A \in \mathcal I'$, then, there exists some $A' \in \mathcal I$ so that $S − A \subseteq A'$, however, $S − B \supseteq S − A \subseteq A$ so $B \in \mathcal I'$.

Last, we prove the exchange property. That is, if we have $B, A \in \mathcal I'$ and $|B| < |A|$, we can find an element $x$ in $A − B$ to add to $B$ so that it stays independent. We will split into two cases:

- The first case is that $|A - B| = 1$. Let $x \in A-B$ be the only element in $A - B$. Since $|A| > |B|$ and $|A - B| = 1$, it follows in this case $B \subset A$. We extend $B$ by $x$ and we have $B \cup \\{x\\} = A \in \mathcal I'$.
- The second case is if the first case does not hold. Let $C$ be a maximal independent set of $\mathcal I$ contained in $S − A$. Pick an aribitrary set of size $|C| − 1$ from some maximal independent set contained in $S - B$, call it $$. Since $D$ is a subset of a maximal independent set, it is also independent, and so, by the exchange property, there is some $y \in C − D$ so that $D \cup \\{y\\}$ is a maximal independent set in $\mathcal I$. Then, we select $x$ to be any element other than $y$ in $A − B$. Then, $S − (B \cup \\{x\\})$ will still contain $D \cup \\{y\\}$. This means that $B \cup \\{x\\}$ is independent in $\mathcal I'$.


question:
 Let $S$ be a finite set and let $S_1, S_2, \ldots, S_k$ be a partition of $S$ into nonempty disjoint subsets. Define the structure $(S, \mathcal I)$ by the condition that $\mathcal I = \\{A: \mid A \cap S_i \mid \le 1$ for $i = 1, 2, \ldots, k\\}$. Show that $(S, \mathcal I)$ is a matroid. That is, the set of all sets $A$ that contain at most one member of each subset in the partition determines the independent sets of a matroid.

answer:
Suppose $X \subset Y$ and $Y \in \mathcal I$. Then $(X \cap S_i) \subset (Y \cap S_i)$ for all $i$, so

$$|X \cap S_i| \le |Y \cap S_i| \le 1$$

for all $1 \le i \le k$. Therefore $\mathcal M$ is closed under inclusion.

Now Let $A, B \in \mathcal I$ with $|A| > |B|$. Then there must exist some $j$ such that $|A \cap S_j| = 1$ but $|B \cap S_j| = 0$. Let $a \in A \cap S_j$. Then $a \notin B$ and $|(B \cup \\{a\\}) \cap S_j| = 1$. Since

$$|(B \cup \\{a\\}) \cap S_i| = |B \cap S_i| \le 1$$

for all $i \ne j$, we must have $B \cup \\{a\\} \in \mathcal I$. Therefore $\mathcal M$ is a matroid.


question:
 Show how to transform the weight function of a weighted matroid problem, where the desired optimal solution is a _minimum-weight_ maximal independent subset, to make it a standard weighted-matroid problem. Argue carefully that your transformation is correct.

answer:
Suppose that $W$ is the largest weight that any one element takes. Then, define the new weight function $w_2(x) = 1 + W - w(x)$. This then assigns a strictly positive weight, and we will show that any independent set that that has maximum weight with respect to $w_2$ will have minimum weight with respect to $w$.

Recall Theorem 16.6 since we will be using it, suppose that for our matriod, all maximal independent sets have size $S$. Then, suppose $M_1$ and $M_2$ are maximal independent sets so that $M_1$ is maximal with respect to $w_2$ and $M_2$ is minimal with respect to $w$. Then, we need to show that $w(M_1) = w(M_2)$. Suppose not to achieve a contradiction, then, by minimality of $M_2$, $w(M_1) > w(M_2)$.

Rewriting both sides in terms of $w_2$, we have

$$w_2(M_2) - (1 + W)S > w_2(M_1) - (1 + W)S,$$

so,

$$w_2(M_2) > w_2(M_1).$$

This however contradicts maximality of $M_1$ with respect to $w_2$. So, we must have that $w(M_1) = w(M_2)$. So, a maximal independent set that has the largest weight with respect to $w_2$ also has the smallest weight with respect to $w$.


question:
 Give a dynamic-programming algorithm for the activity-selection problem, based on recurrence $\text{(16.2)}$. Have your algorithm compute the sizes $c[i, j]$ as defined above and also produce the maximum-size subset of mutually compatible activities.
>
> Assume that the inputs have been sorted as in equation $\text{(16.1)}$. Compare the running time of your solution to the running time of $\text{GREEDY-ACTIVITY-SELECTOR}$.

answer:
```cpp
DYNAMIC-ACTIVITY-SELECTOR(s, f, n)
    let c[0..n + 1, 0..n + 1] and act[0..n + 1, 0..n + 1] be new tables
    for i = 0 to n
        c[i, i] = 0
        c[i, i + 1] = 0
    c[n + 1, n + 1] = 0
    for l = 2 to n + 1
        for i = 0 to n - l + 1
            j = i + l
            c[i, j] = 0
            k = j - 1
            while f[i] < f[k]
                if f[i] ≤ s[k] and f[k] ≤ s[j] and c[i, k] + c[k, j] + 1 > c[i, j]
                    c[i, j] = c[i, k] + c[k, j] + 1
                    act[i, j] = k
                k = k - 1
    print "A maximum size set of mutually compatible activities has size" c[0, n + 1]
    print "The set contains"
    PRINT-ACTIVITIES(c, act, 0, n + 1)
```

```cpp
PRINT-ACTIVITIES(c, act, i, j)
    if c[i, j] > 0
        k = act[i, j]
        print k
        PRINT-ACTIVITIES(c, act, i, k)
        PRINT-ACTIVITIES(c, act, k, j)
```

- $\text{GREEDY-ACTIVITY-SELECTOR}$ runs in $\Theta(n)$ time and
- $\text{DYNAMIC-ACTIVITY-SELECTOR}$ runs in $O(n^3)$ time.


question:
 Suppose that instead of always selecting the first activity to finish, we instead select the last activity to start that is compatible with all previously selected activities. Describe how this approach is a greedy algorithm, and prove that it yields an optimal solution.

answer:
This becomes exactly the same as the original problem if we imagine time running in reverse, so it produces an optimal solution for essentially the same reasons. It is greedy because we make the best looking choice at each step.


question:
 Not just any greedy approach to the activity-selection problem produces a maximum-size set of mutually compatible activities. Give an example to show that the approach of selecting the activity of least duration from among those that are compatible with previously selected activities does not work. Do the same for the approaches of always selecting the compatible activity that overlaps the fewest other remaining activities and always selecting the compatible remaining activity with the earliest start time.

answer:
As a counterexample to the optimality of greedily selecting the shortest, suppose our activity times are $\\{(1, 9), (8, 11), (10, 20)\\}$ then, picking the shortest first, we have to eliminate the other two, where if we picked the other two instead, we would have two tasks not one.

As a counterexample to the optimality of greedily selecting the task that conflicts with the fewest remaining activities, suppose the activity times are $\\{(−1, 1), (2, 5), (0, 3), (0, 3), (0, 3), (4, 7), (6, 9), (8, 11), (8, 11), (8, 11), (10, 12)\\}$. Then, by this greedy strategy, we would first pick $(4, 7)$ since it only has a two conflicts. However, doing so would mean that we would not be able to pick the only optimal solution of $(−1, 1)$, $(2, 5)$, $(6, 9)$, $(10, 12)$.

As a counterexample to the optimality of greedily selecting the earliest start times, suppose our activity times are $\\{(1, 10), (2, 3), (4, 5)\\}$. If we pick the earliest start time, we will only have a single activity, $(1, 10)$, whereas the optimal solution would be to pick the two other activities.


question:
 Suppose that we have a set of activities to schedule among a large number of lecture halls, where any activity can take place in any lecture hall. We wish to schedule all the activities using as few lecture halls as possible. Give an efficient greedy algorithm to determine which activity should use which lecture hall.
>
> (This problem is also known as the **_interval-graph coloring problem_**. We can create an interval graph whose vertices are the given activities and whose edges connect incompatible activities. The smallest number of colors required to color every vertex so that no two adjacent vertices have the same color corresponds to finding the fewest lecture halls needed to schedule all of the given activities.)

answer:
Maintain a set of free (but already used) lecture halls $F$ and currently busy lecture halls $B$. Sort the classes by start time. For each new start time which you encounter, remove a lecture hall from $F$, schedule the class in that room, and add the lecture hall to $B$. If $F$ is empty, add a new, unused lecture hall to $F$. When a class finishes, remove its lecture hall from $B$ and add it to $F$. This is optimal for following reason, suppose we have just started using the mth lecture hall for the first time. This only happens when ever classroom ever used before is in $B$. But this means that there are $m$ classes occurring simultaneously, so it is necessary to have $m$ distinct lecture halls in use.


question:
 Consider a modification to the activity-selection problem in which each activity $a_i$ has, in addition to a start and finish time, a value $v_i$. The objective is no longer to maximize the number of activities scheduled, but instead to maximize the total value of the activities scheduled. That is, we wish to choose a set $A$ of compatible activities such that $\sum_{a_k \in A} v_k$ is maximized. Give a polynomial-time algorithm for this problem.

answer:
Easy and straightforward solution is to run a dynamic programming solution based on the equation $\text{(16.2)}$ where the second case has "1" replaced with "$v_k$". Since the subproblems are still indexed by a pair of activities, and each calculation requires taking the minimum over some set of size $\le |S_{ij}| \in O(n)$. The total runtime is bounded by $O(n^3)$.

However, if we are cunning a little, we can be more efficient and give the algorithm which runs in $O(n\log n)$.

**INPUT:** $n$ activities with values.

**IDEA OF ALGORITHM:**

1. Sort input vector of activities according their finish times in ascending order. Let us denote the activities in this sorted vector by $(a_0, a_1, \dots, a_{n - 1})$.
2. For each $0 \le i < n$ construct partial solution $S_i$. By a partial solution $S_i$, we mean a solution to the problem but considering only activities with indexes lower or equal to $i$. Remember value of each partial solution.
3. Clearly $S_0 = \\{a_0\\}$.
4. We can construct $S_{i + 1}$ as follows. Possible values of $S_{i + 1}$ is either $S_i$ or the solution obtained by joining the activity $a_{i + 1}$ with partial solution $S_j$ where $j < i + 1$ is the index of activity such that $a_j$ is compatible with $a_{i + 1}$ but $a_{j + 1}$ is not compatible with $a_{i + 1}$. Pick the one of these two possible solutions, which has greater value. Ties can be resolved arbitrarily.
5. Therefore we can construct partial solutions in order $S_0, S_1, \dots, S_{n - 1}$ using (3) for $S_0$ and (4) for all the others.
6. Give $S_{n - 1}$ as the solution for problem.

**ANALYSIS OF TIME COMPLEXITY:**

- Sorting of activities can be done in $O(n\log n)$ time.
- Finding the value of $S_0$ is in $O(1)$.
- Any $S_{i + 1}$ can be found in $O(\log n)$. It is thanks to the fact that we have properly sorted activities. Therefore we can for each $i + 1$ find the proper $j$ in $O(\log n)$ using the binary search. If we have the proper $j$, the rest can be done in $O(1)$.
- Therefore, we have $O(n\log n)$ time for constructing of all $S_i$'s.

**IMPLEMENTATION DETAILS:**

- Use the dynamic programming.
- It is important not to remember too much for each $S_i$. Do not construct $S_i$'s directly (you can end up in $\Omega(n^2)$ time if you do so). For each $S_i$ it is sufficient to remember:
    - it's value
    - whether or not it includes the activity $a_i$
    - the value of $j$ (from (4)).
- Using these information obtained by the run of described algorithm you can reconstruct the solution in $O(n)$ time, which does not violate final time complexity.

**PROOF OF CORRECTNESS:** (sketched)

- Clearly, $S_0 = \\{a_0\\}.$
- For $S_{i + 1}$ we argue by (4). Partial solution $S_{i + 1}$ either includes the activity $a_{i + 1}$ or doesn't include it, there is no third way.
    - If it does not include $a_{i + 1}$, then clearly $S_{i + 1} = S_i$.
    - If it includes $a_{i + 1}$, then $S_{i + 1}$ consists of $a_{i + 1}$ and partial solution which uses all activities compatible with $a_{i + 1}$ with indexes lower than $i + 1$. Since activities are sorted according their finish times, activities with indexes $j$ and lower are compatible and activities with index $j + 1$ and higher up to $i + 1$ are not compatible. We do not consider all the other activities for $S_{i + 1}$. Therefore setting $S_{i + 1} = \\{a_{i + 1}\\} \cup S_j$ gives correct answer in this case. The fact that we need $S_j$ and not some other solution for activities with indexes up to $j$ can be easily shown by the standard cut-and-paste argument.
- Since for $S_{n - 1}$ we consider all of the activities, it is actually the solution of the problem.


question:
 Solve the instance of the scheduling problem given in Figure 16.7, but with each penalty $w_i$ replaced by $80 - w_i$.

answer:
$$
\begin{array}{c|ccccccc}
a_i &  1 &  2 &  3 &  4 &  5 &  6 &  7 \\\\
\hline
d_i &  4 &  2 &  4 &  3 &  1 &  4 &  6 \\\\
w_i & 10 & 20 & 30 & 40 & 50 & 60 & 70
\end{array}
$$

We begin by just greedily constructing the matroid, adding the most costly to leave incomplete tasks first. So, we add tasks $7, 6, 5, 4, 3$. Then, in order to schedule tasks $1$ or $2$ we need to leave incomplete more important tasks. So, our final schedule is $\langle 5, 3, 4, 6, 7, 1, 2 \rangle$ to have a total penalty of only $w_1 + w_2 = 30$.


question:
 Show how to use property 2 of Lemma 16.12 to determine in time $O(|A|)$ whether or not a given set $A$ of tasks is independent.

answer:
We provide a pseudocode which grasps main ideas of an algorithm.

```cpp
IS-INDEPENDENT(A)
    n = A.length
    let Nts[0..n] be an array filled with 0s
    for each a in A
        if a.deadline >= n
            Nts[n] = Nts[n] + 1
        else
            Nts[d] = Nts[d] + 1
    for i = 1 to n
        Nts[i] = Nts[i] + Nts[i - 1]
    // at this moment, Nts[i] holds value of N_i(A)
    for i = 1 to n
        if Nts[i] > i
            return false
    return true
```


question: Prove that the fractional knapsack problem has the greedy-choice property.
answer:
Let $I$ be the following instance of the knapsack problem: Let $n$ be the number of items, let $v_i$ be the value of the $i$th item, let $w_i$ be the weight of the $i$th item and let $W$ be the capacity. Assume the items have been ordered in increasing order by $v_i / w_i$ and that $W \ge w_n$.

Let $s = (s_1, s_2, \ldots, s_n)$ be a solution. The greedy algorithm works by assigning $s_n = \min(w_n, W)$, and then continuing by solving the subproblem

$$I' = (n - 1, \\{v_1, v_2, \ldots, v_{n - 1}\\}, \\{w_1, w_2, \ldots, w_{n - 1}\\}, W - w_n)$$

until it either reaches the state $W = 0$ or $n = 0$.

We need to show that this strategy always gives an optimal solution. We prove this by contradiction. Suppose the optimal solution to $I$ is $s_1, s_2, \ldots, s_n$, where $s_n < \min(w_n, W)$. Let $i$ be the smallest number such that $s_i > 0$. By decreasing $s_i$ to $\max(0, W - w_n)$ and increasing $s_n$ by the same amount, we get a better solution. Since this a contradiction the assumption must be false. Hence the problem has the greedy-choice property.


question:
 Give a dynamic-programming solution to the $0$-$1$ knapsack problem that runs in $O(nW)$ time, where $n$ is the number of items and $W$ is the maximum weight of items that the thief can put in his knapsack.

answer:
Suppose we know that a particular item of weight $w$ is in the solution. Then we must solve the subproblem on $n − 1$ items with maximum weight $W − w$. Thus, to take a bottom-up approach we must solve the $0$-$1$ knapsack problem for all items and possible weights smaller than W. We'll build an $n + 1$ by $W + 1$ table of values where the rows are indexed by item and the columns are indexed by total weight. (The first row and column of the table will be a dummy row).

For row $i$ column $j$, we decide whether or not it would be advantageous to include item i in the knapsack by comparing the total value of of a knapsack including items $1$ through $i − 1$ with max weight $j$, and the total value of including items $1$ through $i − 1$ with max weight $j − i.weight$ and also item $i$. To solve the problem, we simply examine the $n$, $W$ entry of the table to determine the maximum value we can achieve. To read off the items we include, start with entry $n$, $W$. In general, proceed as follows: if entry $i$, $j$ equals entry $i - 1$, $j$, don't include item $i$, and examine entry $i - 1$, $j$ next. If entry $i$, $j$ doesn't equal entry $i − 1$, $j$, include item $i$ and examine entry $i − 1$, $j − i$.weight next. See algorithm below for construction of table:

```cpp
0-1-KNAPSACK(n, W)
    Initialize an (n + 1) by (W + 1) table K
    for i = 1 to n
        K[i, 0] = 0
    for j = 1 to W
        K[0, j] = 0
    for i = 1 to n
        for j = 1 to W
            if j < i.weight
                K[i, j] = K[i - 1, j]
            else
                K[i, j] = max(K[i - 1, j], K[i - 1, j - i.weight] + i.value)
```


question:
 Suppose that in a $0$-$1$ knapsack problem, the order of the items when sorted by increasing weight is the same as their order when sorted by decreasing value. Give an efficient algorithm to find an optimal solution to this variant of the knapsack problem, and argue that your algorithm is correct.

answer:
Suppose in an optimal solution we take an item with $v_1$, $w_1$, and drop an item with $v_2$, $w_2$, and $w_1 > w_2$, $v_1 < v_2$, we can substitute $1$ with $2$ and get a better solution. Therefore we should always choose the items with the greatest values.


question:
 Professor Gekko has always dreamed of inline skating across North Dakota. He plans to cross the state on highway U.S. 2, which runs from Grand Forks, on the eastern border with Minnesota, to Williston, near the western border with Montana. The professor can carry two liters of water, and he can skate $m$ miles before running out of water. (Because North Dakota is relatively flat, the professor does not have to worry about drinking water at a greater rate on uphill sections than on flat or downhill sections.) The professor will start in Grand Forks with two full liters of water. His official North Dakota state map shows all the places along U.S. 2 at which he can refill his water and the distances between these locations.
>
> The professor's goal is to minimize the number of water stops along his route across the state. Give an efficient method by which he can determine which water stops he should make. Prove that your strategy yields an optimal solution, and give its running time.

answer:
The greedy solution solves this problem optimally, where we maximize distance we can cover from a particular point such that there still exists a place to get water before we run out. The first stop is at the furthest point from the starting position which is less than or equal to $m$ miles away. The problem exhibits optimal substructure, since once we have chosen a first stopping point $p$, we solve the subproblem assuming we are starting at $p$. Combining these two plans yields an optimal solution for the usual cut-and-paste reasons. Now we must show that this greedy approach in fact yields a first stopping point which is contained in some optimal solution. Let $O$ be any optimal solution which has the professor stop at positions $o_1, o_2, \dots, o_k$. Let $g_1$ denote the furthest stopping point we can reach from the starting point. Then we may replace $o_1$ by $g_2$ to create a modified solution $G$, since $o_2 - o_1 < o_2 - g_1$. In other words, we can actually make it to the positions in $G$ without running out of water. Since $G$ has the same number of stops, we conclude that $g_1$ is contained in some optimal solution. Therefore the greedy strategy works.


question:
 Describe an efficient algorithm that, given a set $\\{x_1, x_2, \ldots, x_n\\}$ of points on the real line, determines the smallest set of unit-length closed intervals that contains all of the given points. Argue that your algorithm is correct.

answer:
Consider the leftmost interval. It will do no good if it extends any further left than the leftmost point, however, we know that it must contain the leftmost point. So, we know that it's left hand side is exactly the leftmost point.

So, we just remove any point that is within a unit distance of the left most point since they are contained in this single interval. Then, we just repeat until all points are covered. Since at each step there is a clearly optimal choice for where to put the leftmost interval, this final solution is optimal.


question: Show how to solve the fractional knapsack problem in $O(n)$ time.
answer:
First compute the value of each item, defined to be it's worth divided by its weight. We use a recursive approach as follows, find the item of median value, which can be done in linear time as shown in chapter 9. Then sum the weights of all items whose value exceeds the median and call it $M$. If $M$ exceeds $W$ then we know that the solution to the fractional knapsack problem lies in taking items from among this collection. In other words, we're now solving the fractional knapsack problem on input of size $n / 2$. On the other hand, if the weight doesn't exceed $W$, then we must solve the fractional knapsack problem on the input of $n / 2$ low-value items, with maximum weight $W − M$. Let $T(n)$ denote the runtime of the algorithm. Since we can solve the problem when there is only one item in constant time, the recursion for the runtime is $T(n) = T(n / 2) + cn$ and $T(1) = d$, which gives runtime of $O(n)$.


question:
 Suppose you are given two sets $A$ and $B$, each containing $n$ positive integers. You can choose to reorder each set however you like. After reordering, let $a_i$ be the $i$th element of set $A$, and let $b_i$ be the $i$th element of set $B$. You then receive a payoff of $\prod_{i = 1}^n a_i^{b_i}$. Give an algorithm that will maximize your payoff. Prove that your algorithm maximizes the payoff, and state its running time.

answer:
Since an idential permutation of both sets doesn't affect this product, suppose that $A$ is sorted in ascending order. Then, we will prove that the product is maximized when $B$ is also sorted in ascending order. To see this, suppose not, that is, there is some $i < j$ so that $a_i < a_j$ and $b_i > b_j$. Then, consider only the contribution to the product from the indices $i$ and $j$. That is, $a_i^{b_i}a_j^{b_j}$, then, if we were to swap the order of $b_i$ and $b_j$, we would have that contribution be $a_i^{b_j}a_j^{b_i}$. we can see that this is larger than the previous expression because it differs by a factor of $\left(\frac{a_j}{a_i}\right)^{b_i - b_j}$ which is bigger than one. So, we couldn't of maximized the product with this ordering on $B$.


question:
 Give a real-world example that requires sorting or a real-world example that requires computing a convex hull.

answer:
- Sorting: browse the price of the restaurants with ascending prices on NTU street.
- Convex hull: computing the diameter of set of points.


question:
 How are the shortest-path and traveling-salesman problems given above similar? How are they different?

answer:
- Similar: finding path with shortest distance.
- Different: traveling-salesman has more constraints.


question:
 Come up with a real-world problem in which only the best solution will do. Then come up with one in which a solution that is "approximately" the best is good enough.

answer:
- Best: find the GCD of two positive integer numbers.
- Approximately: find the solution of differential equations.


answer: Drive navigation.
question:
 Give an example of an application that requires algorithmic content at the application level, and discuss the function of the algorithms involved.


question:
 Suppose we are comparing implementations of insertion sort and merge sort on the same machine. For inputs of size $n$ , insertion sort runs in $8n^2$ steps, while merge sort runs in $64n\lg n$ steps. For which values of $n$ does insertion sort beat merge sort?

answer:
$$
\begin{aligned}
    8n^2 & <   64n\lg n \\\\
     2^n & <   n^8 \\\\
 2 \le n & \le 43.
\end{aligned}
$$


answer: $$ \begin{aligned} 100n^2 & < 2^n \\\\ n & \ge 15. \end{aligned} $$
question:
 What is the smallest value of $n$ such that an algorithm whose running time is $100n^2$ runs faster than an algorithm whose running time is $2^n$ on the same machine?


question:
 Use Strassen's algorithm to compute the matrix product
>
> $$
> \begin{pmatrix}
> 1 & 3 \\\\
> 7 & 5
> \end{pmatrix}
> \begin{pmatrix}
> 6 & 8 \\\\
> 4 & 2
> \end{pmatrix}
> .
> $$
>
> Show your work.

answer:
The first matrices are

$$
\begin{array}{ll}
S_1 =  6 & S_6    =  8 \\\\
S_2 =  4 & S_7    = -2 \\\\
S_3 = 12 & S_8    =  6 \\\\
S_4 = -2 & S_9    = -6 \\\\
S_5 =  6 & S_{10} = 14.
\end{array}
$$

The products are

$$
\begin{aligned}
P_1 & =  1 \cdot 6  =   6 \\\\
P_2 & =  4 \cdot 2  =   8 \\\\
P_3 & =  6 \cdot 12 =  72 \\\\
P_4 & = -2 \cdot 5  = -10 \\\\
P_5 & =  6 \cdot 8  =  48 \\\\
P_6 & = -2 \cdot 6  = -12 \\\\
P_7 & = -6 \cdot 14 = -84.
\end{aligned}
$$

The four matrices are

$$
\begin{aligned}
C_{11} & = 48 + (-10) - 8 + (-12) = 18 \\\\
C_{12} & =  6 + 8 = 14 \\\\
C_{21} & = 72 + (-10) = 62 \\\\
C_{22} & = 48 + 6 - 72 - (-84) = 66.
\end{aligned}
$$

The result is

$$
\begin{pmatrix}
18 & 14 \\\\
62 & 66
\end{pmatrix}
.
$$


question: Write pseudocode for Strassen's algorithm.
answer:
```cpp
STRASSEN(A, B)
    n = A.rows
    if n == 1
        return a[1, 1] * b[1, 1]
    let C be a new n × n matrix
    A[1, 1] = A[1..n / 2][1..n / 2]
    A[1, 2] = A[1..n / 2][n / 2 + 1..n]
    A[2, 1] = A[n / 2 + 1..n][1..n / 2]
    A[2, 2] = A[n / 2 + 1..n][n / 2 + 1..n]
    B[1, 1] = B[1..n / 2][1..n / 2]
    B[1, 2] = B[1..n / 2][n / 2 + 1..n]
    B[2, 1] = B[n / 2 + 1..n][1..n / 2]
    B[2, 2] = B[n / 2 + 1..n][n / 2 + 1..n]
    S[1] = B[1, 2] - B[2, 2]
    S[2] = A[1, 1] + A[1, 2]
    S[3] = A[2, 1] + A[2, 2]
    S[4] = B[2, 1] - B[1, 1]
    S[5] = A[1, 1] + A[2, 2]
    S[6] = B[1, 1] + B[2, 2]
    S[7] = A[1, 2] - A[2, 2]
    S[8] = B[2, 1] + B[2, 2]
    S[9] = A[1, 1] - A[2, 1]
    S[10] = B[1, 1] + B[1, 2]
    P[1] = STRASSEN(A[1, 1], S[1])
    P[2] = STRASSEN(S[2], B[2, 2])
    P[3] = STRASSEN(S[3], B[1, 1])
    P[4] = STRASSEN(A[2, 2], S[4])
    P[5] = STRASSEN(S[5], S[6])
    P[6] = STRASSEN(S[7], S[8])
    P[7] = STRASSEN(S[9], S[10])
    C[1..n / 2][1..n / 2] = P[5] + P[4] - P[2] + P[6]
    C[1..n / 2][n / 2 + 1..n] = P[1] + P[2]
    C[n / 2 + 1..n][1..n / 2] = P[3] + P[4]
    C[n / 2 + 1..n][n / 2 + 1..n] = P[5] + P[1] - P[3] - P[7]
    return C
```


question:
 How would you modify Strassen's algorithm to multiply $n \times n$ matrices in which $n$ is not an exact power of $2$? Show that the resulting algorithm runs in time $\Theta(n^{\lg7})$.

answer:
We can just extend it to an $n \times n$ matrix and pad it with zeroes. It's obviously $\Theta(n^{\lg7})$.


question:
 What is the largest $k$ such that if you can multiply $3 \times 3$ matrices using $k$ multiplications (not assuming commutativity of multiplication), then you can multiply $n \times n$ matrices is time $o(n^{\lg 7})$? What would the running time of this algorithm be?

answer:
Assume $n = 3^m$ for some $m$. Then, using block matrix multiplication, we obtain the recursive running time $T(n) = kT(n / 3) + O(1)$.

By master theorem, we can find the largest $k$ to satisfy $\log_3 k < \lg 7$ is $k = 21$.


question:
 V. Pan has discovered a way of multiplying $68 \times 68$ matrices using $132464$ multiplications, a way of multiplying $70 \times 70$ matrices using $143640$ multiplications, and a way of multiplying $72 \times 72$ matrices using $155424$ multiplications. Which method yields the best asymptotic running time when used in a divide-and-conquer matrix-multiplication algorithm? How does it compare to Strassen's algorithm?

answer:
Using what we know from the last exercise, we need to pick the smallest of the following

$$
\begin{aligned}
\log_{68} 132464 & \approx 2.795128 \\\\
\log_{70} 143640 & \approx 2.795122 \\\\
\log_{72} 155424 & \approx 2.795147.
\end{aligned}
$$

The fastest one asymptotically is $70 \times 70$ using $143640$.


question:
 How quickly can you multiply a $kn \times n$ matrix by an $n \times kn$ matrix, using Strassen's algorithm as a subroutine? Answer the same question with the order of the input matrices reversed.

answer:
- $(kn \times n)(n \times kn)$ produces a $kn \times kn$ matrix. This produces $k^2$ multiplications of $n \times n$ matrices.
- $(n \times kn)(kn \times n)$ produces an $n \times n$ matrix. This produces $k$ multiplications and $k - 1$ additions.


question:
 Show how to multiply the complex numbers $a + bi$ and $c + di$ using only three multiplications of real numbers. The algorithm should take $a$, $b$, $c$ and $d$ as input and produce the real component $ac - bd$ and the imaginary component $ad + bc$ separately.

answer:
The three matrices are

$$
\begin{aligned}
A & = (a + b)(c + d) = ac + ad + bc + bd \\\\
B & = ac \\\\
C & = bd.
\end{aligned}
$$

The result is

$$(B - C) + (A - B - C)i.$$


question: Show that the solution of $T(n) = T(n - 1) + n$ is $O(n^2)$.
answer:
We guess $T(n) \le cn^2$,

$$
\begin{aligned}
T(n) & \le c(n - 1)^2 + n \\\\
     & =   cn^2 - 2cn + c + n \\\\
     & =   cn^2 + n(1 - 2c) + c \\\\
     & \le cn^2,
\end{aligned}
$$

where the last step holds for $c > \frac{1}{2}$.


question: Show that the solution of $T(n) = T(\lceil n / 2 \rceil) + 1$ is $O(\lg n)$.
answer:
We guess $T(n) \le c\lg(n - a)$,

$$
\begin{aligned}
T(n) & \le c\lg(\lceil n / 2 \rceil - a) + 1 \\\\
     & \le c\lg((n + 1) / 2 - a) + 1 \\\\
     & =   c\lg((n + 1 - 2a) / 2) + 1 \\\\
     & =   c\lg(n + 1 - 2a) - c\lg 2 + 1 & (c \ge 1) \\\\
     & \le c\lg(n + 1 - 2a) & (a \ge 1) \\\\
     & \le c\lg(n - a),
\end{aligned}
$$


question:
 We saw that the solution of $T(n) = 2T(\lfloor n / 2 \rfloor) + n$ is $O(n\lg n)$. Show that the solution of this recurrence is also $\Omega(n\lg n)$. Conclude that the solution is $\Theta(n\lg n)$.

answer:
First, we guess $T(n) \le cn\lg n$,

$$
\begin{aligned}
T(n) & \le 2c\lfloor n / 2 \rfloor\lg{\lfloor n / 2 \rfloor} + n \\\\
     & \le cn\lg(n / 2) + n \\\\
     & =   cn\lg n - cn\lg 2 + n \\\\
     & =   cn\lg n + (1 - c)n \\\\
     & \le cn\lg n,
\end{aligned}
$$

where the last step holds for $c \ge 1$.

Next, we guess $T(n) \ge c(n + a)\lg(n + a)$,

$$
\begin{aligned}
T(n) & \ge 2c(\lfloor n / 2 \rfloor + a)(\lg(\lfloor n / 2 \rfloor + a) + n \\\\
     & \ge 2c((n - 1) / 2 + a)(\lg((n - 1) / 2 + a)) + n \\\\
     & =   2c\frac{n - 1 + 2a}{2}\lg\frac{n - 1 + 2a}{2} + n \\\\
     & =   c(n - 1 + 2a)\lg(n - 1 + 2a) - c(n - 1 + 2a)\lg 2 + n \\\\
     & =   c(n - 1 + 2a)\lg(n - 1 + 2a) + (1 - c)n - (2a - 1)c & (0 \le c < 1, n \ge \frac{(2a - 1)c}{1 - c}) \\\\
     & \ge c(n - 1 + 2a)\lg(n - 1 + 2a) & (a \ge 1) \\\\
     & \ge c(n + a)\lg(n + a),
\end{aligned}
$$


question:
 Show that by making a different inductive hyptohesis, we can overcome the difficulty with the boundary condition $T(1) = 1$ for recurrence $\text{(4.19)}$ without adjusting the boundary conditions for the inductive proof.

answer:
We guess $T(n) \le n\lg n + n$,

$$
\begin{aligned}
T(n) & \le 2(c\lfloor n / 2 \rfloor\lg{\lfloor n / 2 \rfloor} + \lfloor n / 2 \rfloor) + n \\\\
     & \le 2c(n / 2)\lg(n / 2) + 2(n / 2) + n \\\\
     & =   cn\lg(n / 2) + 2n \\\\
     & =   cn\lg n - cn\lg{2} + 2n \\\\
     & =   cn\lg n + (2 - c)n \\\\
     & \le cn\lg n + n,
\end{aligned}
$$

where the last step holds for $c \ge 1$.

This time, the boundary condition is

$$T(1) = 1 \le cn\lg n + n = 0 + 1 = 1.$$


question:
 Show that $\Theta(n\lg n)$ is the solution to the "exact" recurrence $\text{(4.3)}$ for merge sort.

answer:
The recurrence is

> $$T(n) = T(\lceil n / 2 \rceil) + T(\lfloor n / 2 \rfloor) + \Theta(n) \tag{4.3}$$

To show $\Theta$ bound, separately show $O$ and $\Omega$ bounds.

- For $O(n\lg n)$, we guess $T(n) \le c(n - 2)\lg(n - 2) - 2c$,

  $$
  \begin{aligned}
  T(n) & \le c(\lceil n / 2 \rceil -2 )\lg(\lceil n / 2 \rceil - 2) + c(\lfloor n / 2 \rfloor - 2)\lg(\lfloor n / 2 \rfloor - 2) + dn \\\\
       & \le c(n / 2 + 1 - 2)\lg(n / 2 + 1 - 2) - 2c + c(n / 2 - 2)\lg(n / 2 - 2) - 2c + dn \\\\
       & \le c(n / 2 - 1 )\lg(n / 2 - 1) + c(n / 2 - 1)\lg(n / 2 - 1) + dn \\\\
       & =   c\frac{n - 2}{2}\lg\frac{n - 2}{2} + c\frac{n - 2}{2}\lg\frac{n - 2}{2} - 4c + dn \\\\
       & =   c(n - 2)\lg\frac{n - 2}{2} - 4c + dn \\\\
       & =   c(n - 2)\lg(n - 2) - c(n - 2) - 4c + dn \\\\
       & =   c(n - 2)\lg(n - 2) + (d - c)n + 2c - 4c \\\\
       & \le c(n - 2)\lg(n - 2) - 2c,
  \end{aligned}
  $$

  where the last step holds for $c > d$.

- For $\Omega(n\lg n)$, we guess $T(n) \ge c(n + 2)\lg (n + 2) + 2c$,

  $$
  \begin{aligned}
  T(n) & \ge c(\lceil n / 2 \rceil +2 )\lg(\lceil n / 2 \rceil + 2) + c(\lfloor n / 2 \rfloor + 2)\lg(\lfloor n / 2 \rfloor + 2) + dn \\\\
       & \ge c(n / 2 + 2)\lg(n / 2 + 2) + 2c + c(n / 2 - 1 + 2)\lg(n / 2 - 1 + 2) + 2c + dn \\\\
       & \ge c(n / 2 + 1)\lg(n / 2 + 1) + c(n / 2 + 1)\lg(n / 2 + 1) + 4c + dn \\\\
       & \ge c\frac{n + 2}{2}\lg\frac{n + 2}{2} + c\frac{n + 2}{2}\lg\frac{n + 2}{2} + 4c + dn \\\\
       & =   c(n + 2)\lg\frac{n + 2}{2} + 4c + dn \\\\
       & =   c(n + 2)\lg(n + 2) - c(n + 2) + 4c + dn \\\\
       & =   c(n + 2)\lg(n + 2) + (d - c)n - 2c + 4c \\\\
       & \ge c(n + 2)\lg(n + 2) + 2c,
  \end{aligned}
  $$

  where the last step holds for $d > c$.


question: Show that the solution to $T(n) = 2T(\lfloor n / 2 \rfloor + 17) + n$ is $O(n\lg n)$.
answer:
We guess $T(n) \le c(n - a)\lg(n - a)$,

$$
\begin{aligned}
T(n) & \le 2c(\lfloor n / 2 \rfloor + 17 - a)\lg(\lfloor n / 2 \rfloor + 17 - a) + n \\\\
     & \le 2c(n / 2 + 17 - a)\lg(n / 2 + 17 - a) + n \\\\
     & =   c(n + 34 - 2a)\lg\frac{n + 34 - 2a}{2} + n \\\\
     & =   c(n + 34 - 2a)\lg(n + 34 - 2a) - c(n + 34 - 2a) + n & (c > 1, n > n_0 = f(a)) \\\\
     & \le c(n + 34 - 2a)\lg(n + 34 - 2a) & (a \ge 34) \\\\
     & \le c(n - a)\lg(n - a).
\end{aligned}
$$


question:
 Using the master method in Section 4.5, you can show that the solution to the recurrence $T(n) = 4T(n / 3) + n$ is $T(n) = \Theta(n^{\log_3 4})$. Show that a substitution proof with the assumption $T(n) \le cn^{\log_3 4}$ fails. Then show how to subtract off a lower-order term to make the substitution proof work.

answer:
We guess $T(n) \le cn^{\log_3 4}$ first,

$$
\begin{aligned}
T(n) & \le 4c(n / 3)^{\log_3 4} + n \\\\
     & =   cn^{\log_3 4} + n.
\end{aligned}
$$

We stuck here.

We guess $T(n) \le cn^{\log_3 4} - dn$ again,

$$
\begin{aligned}
T(n) & \le 4(c(n / 3)^{\log_3 4} - dn / 3) + n \\\\
     & =   4(cn^{\log_3 4} / 4 - dn / 3) + n \\\\
     & =   cn^{\log_3 4} - \frac{4}{3}dn + n \\\\
     & \le cn^{\log_3 4} - dn,
\end{aligned}
$$

where the last step holds for $d \ge 3$.


question:
 Using the master method in Section 4.5, you can show that the solution to the recurrence $T(n) = 4T(n / 2) + n$ is $T(n) = \Theta(n^2)$. Show that a substitution proof with the assumption $T(n) \le cn^2$ fails. Then show how to subtract off a lower-order term to make the substitution proof work.

answer:
First, let's try the guess $T(n) \le cn^2$. Then, we have

$$
\begin{aligned}
T(n) &=   4T(n / 2) + n \\\\
     &\le 4c(n / 2)^2 + n \\\\
     &=   cn^2 + n.
\end{aligned}
$$

We can't proceed any further from the inequality above to conclude $T(n) \le cn^2$.

Alternatively, let us try the guess

$$T(n) \le cn^2 - cn,$$

which subtracts off a lower-order term. Now we have

$$
\begin{aligned}
T(n) &=   4T(n / 2) + n \\\\
     &=   4\left(c(n / 2)^2 - c(n / 2)\right) + n \\\\
     &=   4c(n / 2)^2 - 4c(n / 2) + n \\\\
     &=   cn^2 + (1 - 2c)n \\\\
     &\le cn^2,
\end{aligned}
$$

where the last step holds for $c \ge 1 / 2$.


question:
 Solve the recurrence $T(n) = 3T(\sqrt n) + \log n$ by making a change of variables. Your solution should be asymptotically tight. Do not worry about whether values are integral.

answer:
First,

$$
\begin{aligned}
  T(n) & = 3T(\sqrt n) + \lg n & \text{ let } m = \lg n \\\\
T(2^m) & = 3T(2^{m / 2}) + m \\\\
  S(m) & = 3S(m / 2) + m.
\end{aligned}
$$

Now we guess $S(m) \le cm^{\lg 3} + dm$,

$$
\begin{aligned}
S(m) & \le 3\Big(c(m / 2)^{\lg 3} + d(m / 2)\Big) + m \\\\
     & \le cm^{\lg 3} + (\frac{3}{2}d + 1)m & (d \le -2) \\\\
     & \le cm^{\lg 3} + dm.
\end{aligned}
$$

Then we guess $S(m) \ge cm^{\lg 3} + dm$,

$$
\begin{aligned}
S(m) & \ge 3\Big(c(m / 2)^{\lg 3} + d(m / 2)\Big) + m \\\\
     & \ge cm^{\lg 3} + (\frac{3}{2}d + 1)m & (d \ge -2) \\\\
     & \ge cm^{\lg 3} + dm.
\end{aligned}
$$

Thus,

$$
\begin{aligned}
S(m) & = \Theta(m^{\lg 3}) \\\\
T(n) & = \Theta(\lg^{\lg 3}{n}).
\end{aligned}
$$


question:
 Give a simple and exact expression for $n_j$ in equation $\text{(4.27)}$ for the case in which $b$ is a positive integer instead of an arbitrary real number.

answer:
We state that $\forall{j \ge 0}, n_j = \left \lceil \frac{n}{b^j} \right \rceil$.  

Indeed, for $j = 0$ we have from the recurrence's base case that $n_0 = n = \left \lceil \frac{n}{b^0} \right \rceil$.  
Now, suppose $n_{j - 1} = \left \lceil \frac{n}{b^{j - 1}} \right \rceil$ for some $j > 0$. By definition, $n_j = \left \lceil \frac{n_{j - 1}}{b} \right \rceil$.  
It follows from the induction hypothesis that $n_j = \left \lceil \frac{\left \lceil \frac{n}{b^{j - 1}} \right \rceil}{b} \right \rceil$.  
Since $b$ is a positive integer, equation $\text{(3.4)}$ implies that $\left \lceil \frac{\left \lceil \frac{n}{b^{j - 1}} \right \rceil}{b} \right \rceil = \left \lceil \frac{n}{b^j} \right \rceil$.  
Therefore, $n_j = \left \lceil \frac{n}{b^j} \right \rceil$.

P.S. $n_j$ is obtained by shifting the base $b$ representation $j$ positions to the right, and adding $1$ if any of the $j$ least significant positions are non-zero.


question:
 Show that if $f(n) = \Theta(n^{\log_b a}\lg^k{n})$, where $k \ge 0$, then the master recurrence has solution $T(n) = \Theta(n^{\log_b a}\lg^{k + 1}n)$. For simplicity, confine your analysis to exact powers of $b$.

answer:
$$
\begin{aligned}
            g(n) & = \sum_{j = 0}^{\log_b n - 1} a^j f(n / b^j) \\\\
      f(n / b^j) & = \Theta\Big((n / b^j)^{\log_b a} \lg^k(n / b^j) \Big) \\\\
            g(n) & = \Theta\Big(\sum_{j = 0}^{\log_b n - 1}a^j\big(\frac{n}{b^j}\big)^{\log_b a}\lg^k\big(\frac{n}{b^j}\big)\Big) \\\\
                 & = \Theta(A) \\\\
               A & = \sum_{j = 0}^{\log_b n - 1} a^j \big(\frac{n}{b^j}\big)^{\log_b a}\lg^k\frac{n}{b^j} \\\\
                 & = n^{\log_b a} \sum_{j = 0}^{\log_b n - 1}\Big(\frac{a}{b^{\log_b a}}\Big)^j\lg^k\frac{n}{b^j} \\\\
                 & = n^{\log_b a}\sum_{j = 0}^{\log_b n - 1}\lg^k\frac{n}{b^j} \\\\
                 & = n^{\log_b a} B \\\\
\lg^k\frac{n}{d} & = (\lg n - \lg d)^k = \lg^k{n} + o(\lg^k{n}) \\\\
               B & = \sum_{j = 0}^{\log_b n - 1}\lg^k\frac{n}{b^j} \\\\
                 & = \sum_{j = 0}^{\log_b n - 1}\Big(\lg^k{n} - o(\lg^k{n})\Big) \\\\
                 & = \log_b n\lg^k{n} + \log_b n \cdot o(\lg^k{n}) \\\\
                 & = \Theta(\log_b n\lg^k{n}) \\\\
                 & = \Theta(\lg^{k + 1}{n}) \\\\
            g(n) & = \Theta(A) \\\\
                 & = \Theta(n^{\log_b a}B) \\\\
                 & = \Theta(n^{\log_b a}\lg^{k + 1}{n}).
\end{aligned}
$$


question:
 Show that case 3 of the master method is overstated, in the sense that the regularity condition $af(n / b) \le cf(n)$ for some constant $c < 1$ implies that there exists a constant $\epsilon > 0$ such that $f(n) = \Omega(n^{\log_b a + \epsilon})$.

answer:
$$
\begin{aligned}
           af(n / b) & \le cf(n) \\\\
\Rightarrow f(n / b) & \le \frac{c}{a} f(n) \\\\
    \Rightarrow f(n) & \le \frac{c}{a} f(bn) \\\\
                     & =   \frac{c}{a} \left(\frac{c}{a} f(b^2n)\right) \\\\
                     & =   \frac{c}{a} \left(\frac{c}{a}\left(\frac{c}{a} f(b^3n)\right)\right) \\\\
                     & =   \left(\frac{c}{a}\right)^i f(b^i n) \\\\
\Rightarrow f(b^i n) & \ge \left(\frac{a}{c}\right)^i f(n).
\end{aligned}
$$

Let $n = 1$, then we have

$$f(b^i) \ge \left(\frac{a}{c}\right)^i f(1) \quad (*).$$

Let $b^i = n \Rightarrow i = \log_b n$, then substitue back to equation $(*)$,

$$
\begin{aligned}
f(n) & \ge \left(\frac{a}{c}\right)^{\log_b n} f(1) \\\\
     & \ge n^{\log_b \frac{a}{c}} f(1) \\\\
     & \ge n^{\log_b a + \epsilon} & \text{ where $\epsilon > 0$ because $\frac{a}{c} > a$ (recall that $c < 1$)} \\\\
     & =   \Omega(n^{\log_b a + \epsilon}).
\end{aligned}
$$


question:
 Write pseudocode for the brute-force method of solving the maximum-subarray problem. Your procedure should run in $\Theta(n^2)$ time.

answer:
```cpp
BRUTE-FORCE-FIND-MAXIMUM-SUBARRAY(A)
    n = A.length
    max-sum = -∞
    for l = 1 to n
        sum = 0
        for h = l to n
            sum = sum + A[h]
            if sum > max-sum
                max-sum = sum
                low = l
                high = h
    return (low, high, max-sum)
```


question:
 Implement both the brute-force and recursive algorithms for the maximum-subarray problem on your own computer. What problem size $n_0$ gives the crossover point at which the recursive algorithm beats the brute-force algorithm? Then, change the base case of the recursive algorithm to use the brute-force algorithm whenever the problem size is less than $n_0$. Does that change the crossover point?

answer:
On my computer, $n_0$ is $37$.

If the algorithm is modified to used divide and conquer when $n \ge 37$ and the brute-force approach when $n$ is less, the performance at the crossover point almost doubles. The performance at $n_0 - 1$ stays the same, though (or even gets worse, because of the added overhead).

What I find interesting is that if we set $n_0 = 20$ and used the mixed approach to sort $40$ elements, it is still faster than both.


answer: If the original algorithm returns a negative sum, returning an empty subarray instead.
question:
 Suppose we change the definition of the maximum-subarray problem to allow the result to be an empty subarray, where the sum of the values of an empty subarray is $0$. How would you change any of the algorithms that do not allow empty subarrays to permit an empty subarray to be the result?


question:
 Use the following ideas to develop a nonrecursive, linear-time algorithm for the maximum-subarray problem. Start at the left end of the array, and progress toward the right, keeping track of the maximum subarray seen so far. Knowing a maximum subarray $A[1..j]$, extend the answer to find a maximum subarray ending at index $j + 1$ by using the following observation: a maximum subarray $A[i..j + 1]$, is either a maximum subarray of $A[1..j]$ or a subarray $A[i..j + 1]$, for some $1 \le i \le j + 1$. Determine a maximum subarray of the form $A[i..j + 1]$ in constant time based on knowing a maximum subarray ending at index $j$.

answer:
```cpp
ITERATIVE-FIND-MAXIMUM-SUBARRAY(A)
    n = A.length
    max-sum = -∞
    sum = -∞
    for j = 1 to n
        currentHigh = j
        if sum > 0
            sum = sum + A[j]
        else
            currentLow = j
            sum = A[j]
        if sum > max-sum
            max-sum = sum
            low = currentLow
            high = currentHigh
    return (low, high, max-sum)
```


question:
 Use the master method to give tight asymptotic bounds for the following recurrences:
>
> **a.** $T(n) = 2T(n / 4) + 1$.
>
> **b.** $T(n) = 2T(n / 4) + \sqrt n$.
>
> **c.** $T(n) = 2T(n / 4) + n$.
>
> **d.** $T(n) = 2T(n / 4) + n^2$.

answer:
**a.** $\Theta(n^{\log_4 2}) = \Theta(\sqrt n)$.

**b.** $\Theta(n^{\log_4 2}\lg n) = \Theta(\sqrt n\lg n)$.

**c.** $\Theta(n)$.

**d.** $\Theta(n^2)$.


question:
 Professor Caesar wishes to develop a matrix-multiplication algorithm that is asymptotically faster than Strassen's algorithm. His algorithm will use the divide-and-conquer method, dividing each matrix into pieces of size $n / 4 \times n / 4$, and the divide and combine steps together will take $\Theta(n^2)$ time. He needs to determine how many subproblems his algorithm has to create in order to beat Strassen's algorithm. If his algorithm creates $a$ subproblems, then the recurrence for the running time $T(n)$ becomes $T(n) = aT(n / 4) + \Theta(n^2)$. What is the largest integer value of $a$ for which Professor Caesar's algorithm would be asymptotically faster than Strassen's algorithm?

answer:
Strassen's algorithm has running time of $\Theta(n^{\lg 7})$.

The largest integer $a$ such that $\log_4 a < \lg 7$ is $a = 48$.


question:
 Use the master method to show that the solution to the binary-search recurrence $T(n) = T(n / 2) + \Theta(1)$ is $T(n) = \Theta(\lg n)$. (See exercise 2.3-5 for a description of binary search.)

answer:
$$
\begin{aligned}
   a & = 1, b = 2, \\\\
f(n) & = \Theta(n^{\lg 1}) = \Theta(1), \\\\
T(n) & = \Theta(\lg n).
\end{aligned}
$$


question:
 Can the master method be applied to the recurrence $T(n) = 4T(n / 2) + n^2\lg n$? Why or why not? Give an asymptotic upper bound for this recurrence.

answer:
With $a = 4$, $b = 2$, we have $f(n) = n^2\lg n \ne O(n^{2 - \epsilon}) \ne \Omega(n^{2 + \epsilon})$, so we cannot apply the master method.

We guess $T(n) \le cn^2\lg^2 n$, subsituting $T(n/2) \le c(n/2)^2\lg^2 (n/2)$ into the recurrence yields

$$
\begin{aligned}
T(n) & =   4T(n / 2) + n^2\lg n \\\\
     & \le 4c(n / 2)^2\lg^2(n / 2) + n^2\lg n \\\\
     & =   cn^2\lg(n / 2)\lg n - cn^2\lg(n / 2)\lg 2 + n^2\lg n \\\\
     & =   cn^2\lg^2 n - cn^2\lg n\lg 2 - cn^2\lg(n / 2)\lg 2 + n^2\lg n \\\\
     & =   cn^2\lg^2 n + (1 - c\lg 2)n^2\lg n - cn^2\lg(n / 2)\lg 2 & (c \ge 1/\lg 2) \\\\
     & \le cn^2\lg^2 n - cn^2\lg(n / 2)\lg 2 \\\\
     & \le cn^2\lg^2 n.
\end{aligned}
$$

Exercise 4.6-2 is the general case for this.


question:
 Consider the regularity condition $af(n / b) \le cf(n)$ for some constant $c < 1$, which is part of case 3 of the master theorem. Give an example of constants $a \ge 1$ and $b > 1$ and a function $f(n)$ that satisfies all the conditions in case 3 of the master theorem, except the regularity condition.

answer:
$a = 1$, $b = 2$ and $f(n) = n(2 - \cos n)$.

If we try to prove it,

$$
\begin{aligned}
\frac{n}{2}(2 - \cos\frac{n}{2}) & <   cn \\\\
        \frac{1 - cos(n / 2)}{2} & <   c \\\\
        1 - \frac{cos(n / 2)}{2} & \le c.
\end{aligned}
$$

Since $\min\cos(n / 2) = -1$, this implies that $c \ge 3 / 2$. But $c < 1$.


question:
 Use a recursion tree to determine a good asymptotic upper bound on the recurrence $T(n) = 3T(\lfloor n / 2 \rfloor) + n$. Use the substitution method to verify your answer.

answer:
- The subproblem size for a node at depth $i$ is $n / 2^i$.

    Thus, the tree has $\lg n + 1$ levels and $3^{\lg n} = n^{\lg 3}$ leaves.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, \lg n - 1$, is $3^i(n / 2^i) = (3 / 2)^i n$.

    $$
    \begin{aligned}
    T(n) & = n + \frac{3}{2}n + \Big(\frac{3}{2}\Big)^2 n + \cdots + \Big(\frac{3}{2}\Big)^{\lg n - 1} n + \Theta(n^{\lg 3}) \\\\
        & = \sum_{i = 0}^{\lg n - 1} \Big(\frac{3}{2}\Big)^i n + \Theta(n^{\lg 3}) \\\\
        & = \frac{(3 / 2)^{\lg n} - 1}{(3 / 2) - 1}n + \Theta(n^{\lg 3}) \\\\
        & = 2[(3 / 2)^{\lg n} - 1]n + \Theta(n^{\lg 3}) \\\\
        & = 2[n^{\lg(3 / 2)} - 1]n + \Theta(n^{\lg 3}) \\\\
        & = 2[n^{\lg 3 - \lg 2} - 1]n + \Theta(n^{\lg 3}) \\\\
        & = 2[n^{\lg 3 - 1 + 1} - n] + \Theta(n^{\lg 3}) \\\\
        & = O(n^{\lg 3}).
    \end{aligned}
    $$

- We guess $T(n) \le cn^{\lg 3} - dn$,

    $$
    \begin{aligned}
    T(n) & =   3T(\lfloor n / 2 \rfloor) + n \\\\
         & \le 3 \cdot (c(n / 2)^{\lg 3} - d(n / 2)) + n \\\\
         & =   (3 / 2^{\lg 3})cn^{\lg 3} - (3d / 2)n + n \\\\
         & =   cn^{\lg 3} + (1 - 3d / 2)n \\\\
         & \le cn^{\lg 3} - dn,
    \end{aligned}
    $$

    where the last step holds for $d \ge 2$.


question:
 Use a recursion tree to determine a good asymptotic upper bound on the recurrence $T(n) = T(n / 2) + n^2$. Use the substitution method to verify your answer.

answer:
- The subproblem size for a node at depth $i$ is $n / 2^i$.

    Thus, the tree has $\lg n + 1$ levels and $1^{\lg n} = 1$ leaf.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, \lg{n - 1}$, is $1^i (n / 2^i)^2 = (1 / 4)^i n^2$.

    $$
    \begin{aligned}
    T(n) & = \sum_{i = 0}^{\lg n - 1} \Big(\frac{1}{4}\Big)^i n^2 + \Theta(1) \\\\
         & < \sum_{i = 0}^\infty \Big(\frac{1}{4}\Big)^i n^2 + \Theta(1) \\\\
         & = \frac{1}{1 - (1 / 4)} n^2 + \Theta(1) \\\\
         & = \Theta(n^2).
    \end{aligned}
    $$

- We guess $T(n) \le cn^2$,

    $$
    \begin{aligned}
    T(n) & \le c(n / 2)^2 + n^2 \\\\
         & =   cn^2 / 4 + n^2 \\\\
         & =   (c / 4 + 1)n^2 \\\\
         & \le cn^2,
    \end{aligned}
    $$

    where the last step holds for $c \ge 4 / 3$.


question:
 Use a recursion tree to determine a good asymptotic upper bound on the recurrence $T(n) = 4T(n / 2 + 2) + n$. Use the substitution method to verify your answer.

answer:
- The subproblem size for a node at depth $i$ is $n / 2^i$.

    Thus, the tree has $\lg n + 1$ levels and $4^{\lg n} = n^2$ leaves.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, \lg n - 1$, is $4^i(n / 2^i + 2) = 2^i n + 2 \cdot 4^i$.

    $$
    \begin{aligned}
    T(n) & = \sum_{i = 0}^{\lg n - 1} (2^i n + 2 \cdot 4^i) + \Theta(n^2) \\\\
         & = \sum_{i = 0}^{\lg n - 1} 2^i n + \sum_{i = 0}^{\lg n - 1} 2 \cdot 4^i + \Theta(n^2) \\\\
         & = \frac{2^{\lg n} - 1}{2 - 1}n + 2 \cdot \frac{4^{\lg n} - 1}{4 - 1} + \Theta(n^2) \\\\
         & = (2^{\lg n} - 1)n + \frac{2}{3} (4^{\lg n} - 1) + \Theta(n^2) \\\\
         & = (n - 1)n + \frac{2}{3}(n^2 - 1) + \Theta(n^2) \\\\
         & = \Theta(n^2).
    \end{aligned}
    $$

- We guess $T(n) \le c(n^2 - dn)$,

    $$
    \begin{aligned}
    T(n) & =   4T(n / 2 + 2) + n \\\\
         & \le 4c[(n / 2 + 2)^2 - d(n / 2 + 2)] + n \\\\
         & =   4c(n^2 / 4 + 2n + 4 - dn / 2 - 2d) + n \\\\
         & =   cn^2 + 8cn + 16c - 2cdn - 8cd + n \\\\
         & =   cn^2 - cdn + 8cn + 16c - cdn - 8cd + n \\\\
         & =   c(n^2 - dn) - (cd - 8c - 1)n - (d - 2) \cdot 8c \\\\
         & \le c(n^2 - dn),
    \end{aligned}
    $$

    where the last step holds for $cd - 8c - 1 \ge 0$.


question:
 Use a recursion tree to determine a good asymptotic upper bound on the recurrence $T(n) = 2T(n - 1) + 1$. Use the substitution method to verify your answer.

answer:
- The subproblem size for a node at depth $i$ is $n - i$.

    Thus, the tree has $n + 1$ levels ($i = 0, 1, 2, \dots, n$) and $2^n$ leaves.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, n - 1$, is $2^i$.

    The $n$-th level has $2^n$ leaves each with cost $\Theta(1)$, so the total cost of the $n$-th level is $\Theta(2^n)$.

    Adding the costs of all the levels of the recursion tree we get the following:

    $$
    \begin{aligned}
    T(n) & = \sum_{i = 0}^{n - 1} 2^i + \Theta(2^n)\\\\
         & = \frac{2^n - 1}{2 - 1} + \Theta(2^n)\\\\
         & = 2^n - 1 + \Theta(2^n)\\\\
         & = \Theta(2^n).
    \end{aligned}
    $$

- We guess $T(n) \le c2^n - d$,

    $$
    \begin{aligned}
    T(n) & \le 2(c2^{n - 1} - d) + 1 \\\\
         & = c2^n - 2d + 1 \\\\
         & \le c2^n - d
    \end{aligned}
    $$

    Where the last step holds for $d \ge 1$. Thus $T(n) = O(2^n)$.


question:
 Use a recursion tree to determine a good asymptotic upper bound on the recurrence $T(n) = T(n - 1) + T(n / 2) + n$. Use the substitution method to verify your answer.

answer:
This is a curious one. The tree makes it look like it is exponential in the worst case. The tree is not full (not a complete binary tree of height $n$), but it is not polynomial either. It's easy to show $O(2^n)$ and $\Omega(n^2)$.

To justify that this is a pretty tight upper bound, we'll show that we can't have any other choice. If we have that $T(n) \le cn^k$, when we substitue into the recurrence, the new coefficient for $n^k$ can be as high as $c(1 + \frac{1}{2^k})$ which is bigger than $c$ regardless of how we choose the value $c$.

- We guess $T(n) \le c2^n - 4n$,

    $$
    \begin{aligned}
    T(n) & \le c2^{n - 1} - 4(n - 1) + c2^{n / 2} - 4n / 2 + n \\\\
         & =   c(2^{n - 1} + 2^{n / 2}) - 5n + 4 & (n \ge 1 / 4) \\\\
         & \le c(2^{n - 1} + 2^{n / 2}) - 4n & (n \ge 2)\\\\
         & =   c(2^{n - 1} + 2^{n - 1}) - 4n \\\\
         & \le c2^n - 4n \\\\ & = O(2^n).
    \end{aligned}
    $$

- We guess $T(n) \ge cn^2$,

    $$
    \begin{aligned}
    T(n) & \ge c(n - 1)^2 + c(n / 2)^2 + n \\\\
         & =   cn^2 - 2cn + c + cn^2 / 4 + n \\\\
         & =   (5 / 4)cn^2 + (1 - 2c)n + c \\\\
         & \ge cn^2 + (1 - 2c)n + c & (c \le 1 / 2) \\\\
         & \ge cn^2 \\\\
         & =   \Omega(n^2).
    \end{aligned}
    $$


question:
 Argue that the solution to the recurrence $T(n) = T(n / 3) + T(2n / 3) + cn$, where $c$ is a constant, is $\Omega(n\lg n)$ by appealing to the recursion tree.

answer:
We know that the cost at each level of the tree is $cn$ by examining the tree in figure 4.6. To find a lower bound on the cost of the algorithm, we need a lower bound on the height of the tree.

The shortest simple path from root to leaf is found by following the leftest child at each node. Since we divide by $3$ at each step, we see that this path has length $\log_3 n$. Therefore, the cost of the algorithm is

$$cn(\log_3 n + 1) \ge cn\log_3 n = \frac{c}{\lg 3} n\lg n = \Omega(n\lg n).$$


question:
 Draw the recursion tree for $T(n) = 4T(\lfloor n / 2 \rfloor) + cn$, where $c$ is a constant, and provide a tight asymptotic bound on its solution. Verify your answer with the substitution method.

answer:
- The subproblem size for a node at depth $i$ is $n / 2^i$.

    Thus, the tree has $\lg n + 1$ levels and $4^{\lg n} = n^{\lg 4} = n^2$ leaves.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, \lg n - 1$, is $4^i(cn / 2^i) = 2^icn$.

    $$
    \begin{aligned}
    T(n) & = \sum_{i = 0}^{\lg n - 1} 2^icn + \Theta(n^2) \\\\
         & = \frac{2^{\lg n} - 1}{2 - 1}cn + \Theta(n^2) \\\\
         & = \Theta(n^2).
    \end{aligned}
    $$

- For $O(n^2)$, we guess $T(n) \le dn^2 - cn$,

    $$
    \begin{aligned}
    T(n) & \le 4d(n / 2)^2 - 4c(n / 2) + cn \\\\
         & =   dn^2 - cn.
    \end{aligned}
    $$

- For $\Omega(n^2)$, we guess $T(n) \ge dn^2 - cn$,

    $$
    \begin{aligned}
    T(n) & \ge 4d(n / 2)^2 - 4c(n / 2) + cn \\\\
         & =   dn^2 - cn.
    \end{aligned}
    $$


question:
 Use a recursion tree to give an asymptotically tight solution to the recurrence $T(n) = T(n - a) + T(a) + cn$, where $a \ge 1$ and $c > 0$ are constants.

answer:
- The tree has $n / a + 1$ levels.

    The total cost over all nodes at depth $i$, for $i = 0, 1, 2, \ldots, n / a - 1$, is $c(n - ia)$.

    $$
    \begin{aligned}
    T(n) & = \sum_{i = 0}^{n / a} c(n - ia) + (n / a)ca \\\\
         & = \sum_{i = 0}^{n / a} cn - \sum_{i = 0}^{n / a} cia + (n / a)ca \\\\
         & = cn^2/a - \Theta(n) + \Theta(n) \\\\
         & = \Theta(n^2).
    \end{aligned}
    $$

- For $O(n^2)$, we guess $T(n) \le cn^2$,

    $$
    \begin{aligned}
    T(n) & \le c(n - a)^2 + ca + cn \\\\
         & \le cn^2 - 2can + ca + cn \\\\
         & \le cn^2 - c(2an - a - n) & (a > 1 / 2, n > 2a) \\\\
         & \le cn^2 - cn \\\\
         & \le cn^2 \\\\
         & =   \Theta(n^2).
    \end{aligned}
    $$

- For $\Omega(n^2)$, we guess $T(n) \ge cn^2$,

    $$
    \begin{aligned}
    T(n) & \ge c(n - a)^2 + ca + cn \\\\
         & \ge cn^2 - 2acn + ca + cn \\\\
         & \ge cn^2 - c(2an - a - n) & (a < 1 / 2, n > 2a) \\\\
         & \ge cn^2 + cn \\\\
         & \ge cn^2 \\\\
         & =   \Theta(n^2).
    \end{aligned}
    $$


question:
 Use a recursion tree to give an asymptotically tight solution to the recurrence $T(n) = T(\alpha n) + T((1 - \alpha)n) + cn$, where $\alpha$ is a constant in the range $0 < \alpha < 1$, and $c > 0$ is also a constant.

answer:
We can assume that $0 < \alpha \le 1 / 2$, since otherwise we can let $\beta = 1 − \alpha$ and solve it for $\beta$.

Thus, the depth of the tree is $\log_{1 / \alpha} n$ and each level costs $cn$. And let's guess that the leaves are $\Theta(n)$,

$$
\begin{aligned}
T(n) & = \sum_{i = 0}^{\log_{1 / \alpha} n} cn + \Theta(n) \\\\
     & = cn\log_{1 / \alpha} n + \Theta(n) \\\\
     & = \Theta(n\lg n).
\end{aligned}
$$

We can also show $T(n) = \Theta(n\lg n)$ by substitution.

To prove the upper bound, we guess that $T(n) \le dn\lg n$ for a constant $d > 0$,

$$
\begin{aligned}
T(n) & =   T(\alpha n) + T((1 - \alpha)n) + cn \\\\
     & \le d\alpha n\lg(\alpha n) + d(1 - \alpha)n\lg((1 - \alpha)n) + cn \\\\
     & =   d\alpha n\lg\alpha + d\alpha n\lg n + d(1 - \alpha)n\lg(1 - \alpha) + d(1 - \alpha)n\lg n + cn \\\\
     & =   dn\lg n + dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn \\\\
     & \le dn\lg n,
\end{aligned}
$$

where the last step holds when $d \ge \frac{-c}{\alpha\lg\alpha + (1 - \alpha)\lg(1 - \alpha)}$.

We can achieve this result by solving the inequality

$$
\begin{aligned}
dn\lg n + dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn & \le dn\lg n \\\\
\implies dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn & \le 0 \\\\
\implies d(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) & \le -c \\\\
\implies d & \ge \frac{-c}{\alpha\lg\alpha + (1 - \alpha)\lg(1 - \alpha)},
\end{aligned}
$$

To prove the lower bound, we guess that $T(n) \ge dn\lg n$ for a constant $d > 0$,

$$
\begin{aligned}
T(n) & =   T(\alpha n) + T((1 - \alpha)n) + cn \\\\
     & \ge d\alpha n\lg(\alpha n) + d(1 - \alpha)n\lg((1 - \alpha)n) + cn \\\\
     & =   d\alpha n\lg\alpha + d\alpha n\lg n + d(1 - \alpha)n\lg(1 - \alpha) + d(1 - \alpha)n\lg n + cn \\\\
     & =   dn\lg n + dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn \\\\
     & \ge dn\lg n,
\end{aligned}
$$

where the last step holds when $0 < d \le \frac{-c}{\alpha\lg\alpha + (1 - \alpha)\lg(1 - \alpha)}$.

We can achieve this result by solving the inequality

$$
\begin{aligned}
dn\lg n + dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn & \ge dn\lg n \\\\
\implies dn(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) + cn & \ge 0 \\\\
\implies d(\alpha \lg\alpha + (1 - \alpha) \lg(1 - \alpha)) & \ge -c \\\\
\implies 0 < d & \le \frac{-c}{\alpha\lg\alpha + (1 - \alpha)\lg(1 - \alpha)},
\end{aligned}
$$

Therefore, $T(n) = \Theta(n\lg n)$.


question:
 Suppose that we insert $n$ keys into a hash table of size $m$ using open addressing and uniform hashing. Let $p(n, m)$ be the probability that no collisions occur. Show that $p(n, m) \le e^{-n(n - 1) / 2m}$. ($\textit{Hint:}$ See equation $\text{(3.12)}$.) Argue that when $n$ exceeds $\sqrt m$, the probability of avoiding collisions goes rapidly to zero.

answer:
$$
\begin{aligned}
p(n, m) & = \frac{m}{m} \cdot \frac{m - 1}{m} \cdots \frac{m - n + 1}{m} \\\\
        & = \frac{m \cdot (m - 1) \cdots (m - n + 1)}{m^n}.
\end{aligned}
$$

$$
\begin{aligned}
(m - i) \cdot (m - n + i)
    & =    (m - \frac{n}{2} + \frac{n}{2} - i) \cdot (m - \frac{n}{2} - \frac{n}{2} + i) \\\\
    & =    (m - \frac{n}{2})^2 - (i - \frac{n}{2})^2 \\\\
    & \le  (m - \frac{n}{2})^2.
\end{aligned}
$$

$$
\begin{aligned}
p(n, m) & \le \frac{m \cdot (m - \frac{n}{2})^{n - 1}}{m^n} \\\\
        & =   (1 - \frac{n}{2m}) ^ {n - 1}.
\end{aligned}
$$

Based on equation $\text{(3.12)}$, $e^x \ge 1 + x$,

$$
\begin{aligned}
p(n, m) & \le (e^{-n / 2m})^{n - 1} \\\\
        & =   e^{-n(n - 1) / 2m}.
\end{aligned}
$$


