# problem

question:
 Suppose we use a hash function $h$ to hash $n$ distinct keys into an array $T$ of length $m$. Assuming simple uniform hashing, what is the expected number of collisions? More precisely, what is the expected cardinality of $\\{\\{k, l\\}: k \ne l \text{ and } h(k) = h(l)\\}$?

answer:
Under the assumption of simple uniform hashing, we will use linearity of expectation to compute this.

Suppose that all the keys are totally ordered $\\{k_1, \dots, k_n\\}$. Let $X_i$ be the number of $\ell$'s such that $\ell > k_i$ and $h(\ell) = h(k_i)$. So $X_i$ is the (expected) number of times that key $k_i$ is collided by those keys hashed afterward. Note, that this is the same thing as $\sum_{j > i} \Pr(h(k_j) = h(k_i)) = \sum_{j > i} 1 / m = (n - i) / m$. Then, by linearity of expectation, the number of collisions is the sum of the number of collisions for each possible smallest element in the collision. The expected number of collisions is

$$\sum_{i = 1}^n \frac{n - i}{m} = \frac{n^2 - n(n + 1) / 2}{m} = \frac{n^2 - n}{2m}.$$


question:
 Demonstrate what happens when we insert the keys $5, 28, 19, 15, 20, 33, 12, 17, 10$ into a hash table with collisions resolved by chaining. Let the table have $9$ slots, and let the hash function be $h(k) = k \mod 9$.

answer:
Let us number our slots $0, 1, \dots, 8$.

Then our resulting hash table will look like following:

$$
\begin{array}{c|l}
h(k)     & \text{keys}      \\\\
\hline
0 \mod 9 &                  \\\\
1 \mod 9 & 10 \to 19 \to 28 \\\\
2 \mod 9 & 20               \\\\
3 \mod 9 & 12               \\\\
4 \mod 9 &                  \\\\
5 \mod 9 & 5                \\\\
6 \mod 9 & 33 \to 15        \\\\
7 \mod 9 &                  \\\\
8 \mod 9 & 17
\end{array}
$$


question:
 Professor Marley hypothesizes that he can obtain substantial performance gains by modifying the chaining scheme to keep each list in sorted order. How does the professor's modification affect the running time for successful searches, unsuccessful searches, insertions, and deletions?

answer:
- Successful searches: no difference, $\Theta(1 + \alpha)$.
- Unsuccessful searches: faster but still $\Theta(1 + \alpha)$.
- Insertions: same as successful searches, $\Theta(1 + \alpha)$.
- Deletions: same as before if we use doubly linked lists, $\Theta(1)$.


question:
 Suggest how to allocate and deallocate storage for elements within the hash table itself by linking all unused slots into a free list. Assume that one slot can store a flag and either one element plus a pointer or two pointers. All dictionary and free-list operations should run in $O(1)$ expected time. Does the free list need to be doubly linked, or does a singly linked free list suffice?

answer:
The flag in each slot of the hash table will be $1$ if the element contains a value, and $0$ if it is free. The free list must be doubly linked.

- Search is unmodified, so it has expected time $O(1)$.
- To insert an element $x$, first check if $T[h(x.key)]$ is free. If it is, delete $T[h(x.key)]$ and change the flag of $T[h(x.key)]$ to $1$. If it wasn't free to begin with, simply insert $x.key$ at the start of the list stored there.
- To delete, first check if $x.prev$ and $x.next$ are $\text{NIL}$. If they are, then the list will be empty upon deletion of $x$, so insert $T[h(x.key)]$ into the free list, update the flag of $T[h(x.key)]$ to $0$, and delete $x$ from the list it's stored in. Since deletion of an element from a singly linked list isn't $O(1)$, we must use a doubly linked list.
- All other operations are $O(1)$.


question:
 Suppose that we are storing a set of $n$ keys into a hash table of size $m$. Show that if the keys are drawn from a universe $U$ with $|U| > nm$, then $U$ has a subset of size $n$ consisting of keys that all hash to the same slot, so that the worst-case searching time for hashing with chaining is $\Theta(n)$.

answer:
Suppose the $m - 1$ slots contains at most $n - 1$ elements, then the remaining slot should have

$$|U| - (m - 1)(n - 1) > nm - (m - 1)(n - 1) = n + m - 1 \ge n$$

elements, thus $U$ has a subset of size $n$.


question:
 Suppose we have stored $n$ keys in a hash table of size $m$, with collisions resolved by chaining, and that we know the length of each chain, including the length $L$ of the longest chain. Describe a procedure that selects a key uniformly at random from among the keys in the hash table and returns it in expected time $O(L \cdot (1 + 1 / \alpha))$.

answer:
Choose one of the $m$ spots in the hash table at random. Let $n_k$ denote the number of elements stored at $T[k]$. Next pick a number $x$ from $1$ to $L$ uniformly at random. If $x < n_j$, then return the $x$th element on the list. Otherwise, repeat this process. Any element in the hash table will be selected with probability $1 / mL$, so we return any key with equal probability. Let $X$ be the random variable which counts the number of times we must repeat this process before we stop and $p$ be the probability that we return on a given attempt. Then $E[X] = p(1 + \alpha) + (1 − p)(1 + E[X])$ since we'd expect to take $1 + \alpha$ steps to reach an element on the list, and since we know how many elements are on each list, if the element doesn't exist we'll know right away. Then we have $E[X] = \alpha + 1 / p$. The probability of picking a particular element is $n / mL = \alpha / L$. Therefore, we have

$$
\begin{aligned}
E[X] & = \alpha + L / \alpha \\\\
     & = L(\alpha/L + 1/\alpha) \\\\
     & = O(L(1 + 1/\alpha))
\end{aligned}
$$

since $\alpha \le L$.


question:
 Consider inserting the keys $10, 22, 31, 4, 15, 28, 17, 88, 59$ into a hash table of length $m = 11$ using open addressing with the auxiliary hash function $h'(k) = k$. Illustrate the result of inserting these keys using linear probing, using quadratic probing with $c_1 = 1$ and $c_2 = 3$, and using double hashing with $h_1(k) = k$ and $h_2(k) = 1 + (k \mod (m - 1))$.

answer:
We use $T_t$ to represent each time stamp $t$ starting with $i = 0$, and if encountering a collision, then we iterate $i$ from $i = 1$ to $i = m - 1 = 10$ until there is no collision.

- **Linear probing**:

    $$
    \begin{array}{r|ccccccccc}
    h(k, i) = (k + i) \mod 11 & T_0 & T_1 & T_2 & T_3 & T_4 & T_5 & T_6 & T_7 & T_8 \\\\
    \hline
     0 \mod 11 &    & 22 & 22 & 22 & 22 & 22 & 22 & 22 & 22 \\\\
     1 \mod 11 &    &    &    &    &    &    &    & 88 & 88 \\\\
     2 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     3 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     4 \mod 11 &    &    &    &  4 &  4 &  4 &  4 &  4 &  4 \\\\
     5 \mod 11 &    &    &    &    & 15 & 15 & 15 & 15 & 15 \\\\
     6 \mod 11 &    &    &    &    &    & 28 & 28 & 28 & 28 \\\\
     7 \mod 11 &    &    &    &    &    &    & 17 & 17 & 17 \\\\
     8 \mod 11 &    &    &    &    &    &    &    &    & 59 \\\\
     9 \mod 11 &    &    & 31 & 31 & 31 & 31 & 31 & 31 & 31 \\\\
    10 \mod 11 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10
    \end{array}
    $$

- **Quadradic probing**, it will look identical until there is a collision on inserting the fifth element:

    $$
    \begin{array}{r|ccccccccc}
    h(k, i) = (k + i + 3i^2) \mod 11 & T_0 & T_1 & T_2 & T_3 & T_4 & T_5 & T_6 & T_7 & T_8 \\\\
    \hline
     0 \mod 11 &    & 22 & 22 & 22 & 22 & 22 & 22 & 22 & 22 \\\\
     1 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     2 \mod 11 &    &    &    &    &    &    &    & 88 & 88 \\\\
     3 \mod 11 &    &    &    &    &    &    & 17 & 17 & 17 \\\\
     4 \mod 11 &    &    &    &  4 &  4 &  4 &  4 &  4 &  4 \\\\
     5 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     6 \mod 11 &    &    &    &    &    & 28 & 28 & 28 & 28 \\\\
     7 \mod 11 &    &    &    &    &    &    &    &    & 59 \\\\
     8 \mod 11 &    &    &    &    & 15 & 15 & 15 & 15 & 15 \\\\
     9 \mod 11 &    &    & 31 & 31 & 31 & 31 & 31 & 31 & 31 \\\\
    10 \mod 11 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10
    \end{array}
    $$

Note that there is no way to insert the element $59$ now, because the offsets coming from $c_1 = 1$ and $c_2 = 3$ can only be even, and an odd offset would be required to insert $59$ because $59 \mod 11 = 4$ and all the empty positions are at odd indices.

- **Double hashing**:

    $$
    \begin{array}{r|ccccccccc}
    h(k, i) = (k + i(1 + k \mod 10)) \mod 11 & T_0 & T_1 & T_2 & T_3 & T_4 & T_5 & T_6 & T_7 & T_8 \\\\
    \hline
     0 \mod 11 &    & 22 & 22 & 22 & 22 & 22 & 22 & 22 & 22 \\\\
     1 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     2 \mod 11 &    &    &    &    &    &    &    &    & 59 \\\\
     3 \mod 11 &    &    &    &    &    &    & 17 & 17 & 17 \\\\
     4 \mod 11 &    &    &    &  4 &  4 &  4 &  4 &  4 &  4 \\\\
     5 \mod 11 &    &    &    &    & 15 & 15 & 15 & 15 & 15 \\\\
     6 \mod 11 &    &    &    &    &    & 28 & 28 & 28 & 28 \\\\
     7 \mod 11 &    &    &    &    &    &    &    & 88 & 88 \\\\
     8 \mod 11 &    &    &    &    &    &    &    &    &    \\\\
     9 \mod 11 &    &    & 31 & 31 & 31 & 31 & 31 & 31 & 31 \\\\
    10 \mod 11 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10 & 10
    \end{array}
    $$


question:
 Write pseudocode for $\text{HASH-DELETE}$ as outlined in the text, and modify $\text{HASH-INSERT}$ to handle the special value $\text{DELETED}$.

answer:
```cpp
HASH-DELETE(T, k)
    i = 0
    repeat
        j = h(k, i)
        if T[j] == k
            T[j] = DELETE
            return j
        else i = i + 1
    until T[j] == NIL or i == m
    error "element not exist"
```

By implementing $\text{HASH-DELETE}$ in this way, the $\text{HASH-INSERT}$ need to be modified to treat $\text{NIL}$ slots as empty ones.

```cpp
HASH-INSERT(T, k)
    i = 0
    repeat
        j = h(k, i)
        if T[j] == NIL or T[j] == DELETE
            T[j] = k
            return j
        else i = i + 1
    until i == m
    error "hash table overflow"
```


question:
 Consider an open-address hash table with uniform hashing. Give upper bounds on the expected number of probes in an unsuccessful search and on the expected number of probes in a successful search when the load factor is $3 / 4$ and when it is $7 / 8$.

answer:
- $\alpha = 3 / 4$,

    - unsuccessful: $\frac{1}{1 - \frac{3}{4}} = 4$ probes,
    - successful: $\frac{1}{\frac{3}{4}} \ln\frac{1}{1-\frac{3}{4}} \approx 1.848$ probes.

- $\alpha = 7 / 8$,
  
    - unsuccessful: $\frac{1}{1 - \frac{7}{8}} = 8$ probes,
    - successful: $\frac{1}{\frac{7}{8}} \ln\frac{1}{1 - \frac{7}{8}} \approx 2.377$ probes.


question:
 Suppose that we use double hashing to resolve collisions—that is, we use the hash function $h(k, i) = (h_1(k) + ih_2(k)) \mod m$. Show that if $m$ and $h_2(k)$ have greatest common divisor $d \ge 1$ for some key $k$, then an unsuccessful search for key $k$ examines $(1/d)$th of the hash table before returning to slot $h_1(k)$. Thus, when $d = 1$, so that $m$ and $h_2(k)$ are relatively prime, the search may examine the entire hash table. ($\textit{Hint:}$ See Chapter 31.)

answer:
Suppose $d = \gcd(m, h_2(k))$, the $\text{LCM}$ $l = m \cdot h_2(k) / d$.

Since $d | h_2(k)$, then $m \cdot h_2(k) / d \mod m = 0 \cdot (h_2(k) / d \mod m) = 0$, therefore $(l + ih_2(k)) \mod m = ih_2(k) \mod m$, which means $ih_2(k) \mod m$ has a period of $m / d$.


question:
 Consider an open-address hash table with a load factor $\alpha$. Find the nonzero value $\alpha$ for which the expected number of probes in an unsuccessful search equals twice the expected number of probes in a successful search. Use the upper bounds given by Theorems 11.6 and 11.8 for these expected numbers of probes.

answer:
$$
\begin{aligned}
\frac{1}{1 - \alpha} & = 2 \cdot \frac{1}{\alpha} \ln\frac{1}{1 - \alpha} \\\\
              \alpha & = 0.71533.
\end{aligned}
$$


question:
 Suppose we wish to search a linked list of length $n$, where each element contains a key $k$ along with a hash value $h(k)$. Each key is a long character string. How might we take advantage of the hash values when searching the list for an element with a given key?

answer:
If every element also contained a hash of the long character string, when we are searching for the desired element, we'll first check if the hashvalue of the node in the linked list, and move on if it disagrees. This can increase the runtime by a factor proportional to the length of the long character strings.


question:
 Suppose that we hash a string of $r$ characters into $m$ slots by treating it as a radix-128 number and then using the division method. We can easily represent the number $m$ as a 32-bit computer word, but the string of $r$ characters, treated as a radix-128 number, takes many words. How can we apply the division method to compute the hash value of the character string without using more than a constant number of words of storage outside the string itself?

answer:
```cpp
    sum = 0
    for i = 1 to r
        sum = (sum * 128 + s[i]) % m
```

Use `sum` as the key.


question:
 Consider a version of the division method in which $h(k) = k \mod m$, where $m = 2^p - 1$ and $k$ is a character string interpreted in radix $2^p$. Show that if we can derive string $x$ from string $y$ by permuting its characters, then $x$ and $y$ hash to the same value. Give an example of an application in which this property would be undesirable in a hash function.

answer:
We will show that each string hashes to the sum of it's digits $\mod 2^p − 1$. We will do this by induction on the length of the string.

- Base case

    Suppose the string is a single character, then the value of that character is the value of $k$ which is then taken $\mod m$.

- Inductive step.

    Let $w = w_1w_2$ where $|w_1| \ge 1$ and $|w_2| = 1$. Suppose $h(w_1) = k_1$. Then, $h(w) = h(w_1)2^p + h(w_2) \mod 2^p − 1 = h(w_1) + h(w_2) \mod 2^p − 1$. So, since $h(w_1)$ was the sum of all but the last digit $\mod m$, and we are adding the last digit $\mod m$, we have the desired conclusion.


question:
 Consider a hash table of size $m = 1000$ and a corresponding hash function $h(k) = \lfloor m (kA \mod 1) \rfloor$ for $A = (\sqrt 5 - 1) / 2$. Compute the locations to which the keys $61$, $62$, $63$, $64$, and $65$ are mapped.

answer:
- $h(61) = \lfloor 1000(61 \cdot \frac{\sqrt 5 - 1}{2} \mod 1) \rfloor = 700$.
- $h(62) = \lfloor 1000(62 \cdot \frac{\sqrt 5 - 1}{2} \mod 1) \rfloor = 318$.
- $h(63) = \lfloor 1000(63 \cdot \frac{\sqrt 5 - 1}{2} \mod 1) \rfloor = 936$.
- $h(64) = \lfloor 1000(64 \cdot \frac{\sqrt 5 - 1}{2} \mod 1) \rfloor = 554$.
- $h(65) = \lfloor 1000(65 \cdot \frac{\sqrt 5 - 1}{2} \mod 1) \rfloor = 172$.


question:
 Define a family $\mathcal H$ of hash functions from a finite set $U$ to a finite set $B$ to be **_$\epsilon$-universal_** if for all pairs of distinct elements $k$ and $l$ in $U$,
>
> $$\Pr\\{h(k) = h(l)\\} \le \epsilon,$$
>
> where the probability is over the choice of the hash function $h$ drawn at random from the family $\mathcal H$. Show that an $\epsilon$-universal family of hash functions must have
>
> $$\epsilon \ge \frac{1}{|B|} - \frac{1}{|U|}.$$

answer:
As a simplifying assumption, assume that $|B|$ divides $|U|$. It's just a bit messier if it doesn't divide evenly.

Suppose to a contradiction that $\epsilon > \frac{1}{|B|} - \frac{1}{|U|}$. This means that $\forall$ pairs $k, \ell \in U$, we have that the number $n_{k, \ell}$ of hash functions in $\mathcal H$ that have a collision on those two elements satisfies $n_{k, \ell} \le \frac{|\mathcal H}{|B|} - \frac{|\mathcal H}{|U|}$. So, summing over all pairs of elements in $U$, we have that the total number is $\le \frac{|\mathcal H||U|^2}{2|B|} - \frac{|\mathcal H||U|}{2}$.

Any particular hash function must have that there are at least $|B|\binom{|U| / |B|}{2} = |B|\frac{|U|^2 - |U||B|}{2|B|^2} = \frac{|U|^2}{2|B|} - \frac{|U|}{2}$ colliding pairs for that hash function, summing over all hash functions, we get that there are at least $|\mathcal H| \left(\frac{|U|^2}{2|B|} - \frac{|U|}{2}\right)$ colliding pairs total. Since we have that there are at most some number less than this many, we have a contradiction, and so must have the desired restriction on $\epsilon$.


question:
 Let $U$ be the set of $n$-tuples of values drawn from $\mathbb Z_p$, and let $B = \mathbb Z_p$, where $p$ is prime. Define the hash function $h_b: U \rightarrow B$ for $b \in \mathbb Z_p$ on an input $n$-tuple $\langle a_0, a_1, \ldots, a_{n - 1} \rangle$ from $U$ as
>
> $$h_b(\langle a_0, a_1, \ldots, a_{n - 1} \rangle) = \Bigg(\sum_{j = 0}^{n - 1} a_jb^j \Bigg) \mod p,$$
>
> and let $\mathcal{H} = \\{h_b : b \in \mathbb Z_p\\}$. Argue that $\mathcal H$ is $((n - 1) / p)$-universal according to the definition of $\epsilon$-universal in Exercise 11.3-5. ($\textit{Hint:}$ See Exercise 31.4-4.)

answer:
Fix $b \in \mathbb Z_p$. By exercise 31.4-4, $h_b(x)$ collides with $h_b(y)$ for at most $n - 1$ other $y \in U$. Since there are a total of $p$ possible values that $h_b$ takes on, the
probability that $h_b(x) = h_b(y)$ is bounded from above by $\frac{n - 1}{p}$, since this holds for any value of $b$, $\mathcal H$ is $((n - 1 ) /p)$-universal.


question:
 Suppose that a dynamic set $S$ is represented by a direct-address table $T$ of length $m$. Describe a procedure that finds the maximum element of $S$. What is the worst-case performance of your procedure?

answer:
As the dynamic set $S$ is represented by the direct-address table $T$, for each key $k$ in $S$, there is a slot $k$ in $T$ points to it. If no element with key $k$ in $S$, then $T[k] = \text{NIL}$. Using this property, we can find the maximum element of $S$ by traversing down from the highest slot to seek the first non-$\text{NIL}$ one.

```cpp
MAXIMUM(S)
    return TABLE-MAXIMUM(T, m - 1)
```

```cpp
TABLE-MAXIMUM(T, l)
    if l < 0
        return NIL
    else if DIRECT-ADDRESS-SEARCH(T, l) != NIL
        return l
    else return TABLE-MAXIMUM(T, l - 1)
```

The $\text{TABLE-MAXIMUM}$ procedure gest down and checks $1$ sloc at a time, linearly approaches the solution. In the worst case where $S$ is empty, $\text{TABLE-MAXIMUM}$ examines $m$ slots. Therefore, the worst-case performance of $\text{MAXIMUM}$ is $O(m)$, where $m$ is the length of the direct-address table $T$.


question:
 A **_bit vector_** is simply an array of bits ($0$s and $1$s). A bit vector of length $m$ takes much less space than an array of $m$ pointers. Describe how to use a bit vector to represent a dynamic set of distinct elements with no satellite data. Dictionary operations should run in $O(1)$ time.

answer:
Using the bit vector data structure, we can represent keys less than $m$ by a string of $m$ bits, denoted by $V[0..m - 1]$, in which each position that occupied by the bit $1$, corresponds to a key in the set $S$. If the set contains no element with key $k$, then $V[k] = 0$. For instance, we can store the set $\\{2, 4, 6, 10, 16\\}$ in a bit vector of length $20$:

0-indexed, ordered from left to right:

$$00101010001000001000$$

```cpp
BITMAP-SEARCH(V, k)
    if V[k] != 0
        return k
    else return NIL
```

```cpp
BITMAP-INSERT(V, x)
    V[x] = 1
```

```cpp
BITMAP-DELETE(V, x)
    V[x] = 0
```

Each of these operations takes only $O(1)$ time.


question:
 Suggest how to implement a direct-address table in which the keys of stored elements do not need to be distinct and the elements can have satellite data. All three dictionary operations ($\text{INSERT}$, $\text{DELETE}$, and $\text{SEARCH}$) should run in $O(1)$ time. (Don't forget that $\text{DELETE}$ takes as an argument a pointer to an object to be deleted, not a key.)

answer:
Assuming that fetching an element should return the satellite data of all the stored elements, we can have each key map to a doubly linked list.

- $\text{INSERT}$: appends the element to the list in constant time
- $\text{DELETE}$: removes the element from the linked list in constant time (the element contains pointers to the previous and next element)
- $\text{SEARCH}$: returns the first element, which is a node in a linked list, in constant time


question:
 We wish to implement a dictionary by using direct addressing on a _huge_ array. At the start, the array entries may contain garbage, and initializing the entire array is impractical because of its size. Describe a scheme for implementing a direct-address dictionary on a huge array. Each stored object should use $O(1)$ space; the operations $\text{SEARCH}$, $\text{INSERT}$, and $\text{DELETE}$ should take $O(1)$ time each; and initializing the data structure should take $O(1)$ time. ($\textit{Hint:}$ Use an additional array, treated somewhat like a stack whose size is the number of keys actually stored in the dictionary, to help determine whether a given entry in the huge array is valid or not.)

answer:
The additional data structure will be a stack $S$.

Initially, set $S$ to be empty, and do nothing to initialize the huge array. Each object stored in the huge array will have two parts: the key value, and a pointer to an element of $S$, which contains a pointer back to the object in the huge array.

- To insert $x$, push an element $y$ to the stack which contains a pointer to position $x$ in the huge array. Update position $A[x]$ in the huge array $A$ to contain a pointer to $y$ in $S$.

- To search for $x$, go to position $x$ of $A$ and go to the location stored there. If that location is an element of $S$ which contains a pointer to $A[x]$, then we know $x$ is in $A$. Otherwise, $x \notin A$.

- To delete $x$, invalidate the element of $S$ which is pointed to by $A[x]$. Because there may be "holes" in $S$ now, we need to pop an item from $S$, move it to the position of the "hole", and update the pointer in $A$ accordingly. Each of these takes $O(1)$ time and there are at most as many elements in $S$ as there are valid elements in $A$.


question:
 Professor Williams comes up with a scheme that allows the closest-pair algorithm to check only $5$ points following each point in array $Y'$. The idea is always to place points on line $l$ into set $P_L$. Then, there cannot be pairs of coincident points on line $l$ with one point in $P_L$ and one in $P_R$. Thus, at most $6$ points can reside in the $\delta \times 2\delta$ rectangle. What is the flaw in the professor's scheme?

answer:
In particular, when we select line $l$, we may be unable perform an even split of the vertices. So, we don't neccesarily have that both the left set of points and right set of points have fallen to roughly half. For example, suppose that the points are all arranged on a vertical line, then, when we recurse on the the left set of points, we haven't reduced the problem size at all, let alone by a factor of two. There is also the issue in this setup that you may end up asking about a set of size less than two when looking at the right set of points.


question:
 Show that it actually suffices to check only the points in the $5$ array positions following each point in the array $Y'$.

answer:
Since we only care about the shortest distance, the distance $\delta'$ must be strictly less than $\delta$. The picture in Figure 33.11(b) only illustrates the case of a nonstrict inequality. If we exclude the possibility of points whose $x$ coordinate differs by exactly $\delta$ from $l$, then it is only possible to place at most $6$ points in the $\delta \times 2\delta$ rectangle, so it suffices to check on the points in the $5$ array positions following each point in the array $Y'$.


question:
 We can define the distance between two points in ways other than euclidean. In the plane, the **_$L_m$-distance_** between points $p_1$ and $p_2$ is given by the expression $(|x_1 - x_2|^m + |y_1 - y_2|^m)^{1 / m}$. Euclidean distance, therefore, is $L_2$-distance. Modify the closest-pair algorithm to use the $L_1$-distance, which is also known as the **_Manhattan distance_**.

answer:
In the analysis of the algorithm, most of it goes through just based on the triangle inequality. The only main point of difference is in looking at the number of points that can be fit into a $\delta \times 2\delta$ rectangle. In particular, we can cram in two more points than the eight shown into the rectangle by placing points at the centers of the two squares that the rectangle breaks into. This means that we need to consider points up to $9$ away in $Y'$ instead of $7$ away. This has no impact on the asymptotics of the algorithm and it is the only correction to the algorithm that is needed if we switch from $L_2$ to $L_1$.


question:
 Given two points $p_1$ and $p_2$ in the plane, the $L_\infty$-distance between them is given by $\max(|x_1 - x_2|, |y_1 - y_2|)$. Modify the closest-pair algorithm to use the $L_\infty$-distance.

answer:
We can simply run the divide and conquer algorithm described in the section, modifying the brute force search for $|P| \le 3$ and the check against the next $7$ points in $Y'$ to use the $L_\infty$ distance. Since the $L_\infty$ distance between two points is always less than the euclidean distance, there can be at most $8$ points in the $\delta \times 2\delta$ rectangle which we need to examine in order to determine whether the closest pair is in that box. Thus, the modified algorithm is still correct and has the same runtime.


question:
 Suppose that $\Omega(n)$ of the points given to the closest-pair algorithm are covertical. Show how to determine the sets $P_L$ and $P_R$ and how to determine whether each point of $Y$ is in $P_L$ or $P_R$ so that the running time for the closest-pair algorithm remains $O(n\lg n)$.

answer:
We select the line $l$ so that it is roughly equal, and then, we won't run into any issue if we just pick an arbitrary subset of the vertices that are on the line to go to one side or the other.

Since the analysis of the algorithm allowed for both elements from $P_L$ and $P_R$ to be on the line, we still have correctness if we do this. To determine what values of $Y$ belong to which of the set can be made easier if we select our set going to $P_L$ to be the lowest however many points are needed, and the $P_R$ to be the higher points. Then, just knowing the index of $Y$ that we are looking at, we know whether that point belonged to $P_L$ or to $P_R$.


question:
 Suggest a change to the closest-pair algorithm that avoids presorting the $Y$ array but leaves the running time as $O(n\lg n)$. ($\textit{Hint:}$ Merge sorted arrays $Y_L$ and $Y_R$ to form the sorted array $Y$.)

answer:
In addition to returning the distance of the closest pair, the modify the algorithm to also return the points passed to it, sorted by $y$-coordinate, as $Y$. To do this, merge $Y_L$ and $Y_R$ returned by each of its recursive calls. If we are at the base case, when $n \le 3$, simply use insertion sort to sort the elements by y-coordinate directly. Since each merge takes linear time, this doesn't affect the recursive equation for the runtime.


answer: (Omit!)
question:
 Given two segments $a$ and $b$ that are comparable at $x$, show how to determine in $O(1)$ time which of $a \succeq_x b$ or $b \succeq_x a$ holds. Assume that neither segment is vertical.


answer: No.
question:
 Professor Mason suggests that we modify $\text{ANY-SEGMENTS-INTERSECT}$ so that instead of returning upon finding an intersection, it prints the segments that intersect and continues on to the next iteration of the **for** loop. The professor calls the resulting procedure $\text{PRINT-INTERSECTING-SEGMENTS}$ and claims that it prints all intersections, from left to right, as they occur in the set of line segments. Professor Dixon disagrees, claiming that Professor Mason's idea is incorrect. Which professor is right? Will $\text{PRINT-INTERSECTING-SEGMENTS}$ always find the leftmost intersection first? Will it always find all the intersections?


answer: Same as $\text{ANY-SEGMENTS-INTERSECT}$.
question:
 Give an $O(n\lg n)$-time algorithm to determine whether two simple polygons with a total of $n$ vertices intersect.


answer: Same as $\text{ANY-SEGMENTS-INTERSECT}$.
question:
 A **_disk_** consists of a circle plus its interior and is represented by its center point and radius. Two disks intersect if they have any point in common. Give an $O(n\lg n)$- time algorithm to determine whether any two disks in a set of $n$ intersect.


answer: Treat the intersection points as event points.
question:
 Given a set of $n$ line segments containing a total of $k$ intersections, show how to output all $k$ intersections in $O((n + k) \lg)$ time.


answer: (Omit!)
question:
 Argue that $\text{ANY-SEGMENTS-INTERSECT}$ works correctly even if three or more segments intersect at the same point.


answer: (Omit!)
question:
 Show that $\text{ANY-SEGMENTS-INTERSECT}$ works correctly in the presence of vertical segments if we treat the bottom endpoint of a vertical segment as if it were a left endpoint and the top endpoint as if it were a right endpoint. How does your answer to Exercise 33.2-2 change if we allow vertical segments?


answer: (Omit!)
question:
 Prove that if $p_1 \times p_2$ is positive, then vector $p_1$ is clockwise from vector $p_2$ with respect to the origin $(0, 0)$ and that if this cross product is negative, then $p_1$ is counterclockwise from $p_2$.


answer: $(0, 0), (5, 5), (10, 0)$.
question:
 Professor van Pelt proposes that only the $x$-dimension needs to be tested in line 1 of ON-SEGMENT. Show why the professor is wrong.


answer: (Omit!)
question:
 The **_polar angle_** of a point $p_1$ with respect to an origin point $p_0$ is the angle of the vector $p_1 - p_0$ in the usual polar coordinate system. For example, the polar angle of $(3, 5)$ with respect to $(2, 4)$ is the angle of the vector $(1, 1)$, which is $45$ degrees or $\pi / 4$ radians. The polar angle of $(3, 3)$ with respect to $(2, 4)$ is the angle of the vector $(1, 1)$, which is $315$ degrees or $7\pi / 4$ radians. Write pseudocode to sort a sequence $\langle p_1, p_2, \ldots, p_n \rangle$ of $n$ points according to their polar angles with respect to a given origin point $p_0$. Your procedure should take $O(n\lg n)$ time and use cross products to compare angles.


question:
 Show how to determine in $O(n^2 \lg n)$ time whether any three points in a set of $n$ points are colinear.

answer:
Based on exercise 33.1-3, for each point $p_i$, let $p_i$ be $p_0$ and sort other points according to their polar angles mod $\pi$. Then scan linearly to see whether two points have the same polar angle. $O(n \cdot n\lg n) = O(n^2 \lg n)$.


question:
 A **_polygon_** is a piecewise-linear, closed curve in the plane. That is, it is a curve ending on itself that is formed by a sequence of straight-line segments, called the **_sides_** of the polygon. A point joining two consecutive sides is a **_vertex_** of the polygon. If the polygon is **_simple_**, as we shall generally assume, it does not cross itself. The set of points in the plane enclosed by a simple polygon forms the **_interior_** of the polygon, the set of points on the polygon itself forms its **_boundary_**, and the set of points surrounding the polygon forms its **_exterior_**. A simple polygon is convex if, given any two points on its boundary or in its interior, all points on the line segment drawn between them are contained in the polygon's boundary or interior. A vertex of a convex polygon cannot be expressed as a convex combination of any two distinct points on the boundary or in the interior of the polygon.

answer:
> Professor Amundsen proposes the following method to determine whether a sequence $\langle p_1, p_2, \ldots, p_{n - 1} \rangle$ of $n$ points forms the consecutive vertices of a convex polygon. Output "yes" if the set ${ \angle p_i p_{i + 1} p_{i + 2}: i = 0, 1, \ldots, n - 1 }$, where subscript addition is performed modulo $n$, does not contain both left turns and right turns; otherwise, output "no." Show that although this method runs in linear time, it does not always produce the correct answer. Modify the professor's method so that it always produces the correct answer in linear time.

A line.


question:
 Given a point $p_0 = (x_0, y_0)$, the **_right horizontal ray_** from $p_0$ is the set of points ${ p_i = (x_i, y_i) : x_i \ge x_0 ~\text{and}~ y_i = y_0 }$, that is, it is the set of points due right of $p_0$ along with $p_0$ itself. Show how to determine whether a given right horizontal ray from $p_0$ intersects a line segment $\overline{p_1 p_2}$ in $O(1)$ time by reducing the problem to that of determining whether two line segments intersect.

answer:
$p_1.y = p_2.y = 0$ and $\max(p_1.x, p_2.x) \ge 0$.

or

$\text{sign}(p_1.y) \ne \text{sign}(p_2.y)$ and \$\displaystyle p_1.y \cdot \frac{p_1.x - p_2.x}{p_1.y - p_2.y} \ge 0{equation}


answer: Based on exercise 33.1-6, use $p_i - p_0$ as $p_i$.
question:
 One way to determine whether a point $p_0$ is in the interior of a simple, but not necessarily convex, polygon $P$ is to look at any ray from $p_0$ and check that the ray intersects the boundary of $P$ an odd number of times but that $p_0$ itself is not on the boundary of $P$. Show how to compute in $\Theta(n)$ time whether a point $p_0$ is in the interior of an $n$-vertex polygon $P$. (Hint: Use Exercise 33.1-6. Make sure your algorithm is correct when the ray intersects the polygon boundary at a vertex and when the ray overlaps a side of the polygon.)


question:
 Show how to compute the area of an $n$-vertex simple, but not necessarily convex, polygon in $\Theta(n)$ time. (See Exercise 33.1-5 for definitions pertaining to polygons.)

answer:
Half of the sum of the cross products of ${\overline{p_1 p_i}, \overline{p_1 p_{i + 1}} ~|~ i \in [2, n - 1] }$.


question:
 Prove that in the procedure $\text{GRAHAM-SCAN}$, points $p_1$ and $p_m$ must be vertices of $\text{CH}(Q)$.

answer:
To see this, note that $p_1$ and $p_m$ are the points with the lowest and highest polar angle with respect to $p_0$. By symmetry, we may just show it for $p_1$ and we would also have it for $p_m$ just by reflecting the set of points across a vertical line.

To see a contradiction, suppose that we have the convex hull doesn't contain $p_1$. Then, let $p$ be the point in the convex hull that has the lowest polar angle with respect to $p_0$. If $p$ is on the line from $p_0$ to $p_1$, we could replace it with $p_1$ and have a convex hull, meaning we didn't start with a convex hull.

If we have that it is not on that line, then there is no way that the convex hull given contains $p_1$, also contradicting the fact that we had selected a convex hull


question:
 Consider a model of computation that supports addition, comparison, and multiplication and for which there is a lower bound of $\Omega(n\lg n)$ to sort $n$ numbers. Prove that $\Omega(n\lg n)$ is a lower bound for computing, in order, the vertices of the convex hull of a set of $n$ points in such a model.

answer:
Let our $n$ numbers be $a_1, a_2, \dots, a_n$ and $f$ be a strictly convex function, such as $e^x$. Let $p_i = (a_i, f(a_i))$. Compute the convex hull of $p_1, p_2, \dots, p_n$. Then every point is in the convex hull. We can recover the numbers themselves by looking at the $x$-coordinates of the points in the order returned by the convex-hull algorithm, which will necessarily be a cyclic shift of the numbers in increasing order, so we can recover the proper order in linear time.

In an algorithm such as $\text{GRAHAM-SCAN}$ which starts with the point with minimum $y$-coordinate, the order returned actually gives the numbers in increasing order.


question:
 Given a set of points $Q$, prove that the pair of points farthest from each other must be vertices of $\text{CH}(Q)$.

answer:
Suppose that $p$ and $q$ are the two furthest apart points. Also, to a contradiction, suppose, without loss of generality that $p$ is on the interior of the convex hull. Then, construct the circle whose center is $q$ and which has $p$ on the circle. Then, if we have that there are any vertices of the convex hull that are outside this circle, we could pick that vertex and $q$, they would have a higher distance than between $p$ and $q$. So, we know that all of the vertices of the convex hull lie inside the circle. This means that the sides of the convex hull consist of line segments that are contained within the circle. So, the only way that they could contain $p$, a point on the circle is if it was a vertex, but we suppsed that $p$ wasn't a vertex of the convex hull, giving us our contradiction.


question:
 For a given polygon $P$ and a point $q$ on its boundary, the **_shadow_** of $q$ is the set of points $r$ such that the segment $\overline{qr}$ is entirely on the boundary or in the interior of $P$. As Figure 33.10 illustrates, a polygon $P$ is **_star-shaped_** if there exists a point $p$ in the interior of $P$ that is in the shadow of every point on the boundary of $P$. The set of all such points $p$ is called the **_kernel_** of $P$. Given an $n$-vertex, star-shaped polygon $P$ specified by its vertices in counterclockwise order, show how to compute $\text{CH}(P)$ in $O(n)$ time.

answer:
We simply run $\text{GRAHAM-SCAN}$ but without sorting the points, so the runtime becomes $O(n)$. To prove this, we'll prove the following loop invariant: At the start of each iteration of the for loop of lines 7-10, stack $S$ consists of, from bottom to top, exactly the vertices of $\text{CH}(Q_{i - 1})$. The proof is quite similar to the proof of correctness. The invariant holds the first time we execute line 7 for the same reasons outline in the section. At the start of the $i$th iteration, $S$ contains $\text{CH}(Q_{i - 1})$. Let $p_j$ be the top point on $S$ after executing the while loop of lines 8-9, but before $p_i$ is pushed, and let pk be the point just below $p_j$ on $S$. At this point, $S$ contains $\text{CH}(Q_j)$ in counterclockwise order from bottom to top. Thus, when we push $p_i$, $S$ contains exactly the vertices of $\text{CH}(Q_j \cup \\{p_i\\})$.

We now show that this is the same set of points as $\text{CH}(Q_i)$. Let $p_t$ be any point that was popped from $S$ during iteration $i$ and $p_r$ be the point just below $p_t$ on stack $S$ at the time $p_t$ was popped. Let $p$ be a point in the kernel of $P$. Since the angle $\angle p_rp_tp_i$ makes a nonelft turn and $P$ is star shaped, $p_t$ must be in the interior or on the boundary of the triangle formed by $p_r$, $p_i$, and $p$. Thus, $p_t$ is not in the convex hull of $Q_i$, so we have $\text{CH}(Q_i - \\{p_t\\}) = \text{CH}(Q_i)$. Applying this equality repeatedly for each point removed from $S$ in the while loop of lines 8-9, we have $\text{CH}(Q_j \cup \\{p_i\\}) = \text{CH}(Q_i)$.

When the loop terminates, the loop invariant implies that $S$ consists of exactly the vertices of $\text{CH}(Q_m)$ in counterclockwise order, proving correctness.


question:
 In the **_on-line convex-hull problem_**, we are given the set $Q$ of $n$ points one point at a time. After receiving each point, we compute the convex hull of the points seen so far. Obviously, we could run Graham's scan once for each point, with a total running time of $O(n^2\lg n)$. Show how to solve the on-line convex-hull problem in a total of $O(n^2)$ time.

answer:
Suppose that we have a convex hull computed from the previous stage $\\{q_0, q_1, \dots, q_m\\}$, and we want to add a new vertex, $p$ in and keep track of how we should change the convex hull.

First, process the vertices in a clockwise manner, and look for the first time that we would have to make a non-left to get to $p$. This tells us where to start cutting vertices out of the convex hull. To find out the upper bound on the vertices that we need to cut out, turn around, start processing vertices in a clockwise manner and see the first time that we would need to make a non-right.

Then, we just remove the vertices that are in this set of vertices and replace the with $p$. There is one last case to consider, which is when we end up passing ourselves when we do our clockwise sweep.

Then we just remove no vertices and add $p$ in in between the two vertices that we had found in the two sweeps. Since for each vertex we add we are only considering each point in the previous step's convex hull twice, the runtime is $O(nh) = O(n^2)$ where h is the number of points in the convex hull.


answer: (Omit!)
question:
 Show how to implement the incremental method for computing the convex hull of $n$ points so that it runs in $O(n\lg n)$ time.


answer: (Omit!)
question:
 Draw the computation dag for computing $\text{P-SQUARE-MATRIX-MULTIPLY}$ on $2 \times 2$ matrices, labeling how the vertices in your diagram correspond to strands in the execution of the algorithm. Use the convention that spawn and call edges point downward, continuation edges point horizontally to the right, and return edges point upward. Assuming that each strand takes unit time, analyze the work, span, and parallelism of this computation.


answer: (Removed)
question:
 Give pseudocode for a multithreaded algorithm that multiplies two $n \times n$ matrices with work $\Theta(n^3)$ but span only $\Theta(\lg n)$. Analyze your algorithm.


answer: (Removed)
question:
 Give pseudocode for an efficient multithreaded algorithm that multiplies a $p \times q$ matrix by a $q \times r$ matrix. Your algorithm should be highly parallel even if any of $p$, $q$, and $r$ are $1$. Analyze your algorithm.


question:
 Give pseudocode for an efficient multithreaded algorithm that transposes an $n \times n$ matrix in place by using divide-and-conquer to divide the matrix recursively into four $n / 2 \times n / 2$ submatrices. Analyze your algorithm.

answer:
```cpp
P-MATRIX-TRANSPOSE(A)
    n = A.rows
    if n == 1
        return
    partition A into n / 2 ✕ n / 2 submatrices A11, A12, A21, A22
    spawn P-MATRIX-TRANSPOSE(A11)
    spawn P-MATRIX-TRANSPOSE(A12)
    spawn P-MATRIX-TRANSPOSE(A21)
    P-MATRIX-TRANSPOSE(A22)
    sync
    // exchange A12 with A21
    parallel for i = 1 to n / 2
        parallel for j = 1 + n / 2 to n
            exchange A[i, j] with A[i + n / 2, j - n / 2]
```

- span: $T(n) = T(n / 2) + O(\lg n) = O(\lg^2 n)$.
- work: $T(n) = 4T(n / 2) + O(n^2) = O(n^2\lg n)$.


answer: (Removed)
question:
 Give pseudocode for an efficient multithreaded implementation of the Floyd-Warshall algorithm (see Section 25.2), which computes shortest paths between all pairs of vertices in an edge-weighted graph. Analyze your algorithm.


answer: (Removed)
question:
 Suppose that we spawn $\text{P-FIB}(n - 2)$ in line 4 of $\text{P-FIB}$, rather than calling it as is done in the code. What is the impact on the asymptotic work, span, and parallelism?


answer: - Work: $T_1 = 29$. - Span: $T_\infty = 10$. - Parallelism: $T_1 / T_\infty \approx 2.9$.
question:
 Draw the computation dag that results from executing $\text{P-FIB}(5)$. Assuming that each strand in the computation takes unit time, what are the work, span, and parallelism of the computation? Show how to schedule the dag on 3 processors using greedy scheduling by labeling each strand with the time step in which it is executed.


question:
 Prove that a greedy scheduler achieves the following time bound, which is slightly stronger than the bound proven in Theorem 27.1:
>
> $$T_P \le \frac{T_1 - T_\infty}{P} + T_\infty. \tag{27.5}$$

answer:
Suppose that there are x incomplete steps in a run of the program. Since each of these steps causes at least one unit of work to be done, we have that there is at most $(T_1 - x)$ units of work done in the complete steps. Then, we suppose by contradiction that the number of complete steps is strictly greater than $\lfloor (T_1 - x) / P \rfloor$. Then, we have that the total amount of work done during the complete steps is

$$P \cdot (\lfloor (T_1 - x) / P \rfloor + 1) = P \lfloor (T_1 - x) / P = (T_1 - x) - ((T_1 - x) \mod P) + P > T_1 - x.$$

This is a contradiction because there are only $(T_1 - x)$ units of work done during complete steps, which is less than the amount we would be doing. Notice that since $T_\infty$ is abound on the total number of both kinds of steps, it is a bound on the number of incomplete steps, $x$, so,

$$T_P \le \lfloor (T_1 - x) / P \rfloor + x \le \lfloor (T_1 - T_\infty) / P \rfloor + T_\infty.$$

Where the second inequality comes by noting that the middle expression, as a function of $x$ is monotonically increasing, and so is bounded by the largest value of $x$ that is possible, namely $T_\infty$.


question:
 Construct a computation dag for which one execution of a greedy scheduler can take nearly twice the time of another execution of a greedy scheduler on the same number of processors. Describe how the two executions would proceed.

answer:
The computation is given in the image below. Let vertex $u$ have degree $k$, and assume that there are $m$ vertices in each vertical chain. Assume that this is executed on $k$ processors. In one execution, each strand from among the $k$ on the left is executed concurrently, and then the $m$ strands on the right are executed one at a time. If each strand takes unit time to execute, then the total computation takes $2m$ time. On the other hand, suppose that on each time step of the computation, $k - 1$ strands from the left (descendants of $u$) are executed, and one from the right (a descendant of $v$), is executed. If each strand take unit time to executed, the total computation takes $m + m / k$. Thus, the ratio of times is $2m / (m + m / k) = 2 / (1 + 1 / k)$. As $k$ gets large, this approaches $2$ as desired.


answer: (Removed)
question:
 Professor Karan measures her deterministic multithreaded algorithm on $4$, $10$, and $64$ processors of an ideal parallel computer using a greedy scheduler. She claims that the three runs yielded $T_4 = 80$ seconds, $T_{10} = 42$ seconds, and $T_{64} = 10$ seconds. Argue that the professor is either lying or incompetent. ($\textit{Hint:}$ Use the work law $\text{(27.2)}$, the span law $\text{(27.3)}$, and inequality $\text{(27.5)}$ from Exercise 27.1-3.)


answer: (Removed)
question:
 Give a multithreaded algorithm to multiply an $n \times n$ matrix by an $n$-vector that achieves $\Theta(n^2 / \lg n)$ parallelism while maintaining $\Theta(n^2)$ work.


answer: (Removed)
question:
 Consider the following multithreaded pseudocode for transposing an $n \times n$ matrix $A$ in place:
>
> ```cpp
> P-TRANSPOSE(A)
>     n = A.rows
>     parallel for j = 2 to n
>         parallel for i = 1 to j - 1
>             exchange a[i, j] with a[j, i]
> ```
>
> Analyze the work, span, and parallelism of this algorithm.


answer: (Removed)
question:
 Suppose that we replace the **parallel for** loop in line 3 of $\text{P-TRANSPOSE}$ (see Exercise 27.1-7) with an ordinary **for** loop. Analyze the work, span, and parallelism of the resulting algorithm.


answer: (Removed)
question:
 For how many processors do the two versions of the chess programs run equally fast, assuming that $T_P = T_1 / P + T_\infty$?


question: Explain how to coarsen the base case of $\text{P-MERGE}$.
answer:
Replace the condition on line 2 with a check that $n < k$ for some base case size $k$. And instead of just copying over the particular element of $A$ to the right spot in $B$, you would call a serial sort on the remaining segment of $A$ and copy the result of that over into the right spots in $B$.


question:
 Instead of finding a median element in the larger subarray, as $\text{P-MERGE}$ does, consider a variant that finds a median element of all the elements in the two sorted subarrays using the result of Exercise 9.3-8. Give pseudocode for an efficient multithreaded merging procedure that uses this median-finding procedure. Analyze your algorithm.

answer:
By a slight modification of exercise 9.3-8 we can find we can find the median of all elements in two sorted arrays of total length $n$ in $O(\lg n)$ time. We'll modify $\text{P-MERGE}$ to use this fact. Let $\text{MEDIAN}(T, p_1, r_1, p_2, r_2)$ be the function which returns a pair, $q$, where $q.pos$ is the position of the median of all the elements $T$ which lie between positions $p_1$ and $r_1$, and between positions $p_2$ and $r_2$, and $q.arr$ is $1$ if the position is between $p_1$ and $r_1$, and $2$ otherwise.

```cpp
P-MEDIAN-MERGE(T, p[1], r[1], p[2], r[2], A, p[3])
    n[1] = r[1] - p[1] + 1
    n[2] = r[2] - p[2] + 1
    if n[1] < n[2]              // ensure that n[1] ≥ n[2]
        exchange p[1] with p[2]
        exchange r[1] with r[2]
        exchange n[1] with n[2]
    if n[1] == 0              // both empty?
        return
    q = MEDIAN(T, p[1], r[1], p[2], r[2])
    if q.arr == 1
        q[2] = BINARY-SEARCH(T[q.pos], T, p[2], r[2])
        q[3] = p[3] + q.pos - p[1] + q[2] - p[2]
        A[q[3]] = T[q.pos]
        spawn P-MEDIAN-MERGE(T, p[1], q.pos - 1, p[2], q[2] - 1, A, p[3])
        P-MEDIAN-MERGE(T, q.pos + 1, r[1], q[2] + 1, r[2], A, p[3])
        sync
    else
        q[2] = BINARY-SEARCH(T[q.pos], T, p[1], r[1])
        q[3] = p[3] + q.pos - p[2] + q[2] - p[1]
        A[q[3]] = T[q.pos]
        spawn P-MEDIAN-MERGE(T, p[1], q[2] - 1, p[2], q.pos - 1, A, p[3])
        P-MEDIAN-MERGE(T, q[2] + 1, r[1], q.pos + 1, r[2], A, p[3])
        sync
```

The work is characterized by the recurrence $T_1(n) = O(\lg n) + 2T_1(n / 2)$, whose solution tells us that $T_1(n) = O(n)$. The work is at least $\Omega(n)$ since we need to examine each element, so the work is $\Theta(n)$. The span satisfies the recurrence

$$
\begin{aligned}
T_\infty(n) & = O(\lg n) + O(\lg n / 2) + T_\infty(n / 2) \\\\
            & = O(\lg n) + T_\infty(n / 2) \\\\
            & = \Theta(\lg^2 n),
\end{aligned}
$$

by exercise 4.6-2.


question:
 Give an efficient multithreaded algorithm for partitioning an array around a pivot, as is done by the $\text{PARTITION}$ procedure on page 171. You need not partition the array in place. Make your algorithm as parallel as possible. Analyze your algorithm. ($\textit{Hint:}$ You may need an auxiliary array and may need to make more than one pass over the input elements.)

answer:
Suppose that there are $c$ different processors, and the array has length $n$
and you are going to use its last element as a pivot. Then, look at each chunk
of size $\lceil \frac{n}{c} \rceil$ of entries before the last element, give one to each processor. Then, each counts the number of elements that are less than the pivot. Then, we compute all the running sums of these values that are returned. This can be done easily by considering all of the subarrays placed along the leaves of a binary tree, and then summing up adjacent pairs. This computation can be done in time $\lg(\min\\{c, n\\})$ since it's the log of the number of leaves. From there, we can compute all the running sums for each of the subarrays also in logarithmic time. This is by keeping track of the sum of all more left cousins of each internal node, which is found by adding the left sibling's sum vale to the left cousin value of the parent, with the root's left cousin value initiated to $0$. This also just takes time the depth of the tree, so is $\lg(\min\\{c, n\\})$. Once all of these values are computed at the root, it is the index that the subarray's elements less than the pivot should be put. To find the position where the subarray's elements larger than the root should be put, just put it at twice the sum value of the root minus the left cousin value for that subarray. Then, the time taken is just $O(\frac{n}{c})$. By doing this procedure, the total work is just $O(n)$, and the span is $O(\lg n)$, and so has parallelization of $O(\frac{n}{\lg n})$. This whole process is split across the several algoithms appearing here.


question:
 Give a multithreaded version of $\text{RECURSIVE-FFT}$ on page 911. Make your implementation as parallel as possible. Analyze your algorithm.

answer:
```cpp
P-RECURSIVE-FFT(a)
    n = a.length
    if n == 1
        return a
    w[n] = e^{2 * π * i / n}
    w = 1
    a(0) = [a[0], a[2]..a[n - 2]]
    a(1) = [a[1], a[3]..a[n - 1]]
    y(0) = spawn P-RECURSIVE-FFT(a[0])
    y(1) = P-RECURSIVE-FFT(a[1])
    sync
    parallel for k = 0 to n / 2 - 1
        y[k] = y[k](0) + w * y[k](1)
        y[k + n / 2] = y[k](0) - w * y[k](1)
        w = w * w[n]
    return y
```

$\text{P-RECURSIVE-FFT}$ parallelized over the two recursive
calls, having a parallel for works because each of the iterations of the **for** loop touch independent sets of variables. The span of the procedure is only $\Theta(\lg n)$ giving it a parallelization of $\Theta(n)$.


question:
 Give a multithreaded version of $\text{RANDOMIZED-SELECT}$ on page 216. Make your implementation as parallel as possible. Analyze your algorithm. ($\textit{Hint:}$ Use the partitioning algorithm from Exercise 27.3-3.)

answer:
Randomly pick a pivot element, swap it with the last element, so that it is in the correct format for running the procedure described in 27.3-3. Run partition from problem 27.3-3. As an intermediate step, in that procedure, we compute the number of elements less than the pivot ($T.root.sum$), so keep track of that value after the end of partition. Then, if we have that it is less than $k$, recurse on the subarray that was greater than or equal to the pivot, decreasing the order statistic of the element to be selected by $T.root.sum$. If it is larger than the order statistic of the element to be selected, then leave it unchanged and recurse on the subarray that was formed to be less than the pivot. A lot of the analysis in section 9.2 still applies, except replacing the timer needed for partitioning with the runtime of the algorithm in problem 27.3-3. The work is unchanged from the serial case because when $c = 1$, the algorithm reduces to the serial algorithm for partitioning. For span, the $O(n)$ term in the equation half way down page 218 can be replaced with an $O(\lg n)$ term. It can be seen with the substitution method that the solution to this is logarithmic

$$E[T(n)] \le \frac{2}{n} \sum_{k = \lfloor n / 2 \rfloor}^{n - 1} C\lg k + O(\lg n) \le O(\lg n).$$

So, the total span of this algorithm will still just be $O(\lg n)$.


question:
 Show how to multithread $\text{SELECT}$ from Section 9.3. Make your implementation as parallel as possible. Analyze your algorithm.

answer:
Let $\text{MEDIAN}(A)$ denote a brute force method which returns the median element of the array $A$. We will only use this to find the median of small arrays, in particular, those of size at most $5$, so it will always run in constant time. We also let $A[i..j]$ denote the array whose elements are $A[i], A[i + 1], \ldots, A[j]$. The function $\text{P-PARTITION}(A, x)$ is a multithreaded function which partitions $A$ around the input element $x$ and returns the number of elements in $A$ which are less than or equal to $x$. Using a parallel **for** loop, its span is logarithmic in the number of elements in $A$. The work is the same as the serialization, which is $\Theta(n)$ according to section 9.3. The span satisfies the recurrence

$$
\begin{aligned}
T_\infty(n) & =   \Theta(lg n / 5) + T_\infty(n / 5) + \Theta(\lg n) + T_\infty(7n / 10 + 6) \\\\
            & \le \Theta(\lg n) + T_\infty(n / 5) + T_\infty(7n / 10 + 6).
\end{aligned}
$$

Using the substitution method we can show that $T_\infty(n) = O(n^\epsilon)$ for some $\epsilon < 1$. In particular, $\epsilon = 0.9$ works. This gives a parallelization of $\Omega(n^0.1)$.

```cpp
P-SELECT(A, i)
    if n == 1
        return A[1]
    let T[1..floor(n / 5)] be a new array
    parallel for i = 0 to floor(n / 5) - 1
        T[i + 1] = MEDIAN(A[i * floor(n / 5)..i * floor(n / 5) + 4])
    if n / 5 is not an integer
        T[floor(n / 5)] = MEDIAN(A[5 * floor(n / 5)..n])
    x = P-SELECT(T, ceil(n / 5))
    k = P-PARTITION(A, x)
    if k == i
        return x
    else if i < k
        P-SELECT(A[1..k - 1], i)
    else
        P-SELECT(A[k + 1..n], i - k)
```


question:
 Using Figure 2.4 as a model, illustrate the operation of merge sort on the array $A = \langle 3, 41, 52, 26, 38, 57, 9, 49 \rangle$.

answer:
$$[3] \quad [41] \quad [52] \quad [26] \quad [38] \quad [57] \quad [9] \quad [49]$$

$$\downarrow$$

$$[3|41] \quad [26|52] \quad [38|57] \quad [9|49]$$

$$\downarrow$$

$$[3|26|41|52] \quad [9|38|49|57]$$

$$\downarrow$$

$$[3|9|26|38|41|49|52|57]$$


question:
 Rewrite the $\text{MERGE}$ procedure so that it does not use sentinels, instead stopping once either array $L$ or $R$ has had all its elements copied back to $A$ and then copying the remainder of the other array back into $A$.

answer:
```cpp
MERGE(A, p, q, r)
    n1 = q - p + 1
    n2 = r - q
    let L[1..n1] and R[1..n2] be new arrays
    for i = 1 to n1
        L[i] = A[p + i - 1]
    for j = 1 to n2
        R[j] = A[q + j]
    i = 1
    j = 1
    for k = p to r
        if i > n1
            A[k] = R[j]
            j = j + 1
        else if j > n2
            A[k] = L[i]
            i = i + 1
        else if L[i] ≤ R[j]
            A[k] = L[i]
            i = i + 1
        else
            A[k] = R[j]
            j = j + 1
```


question:
 Use mathematical induction to show that when $n$ is an exact power of $2$, the solution of the recurrence
>
> $$
> T(n) =
> \begin{cases}
>     2             & \text{if } n = 2, \\\\
>     2T(n / 2) + n & \text{if } n = 2^k, \text{for } k > 1
> \end{cases}
> $$
>
> is $T(n) = n\lg n$.

answer:
- Base case

  For $n = 2^1$, $T(n) = 2\lg 2 = 2$.

- Suppose $n = 2^k$, $T(n) = n\lg n = 2^k \lg 2^k = 2^kk$.

  For $n = 2^{k + 1}$,

  $$
  \begin{aligned}
  T(n) & = 2T(2^{k + 1} / 2) + 2^{k + 1} \\\\
       & = 2T(2^k) + 2^{k + 1} \\\\
       & = 2 \cdot 2^kk + 2^{k + 1} \\\\
       & = 2^{k + 1}(k + 1) \\\\
       & = 2^{k + 1} \lg 2^{k + 1} \\\\
       & = n\lg n.
  \end{aligned}
  $$

By P.M.I., $T(n) = n\lg n$, when $n$ is an exact power of $2$.


question:
 We can express insertion sort as a recursive procedure as follows. In order to sort $A[1..n]$, we recursively sort $A[1..n - 1]$ and then insert $A[n]$ into the sorted array $A[1..n - 1]$. Write a recurrence for the running time of this recursive version of insertion sort.

answer:
It takes $\Theta(n)$ time in the worst case to insert $A[n]$ into the sorted array $A[1..n - 1]$. Therefore, the recurrence

$$
T(n) = \begin{cases}
    \Theta(1)            & \text{if } n = 1, \\\\
    T(n - 1) + \Theta(n) & \text{if } n > 1.
\end{cases}
$$

The solution of the recurrence is $\Theta(n^2)$.


question:
 Referring back to the searching problem (see Exercise 2.1-3), observe that if the sequence $A$ is sorted, we can check the midpoint of the sequence against $v$ and eliminate half of the sequence from further consideration. The **_binary search_** algorithm repeats this procedure, halving the size of the remaining portion of the sequence each time. Write pseudocode, either iterative or recursive, for binary search. Argue that the worst-case running time of binary search is $\Theta(\lg n)$.

answer:
- Iterative:

  ```cpp
  ITERATIVE-BINARY-SEARCH(A, v, low, high)
      while low ≤ high
          mid = floor((low + high) / 2)
          if v == A[mid]
              return mid
          else if v > A[mid]
              low = mid + 1
          else high = mid - 1
      return NIL
  ```

- Recursive:

  ```cpp
  RECURSIVE-BINARY-SEARCH(A, v, low, high)
      if low > high
          return NIL
      mid = floor((low + high) / 2)
      if v == A[mid]
          return mid
      else if v > A[mid]
          return RECURSIVE-BINARY-SEARCH(A, v, mid + 1, high)
      else return RECURSIVE-BINARY-SEARCH(A, v, low, mid - 1)
  ```

Each time we do the comparison of $v$ with the middle element, the search range continues with range halved.

The recurrence

$$
T(n) = \begin{cases}
    \Theta(1)            & \text{if } n = 1, \\\\
    T(n / 2) + \Theta(1) & \text{if } n > 1.
\end{cases}
$$

The solution of the recurrence is $T(n) = \Theta(\lg n)$.


question:
 Observe that the **while** loop of lines 5–7 of the $\text{INSERTION-SORT}$ procedure in Section 2.1 uses a linear search to scan (backward) through the sorted subarray $A[i..j - 1]$. Can we use a binary search (see Exercise 2.3-5) instead to improve the overall worst-case running time of insertion sort to $\Theta(n\lg n)$?

answer:
Each time the **while** loop of lines 5-7 of $\text{INSERTION-SORT}$ scans backward through the sorted array $A[1..j - 1]$. The loop not only searches for the proper place for $A[j]$, but it also moves each of the array elements that are bigger than $A[j]$ one position to the right (line 6). These movements takes $\Theta(j)$ time, which occurs when all the $j - 1$ elements preceding $A[j]$ are larger than $A[j]$. The running time of using binary search to search is $\Theta(\lg j)$, which is still dominated by the running time of moving element $\Theta(j)$.

Therefore, we can't improve the overrall worst-case running time of insertion sort to $\Theta(n\lg n)$.


question:
 Describe a $\Theta(n\lg n)$-time algorithm that, given a set $S$ of $n$ integers and another integer $x$, determines whether or not there exist two elements in $S$ whose sum is exactly $x$.

answer:
First, sort $S$, which takes $\Theta(n\lg n)$.
Then, for each element $s_i$ in $S$, $i = 1, \dots, n$, search $A[i + 1..n]$ for $s_i' = x - s_i$ by binary search, which takes $\Theta(\lg n)$.

- If $s_i'$ is found, return its position;
- otherwise, continue for next iteration.

The time complexity of the algorithm is $\Theta(n\lg n) + n \cdot \Theta(\lg n) = \Theta(n\lg n)$.


question:
 Using Figure 2.2 as a model, illustrate the operation of $\text{INSERTION-SORT}$ on the array $A = \langle 31, 41, 59, 26, 41, 58 \rangle$.

answer:
![2.1-1-1](../img/2.1-1-1.png)

The operation of $\text{INSERTION-SORT}$ on the array $A = \langle 31, 41, 59, 26, 41, 58 \rangle$. Array indices appear above the rectangles, and values stored in the array positions appear within the rectangles.

(a)-(e) are iterations of the for loop of lines 1-8.

In each iteration, the black rectangle holds the key taken from $A[i]$, which is compared with the values in shaded rectangles to its left in the test of line 5. Dotted arrows show array values moved one position to the right in line 6. and solid arrows indicate where the key moves to in line 8.

(f) is the final sorted array.

The changes of array A during traversal:
$$
\begin{aligned}
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\\\
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\\\
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\\\
A & = \langle 26, 31, 41, 59, 41, 58 \rangle \\\\
A & = \langle 26, 31, 41, 41, 59, 58 \rangle \\\\
A & = \langle 26, 31, 41, 41, 58, 59 \rangle
\end{aligned}
$$


question:
 Rewrite the $\text{INSERTION-SORT}$ procedure to sort into nonincreasing instead of nondecreasing order.

answer:
```cpp
INSERTION-SORT(A)
    for j = 2 to A.length
        key = A[j]
        i = j - 1
        while i > 0 and A[i] < key
            A[i + 1] = A[i]
            i = i - 1
        A[i + 1] = key
```


question:
 Consider the **_searching problem_**:
>
> **Input**: A sequence of $n$ numbers $A = \langle a_1, a_2, \ldots, a_n \rangle$ and a value $v$.
>
> **Output:** An index $i$ such that $v = A[i]$ or the special value $\text{NIL}$ if $v$ does not appear in $A$.
>
> Write pseudocode for **_linear search_**, which scans through the sequence, looking for $v$. Using a loop invariant, prove that your algorithm is correct. Make sure that your loop invariant fulfills the three necessary properties.

answer:
```cpp
LINEAR-SEARCH(A, v)
    for i = 1 to A.length
       if A[i] == v
            return i
    return NIL
```

**Loop invariant:** At the start of each iteration of the **for** loop, the subarray $A[1..i - 1]$ consists of elements that are different than $v$.

**Initialization:** Before the first loop iteration ($i = 1$), the subarray is the empty array, so the proof is trivial.

**Maintenance:** During each loop iteration, we compare $v$ with $A[i]$. If they are the same, we return $i$, which is the correct result. Otherwise, we continue to the next iteration of the loop. At the end of each loop iteration, we know the subarray $A[1..i]$ does not contain $v$, so the loop invariant holds true. Incrementing $i$ for the next iteration of the **for** loop then preserves the loop invariant.

**Termination:** The loop terminates when $i > A.length = n$. Since $i$ increases by $1$, we must have $i = n + 1$ at that time. Substituting $n + 1$, for $i$ in the wording of the loop invariant, we have that the subarray $A[1..n]$ consists of elements that are different than $v$. Thus, we return $\text{NIL}$. Observing that $A[1..n]$, we conclude that the entire array does not have any element equal to $v$. Hence the algorithm is correct.


question:
 Consider the problem of adding two $n$-bit binary integers, stored in two $n$-element arrays $A$ and $B$. The sum of the two integers should be stored in binary form in an $(n + 1)$-element array $C$. State the problem formally and write pseudocode for adding the two integers.

answer:
**Input:** An array of booleans $A = \langle a_1, a_2, \ldots, a_n \rangle$ and an array of booleans $B = \langle b_1, b_2, \ldots, b_n \rangle$, each representing an integer stored in binary format (each digit is a number, either $0$ or $1$, **least-significant digit first**) and each of length $n$.

**Output:** An array $C = \langle c_1, c_2, \ldots, c_{n + 1} \rangle$ such that $C' = A' + B'$ where $A'$, $B'$ and $C'$ are the integers, represented by $A$, $B$ and $C$.

```cpp
ADD-BINARY(A, B)
    carry = 0
    for i = 1 to A.length
        sum = A[i] + B[i] + carry
        C[i] = sum % 2  // remainder
        carry = sum / 2 // quotient
    C[A.length + 1] = carry
    return C
```


question:
 Consider sorting $n$ numbers stored in array $A$ by first finding the smallest element of $A$ and exchanging it with the element in $A[1]$. Then find the second smallest element of $A$, and exchange it with $A[2]$. Continue in this manner for the first $n - 1$ elements of $A$. Write pseudocode for this algorithm, which is known as **_selection sort_**. What loop invariant does this algorithm maintain? Why does it need to run for only the first $n - 1$ elements, rather than for all $n$ elements? Give the best-case and worst-case running times of selection sort in $\Theta$-notation.

answer:
- Pseudocode:

    ```cpp
    n = A.length
    for i = 1 to n - 1
        minIndex = i
        for j = i + 1 to n
            if A[j] < A[minIndex]
                minIndex = j
        swap(A[i], A[minIndex])
    ```

- Loop invariant:

    At the start of the loop in line 1, the subarray $A[1..i - 1]$ consists of the smallest $i - 1$ elements in array $A$ with sorted order.

- Why does it need to run for only the first $n - 1$ elements, rather than for all $n$ elements?

    After $n - 1$ iterations, the subarray $A[1..n - 1]$ consists of the smallest $i - 1$ elements in array $A$ with sorted order. Therefore, $A[n]$ is already the largest element.

- Running time: $\Theta(n^2)$.


question:
 Consider linear search again (see Exercise 2.1-3). How many elements of the in- put sequence need to be checked on the average, assuming that the element being searched for is equally likely to be any element in the array? How about in the worst case? What are the average-case and worst-case running times of linear search in $\Theta$-notation? Justify your answers.

answer:
If the element is present in the sequence, half of the elements are likely to be checked before it is found in the average case. In the worst case, all of them will be checked. That is, $n / 2$ checks for the average case and $n$ for the worst case. Both of them are $\Theta(n)$.


question: How can we modify almost any algorithm to have a good best-case running time?
answer:
You can modify any algorithm to have a best case time complexity by adding a special case. If the input matches this special case, return the pre-computed answer.


answer: (Removed)
question:
 Let $(u, v)$ be a minimum-weight edge in a connected graph $G$. Show that $(u, v)$ belongs to some minimum spanning tree of $G$.


question:
 Professor Sabatier conjectures the following converse of Theorem 23.1. Let $G = (V, E)$ be a connected, undirected graph with a real-valued weight function $w$ defined on $E$. Let $A$ be a subset of $E$ that is included in some minimum spanning tree for $G$, let $(S, V - S)$ be any cut of $G$ that respects $A$, and let $(u, v)$ be a safe edge for $A$ crossing $(S, V - S)$. Then, $(u, v)$ is a light edge for the cut. Show that the professor's conjecture is incorrect by giving a counterexample.

answer:
Let $G$ be the graph with $4$ vertices: $u, v, w, z$. Let the edges of the graph be $(u, v), (u, w), (w, z)$ with weights $3$, $1$, and $2$ respectively.

Suppose $A$ is the set $\\{(u, w)\\}$. Let $S = A$. Then $S$ clearly respects $A$. Since $G$ is a tree, its minimum spanning tree is itself, so $A$ is trivially a subset of a minimum spanning tree.

Moreover, every edge is safe. In particular, $(u, v)$ is safe but not a light edge for the cut. Therefore Professor Sabatier's conjecture is false.


question:
 Show that if an edge $(u, v)$ is contained in some minimum spanning tree, then it is a light edge crossing some cut of the graph.

answer:
Let $T_0$ and $T_1$ be the two trees that are obtained by removing edge $(u, v)$ from a $\text{MST}$. Suppose that $V_0$ and $V_1$ are the vertices of $T_0$ and $T_1$ respectively.

Consider the cut which separates $V_0$ from $V_1$. Suppose to a contradiction that there is some edge that has weight less than that of $(u, v)$ in this cut. Then, we could construct a minimum spanning tree of the whole graph by adding that edge to $T_1 \cup T_0$. This would result in a minimum spanning tree that has weight less than the original minimum spanning tree that contained $(u, v)$.


answer: (Removed)
question:
 Give a simple example of a connected graph such that the set of edges $\\{(u, v):$ there exists a cut $(S, V - S)$ such that $(u, v)$ is a light edge crossing $(S, V - S)\\}$ does not form a minimum spanning tree.


question:
 Let $e$ be a maximum-weight edge on some cycle of connected graph $G = (V, E)$. Prove that there is a minimum spanning tree of $G' = (V, E - \\{e\\})$ that is also a minimum spanning tree of $G$. That is, there is a minimum spanning tree of $G$ that does not include $e$.

answer:
Let $A$ be any cut that causes some vertices in the cycle on once side of the cut, and some vertices in the cycle on the other. For any of these cuts, we know that the edge $e$ is not a light edge for this cut. Since all the other cuts won't have the edge $e$ crossing it, we won't have that the edge is light for any of those cuts either. This means that we have that e is not safe.


answer: (Removed)
question:
 Show that a graph has a unique minimum spanning tree if, for every cut of the graph, there is a unique light edge crossing the cut. Show that the converse is not true by giving a counterexample.


question:
 Argue that if all edge weights of a graph are positive, then any subset of edges that connects all vertices and has minimum total weight must be a tree. Give an example to show that the same conclusion does not follow if we allow some weights to be nonpositive.

answer:
First, we show that the subset of edges of minimum total weight that connects all the vertices is a tree. To see this, suppose not, that it had a cycle. This would mean that removing any of the edges in this cycle would mean that the remaining edges would still connect all the vertices, but would have a total weight that's less by the weight of the edge that was removed. This would contradict the minimality of the total weight of the subset of vertices. Since the subset of edges forms a tree, and has minimal total weight, it must also be a minimum spanning tree.

To see that this conclusion is not true if we allow negative edge weights, we provide a construction. Consider the graph $K_3$ with all edge weights equal to $-1$. The only minimum weight set of edges that connects the graph has total weight $-3$, and consists of all the edges. This is clearly not a $\text{MST}$ because it is not a tree, which can be easily seen because it has one more edge than a tree on three vertices should have. Any $\text{MST}$ of this weighted graph must have weight that is at least $-2$.


question:
 Let $T$ be a minimum spanning tree of a graph $G$, and let $L$ be the sorted list of the edge weights of $T$. Show that for any other minimum spanning tree $T'$ of $G$, the list $L$ is also the sorted list of edge weights of $T'$.

answer:
Suppose that $L'$ is another sorted list of edge weights of a minimum spanning tree. If $L' \ne L$, there must be a first edge $(u, v)$ in $T$ or $T'$ which is of smaller weight than the corresponding edge $(x, y)$ in the other set. Without loss of generality, assume $(u, v)$ is in $T$.

Let $C$ be the graph obtained by adding $(u, v)$ to $L'$. Then we must have introduced a cycle. If there exists an edge on that cycle which is of larger weight than $(u, v)$, we can remove it to obtain a tree $C'$ of weight strictly smaller than the weight of $T'$, contradicting the fact that $T'$ is a minimum spanning tree.

Thus, every edge on the cycle must be of lesser or equal weight than $(u, v)$. Suppose that every edge is of strictly smaller weight. Remove $(u, v)$ from $T$ to disconnect it into two components. There must exist some edge besides $(u, v)$ on the cycle which would connect these, and since it has smaller weight we can use that edge instead to create a spanning tree with less weight than $T$, a contradiction. Thus, some edge on the cycle has the same weight as $(u, v)$. Replace that edge by $(u, v)$. The corresponding lists $L$ and $L'$ remain unchanged since we have swapped out an edge of equal weight, but the number of edges which $T$ and $T'$ have in common has increased by $1$.

If we continue in this way, eventually they must have every edge in common, contradicting the fact that their edge weights differ somewhere. Therefore all minimum spanning trees have the same sorted list of edge weights.


question:
 Let $T$ be a minimum spanning tree of a graph $G = (V, E)$, and let $V'$ be a subset of $V$. Let $T'$ be the subgraph of $T$ induced by $V'$, and let $G'$ be the subgraph of $G$ induced by $V'$. Show that if $T'$ is connected, then $T'$ is a minimum spanning tree of $G'$.

answer:
Suppose that there was some cheaper spanning tree than $T'$. That is, we have that there is some $T''$ so that $w(T'') < w(T')$. Then, let $S$ be the edges in $T$ but not in $T'$. We can then construct a minimum spanning tree of $G$ by considering $S \cup T''$. This is a spanning tree since $S \cup T'$ is, and $T''$ makes all the vertices in $V'$ connected just like $T'$ does.

However, we have that

$$w(S \cup T'') = w(S) + w(T'') < w(S) + w(T') = w(S \cup T') = w(T).$$

This means that we just found a spanning tree that has a lower total weight than a minimum spanning tree. This is a contradiction, and so our assumption that there was a spanning tree of $V'$ cheaper than $T'$ must be false.


answer: (Removed)
question:
 Given a graph $G$ and a minimum spanning tree $T$, suppose that we decrease the weight of one of the edges in $T$. Show that $T$ is still a minimum spanning tree for $G$. More formally, let $T$ be a minimum spanning tree for $G$ with edge weights given by weight function $w$. Choose one edge $(x, y) \in T$ and a positive number $k$, and define the weight function $w'$ by
>
> $$
> w'(u, v) =
> \begin{cases}
> w(u, v)     & \text{ if }(u, v) \ne (x, y), \\\\
> w(x, y) - k & \text{ if }(u, v) =   (x, y).
> \end{cases}
> $$
>
> Show that $T$ is a minimum spanning tree for $G$ with edge weights given by $w'$.


question:
 Given a graph $G$ and a minimum spanning tree $T$, suppose that we decrease the weight of one of the edges not in $T$. Give an algorithm for finding the minimum spanning tree in the modified graph.

answer:
If we were to add in this newly decreased edge to the given tree, we would be creating a cycle. Then, if we were to remove any one of the edges along this cycle, we would still have a spanning tree. This means that we look at all the weights along this cycle formed by adding in the decreased edge, and remove the edge in the cycle of maximum weight. This does exactly what we want since we could only possibly want to add in the single decreased edge, and then, from there we change the graph back to a tree in the way that makes its total weight minimized.


question:
 Kruskal's algorithm can return different spanning trees for the same input graph $G$, depending on how it breaks ties when the edges are sorted into order. Show that for each minimum spanning tree $T$ of $G$, there is a way to sort the edges of $G$ in Kruskal's algorithm so that the algorithm returns $T$.

answer:
Suppose that we wanted to pick $T$ as our minimum spanning tree. Then, to obtain this tree with Kruskal's algorithm, we will order the edges first by their weight, but then will resolve ties in edge weights by picking an edge first if it is contained in the minimum spanning tree, and treating all the edges that aren't in $T$ as being slightly larger, even though they have the same actual weight.

With this ordering, we will still be finding a tree of the same weight as all the minimum spanning trees $w(T)$. However, since we prioritize the edges in $T$, we have that we will pick them over any other edges that may be in other minimum spanning trees.


question:
 Suppose that we represent the graph $G = (V, E)$ as an adjacency matrix. Give a simple implementation of Prim's algorithm for this case that runs in $O(V^2)$ time.

answer:
At each step of the algorithm we will add an edge from a vertex in the tree created so far to a vertex not in the tree, such that this edge has minimum weight. Thus, it will be useful to know, for each vertex not in the tree, the edge from that vertex to some vertex in the tree of minimal weight. We will store this information in an array $A$, where $A[u] = (v, w)$ if $w$ is the weight of $(u, v)$ and is minimal among the weights of edges from $u$ to some vertex $v$ in the tree built so far. We'll use $A[u].1$ to access $v$ and $A[u].2$ to access $w$.

```cpp
PRIM-ADJ(G, w, r)
    initialize A with every entry = (NIL, ∞)
    T = {r}
    for i = 1 to V
        if Adj[r, i] != 0
            A[i] = (r, w(r, i))
    for each u in V - T
        k = min(A[i].2)
        T = T ∪ {k}
        k.π = A[k].1
        for i = 1 to V
            if Adf[k, i] != 0 and Adj[k, i] < A[i].2
                A[i] = (k, Adj[k, i])
```


question:
 For a sparse graph $G = (V, E)$, where $|E| = \Theta(V)$, is the implementation of Prim's algorithm with a Fibonacci heap asymptotically faster than the binary-heap implementation? What about for a dense graph, where $|E| = \Theta(V^2)$? How must the sizes $|E|$ and $|V|$ be related for the Fibonacci-heap implementation to be asymptotically faster than the binary-heap implementation?

answer:
Prim's algorithm implemented with a Binary heap has runtime $O((V + E)\lg V)$, which in the sparse case, is just $O(V\lg V)$. The implementation with Fibonacci heaps is

$$O(E + V\lg V) = O(V + V\lg V) = O(V \lg V).$$

- In the sparse case, the two algorithms have the same asymptotic runtimes.
- In the dense case.

    - The binary heap implementation has a runtime of

        $$O((V + E)\lg V) = O((V + V^2)\lg V) = O(V^2\lg V).$$

    - The Fibonacci heap implementation has a runtime of

        $$O(E + V\lg V) = O(V^2 + V\lg V) = O(V^2).$$

    So, in the dense case, we have that the Fibonacci heap implementation is asymptotically faster.

- The Fibonacci heap implementation will be asymptotically faster so long as $E = \omega(V)$. Suppose that we have some function that grows more quickly than linear, say $f$, and $E = f(V)$.
- The binary heap implementation will have runtime of

    $$O((V + E)\lg V) = O((V + f(V))\lg V) = O(f(V)\lg V).$$

However, we have that the runtime of the Fibonacci heap implementation will have runtime of

$$O(E + V\lg V) = O(f(V) + V\lg V).$$

This runtime is either $O(f(V))$ or $O(V\lg V)$ depending on if $f(V)$ grows more or less quickly than $V\lg V$ respectively.

In either case, we have that the runtime is faster than $O(f(V)\lg V)$.


answer: (Removed)
question:
 Suppose that all edge weights in a graph are integers in the range from $1$ to $|V|$. How fast can you make Kruskal's algorithm run? What if the edge weights are integers in the range from $1$ to $W$ for some constant $W$?


question:
 Suppose that all edge weights in a graph are integers in the range from $1$ to $|V|$. How fast can you make Prim's algorithm run? What if the edge weights are integers in the range from $1$ to $W$ for some constant $W$?

answer:
For the first case, we can use a van Emde Boas tree to improve the time bound to $O(E \lg \lg V)$. Comparing to the Fibonacci heap implementation, this improves the asymptotic running time only for sparse graphs, and it cannot improve the running time polynomially. An advantage of this implementation is that it may have a lower overhead.

For the second case, we can use a collection of doubly linked lists, each corresponding to an edge weight. This improves the bound to $O(E)$.


question:
 Suppose that the edge weights in a graph are uniformly distributed over the halfopen interval $[0, 1)$. Which algorithm, Kruskal's or Prim's, can you make run faster?

answer:
For input drawn from a uniform distribution I would use bucket sort with Kruskal's algorithm, for expected linear time sorting of edges by weight. This would achieve expected runtime $O(E\alpha(V))$.


answer: (Removed)
question:
 Suppose that a graph $G$ has a minimum spanning tree already computed. How quickly can we update the minimum spanning tree if we add a new vertex and incident edges to $G$?


question:
 Professor Borden proposes a new divide-and-conquer algorithm for computing minimum spanning trees, which goes as follows. Given a graph $G = (V, E)$, partition the set $V$ of vertices into two sets $V_1$ and $V_2$ such that $|V_1|$ and $|V_2|$ differ by at most $1$. Let $E_1$ be the set of edges that are incident only on vertices in $V_1$, and let $E_2$ be the set of edges that are incident only on vertices in $V_2$. Recursively solve a minimum-spanning-tree problem on each of the two subgraphs $G_1 = (V_1, E_1)$ and $G_2 = (V_2, E_2)$. Finally, select the minimum-weight edge in $E$ that crosses the cut $(V_1, V_2)$, and use this edge to unite the resulting two minimum spanning trees into a single spanning tree.
>
> Either argue that the algorithm correctly computes a minimum spanning tree of $G$, or provide an example for which the algorithm fails.

answer:
The algorithm fails. Suppose $E = \\{(u, v), (u, w), (v, w)\\}$, the weight of $(u, v)$ and $(u, w)$ is $1$, and the weight of $(v, w)$ is $1000$, partition the set into two sets $V_1 = \\{u\\}$ and $V_2 = \\{v, w\\}$.


answer: (Omit!)
question:
 Consider the language $\text{GRAPH-ISOMORPHISM}$ $= \\{\langle G_1, G_2 \rangle: G_1$ and $G_2$ are isomorphic graphs$\\}$. Prove that $\text{GRAPH-ISOMORPHISM} \in \text{NP}$ by describing a polynomial-time algorithm to verify the language.


answer: (Omit!)
question:
 Prove that if $G$ is an undirected bipartite graph with an odd number of vertices, then $G$ is nonhamiltonian.


answer: (Omit!)
question:
 Show that if $\text{HAM-CYCLE} \in P$, then the problem of listing the vertices of a hamiltonian cycle, in order, is polynomial-time solvable.


answer: (Omit!)
question:
 Prove that the class $\text{NP}$ of languages is closed under union, intersection, concatenation, and Kleene star. Discuss the closure of $\text{NP}$ under complement.


answer: (Omit!)
question:
 Show that any language in $\text{NP}$ can be decided by an algorithm running in time $2^{O(n^k)}$ for some constant $k$.


answer: (Omit!)
question:
 A **_hamiltonian path_** in a graph is a simple path that visits every vertex exactly once. Show that the language $\text{HAM-PATH}$ $= \\{\langle G, u, v \rangle:$ there is a hamiltonian path from $u$ to $v$ in graph $G\\}$ belongs to $\text{NP}$.


answer: (Omit!)
question:
 Show that the hamiltonian-path problem from Exercise 34.2-6 can be solved in polynomial time on directed acyclic graphs. Give an efficient algorithm for the problem.


answer: (Omit!)
question:
 Let $\phi$ be a boolean formula constructed from the boolean input variables $x_1, x_2, \dots, x_k$, negations ($\neg$), ANDs ($\vee$), ORs ($\wedge$), and parentheses. The formula $\phi$ is a **_tautology_** if it evaluates to $1$ for every assignment of $1$ and $0$ to the input variables. Define $\text{TAUTOLOGY}$ as the language of boolean formulas that are tautologies. Show that $\text{TAUTOLOGY} \in \text{co-NP}$.


answer: (Omit!)
question:
 Let $G$ be a connected, undirected graph with at least $3$ vertices, and let $G^3$ be the graph obtained by connecting all pairs of vertices that are connected by a path in $G$ of length at most $3$. Prove that $G^3$ is hamiltonian. ($\textit{Hint:}$ Construct a spanning tree for $G$, and use an inductive argument.)


question:
 Define the optimization problem $\text{LONGEST-PATH-LENGTH}$ as the relation that associates each instance of an undirected graph and two vertices with the number of edges in a longest simple path between the two vertices. Define the decision problem $\text{LONGEST-PATH}$ $= \\{\langle G, u, v, k\rangle: G = (V, E)$ is an undirected graph, $u, v \in V, k \ge 0$ is an integer, and there exists a simple path from $u$ to $v$ in $G$ consisting of at least $k$ edges $\\}$. Show that the optimization problem $\text{LONGEST-PATH-LENGTH}$ can be solved in polynomial time if and only if $\text{LONGEST-PATH} \in P$.

answer:
Showing that $\text{LONGEST-PATH-LENGTH}$ being polynomial implies that $\text{LONGEST-PATH}$ is polynomial is trivial, because we can just compute the length of the longest path and reject the instance of $\text{LONGEST-PATH}$ if and only if $k$ is larger than the number we computed as the length of the longest path.

Since we know that the number of edges in the longest path length is between $0$ and $|E|$, we can perform a binary search for it's length. That is, we construct an instance of $\text{LONGEST-PATH}$ with the given parameters along with $k = \frac{|E|}{2}$. If we hear yes, we know that the length of the longest path is somewhere above the halfway point. If we hear no, we know it is somewhere below. Since each time we are halving the possible range, we have that the procedure can require $O(\lg |E|)$ many steps. However, running a polynomial time subroutine $\lg n$ many times still gets us a polynomial time procedure, since we know that with this procedure we will never be feeding output of one call of $\text{LONGEST-PATH}$ into the next.


question:
 Give a formal definition for the problem of finding the longest simple cycle in an undirected graph. Give a related decision problem. Give the language corresponding to the decision problem.

answer:
The problem $\text{LONGST-SIMPLE-CYCLE}$ is the relation that associates each instance of a graph with the longest simple cycle contained in that graph. The decision problem is, given $k$, to determine whether or not the instance graph has a simple cycle of length at least $k$. If yes, output $1$. Otherwise, output $0$. The language corresponding to the decision problem is the set of all $\langle G, k\rangle$ such that $G = (V, E)$ is an undirected graph, $k \ge 0$ is an integer, and there exists a simple cycle in $G$ consisting of at least $k$ edges.


question:
 Give a formal encoding of directed graphs as binary strings using an adjacency-matrix representation. Do the same using an adjacency-list representation. Argue that the two representations are polynomially related.

answer:
A formal encoding of the adjacency matrix representation is to first encode an integer $n$ in the usual binary encoding representing the number of vertices. Then, there will be $n^2$ bits following. The value of bit $m$ will be $1$ if there is an edge from vertex $\lfloor m / n \rfloor$ to vertex $(m \mod n)$, and $0$ if there is not such an edge.

An encoding of the adjacency list representation is a bit more finessed. We'll be using a different encoding of integers, call it $g(n)$. In particular, we will place a $0$ immediately after every bit in the usual representation. Since this only doubles the length of the encoding, it is still polynomially related. Also, the reason we will be using this encoding is because any sequence of integers encoded in this way cannot contain the string $11$ and must contain at least one $0$. Suppose that we have a vertex with edges going to the vertices indexed by $i_1, i_2, i_3, \dots, i_k$. Then, the encoding corresponding to that vertex is $g(i_1)11g(i_2)11 \dots 11g(i_k)1111$. Then, the encoding of the entire graph will be the concatenation of all the encodings of the vertices. As we are reading through, since we used this encoding of the indices of the vertices, we won’t ever be confused about where each of the vertex indices ends or when we are moving on to the next vertex's list.

To go from the list to matrix representation, we can read off all the adjacent vertices, store them, sort them, and then output a row of the adjacency matrix. Since there is some small constant amount of space for the adjacency list representation for each vertex in the graph, the size of the encoding blows up by at most a factor of $O(n)$, which means that the size of the encoding overall is at most squared.

To go in the other direction, it is just a matter of keeping track of the positions in a given row that have $1$'s, encoding those numerical values in the way described, and doing this for each row. Since we are only increasing the size of the encoding by a factor of at most $O(\lg n)$ (which happens in the dense graph case), we have that both of them are polynomially related.


question:
 Is the dynamic-programming algorithm for the 0-1 knapsack problem that is asked for in Exercise 16.2-2 a polynomial-time algorithm? Explain your answer.

answer:
This isn't a polynomial-time algorithm. Recall that the algorithm from Exercise 16.2-2 had running time $\Theta(nW)$ where $W$ was the maximum weight supported by the knapsack. Consider an encoding of the problem. There is a polynomial encoding of each item by giving the binary representation of its index, worth, and weight, represented as some binary string of length $a = \Omega(n)$. We then encode $W$, in polynomial time. This will have length $\Theta(\lg W) = b$. The solution to this problem of length $a + b$ is found in time $\Theta(nW) = \Theta(a \cdot 2^b)$. Thus, the algorithm is actually exponential.


answer: (Omit!)
question:
 Show that if an algorithm makes at most a constant number of calls to polynomial-time subroutines and performs an additional amount of work that also takes polynomial time, then it runs in polynomial time. Also show that a polynomial number of calls to polynomial-time subroutines may result in an exponential-time algorithm.


answer: (Omit!)
question:
 Show that the class $P$, viewed as a set of languages, is closed under union, intersection, concatenation, complement, and Kleene star. That is, if $L_1, L_2 \in P$, then $L_1 \cup L_2 \in P$, $L_1 \cap L_2 \in P$, $L_1L_2 \in P$, $\bar L_1 \in P$, and $L_1^\* \in P$.


answer: (Omit!)
question:
 Show that the $\le_\text P$ relation is a transitive relation on languages. That is, show that if $L_1 \le_\text P L_2$ and $L_2 \le_\text P L_3$, then $L_1 \le_\text P L_3$.


answer: (Omit!)
question:
 Show that we could have used a satisfying assignment as a certificate in an alternative proof of Lemma 34.5. Which certificate makes for an easier proof?


answer: (Omit!)
question:
 The proof of Lemma 34.6 assumes that the working storage for algorithm A occupies a contiguous region of polynomial size. Where in the proof do we exploit this assumption? Argue that this assumption does not involve any loss of generality.


answer: (Omit!)
question:
 A language $L$ is **_complete_** for a language class $C$ with respect to polynomial-time reductions if $L \in C$ and $L' \le_\text P L$ for all $L' \in C$. Show that $\emptyset$ and $\\{0, 1\\}^\*$ are the only languages in $\text P$ that are not complete for $\text P$ with respect to polynomial-time reductions.


answer: (Omit!)
question:
 Show that, with respect to polynomial-time reductions (see Exercise 34.3-6), $L$ is complete for $\text{NP}$ if and only if $\bar L$ is complete for \text{co-NP}\$.


answer: (Omit!)
question:
 The reduction algorithm $F$ in the proof of Lemma 34.6 constructs the circuit $C = f(x)$ based on knowledge of $x$, $A$, and $k$. Professor Sartre observes that the string $x$ is input to $F$, but only the existence of $A$, $k$, and the constant factor implicit in the $O(n^k)$ running time is known to $F$ (since the language $L$ belongs to $\text{NP}$), not their actual values. Thus, the professor concludes that $F$ can't possibly construct the circuit $C$ and that the language $\text{CIRCUIT-SAT}$ is not necessarily $\text{NP-hard}$. Explain the flaw in the professor's reasoning.


answer: (Omit!)
question:
 The **_subgraph-isomorphism problem_** takes two undirected graphs $G_1$ and $G_2$, and it asks whether $G_1$ is isomorphic to a subgraph of $G_2$. Show that the subgraphisomorphism problem is $\text{NP-complete}$.


answer: (Omit!)
question:
 Given an integer $m \times n$ matrix $A$ and an integer $m$-vector $b$, the **_0-1 integerprogramming problem_** asks whether there exists an integer $n$-vector $x$ with elements in the set $\\{0, 1\\}$ such that $Ax \le b$. Prove that 0-1 integer programming is $\text{NP-complete}$. ($\textit{Hint:}$ Reduce from $\text{3-CNF-SAT}$.)


answer: (Omit!)
question:
 The integer **_linear-programming problem_** is like the 0-1 integer-programming problem given in Exercise 34.5-2, except that the values of the vector $x$ may be any integers rather than just $0$ or $1$. Assuming that the 0-1 integer-programming problem is $\text{NP-hard}$, show that the integer linear-programming problem is $\text{NP-complete}$.


answer: (Omit!)
question:
 Show how to solve the subset-sum problem in polynomial time if the target value $t$ is expressed in unary.


answer: (Omit!)
question:
 The **_set-partition problem_** takes as input a set $S$ of numbers. The question is whether the numbers can be partitioned into two sets $A$ and $\bar A = S - A$ such that $\sum_{x \in A} x = \sum_{x \in \bar A} x$. Show that the set-partition problem is $\text{NP-complete}$.


answer: (Omit!)
question:
 The **_longest-simple-cycle problem_** is the problem of determining a simple cycle (no repeated vertices) of maximum length in a graph. Formulate a related decision problem, and show that the decision problem is $\text{NP-complete}$.


answer: (Omit!)
question:
 In the **_half 3-CNF satisfiability_** problem, we are given a $\text{3-CNF}$ formula $\phi$ with $n$ variables and $m$ clauses, where $m$ is even. We wish to determine whether there exists a truth assignment to the variables of $\phi$ such that exactly half the clauses evaluate to $0$ and exactly half the clauses evaluate to $1$. Prove that the half $\text{3-CNF}$ satisfiability problem is $\text{NP-complete}$.


answer: (Omit!)
question:
 Consider the straightforward (nonpolynomial-time) reduction in the proof of Theorem 34.9. Describe a circuit of size $n$ that, when converted to a formula by this method, yields a formula whose size is exponential in $n$.


answer: (Omit!)
question:
 Show the $\text{3-CNF}$ formula that results when we use the method of Theorem 34.10 on the formula $\text{(34.3)}$.


answer: (Omit!)
question:
 Professor Jagger proposes to show that $\text{SAT} \le_\text P \text{3-CNF-SAT}$ by using only the truth-table technique in the proof of Theorem 34.10, and not the other steps. That is, the professor proposes to take the boolean formula $\phi$, form a truth table for its variables, derive from the truth table a formula in $\text{3-DNF}$ that is equivalent to $\neg\phi$, and then negate and apply DeMorgan's laws to produce a $\text{3-CNF}$ formula equivalent to $\phi$. Show that this strategy does not yield a polynomial-time reduction.


answer: (Omit!)
question:
 Show that the problem of determining whether a boolean formula is a tautology is complete for $\text{co-NP}$. ($\textit{Hint:}$ See Exercise 34.3-7.)


answer: (Omit!)
question:
 Show that the problem of determining the satisfiability of boolean formulas in disjunctive normal form is polynomial-time solvable.


answer: (Omit!)
question:
 Suppose that someone gives you a polynomial-time algorithm to decide formula satisfiability. Describe how to use this algorithm to find satisfying assignments in polynomial time.


answer: (Omit!)
question:
 Let $\text{2-CNF-SAT}$ be the set of satisfiable boolean formulas in $\text{CNF}$ with exactly $2$ literals per clause. Show that $\text{2-CNF-SAT} \in P$. Make your algorithm as efficient as possible. ($\textit{Hint:}$ Observe that $x \vee y$ is equivalent to $\neg x \to y$. Reduce $\text{2-CNF-SAT}$ to an efficiently solvable problem on a directed graph.)


question:
 Run the Bellman-Ford algorithm on the directed graph of Figure 24.4, using vertex $z$ as the source. In each pass, relax edges in the same order as in the figure, and show the $d$ and $\pi$ values after each pass. Now, change the weight of edge $(z, x)$ to $4$ and run the algorithm again, using $s$ as the source.

answer:
- Using vertex $z$ as the source:

    - $d$ values:

        $$
        \begin{array}{cccccc}
        s & t & x & y & z \\\\
        \hline
        \infty & \infty & \infty & \infty & 0 \\\\
        2 & \infty & 7 & \infty & 0 \\\\
        2 & 5 & 7 & 9 & 0 \\\\
        2 & 5 & 6 & 9 & 0 \\\\
        2 & 4 & 6 & 9 & 0
        \end{array}
        $$

    - $\pi$ values:

        $$
        \begin{array}{cccccc}
        s & t & x & y & z \\\\
        \hline
        \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} \\\\
        z & \text{NIL} & z & \text{NIL} & \text{NIL} \\\\
        z & x & z & s & \text{NIL} \\\\
        z & x & y & s & \text{NIL} \\\\
        z & x & y & s & \text{NIL}
        \end{array}
        $$

- Changing the weight of edge $(z, x)$ to $4$:

    - $d$ values:

        $$
        \begin{array}{cccccc}
        s & t & x & y & z \\\\
        \hline
        0 & \infty & \infty & \infty & \infty \\\\
        0 & 6 & \infty & 7 & \infty \\\\
        0 & 6 & 4 & 7 & 2 \\\\
        0 & 2 & 4 & 7 & 2 \\\\
        0 & 2 & 4 & 7 & -2
        \end{array}
        $$

    - $\pi$ values:

        $$
        \begin{array}{cccccc}
        s & t & x & y & z \\\\
        \hline
        \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} \\\\
        \text{NIL} & s & \text{NIL} & s & \text{NIL} \\\\
        \text{NIL} & s & y & s & t \\\\
        \text{NIL} & x & y & s & t \\\\
        \text{NIL} & x & y & s & t
        \end{array}
        $$

        Consider edge $(z, x)$, it'll return $\text{FALSE}$ since $x.d = 4 > z.d + w(z, x) = -2 + 4$.


question: Prove Corollary 24.3.
answer:
Suppose there is a path from $s$ to $v$. Then there must be a shortest such path of length $\delta(s, v)$. It must have finite length since it contains at most $|V| - 1$ edges and each edge has finite length. By Lemma 24.2, $v.d = \delta(s, v) < \infty$ upon termination.

On the other hand, suppose $v.d < \infty$ when $\text{BELLMAN-FORD}$ terminates. Recall that $v.d$ is monotonically decreasing throughout the algorithm, and $\text{RELAX}$ will update $v.d$ only if $u.d + w(u, v) < v.d$ for some $u$ adjacent to $v$. Moreover, we update $v.\pi = u$ at this point, so $v$ has an ancestor in the predecessor subgraph. Since this is a tree rooted at $s$, there must be a path from $s$ to $v$ in this tree. Every edge in the tree is also an edge in $G$, so there is also a path in $G$ from $s$ to $v$.


question:
 Given a weighted, directed graph $G = (V, E)$ with no negative-weight cycles, let $m$ be the maximum over all vertices $v \in V$ of the minimum number of edges in a shortest path from the source $s$ to $v$. (Here, the shortest path is by weight, not the number of edges.) Suggest a simple change to the Bellman-Ford algorithm that allows it to terminate in $m + 1$ passes, even if $m$ is not known in advance.

answer:
By the upper bound theory, we know that after $m$ iterations, no $d$ values will ever change. Therefore, no $d$ values will change in the $(m + 1)$-th iteration. However, we do not know the exact $m$ value in advance, we cannot make the algorithm iterate exactly $m$ times and then terminate. If we try to make the algorithm stop when every $d$ values do not change anymore, then it will stop after $m + 1$ iterations.


question:
 Modify the Bellman-Ford algorithm so that it sets $v.d$ to $-\infty$ for all vertices $v$ for which there is a negative-weight cycle on some path from the source to $v$.

answer:
```cpp
BELLMAN-FORD'(G, w, s)
    INITIALIZE-SINGLE-SOURCE(G, s)
    for i = 1 to |G.V| - 1
        for each edge (u, v) ∈ G.E
            RELAX(u, v, w)
    for each edge(u, v) ∈ G.E
        if v.d > u.d + w(u, v)
            mark v
    for each vertex u ∈ marked vertices
        DFS-MARK(u)
```

```cpp
DFS-MARK(u)
    if u != NIL and u.d != -∞
        u.d = -∞
        for each v in G.Adj[u]
            DFS-MARK(v)
```

After running $\text{BELLMAN-FORD}'$, run $\text{DFS}$ with all vertices on negative-weight cycles as source vertices. All the vertices that can be reached from these vertices should have their $d$ attributes set to $-\infty$.


question:
 Let $G = (V, E)$ be a weighted, directed graph with weight function $w : E \rightarrow \mathbb R$. Give an $O(VE)$-time algorithm to find, for each vertex $v \in V$, the value $\delta^*(v) = \min_{u \in V} \\{\delta(u, v)\\}$.

answer:
```cpp
RELAX(u, v, w)
    if v.d > min(w(u, v), w(u, v) + u.d)
        v.d = min(w(u, v), w(u, v) + u.d)
        v.π = u.π
```


question:
 Suppose that a weighted, directed graph $G = (V, E)$ has a negative-weight cycle. Give an efficient algorithm to list the vertices of one such cycle. Prove that your algorithm is correct.

answer:
Based on exercise 24.1-4, $\text{DFS}$ from a vertex $u$ that $u.d = -\infty$, if the weight sum on the search path is negative and the next vertex is $\text{BLACK}$, then the search path forms a negative-weight cycle.


question:
 Find a feasible solution or determine that no feasible solution exists for the following system of difference constraints:
>
> $$
> \begin{aligned}
> x_1 - x_2 & \le & 1,  \\\\
> x_1 - x_4 & \le & -4, \\\\
> x_2 - x_3 & \le & 2,  \\\\
> x_2 - x_5 & \le & 7,  \\\\
> x_2 - x_6 & \le & 5,  \\\\
> x_3 - x_6 & \le & 10, \\\\
> x_4 - x_2 & \le & 2,  \\\\
> x_5 - x_1 & \le & -1, \\\\
> x_5 - x_4 & \le & 3,  \\\\
> x_6 - x_3 & \le & 8
> \end{aligned}
> $$

answer:
Our vertices of the constraint graph will be

$$\\{v_0, v_1, v_2, v_3, v_4, v_5, v_6\\}.$$

The edges will be

$$(v_0, v_1), (v_0, v_2), (v_0, v_3), (v_0, v_4), (v_0, v_5), (v_0, v_6), (v_2, v_1), (v_4, v_1), (v_3, v_2), (v_5, v_2), (v_6, v_2), (v_6, v_3),$$

with edge weights

$$0, 0, 0, 0, 0, 0, 1, -4, 2, 7, 5, 10, 2, -1, 3, -8$$

respectively. Then, computing

$$(\delta(v_0, v_1), \delta(v_0, v_2), \delta(v_0, v_3), \delta(v_0, v_4), \delta(v_0, v_5), \delta(v_0, v_6)),$$

we get

$$(-5, -3, 0, -1, -6, -8),$$

which is a feasible solution by Theorem 24.9.


question:
 Find a feasible solution or determine that no feasible solution exists for the following system of difference constraints:
>
> $$
> \begin{aligned}
> x_1 - x_2 & \le &4, \\\\
> x_1 - x_5 & \le &5, \\\\
> x_2 - x_4 & \le &-6, \\\\
> x_3 - x_2 & \le &1, \\\\
> x_4 - x_1 & \le &3, \\\\
> x_4 - x_3 & \le &5, \\\\
> x_4 - x_5 & \le &10, \\\\
> x_5 - x_3 & \le &-4, \\\\
> x_5 - x_4 & \le &-8.
> \end{aligned}
> $$

answer:
There is no feasible solution because the constraint graph contains a negative-weight cycle: $(v_1, v_4, v_2, v_3, v_5, v_1)$ has weight $-1$.


question: Can any shortest-path weight from the new vertex $v_0$ in a constraint graph be positive? Explain.
answer:
No, it cannot be positive. This is because for every vertex $v \ne v_0$, there is an edge $(v_0, v)$ with weight zero. So, there is some path from the new vertex to every other of weight zero. Since $\delta(v_0, v)$ is a minimum weight of all paths, it cannot be greater than the weight of this weight zero path that consists of a single edge.


question:
 Show how to modify the Bellman-Ford algorithm slightly so that when we use it to solve a system of difference constraints with $m$ inequalities on $n$ unknowns, the running time is $O(nm)$.

answer:
We can follow the advice of problem 14.4-7 and solve the system of constraints on a modified constraint graph in which there is no new vertex $v_0$. This is simply done by initializing all of the vertices to have a $d$ value of $0$ before running the iterated relaxations of Bellman Ford. Since we don't add a new vertex and the $n$ edges going from it to to vertex corresponding to each variable, we are just running Bellman Ford on a graph with $n$ vertices and $m$ edges, and so it will have a runtime of $O(mn)$.


question:
 Suppose that in addition to a system of difference constraints, we want to handle **_equality constraints_** of the form $x_i = x_j + b_k$. Show how to adapt the Bellman-Ford algorithm to solve this variety of constraint system.

answer:
To obtain the equality constraint $x_i = x_j + b_k$ we simply use the inequalities $x_i - x_j \le b_k$ and $x_j - x_i \le -bk$, then solve the problem as usual.


answer: (Removed)
question:
 Show how to solve a system of difference constraints by a Bellman-Ford-like algorithm that runs on a constraint graph without the extra vertex $v_0$.


question:
 Let $Ax \le b$ be a system of $m$ difference constraints in $n$ unknowns. Show that the Bellman-Ford algorithm, when run on the corresponding constraint graph, maximizes $\sum_{i = 1}^n x_i$ subject to $Ax \le b$ and $x_i \le 0$ for all $x_i$.

answer:
Bellman-Ford correctly solves the system of difference constraints so $Ax \le b$ is always satisfied. We also have that $x_i = \delta(v_0, v_i) \le w(v_0, v_i) = 0$ so $x_i \le 0$ for all $i$. To show that $\sum x_i$ is maximized, we'll show that for any feasible solution $(y_1, y_2, \ldots, y_n)$ which satisfies the constraints we have $yi \le \delta(v_0, v_i) = x_i$. Let $v_0, v_{i_1}, \ldots, v_{i_k}$ be a shortest path from $v_0$ to $v_i$ in the constraint graph. Then we must have the constraints $y_{i_2} - y_{i_1} \le w(v_{i_1}, v_{i_2}), \ldots, y_{i_k} - y_{i_{k - 1}} \le w(v_{i_{k - 1}},v_{i_k})$. Summing these up we have

$$y_i \le y_i - y_1 \le \sum_{m = 2}^k w(v_{i_m}, v_{i_{m - 1}}) = \delta(v_0, v_i) = x_i.$$


question:
 Show that the Bellman-Ford algorithm, when run on the constraint graph for a system $Ax \le b$ of difference constraints, minimizes the quantity $(\max\\{x_i\\} - \min\\{x_i\\})$ subject to $Ax \le b$. Explain how this fact might come in handy if the algorithm is used to schedule construction jobs.

answer:
We can see that the Bellman-Ford algorithm run on the graph whose construction is described in this section causes the quantity $\max\\{x_i\\} - \min\\{x_i\\}$ to be minimized. We know that the largest value assigned to any of the vertices in the constraint graph is a $0$. It is clear that it won't be greater than zero, since just the single edge path to each of the vertices has cost zero. We also know that we cannot have every vertex having a shortest path with negative weight. To see this, notice that this would mean that the pointer for each vertex has it's $p$ value going to some other vertex that is not the source. This means that if we follow the procedure for reconstructing the shortest path for any of the vertices, we have that it can never get back to the source, a contradiction to the fact that it is a shortest path from the source to that vertex.

Next, we note that when we run Bellman-Ford, we are maximizing $\min\\{x_i\\}$. The shortest distance in the constraint graphs is the bare minimum of what is required in order to have all the constraints satisfied, if we were to increase any of the values we would be violating a constraint.

This could be in handy when scheduling construction jobs because the quantity $\max\\{x_i\\} - \min\\{x_i\\}$ is equal to the difference in time between the last task and the first task. Therefore, it means that minimizing it would mean that the total time that all the jobs takes is also minimized. And, most people want the entire process of construction to take as short of a time as possible.


answer: (Removed)
question:
 Suppose that every row in the matrix $A$ of a linear program $Ax \le b$ corresponds to a difference constraint, a single-variable constraint of the form $x_i \le b_k$, or a singlevariable constraint of the form $-x_i \le b_k$. Show how to adapt the Bellman-Ford algorithm to solve this variety of constraint system.


question:
 Give an efficient algorithm to solve a system $Ax \le b$ of difference constraints when all of the elements of $b$ are real-valued and all of the unknowns $x_i$ must be integers.

answer:
To do this, just take the floor of (largest integer that is less than or equal to) each of the $b$ values and solve the resulting integer difference problem. These modified constraints will be admitting exactly the same set of assignments since we required that the solution have integer values assigned to the variables. This is because since the variables are integers, all of their differences will also be integers. For an integer to be less than or equal to a real number, it is necessary and sufficient for it to be less than or equal to the floor of that real number.


question:
 Give an efficient algorithm to solve a system $Ax \le b$ of difference constraints when all of the elements of $b$ are real-valued and a specified subset of some, but not necessarily all, of the unknowns $x_i$ must be integers.

answer:
To solve the problem of $Ax \le b$ where the elements of $b$ are real-valued we carry out the same procedure as before, running Bellman-Ford, but allowing our edge weights to be real-valued. To impose the integer condition on the $x_i$'s, we modify the $\text{RELAX}$ procedure. Suppose we call $\text{RELAX}(v_i, v_j, w)$ where $v_j$ is required to be integral valued. If $v_j.d > \lfloor v_i.d + w(v_i, v_j) \rfloor$, set $v_j.d = \lfloor v_i.d + w(v_i, v_j) \rfloor$. This guarantees that the condition that $v_j.d - v_i.d \le w(v_i, v_j)$ as desired. It also ensures that $v_j$ is integer valued. Since the triangle inequality still holds, $x = (v_1.d, v_2.d, \ldots, v_n.d)$ is a feasible solution for the system, provided that $G$ contains no negative weight cycles.


question:
 Give two shortest-paths trees for the directed graph of Figure 24.2 (on page 648) other than the two shown.

answer:
Since the induced shortest path trees on $\\{s, t, y\\}$ and on $\\{t, x, y, z\\}$ are independent and have to possible configurations each, there are four total arising from that. So, we have the two not shown in the figure are the one consisting of the edges $\\{(s, t), (s, y), (y, x), (x, z)\\}$ and the one consisting of the edges $\\{(s, t), (t, y), (t, x), (y, z)\\}$.


question:
 Give an example of a weighted, directed graph $G = (V, E)$ with weight function $w: E \rightarrow \mathbb R$ and source vertex $s$ such that $G$ satisfies the following property: For every edge $(u, v) \in E$, there is a shortest-paths tree rooted at $s$ that contains $(u, v)$ and another shortest-paths tree rooted at $s$ that does not contain $(u, v)$.

answer:
Let $G$ have $3$ vertices $s$, $x$, and $y$. Let the edges be $(s, x)$, $(s, y)$, and $(x, y)$ with weights $1$, $1$, and $0$ respectively. There are $3$ possible trees on these vertices rooted at $s$, and each is a shortest paths tree which gives $\delta(s, x) = \delta(s, y) = 1$.


question:
 Embellish the proof of Lemma 24.10 to handle cases in which shortest-path weights are $\infty$ or $-\infty$.

answer:
To modify Lemma 24.10 to allow for possible shortest path weights of $\infty$ and $-\infty$, we need to define our addition as $\infty + c = \infty$, and $-\infty + c = -\infty$. This will make the statement behave correctly, that is, we can take the shortest path from $s$ to $u$ and tack on the edge $(u, v)$ to the end. That is, if there is a negative weight cycle on your way to $u$ and there is an edge from $u$ to $v$, there is a negative weight cycle on our way to $v$. Similarly, if we cannot reach $v$ and there is an edge from $u$ to $v$, we cannot reach $u$.


answer: (Removed)
question:
 Let $G = (V, E)$ be a weighted, directed graph with source vertex $s$, and let $G$ be initialized by $\text{INITIALIZE-SINGLE-SOURCE}(G, s)$. Prove that if a sequence of relaxation steps sets $s.\pi$ to a non-$\text{NIL}$ value, then $G$ contains a negative-weight cycle.


question:
 Let $G = (V, E)$ be a weighted, directed graph with no negative-weight edges. Let $s \in V$ be the source vertex, and suppose that we allow $v.\pi$ to be the predecessor of $v$ on _any_ shortest path to $v$ from source $s$ if $v \in V - \\{s\\}$ is reachable from $s$, and $\text{NIL}$ otherwise. Give an example of such a graph $G$ and an assignment of $\pi$ values that produces a cycle in $G_\pi$. (By Lemma 24.16, such an assignment cannot be produced by a sequence of relaxation steps.)

answer:
Suppose that we have a grap hon three vertices $\\{s, u, v\\}$ and containing edges $(s, u), (s, v), (u, v), (v, u)$ all with weight $0$. Then, there is a shortest path from $s$ to $v$ of $s$, $u$, $v$ and a shortest path from $s$ to $u$ of $s$ $v$, $u$. Based off of these, we could set $v.\pi = u$ and $u.\pi = v$. This then means that there is a cycle consisting of $u, v$ in $G_\pi$.


question:
 Let $G = (V, E)$ be a weighted, directed graph with weight function $w: E \rightarrow \mathbb R$ and no negative-weight cycles. Let $s \in V$ be the source vertex, and let $G$ be initialized by $\text{INITIALIZE-SINGLE-SOURCE}(G, s)$. Prove that for every vertex $v \in V_\pi$, there exists a path from $s$ to $v$ in $G_\pi$ and that this property is maintained as an invariant over any sequence of relaxations.

answer:
We will prove this by induction on the number of relaxations performed. For the base-case, we have just called $\text{INITIALIZE-SINGLE-SOURCE}(G, s)$. The only vertex in $V_\pi$ is $s$, and there is trivially a path from $s$ to itself. Now suppose that after any sequence of $n$ relaxations, for every vertex $v \in V_\pi$ there exists a path from $s$ to $v$ in $G_\pi$. Consider the $(n + 1)$th relaxation. Suppose it is such that $v.d > u.d + w(u, v)$. When we relax $v$, we update $v.\pi = u.\pi$. By the induction hypothesis, there was a path from $s$ to $u$ in $G_\pi$. Now $v$ is in $V_\pi$, and the path from $s$ to $u$, followed by the edge $(u,v) = (v.\pi, v)$ is a path from s to $v$ in $G_\pi$, so the claim holds.


answer: (Removed)
question:
 Let $G = (V, E)$ be a weighted, directed graph that contains no negative-weight cycles. Let $s \in V$ be the source vertex, and let $G$ be initialized by $\text{INITIALIZE-SINGLE-SOURCE}(G, s)$. Prove that there exists a sequence of $|V| - 1$ relaxation steps that produces $v.d = \delta(s, v)$ for all $v \in V$.


answer: (Removed)
question:
 Let $G$ be an arbitrary weighted, directed graph with a negative-weight cycle reachable from the source vertex $s$. Show how to construct an infinite sequence of relaxations of the edges of $G$ such that every relaxation causes a shortest-path estimate to change.


question:
 Run Dijkstra's algorithm on the directed graph of Figure 24.2, first using vertex $s$ as the source and then using vertex $z$ as the source. In the style of Figure 24.6, show the $d$ and $\pi$ values and the vertices in set $S$ after each iteration of the **while** loop.

answer:
- $s$ as the source:

    - $d$ values:

        $$
        \begin{array}{ccccc}
        s & t & x & y & z \\\\
        \hline
        0 & 3 & \infty & 5 & \infty \\\\
        0 & 3 & 9 & 5 & \infty \\\\
        0 & 3 & 9 & 5 & 11 \\\\
        0 & 3 & 9 & 5 & 11 \\\\
        0 & 3 & 9 & 5 & 11
        \end{array}
        $$

    - $\pi$ values:

        $$
        \begin{array}{ccccc}
        s & t & x & y & z \\\\
        \hline
        \text{NIL} & s & \text{NIL} & \text{NIL} & \text{NIL} \\\\
        \text{NIL} & s & t & s & \text{NIL} \\\\
        \text{NIL} & s & t & s & y \\\\
        \text{NIL} & s & t & s & y \\\\
        \text{NIL} & s & t & s & y
        \end{array}
        $$

- $z$ as the source:

    - $d$ values:

        $$
        \begin{array}{ccccc}
        s & t & x & y & z \\\\
        \hline
        3 & \infty & 7 & \infty & 0 \\\\
        3 & 6 & 7 & 8 & 0           \\\\
        3 & 6 & 7 & 8 & 0           \\\\
        3 & 6 & 7 & 8 & 0           \\\\
        3 & 6 & 7 & 8 & 0
        \end{array}
        $$

    - $\pi$ values:

        $$
        \begin{array}{ccccc}
        s & t & x & y & z \\\\
        \hline
        z & \text{NIL} & z & \text{NIL} & \text{NIL} \\\\
        z & s & z & s & \text{NIL}     \\\\
        z & s & z & s & \text{NIL}     \\\\
        z & s & z & s & \text{NIL}     \\\\
        z & s & z & s & \text{NIL}
        \end{array}
        $$


question:
 Give a simple example of a directed graph with negative-weight edges for which Dijkstra's algorithm produces incorrect answers. Why doesn't the proof of Theorem 24.6 go through when negative-weight edges are allowed?

answer:
Consider any graph with a negative cycle. $\text{RELAX}$ is called a finite number of times but the distance to any vertex on the cycle is $-\infty$, so Dijkstra's algorithm cannot possibly be correct here. The proof of theorem 24.6 doesn't go through because we can no longer guarantee that

$$\delta(s, y) \le \delta(s, u).$$


question:
 Suppose we change line 4 of Dijkstra's algorithm to the following.
>
> ```cpp
>  4  while |Q| > 1
> ```
>
> This change causes the **while** loop to execute $|V| - 1$ times instead of $|V|$ times. Is this proposed algorithm correct?

answer:
Yes, the algorithm is correct. Let $u$ be the leftover vertex that does not get extracted from the priority queue $Q$. If $u$ is not reachable from $s$, then

$$u.d = \delta(s, u) = \infty.$$

If $u$ is reachable from $s$, then there is a shortest path

$$p = s \rightarrow x \rightarrow u.$$

When the node $x$ was extracted,

$$x.d = \delta(s, x)$$

and then the edge $(x, u)$ was relaxed; thus,

$$u.d = \delta(s, u).$$


answer: (Removed)
question:
 Professor Gaedel has written a program that he claims implements Dijkstra's algorithm. The program produces $v.d$ and $v.\pi$ for each vertex $v \in V$. Give an $O(V + E)$-time algorithm to check the output of the professor's program. It should determine whether the $d$ and $\pi$ attributes match those of some shortest-paths tree. You may assume that all edge weights are nonnegative.


answer: (Removed)
question:
 Professor Newman thinks that he has worked out a simpler proof of correctness for Dijkstra's algorithm. He claims that Dijkstra's algorithm relaxes the edges of every shortest path in the graph in the order in which they appear on the path, and therefore the path-relaxation property applies to every vertex reachable from the source. Show that the professor is mistaken by constructing a directed graph for which Dijkstra's algorithm could relax the edges of a shortest path out of order.


answer: (Removed)
question:
 We are given a directed graph $G = (V, E)$ on which each edge $(u, v) \in E$ has an associated value $r(u, v)$, which is a real number in the range $0 \le r(u, v) \le 1$ that represents the reliability of a communication channel from vertex $u$ to vertex $v$. We interpret $r(u, v)$ as the probability that the channel from $u$ to $v$ will not fail, and we assume that these probabilities are independent. Give an efficient algorithm to find the most reliable path between two given vertices.


answer: $V + \sum_{(u, v) \in E} w(u, v) - E$.
question:
 Let $G = (V, E)$ be a weighted, directed graph with positive weight function $w: E \rightarrow \\{1, 2, \ldots, W\\}$ for some positive integer $W$, and assume that no two vertices have the same shortest-path weights from source vertex $s$. Now suppose that we define an unweighted, directed graph $G' = (V \cup V', E')$ by replacing each edge $(u, v) \in E$ with $w(u, v)$ unit-weight edges in series. How many vertices does $G'$ have? Now suppose that we run a breadth-first search on $G'$. Show that the order in which the breadth-first search of $G'$ colors vertices in $V$ black is the same as the order in which Dijkstra's algorithm extracts the vertices of $V$ from the priority queue when it runs on $G$.


answer: (Removed)
question:
 Let $G = (V, E)$ be a weighted, directed graph with nonnegative weight function $w: E \rightarrow \\{0, 1, \ldots, W\\}$ for some nonnegative integer $W$. Modify Dijkstra's algorithm to compute the shortest paths from a given source vertex s in $O(WV + E)$ time.


answer: (Removed)
question:
 Modify your algorithm from Exercise 24.3-8 to run in $O((V + E) \lg W)$ time. ($\textit{Hint:}$ How many distinct shortest-path estimates can there be in $V - S$ at any point in time?)


question:
 Suppose that we are given a weighted, directed graph $G = (V, E)$ in which edges that leave the source vertex $s$ may have negative weights, all other edge weights are nonnegative, and there are no negative-weight cycles. Argue that Dijkstra's algorithm correctly finds shortest paths from $s$ in this graph.

answer:
The proof of correctness, Theorem 24.6, goes through exactly as stated in the text. The key fact was that $\delta(s, y) \le \delta(s, u)$. It is claimed that this holds because there are no negative edge weights, but in fact that is stronger than is needed. This always holds if $y$ occurs on a shortest path from $s$ to $u$ and $y \ne s$ because all edges on the path from $y$ to $u$ have nonnegative weight. If any had negative weight, this would imply that we had "gone back" to an edge incident with $s$, which implies that a cycle is involved in the path, which would only be the case if it were a negative-weight cycle. However, these are still forbidden.


question:
 Run $\text{DAG-SHORTEST-PATHS}$ on the directed graph of Figure 24.5, using vertex $r$ as the source.

answer:
- $d$ values:

    $$
    \begin{array}{cccccc}
    r & s & t & x & y & z \\\\
    \hline
    0 & \infty & \infty & \infty & \infty & \infty \\\\
    0 & 5 & 3 & \infty & \infty & \infty \\\\
    0 & 5 & 3 & 11 & \infty & \infty \\\\
    0 & 5 & 3 & 10 & 7 & 5 \\\\
    0 & 5 & 3 & 10 & 7 & 5 \\\\
    0 & 5 & 3 & 10 & 7 & 5
    \end{array}
    $$

- $\pi$ values:

    $$
    \begin{array}{cccccc}
    r & s & t & x & y & z \\\\
    \hline
    \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} & \text{NIL} \\\\
    \text{NIL} & r & r & \text{NIL} & \text{NIL} & \text{NIL} \\\\
    \text{NIL} & r & r & s & \text{NIL} & \text{NIL} \\\\
    \text{NIL} & r & r & t & t & t \\\\
    \text{NIL} & r & r & t & t & t \\\\
    \text{NIL} & r & r & t & t & t
    \end{array}
    $$


question:
 Suppose we change line 3 of $\text{DAG-SHORTEST-PATHS}$ to read
>
> ```cpp
>  3  for the first |V| - 1 vertices, taken in topologically sorted order
> ```
>
> Show that the procedure would remain correct.

answer:
When we reach vertex $v$, the last vertex in the topological sort, it must have $out\text-degree$ $0$. Otherwise there would be an edge pointing from a later vertex to an earlier vertex in the ordering, a contradiction. Thus, the body of the for-loop of line 4 is never entered for this final vertex, so we may as well not consider it.


question:
 The PERT chart formulation given above is somewhat unnatural. In a more natural structure, vertices would represent jobs and edges would represent sequencing constraints; that is, edge $(u, v)$ would indicate that job $u$ must be performed before job $v$. We would then assign weights to vertices, not edges. Modify the $\text{DAG-SHORTEST-PATHS}$ procedure so that it finds a longest path in a directed acyclic graph with weighted vertices in linear time.

answer:
There are two ways to transform a PERT chart $G = (V, E)$ with weights on the vertices to a PERT chart $G' = (V', E')$ with weights on edges. Both ways satisfy $|V'| \le 2|V|$ and $|E'| \le |V| + |E|$, so we can scan $G'$ using the same algorithm to find the longest path through a directed acyclic graph.

In the first way, we transform each vertex $v \in V$ into two vertices $v'$ and $v''$ in $V'$. All edges in $E$ that enters $V$ will also enter $V'$ in $E'$, and all edges in $E$ that leaves $V$ will leave $V''$ in $E'$. Thus, if $(u, v) \in E$, then $(u'', v') \in E'$. All such edges have weight 0, so we can put edges $(v', v'')$ into $E'$ for all vertices $v \in V$, and these edges are given the weight of the corresponding vertex $v$ in $G$. Finally, we get $|V'| \le 2|V|$ and $|E'| \le |V| + |E|$, and the edge weight of each path in $G'$ equals the vertex weight of the corresponding path in $G$.

In the second way, we leave vertices in $V$, but try to add one new source vertex $s$ to $V'$, given that $V' = V \cup \\{s\\}$. All edges of $E$ are in $E'$, and $E'$ also includes an edge $(s, v)$ for every vertex $v \in V$ that has in-degree 0 in $G$. Thus, the only vertex with in-degree 0 in $G'$ is the new source $s$. The weight of edge $(u, v) \in E'$ is the weight of vertex $v$ in $G$. We have the weight of each entering edge in $G'$ is the weight of the vertex it enters in $G$.


question:
 Give an efficient algorithm to count the total number of paths in a directed acyclic graph. Analyze your algorithm.

answer:
We will compute the total number of paths by counting the number of paths whose start point is at each vertex $v$, which will be stored in an attribute $v.paths$. Assume that initial we have $v.paths = 0$ for all $v \in V$. Since all vertices adjacent to $u$ occur later in the topological sort and the final vertex has no neighbors, line 4 is well-defined. Topological sort takes $O(V + E)$ and the nested for-loops take $O(V + E)$ so the total runtime is $O(V + E)$.

```cpp
PATHS(G)
    topologically sort the vertices of G
    for each vertex u, taken in topologically sorted order
        for each v ∈ G.Adj[u]
            v.paths = u.paths + 1 + v.paths
    return the sum of all paths attributes
```


question:
 Illustrate the operation $\text{HEAP-EXTRACT-MAX}$ on the heap $A = \langle 15, 13, 9, 5, 12, 8, 7, 4, 0, 6, 2, 1 \rangle$.

answer:
1. Original heap.

    ![](../img/6.5-1-1.png)

2. Extract the max node $15$, then move $1$ to the top of the heap.

    ![](../img/6.5-1-2.png)

3. Since $13 > 9 > 1$, swap $1$ and $13$.

    ![](../img/6.5-1-3.png)

4. Since $12 > 5 > 1$, swap $1$ and $12$.

    ![](../img/6.5-1-4.png)

5. Since $6 > 2 > 1$, swap $1$ and $6$.

    ![](../img/6.5-1-5.png)


question:
 Illustrate the operation of $\text{MAX-HEAP-INSERT}(A, 10)$ on the heap $A = \langle 15, 13, 9, 5, 12, 8, 7, 4, 0, 6, 2, 1 \rangle$.

answer:
1. Original heap.

    ![](../img/6.5-2-1.png)

2. Since $\text{MAX-HEAP-INSERT}(A, 10)$ is called, we append a node assigned value $-\infty$.

    ![](../img/6.5-2-2.png)

3. Update the $key$ value of the new node.

    ![](../img/6.5-2-3.png)

4. Since the parent $key$ is smaller than $10$, the nodes are swapped.

    ![](../img/6.5-2-4.png)

5. Since the parent $key$ is smaller than $10$, the nodes are swapped.

    ![](../img/6.5-2-5.png)


question:
 Write pseudocode for the procedures $\text{HEAP-MINIMUM}$, $\text{HEAP-EXTRACT-MIN}$, $\text{HEAP-DECREASE-KEY}$, and $\text{MIN-HEAP-INSERT}$ that implement a min-priority queue with a min-heap.

answer:
```cpp
HEAP-MINIMUM(A)
    return A[1]
```

```cpp
HEAP-EXTRACT-MIN(A)
    if A.heap-size < 1
        error "heap underflow"
    min = A[1]
    A[1] = A[A.heap-size]
    A.heap-size = A.heap-size - 1
    MIN-HEAPIFY(A, 1)
    return min
```

```cpp
HEAP-DECREASE-KEY(A, i, key)
    if key > A[i]
        error "new key is larger than current key"
    A[i] = key
    while i > 1 and A[PARENT(i)] > A[i]
        exchange A[i] with A[PARENT(i)]
        i = PARENT(i)
```

```cpp
MIN-HEAP-INSERT(A, key)
    A.heap-size = A.heap-size + 1
    A[A.heap-size] = ∞
    HEAP-DECREASE-KEY(A, A.heap-size, key)
```


answer: In order to pass the guard clause. Otherwise we have to drop the check if $key < A[i]$.
question:
 Why do we bother setting the key of the inserted node to $-\infty$ in line 2 of $\text{MAX-HEAP-INSERT}$ when the next thing we do is increase its key to the desired value?


question:
 Argue the correctness of $\text{HEAP-INCREASE-KEY}$ using the following loop invariant:
>
> At the start of each iteration of the **while** loop of lines 4-6, the subarray $A[1 ..A.heap\text-size]$ satisfies the max-heap property, except that there may be one violation: $A[i]$ may be larger than $A[\text{PARENT}(i)]$.
>
> You may assume that the subarray $A[1..A.heap\text-size]$ satisfies the max-heap property at the time $\text{HEAP-INCREASE-KEY}$ is called.

answer:
**Initialization:** $A$ is a heap except that $A[i]$ might be larger that it's parent, because it has been modified. $A[i]$ is larger than its children, because otherwise the guard clause would fail and the loop will not be entered (the new value is larger than the old value and the old value is larger than the children).

**Maintenance:** When we exchange $A[i]$ with its parent, the max-heap property is satisfied except that now $A[\text{PARENT}(i)]$ might be larger than its parent. Changing $i$ to its parent maintains the invariant.

**Termination:** The loop terminates whenever the heap is exhausted or the max-heap property for $A[i]$ and its parent is preserved. At the loop termination, $A$ is a max-heap.


question:
 Each exchange operation on line 5 of $\text{HEAP-INCREASE-KEY}$ typically requires three assignments. Show how to use the idea of the inner loop of $\text{INSERTION-SORT}$ to reduce the three assignments down to just one assignment.

answer:
```cpp
HEAP-INCREASE-KEY(A, i, key)
    if key < A[i]
        error "new key is smaller than current key"
    while i > 1 and A[PARENT(i)] < key
        A[i] = A[PARENT(i)]
        i = PARENT(i)
    A[i] = key
```


question:
 Show how to implement a first-in, first-out queue with a priority queue. Show how to implement a stack with a priority queue. (Queues and stacks are defined in section 10.1).

answer:
Both are simple. For a stack we keep adding elements in increasing priority, while in a queue we add them in decreasing priority. For the stack we can set the new priority to $\text{HEAP-MAXIMUM}(A) + 1$. For the queue we need to keep track of it and decrease it on every insertion.

Both are not very efficient. Furthermore, if the priority can overflow or underflow, so will eventually need to reassign priorities.


question:
 The operation $\text{HEAP-DELETE}(A, i)$ deletes the item in node $i$ from heap $A$. Give an implementation of $\text{HEAP-DELETE}$ that runs in $O(\lg n)$ time for an $n$-element max-heap.

answer:
```cpp
HEAP-DELETE(A, i)
    if A[i] > A[A.heap-size]
        A[i] = A[A.heap-size]
        MAX-HEAPIFY(A, i)
    else
        HEAP-INCREASE-KEY(A, i, A[A.heap-size])
    A.heap-size = A.heap-size - 1
```

**Note:** The following algorithm is wrong. For example, given an array $A = [15, 7, 9, 1, 2, 3, 8]$ which is a max-heap, and if we delete $A[5] = 2$, then it will fail.

```cpp
HEAP-DELETE(A, i)
    A[i] = A[A.heap-size]
    A.heap-size = A.heap-size - 1
    MAX-HEAPIFY(A, i)
```

- before:

    ```
          15
        /    \
       7      9
      / \    / \
     1   2  3   8
    ```

- after (which is wrong since $8 > 7$ violates the max-heap property):

    ```
          15
        /    \
       7      9
      / \    /
     1   8  3
    ```


question:
 Give an $O(n\lg k)$-time algorithm to merge $k$ sorted lists into one sorted list, where $n$ is the total number of elements in all the input lists. ($\textit{Hint:}$ Use a min-heap for $k$-way merging.)

answer:
We take one element of each list and put it in a min-heap. Along with each element we have to track which list we took it from. When merging, we take the minimum element from the heap and insert another element off the list it came from (unless the list is empty). We continue until we empty the heap.

We have $n$ steps and at each step we're doing an insertion into the heap, which is $\lg k$.

Suppose that sorted lists on input are all nonempty, we have the following pseudocode.

```cpp
def MERGE-SORTED-LISTS(lists)
    n = lists.length
    // Take the lowest element from each of lists together with an index of the list and make list of such pairs.
    // Pairs are of "type" (element-value, index-of-list)
    let lowest-from-each be an empty array
    for i = 1 to n
        add (lists[i][0], i) to lowest-from-each
        delete lists[i][0]
    // This makes min-heap from list lowest-from-each.
    // We are sssuming that pairs of "type" (element-value, index-of-list) are compared according to the values of elements.
    A = MIN-HEAP(lowest-from-each)
    let merged-lists be an empty array
    while not !min-heap.EMPTY()
        element-value, index-of-list = HEAP-EXTRACT-MIN(A)
        add element-value to merged-lists
        if lists[index-of-list].length > 0
            MIN-HEAP-INSERT(A, (lists[index-of-list][0], index-of-list))
            delete lists[index-of-list][0]
    return merged-lists
```


question: What are the minimum and maximum numbers of elements in a heap of height $h$?
answer:
At least $2^h$ and at most $2^{h + 1} − 1$. Can be seen because a complete binary tree of depth $h − 1$ has $\sum_{i = 0}^{h - 1} 2^i = 2^h - 1$ elements, and the number of elements in a heap of depth $h$ is between the number for a complete binary tree of depth $h − 1$ exclusive and the number in a complete binary tree of depth $h$ inclusive.


question: Show that an $n$-element heap has height $\lfloor \lg n \rfloor$.
answer:
Write $n = 2^m − 1 + k$ where $m$ is as large as possible. Then the heap consists of a complete binary tree of height $m − 1$, along with $k$ additional leaves along the bottom. The height of the root is the length of the longest simple path to one of these $k$ leaves, which must have length $m$. It is clear from the way we defined $m$ that $m = \lfloor \lg n\rfloor$.


question:
 Show that in any subtree of a max-heap, the root of the subtree contains the largest value occuring anywhere in the subtree.

answer:
If the largest element in the subtree were somewhere other than the root, it has a parent that is in the subtree. So, it is larger than it's parent, so, the heap property is violated at the parent of the maximum element in the subtree.


question: Where in a max-heap might the smallest element reside, assuming that all elements are distinct?
answer:
In any of the leaves, that is, elements with index $\lfloor n / 2 \rfloor + k$, where $k \geq 1$ (see exercise 6.1-7), that is, in the second half of the heap array.


question: Is an array that is in sorted order a min-heap?
answer:
Yes. For any index $i$, both $\text{LEFT}(i)$ and $\text{RIGHT}(i)$ are larger and thus the elements indexed by them are greater or equal to $A[i]$ (because the array is sorted.)


question:
 Show that, with the array representation for sorting an $n$-element heap, the leaves are the nodes indexed by $\lfloor n / 2 \rfloor + 1, \lfloor n / 2 \rfloor + 2, \ldots, n$.

answer:
Let's take the left child of the node indexed by $\lfloor n / 2 \rfloor + 1$.

$$
\begin{aligned}
\text{LEFT}(\lfloor n / 2 \rfloor + 1)
    & = 2(\lfloor n / 2 \rfloor + 1) \\\\
    & > 2(n / 2 - 1) + 2 \\\\
    & = n - 2 + 2 \\\\
    & = n.
\end{aligned}
$$

Since the index of the left child is larger than the number of elements in the heap, the node doesn't have childrens and thus is a leaf. Same goes for all nodes with larger indices.

Note that if we take element indexed by $\lfloor n / 2 \rfloor$, it will not be a leaf. In case of even number of nodes, it will have a left child with index $n$ and in the case of odd number of nodes, it will have a left child with index $n - 1$ and a right child with index $n$.

This makes the number of leaves in a heap of size $n$ equal to $\lceil n / 2 \rceil$.


question:
 Using figure 6.4 as a model, illustrate the operation of $\text{HEAPSORT}$ on the array $A = \langle 5, 13, 2, 25, 7, 17, 20, 8, 4 \rangle$.

answer:
$$
\begin{aligned}
\langle  5, 13,  2, 25, 7, 17, 20,  8,  4 \rangle \\\\
\langle  5, 13, 20, 25, 7, 17,  2,  8,  4 \rangle \\\\
\langle  5, 25, 20, 13, 7, 17,  2,  8,  4 \rangle \\\\
\langle 25,  5, 20, 13, 7, 17,  2,  8,  4 \rangle \\\\
\langle 25, 13, 20,  5, 7, 17,  2,  8,  4 \rangle \\\\
\langle 25, 13, 20,  8, 7, 17,  2,  5,  4 \rangle \\\\
\langle  4, 13, 20,  8, 7, 17,  2,  5, 25 \rangle \\\\
\langle 20, 13,  4,  8, 7, 17,  2,  5, 25 \rangle \\\\
\langle 20, 13, 17,  8, 7,  4,  2,  5, 25 \rangle \\\\
\langle  5, 13, 17,  8, 7,  4,  2, 20, 25 \rangle \\\\
\langle 17, 13,  5,  8, 7,  4,  2, 20, 25 \rangle \\\\
\langle  2, 13,  5,  8, 7,  4, 17, 20, 25 \rangle \\\\
\langle 13,  2,  5,  8, 7,  4, 17, 20, 25 \rangle \\\\
\langle 13,  8,  5,  2, 7,  4, 17, 20, 25 \rangle \\\\
\langle  4,  8,  5,  2, 7, 13, 17, 20, 25 \rangle \\\\
\langle  8,  4,  5,  2, 7, 13, 17, 20, 25 \rangle \\\\
\langle  8,  7,  5,  2, 4, 13, 17, 20, 25 \rangle \\\\
\langle  4,  7,  5,  2, 8, 13, 17, 20, 25 \rangle \\\\
\langle  7,  4,  5,  2, 8, 13, 17, 20, 25 \rangle \\\\
\langle  2,  4,  5,  7, 8, 13, 17, 20, 25 \rangle \\\\
\langle  5,  4,  2,  7, 8, 13, 17, 20, 25 \rangle \\\\
\langle  2,  4,  5,  7, 8, 13, 17, 20, 25 \rangle \\\\
\langle  4,  2,  5,  7, 8, 13, 17, 20, 25 \rangle \\\\
\langle  2,  4,  5,  7, 8, 13, 17, 20, 25 \rangle
\end{aligned}
$$


question:
 Argue the correctness of $\text{HEAPSORT}$ using the following loop invariant:
>
> At the start of each iteration of the **for** loop of lines 2-5, the subarray $A[1..i]$ is a max-heap containing the $i$ smallest elements of $A[1..n]$, and the subarray $A[i + 1..n]$ contains the $n - i$ largest elements of $A[1..n]$, sorted.

answer:
**Initialization:** The subarray $A[i + 1..n]$ is empty, thus the invariant holds.

**Maintenance:** $A[1]$ is the largest element in $A[1..i]$ and it is smaller than the elements in $A[i + 1..n]$. When we put it in the $i$th position, then $A[i..n]$ contains the largest elements, sorted. Decreasing the heap size and calling $\text{MAX-HEAPIFY}$ turns $A[1..i - 1]$ into a max-heap. Decrementing $i$ sets up the invariant for the next iteration.

**Termination:** After the loop $i = 1$. This means that $A[2..n]$ is sorted and $A[1]$ is the smallest element in the array, which makes the array sorted.


question:
 What is the running time of $\text{HEAPSORT}$ on an array $A$ of length $n$ that is already sorted in increasing order? What about decreasing order?

answer:
Both of them are $\Theta(n\lg n)$.

If the array is sorted in increasing order, the algorithm will need to convert it to a heap that will take $O(n)$. Afterwards, however, there are $n - 1$ calls to $\text{MAX-HEAPIFY}$ and each one will perform the full $\lg k$ operations. Since:

$$\sum_{k = 1}^{n - 1}\lg k = \lg((n - 1)!) = \Theta(n\lg n).$$

Same goes for decreasing order. $\text{BUILD-MAX-HEAP}$ will be faster (by a constant factor), but the computation time will be dominated by the loop in $\text{HEAPSORT}$, which is $\Theta(n\lg n)$.


question: Show that the worst-case running time of $\text{HEAPSORT}$ is $\Omega(n\lg n)$.
answer:
This is essentially the first part of exercise 6.4-3. Whenever we have an array that is already sorted, we take linear time to convert it to a max-heap and then $n\lg n$ time to sort it.


question:
 Show that when all elements are distinct, the best-case running time of $\text{HEAPSORT}$ is $\Omega(n\lg n)$.

answer:
This proved to be quite tricky. My initial solution was wrong. Also, heapsort appeared in 1964, but the lower bound was proved by Schaffer and Sedgewick in 1992. It's evil to put this an exercise.

Let's assume that the heap is a full binary tree with $n = 2^k - 1$. There are $2^{k - 1}$ leaves and $2^{k - 1} - 1$ inner nodes.

Let's look at sorting the first $2^{k - 1}$ elements of the heap. Let's consider their arrangement in the heap and color the leaves to be red and the inner nodes to be blue. The colored nodes are a subtree of the heap (otherwise there would be a contradiction). Since there are $2^{k - 1}$ colored nodes, at most $2^{k - 2}$ are red, which means that at least $2^{k - 2} - 1$ are blue.

While the red nodes can jump directly to the root, the blue nodes need to travel up before they get removed. Let's count the number of swaps to move the blue nodes to the root. The minimal case of swaps is when

1. there are $2^{k - 2} - 1$ blue nodes and
2. they are arranged in a binary tree.

If there are $d$ such blue nodes, then there would be $i = \lg d$ levels, each containing $2^i$ nodes with length $i$. Thus the number of swaps is,

$$\sum_{i = 0}^{\lg d}i2^i = 2 + (\lg d - 2)2^{\lg d} = \Omega(d\lg d).$$

And now for a lazy (but cute) trick. We've figured out a tight bound on sorting half of the heap. We have the following recurrence:

$$T(n) = T(n / 2) + \Omega(n\lg n).$$

Applying the master method, we get that $T(n) = \Omega(n\lg n)$.


question:
 Using figure 6.3 as a model, illustrate the operation of $\text{BUILD-MAX-HEAP}$ on the array $A = \langle 5, 3, 17, 10, 84, 19, 6, 22, 9 \rangle$.

answer:
$$
\begin{aligned}
\langle  5,  3, 17, 10, 84, 19, 6, 22, 9 \rangle \\\\
\langle  5,  3, 17, 22, 84, 19, 6, 10, 9 \rangle \\\\
\langle  5,  3, 19, 22, 84, 17, 6, 10, 9 \rangle \\\\
\langle  5, 84, 19, 22,  3, 17, 6, 10, 9 \rangle \\\\
\langle 84,  5, 19, 22,  3, 17, 6, 10, 9 \rangle \\\\
\langle 84, 22, 19,  5,  3, 17, 6, 10, 9 \rangle \\\\
\langle 84, 22, 19, 10,  3, 17, 6,  5, 9 \rangle \\\\
\end{aligned}
$$


question:
 Why do we want the loop index $i$ in line 2 of $\text{BUILD-MAX-HEAP}$ to decrease from $\lfloor A.length / 2 \rfloor$ to $1$ rather than increase from $1$ to $\lfloor A.length/2 \rfloor$?

answer:
Otherwise we won't be allowed to call $\text{MAX-HEAPIFY}$, since it will fail the condition of having the subtrees be max-heaps. That is, if we start with $1$, there is no guarantee that $A[2]$ and $A[3]$ are roots of max-heaps.


question:
 Show that there are at most $\lceil n / 2^{h + 1} \rceil$ nodes of height $h$ in any $n$-element heap.

answer:
From 6.1-7, we know that the leaves of a heap are the nodes indexed by

$$\left\lfloor n / 2 \right\rfloor + 1, \left\lfloor n / 2 \right\rfloor + 2, \dots, n.$$

Note that those elements corresponds to the second half of the heap array (plus the middle element if $n$ is odd). Thus, the number of leaves in any heap of size $n$ is $\left\lceil n / 2 \right\rceil$. Let's prove by induction. Let $n_h$ denote the number of nodes at height $h$. The upper bound holds for the base since $n_0 = \left\lceil n / 2 \right\rceil$ is exactly the number of leaves in a heap of size $n$.

Now assume it holds for $h − 1$. We have prove that it also holds for $h$. Note that if $n_{h - 1}$ is even each node at height $h$ has exactly two children, which implies $n_h = n_{h - 1} / 2 = \left\lfloor n_{h - 1} / 2 \right\rfloor$. If $n_{h - 1}$ is odd, one node at height $h$ has one child and the remaining has two children, which also implies $n_h = \left\lfloor n_{h - 1} / 2 \right\rfloor + 1 = \left\lceil n_{h - 1} / 2 \right\rceil$. Thus, we have

$$
\begin{aligned}
n_h & =   \left\lceil \frac{n_{h - 1}}{2} \right\rceil \\\\
    & \le \left\lceil \frac{1}{2} \cdot \left\lceil \frac{n}{2^{(h - 1) + 1}} \right\rceil \right\rceil \\\\
    & =   \left\lceil \frac{1}{2} \cdot \left\lceil \frac{n}{2^h} \right\rceil \right\rceil \\\\
    & =   \left\lceil \frac{n}{2^{h + 1}} \right\rceil,
\end{aligned}
$$

which implies that it holds for $h$.


question:
 Using figure 6.2 as a model, illustrate the operation of $\text{MAX-HEAPIFY}(A, 3)$ on the array $A = \langle 27, 17, 3, 16, 13, 10, 1, 5, 7, 12, 4, 8, 9, 0 \rangle$.

answer:
$$
\begin{aligned}
\langle 27, 17, 3,  16, 13, 10,1, 5, 7, 12, 4, 8, 9, 0 \rangle \\\\
\langle 27, 17, 10, 16, 13, 3, 1, 5, 7, 12, 4, 8, 9, 0 \rangle \\\\
\langle 27, 17, 10, 16, 13, 9, 1, 5, 7, 12, 4, 8, 3, 0 \rangle \\\\
\end{aligned}
$$


question:
 Starting with the procedure $\text{MAX-HEAPIFY}$, write pseudocode for the procedure $\text{MIN-HEAPIFY}(A, i)$, which performs the corresponding manipulation on a min-heap. How does the running time of $\text{MIN-HEAPIFY}$ compare to that of $\text{MAX-HEAPIFY}$?

answer:
```cpp
MIN-HEAPIFY(A, i)
    l = LEFT(i)
    r = RIGHT(i)
    if l ≤ A.heap-size and A[l] < A[i]
        smallest = l
    else smallest = i
    if r ≤ A.heap-size and A[r] < A[smallest]
        smallest = r
    if smallest != i
        exchange A[i] with A[smallest]
        MIN-HEAPIFY(A, smallest)
```

The running time is the same. Actually, the algorithm is the same with the exceptions of two comparisons and some names.


question:
 What is the effect of calling $\text{MAX-HEAPIFY}(A, i)$ when the element $A[i]$ is larger than its children?

answer:
No effect. The comparisons are carried out, $A[i]$ is found to be largest and the procedure just returns.


question: What is the effect of calling $\text{MAX-HEAPIFY}(A, i)$ for $i > A.heap\text-size / 2$?
answer:
No effect. In that case, it is a leaf. Both $\text{LEFT}$ and $\text{RIGHT}$ return values that fail the comparison with the heap size and $i$ is stored in largest. Afterwards the procedure just returns.


question:
 The code for $\text{MAX-HEAPIFY}$ is quite efficient in terms of constant factors, except possibly for the recursive call in line 10, which might cause some compilers to produce inefficient code. Write an efficient $\text{MAX-HEAPIFY}$ that uses an iterative control construct (a loop) instead of recursion.

answer:
```cpp
MAX-HEAPIFY(A, i)
    while true
        l = LEFT(i)
        r = RIGHT(i)
        if l ≤ A.heap-size and A[l] > A[i]
            largest = l
        else largest = i
        if r ≤ A.heap-size and A[r] > A[largest]
            largest = r
        if largest == i
            return
        exchange A[i] with A[largest]
        i = largest
```


question:
 Show that the worst-case running time of $\text{MAX-HEAPIFY}$ on a heap of size $n$ is $\Omega(\lg n)$. ($\textit{Hint:}$ For a heap with $n$ nodes, give node values that cause $\text{MAX-HEAPIFY}$ to be called recursively at every node on a simple path from the root down to a leaf.)

answer:
Consider the heap resulting from $A$ where $A[1] = 1$ and $A[i] = 2$ for $2 \le i \le n$. Since $1$ is the smallest element of the heap, it must be swapped through each level of the heap until it is a leaf node. Since the heap has height $\lfloor \lg n\rfloor$, $\text{MAX-HEAPIFY}$ has worst-case time $\Omega(\lg n)$.


question:
 Professor Marceau objects to the loop invariant used in the proof of Lemma 5.5. He questions whether it is true prior to the first iteration. He reasons that we could just as easily declare that an empty subarray contains no $0$-permutations. Therefore, the probability that an empty subarray contains a $0$-permutation should be $0$, thus invalidating the loop invariant prior to the first iteration. Rewrite the procedure $\text{RANDOMIZE-IN-PLACE}$ so that its associated loop invariant applies to a nonempty subarray prior to the first iteration, and modify the proof of Lemma 5.5 for your procedure.

answer:
Modify the algorithm by unrolling the $i = 1$ case.

```cpp
swap(A[1], A[RANDOM(1, n)])
for i = 2 to n
    swap(A[i], A[RANDOM(i, n)])
```

Modify the proof of Lemma 5.5 by starting with $i = 2$ instead of $i = 1$. This resolves the issue of $0$-permutations.


question:
 Professor Kelp decides to write a procedure that produces at random any permutation besides the identity permutation. He proposes the following procedure:
>
> ```cpp
> PERMUTE-WITHOUT-IDENTITY(A)
>     n = A.length
>     for i = 1 to n - 1
>         swap A[i] with A[RANDOM(i + 1, n)]
> ```
>
> Does this code do what Professor Kelp intends?

answer:
The code does not do what he intends. Suppose $A = [1, 2, 3]$. If the algorithm worked as proposed, then with nonzero probability the algorithm should output $[3, 2, 1]$. On the first iteration we swap $A[1]$ with either $A[2]$ or $A[3]$. Since we want $[3, 2, 1]$ and will never again alter $A[1]$, we must necessarily swap with $A[3]$. Now the current array is $[3, 2, 1]$. On the second (and final) iteration, we have no choice but to swap $A[2]$ with $A[3]$, so the resulting array is $[3, 1, 2]$. Thus, the procedure cannot possibly be producing random non-identity permutations.


question:
 Suppose that instead of swapping element $A[i]$ with a random element from the subarray $A[i..n]$, we swapped it with a random element from anywhere in the array:
>
> ```cpp
> PERMUTE-WITH-ALL(A)
>     n = A.length
>     for i = 1 to n
>         swap A[i] with A[RANDOM(1, n)]
> ```
>
> Does this code produce a uniform random permutation? Why or why not?

answer:
Consider the case of $n = 3$ in running the algorithm, three IID choices will be made, and so you'll end up having $27$ possible end states each with equal probability. There are $3! = 6$ possible orderings, these should appear equally often, but this can't happen because $6$ does not divide $27$.


question:
 Professor Armstrong suggests the following procedure for generating a uniform random permutation:
>
> ```cpp
> PERMUTE-BY-CYCLIC(A)
>     n = A.length
>     let B[1..n] be a new array
>     offset = RANDOM(1, n)
>     for i = 1 to n
>         dest = i + offset
>         if dest > n
>             dest = dest - n
>         B[dest] = A[i]
>     return B
> ```
>
> Show that each element $A[i]$ has a $1 / n$ probability of winding up in any particular position in $B$. Then show that Professor Armstrong is mistaken by showing that the resulting permutation is not uniformly random.

answer:
Fix a position $j$ and an index $i$. We'll show that the probability that $A[i]$ winds up in position $j$ is $1 / n$. The probability $B[j] = A[i]$ is the probability that $dest = j$, which is the probability that $i + offset$ or $i + offset − n$ is equal to $j$, which is $1 / n$.

This algorithm can't possibly return a random permutation because it doesn't change the relative positions of the elements; it merely cyclically permutes the whole permutation.

For instance, suppose $A = [1, 2, 3]$,

- if $offset = 1$, $B = [3, 1, 2]$,
- if $offset = 2$, $B = [2, 3, 1]$,
- if $v = 3$, $B = [1, 2, 3]$.

Thus, the algorithm will never produce $B = [1, 3, 2]$, so the resulting permutation cannot be uniformly random.


question:
 Prove that in the array $P$ in procedure $\text{PERMUTE-BY-SORTING}$, the probability that all elements are unique is at least $1 - 1 / n$.

answer:
Let $\Pr\\{j\\}$ be the probability that the element with index $j$ is unique. If there are $n^3$ elements, then the $\Pr\\{j\\} = 1 - \frac{j - 1}{n^3}$.

$$
\begin{aligned}
\Pr\\{1 \cap 2 \cap 3 \cap \ldots\\}
    & =   \Pr\\{1\\} \cdot \Pr\\{2 \mid 1\\} \cdot \Pr\\{3 \mid 1 \cap 2\\} \cdots \\\\
    & =   1 (1 - \frac{1}{n^3})(1 - \frac{2}{n^3})(1 - \frac{3}{n^3}) \cdots \\\\
    & \ge 1 (1 - \frac{n}{n^3}) (1 - \frac{n}{n^3})(1 - \frac{n}{n^3}) \cdots \\\\
    & \ge (1 - \frac{1}{n^2})^n \\\\
    & \ge 1 - \frac{1}{n}, \\\\
\end{aligned}
$$

where the last step holds for $(1 - x)^n \ge 1 - nx$.


question:
 Explain how to implement the algorithm $\text{PERMUTE-BY-SORTING}$ to handle the case in which two or more priorities are identical. That is, your algorithm should produce a uniform random permutation, even if two or more priorities are identical.

answer:
```cpp
PERMUTE-BY-SORTING(A)
    let P[1..n] be a new array
    for i = 1 to n
        P[i] = i
    for i = 1 to n
        swap P[i] with P[RANDOM(i, n)]
```


question:
 Suppose we want to create a **_random sample_** of the set $\\{1, 2, 3, \ldots, n\\}$, that is, an $m$-element subset $S$, where $0 \le m \le n$, such that each $m$-subset is equally likely to be created. One way would be to set $A[i] = i$ for $i = 1, 2, 3, \ldots, n$, call $\text{RANDOMIZE-IN-PLACE}(A)$, and then take just the first $m$ array elements. This method would make $n$ calls to the $\text{RANDOM}$ procedure. If $n$ is much larger than $m$, we can create a random sample with fewer calls to $\text{RANDOM}$. Show that the following recursive procedure returns a random $m$-subset $S$ of $\\{1, 2, 3, \ldots, n\\}$, in which each $m$-subset is equally likely, while making only $m$ calls to $\text{RANDOM}$:
>
> ```cpp
> RANDOM-SAMPLE(m, n)
>     if m == 0
>         return Ø
>     else S = RANDOM-SAMPLE(m - 1, n - 1)
>         i = RANDOM(1, n)
>         if i ∈ S
>             S = S ∪ {n}
>         else S = S ∪ {i}
>         return S
> ```

answer:
We prove that it produces a random $m$ subset by induction on $m$. It is obviously true if $m = 0$ as there is only one size $m$ subset of $[n]$. Suppose $S$ is a uniform $m − 1$ subset of $n − 1$, that is, $\forall j \in [n - 1]$, $\Pr[j \in S] = \frac{m - 1}{n - 1}$.

If we let $S'$ denote the returned set, suppose first $j \in [n − 1]$,

$$
\begin{aligned}
\Pr[j \in S']
    & = \Pr[j \in S] + \Pr[j \notin S \wedge i = j] \\\\
    & = \frac{m - 1}{n - 1} + \Pr[j \notin S]\Pr[i = j] \\\\
    & = \frac{m - 1}{n - 1} + \left(1 - \frac{m - 1}{n - 1}\right) \frac{1}{n} \\\\
    & = \frac{n(m - 1) + n - m}{(n - 1)n} \\\\
    & = \frac{nm - m}{(n - 1)n} = \frac{m}{n}.
\end{aligned}
$$

Since the constructed subset contains each of $[n − 1]$ with the correct probability, it must also contain $n$ with the correct probability because the probabilities sum to $1$.


question:
 How many people must there be in a room before the probability that someone has the same birthday as you do is at least $1 / 2$? How many people must there be before the probability that at least two people have a birthday on July 4 is greater than $1 / 2$?

answer:
The probability of a person not having the same birthday as me is $(n - 1) / n$. The probability of $k$ people not having the same birthday as me is that, squared. We apply the same approach as the text - we take the complementary event and solve it for $k$,

$$
\begin{aligned}
        1 - \big(\frac{n - 1}{n}\big)^k & \ge \frac{1}{2} \\\\
            \big(\frac{n - 1}{n}\big)^k & \le \frac{1}{2} \\\\
          k\lg\big(\frac{n - 1}{n}\big) & \ge \lg\frac{1}{2} \\\\
k = \frac{\log(1 / 2)}{\log(364 / 365)} & \approx 253.
\end{aligned}
$$

As for the other question,

$$
\begin{aligned}
\Pr\\{\text{2 born on Jul 4}\\}
    & = 1 - \Pr\\{\text{1 born on Jul 4}\\} - \Pr\\{\text{0 born on Jul 4}\\} \\\\
    & = 1 - \frac{k}{n}\big(\frac{n - 1}{n}\big)^{k - 1} - \big(\frac{n - 1}{n}\big)^k \\\\
    & = 1 - \big(\frac{n - 1}{n}\big)^{k - 1}\big(\frac{n + k - 1}{n}\big).
\end{aligned}
$$

Writing a Ruby programme to find the closest integer, we get $115$.


question:
 Suppose that we toss balls into $b$ bins until some bin contains two balls. Each toss is independent, and each ball is equally likely to end up in any bin. What is the expected number of ball tosses?

answer:
This is just a restatement of the birthday problem. I consider this all that needs to be said on this subject.


answer: Pairwise independence is enough. It's sufficient for the derivation after $\text{(5.6)}$.
question:
 For the analysis of the birthday paradox, is it important that the birthdays be mutually independent, or is pairwise independence sufficient? Justify your answer.


question:
 How many people should be invited to a party in order to make it likely that there are $three$ people with the same birthday?

answer:
The answer is $88$. I reached it by trial and error. But let's analyze it with indicator random variables.

Let $X_{ijk}$ be the indicator random variable for the event of the people with indices $i$, $j$ and $k$ have the same birthday. The probability is $1 / n^2$. Then,

$$
\begin{aligned}
\text E[X]
    & = \sum_{i = 1}^n\sum_{j = i + 1}^n \sum_{k = j + 1}^n X_{ijk} \\\\
    & = \sum_{i = 1}^n\sum_{j = i + 1}^n \sum_{k = j + 1}^n \frac{1}{n^2} \\\\
    & = \binom{n}{3}\frac{1}{n^2} \\\\
    & = \frac{k(k - 1)(k - 2)}{6n^2}.
\end{aligned}
$$

Solving this yields $94$. It's a bit more, but again, indicator random variables are approximate.

Finding more commentary online is tricky.


question:
 What is the probability that a $k$-string over a set of size $n$ forms a $k$-permutation? How does this question relate to the birthday paradox?

answer:
$$
\begin{aligned}
\Pr\\{k\text{-perm in }n\\}
    & = 1 \cdot \frac{n - 1}{n} \cdot \frac{n - 2}{n} \cdots \frac{n - k + 1}{n} \\\\
    & = \frac{n!}{n^k(n - k)!}.
\end{aligned}
$$

This is the complementary event to the birthday problem, that is, the chance of $k$ people have distinct birthdays.


question:
 Suppose that $n$ balls are tossed into $n$ bins, where each toss is independent and the ball is equally likely to end up in any bin. What is the expected number of empty bins? What is the expected number of bins with exactly one ball?

answer:
Let $X_i$ be the indicator variable that bin $i$ is empty after all balls are tossed and $X$ be the random variable that gives the number of empty bins. Thus we have

$$E[X] = \sum_{i = 1}^n E[X_i] = \sum_{i = 1}^n \bigg(\frac{n - 1}{n}\bigg)^n = n\bigg(\frac{n - 1}{n}\bigg)^n.$$

Let $X_i$ be the indicator variable that bin $i$ contains exactly $1$ ball after all balls are tossed and $X$ be the random variable that gives the number of bins containing exactly $1$ ball. Thus we have

$$E[X] = \sum_{i = 1}^n E[X_i] = \sum_{i = 1}^n \binom{n}{1}\bigg(\frac{n - 1}{n}\bigg)^{n - 1} \frac{1}{n} = n\bigg(\frac{n - 1}{n}\bigg)^{n - 1},$$

because we need to choose which toss will go into bin $i$, then multiply by the probability that that toss goes into that bin and the remaining $n − 1$ tosses avoid it.


question:
 Sharpen the lower bound on streak length by showing that in $n$ flips of a fair coin, the probability is less than $1 / n$ that no streak longer than $\lg n - 2\lg\lg n$ consecutive heads occurs.

answer:
We split up the n flips into $n / s$ groups where we pick $s = \lg(n) - 2 \lg(\lg(n))$. We will show that at least one of these groups comes up all heads with probability at least $\frac{n - 1}{n}$. So, the probability the group starting in position $i$ comes up all heads is

$$\Pr(A_{i,\lg n - 2\lg(\lg n)}) = \frac{1}{2^{\lg n - 2\lg(\lg n)}} = \frac{\lg n^2}{n}.$$

Since the groups are based of of disjoint sets of IID coin flips, these probabilities
are independent. so,

$$
\begin{aligned}
\Pr(\bigwedge\neg A_{i,\lg n - 2\lg(\lg n)})
    & =   \prod_i\Pr(\neg A_{i,\lg n - 2\lg(\lg n)}) \\\\
    & =   \Big(1-\frac{\lg n^2}{n}\Big)^{\frac{n}{\lg n - 2\lg(\lg n)}} \\\\
    & \le e^{-\frac{\lg n^2}{\lg n - 2\lg(\lg n)}} \\\\ &= \frac{1}{n} e^{\frac{-2\lg(\lg n)\lg n}{\lg n - 2\lg(\lg n)}} \\\\
    & =   n^{-1-\frac{2\lg(\lg n)}{\lg n - 2\lg(\lg n)}} \\\\
& < n^{-1}.
\end{aligned}
$$

Showing that the probability that there is no run of length at least $\lg n - 2\lg(\lg n)$ to be $< \frac{1}{n}$.


question:
 Show that the assumption that we are always able to determine which candidate is best in line 4 of procedure $\text{HIRE-ASSISTANT}$ implies that we know a total order on the ranks of the candidates.

answer:
A total order is a partial order that is a total relation $(\forall a, b \in A:aRb \text{ or } bRa)$.
A relation is a partial order if it is reflexive, antisymmetric and transitive.

Assume that the relation is good or better.

- **Reflexive:** This is a bit trivial, but everybody is as good or better as themselves.
- **Transitive:** If $A$ is better than $B$ and $B$ is better than $C$, then $A$ is better than $C$.
- **Antisymmetric:** If $A$ is better than $B$, then $B$ is not better than $A$.

So far we have a partial order.

Since we assume we can compare any two candidates, then comparison must be a total relation and thus we have a total order.


question:
 Describe an implementation of the procedure $\text{RANDOM}(a, b)$ that only makes calls to $\text{RANDOM}(0, 1)$. What is the expected running time of your procedure, as a function of $a$ and $b$?

answer:
As $(b - a)$ could be any number, we need at least $\lceil \lg(b - a) \rceil$ bits to represent the number. We set $\lceil \lg(b - a) \rceil$ as $k$. Basically, we need to call $\text{RANDOM}(0, 1)$ $k$ times. If the number represented by binary is bigger than $b - a$, it's not valid number and we give it another try, otherwise we return that number.

```cpp
RANDOM(a, b)
    range = b - a
    bits = ceil(log(2, range))
    result = 0
    for i = 0 to bits - 1
        r = RANDOM(0, 1)
        result = result + r << i
    if result > range
        return RANDOM(a, b)
    else return a + result
```

The expectation of times of calling procedure $\text{RANDOM}(a, b)$ is $\frac{2^k}{b - a}$. $\text{RANDOM}(0, 1)$ will be called $k$ times in that procedure.

The expected running time is $\Theta(\frac{2^k}{b - a} \cdot k)$, $k$ is $\lceil \lg(b - a) \rceil$.
Considering $2^k$ is less than $2 \cdot (b - a)$, so the running time is $O(k)$.


question:
 Suppose that you want to output $0$ with probability $1 / 2$ and $1$ with probability $1 / 2$. At your disposal is a procedure $\text{BIASED-RANDOM}$, that outputs either $0$ or $1$. It outputs $1$ with some probability $p$ and $0$ with probability $1 - p$, where $0 < p < 1$, but you do not know what $p$ is. Give an algorithm that uses $\text{BIASED-RANDOM}$ as a subroutine, and returns an unbiased answer, returning $0$ with probability $1 / 2$ and $1$ with probability $1 / 2$. What is the expected running time of your algorithm as a function of $p$?

answer:
There are 4 outcomes when we call $\text{BIASED-RANDOM}$ twice, i.e., $00$, $01$, $10$, $11$.

The strategy is as following:

- $00$ or $11$: call $\text{BIASED-RANDOM}$ twice again
- $01$: output $0$
- $10$: output $1$

We can calculate the probability of each outcome:

- $\Pr\\{00 | 11\\} = p^2 + (1 - p)^2$
- $\Pr\\{01\\} = (1 - p)p$
- $\Pr\\{10\\} = p(1 - p)$

Since there's no other way to return a value, it returns $0$ and $1$ both with probability $1 / 2$.

The pseudo code is as follow:

```cpp
UNBIASED-RANDOM
    while true
        x = BIASED-RANDOM
        y = BIASED-RANDOM
        if x != y
            return x
```

This algorithm actually uses the equivalence of the probability of occurrence of $01$ and $10$, and subtly converts the unequal $00$ and $11$ to $01$ and $10$, thus eliminating the probability that its probability is not equivalent.

Each iteration is a Bernoulli trial, where "success" means that the iteration does return a value.

We can view each iteration as a Bernoulli trial, where "success" means that the iteration returns a value.

$$
\begin{aligned}
\Pr\\{\text{success}\\}
    & = \Pr\\{0\text{ is returned}\\} + \Pr\\{1\text{ is returned}\\} \\\\
    & = 2p(1 - p).
\end{aligned}
$$

The expected number of trials for this scenario is $1 / (2p(1 - p))$. Thus, the expected running time of $\text{UNBIASED-RANDOM}$ is $\Theta(1 / (2p(1 - p))$.


question:
 In $\text{HIRE-ASSISTANT}$, assuming that the candidates are presented in a random order, what is the probability that you hire exactly one time? What is the probability you hire exactly $n$ times?

answer:
You will hire exactly one time if the best candidate is at first. There are $(n − 1)!$ orderings with the best candidate being at first, so the probability that you hire exactly one time is $\frac{(n - 1)!}{n!} = \frac{1}{n}$.

You will hire exactly $n$ times if the candidates are presented in increasing order. There is only an ordering for this situation, so the probability that you hire exactly $n$ times is $\frac{1}{n!}$.


question:
 In $\text{HIRE-ASSISTANT}$, assuming that the candidates are presented in a random order, what is the probability that you hire exactly twice?

answer:
Note that

- Candidate $1$ is always hired
- The best candidate (candidate whose rank is $n$) is always hired
- If the best candidate is candidate $1$, then that's the only candidate hired.

In order for $\text{HIRE-ASSISTANT}$ to hire exactly twice, candidate $1$ should have rank $i$, where $1 \le i \le n - 1$, and all candidates whose ranks are $i + 1, i + 2, \dots, n - 1$ should be interviewed after the candidate whose rank is $n$ (the best candidate).

Let $E_i$ be the event in which candidate $1$ have rank $i$, so we have $P(E_i) = 1 / n$ for $1 \le i \le n$.

Our goal is to find for $1 \le i \le n - 1$, given $E_i$ occurs, i.e., candidate $1$ has rank $i$, the candidate whose rank is $n$ (the best candidate) is the first one interviewed out of the $n - i$ candidates whose ranks are $i + 1, i + 2, \dots, n$.

So,

$$\sum_{i = 1}^{n - 1} P(E_i) \cdot \frac{1}{n - i} = \sum_{i = 1}^{n - 1} \frac{1}{n} \cdot \frac{1}{n - i}.$$


question: Use indicator random variables to compute the expected value of the sum of $n$ dice.
answer:
Expectation of a single dice $X_i$ is

$$
\begin{aligned}
\text E[X_k] & = \sum_{i = 1}^6 i \Pr\\{X_k = i\\} \\\\
             & = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} \\\\
             & = \frac{21}{6} \\\\
             & = 3.5.
\end{aligned}
$$

As for multiple dices,

$$
\begin{aligned}
\text E[X] & = \text E\Bigg[\sum_{i = 1}^n X_i \Bigg] \\\\
           & = \sum_{i = 1}^n \text E[X_i] \\\\
           & = \sum_{i = 1}^n 3.5 \\\\
           & = 3.5 \cdot n.
\end{aligned}
$$


question:
 Use indicator random variables to solve the following problem, which is known as the **_hat-check problem_**. Each of $n$ customers gives a hat to a hat-check person at a restaurant. The hat-check person gives the hats back to the customers in a random order. What is the expected number of customers who get back their hat?

answer:
Let $X$ be the number of customers who get back their own hat and $X_i$ be the indicator random variable that customer $i$ gets his hat back. The probability that an individual gets his hat back is $\frac{1}{n}$. Thus we have

$$E[X] = E\Bigg[\sum_{i = 1}^n X_i\Bigg] = \sum_{i = 1}^n E[X_i] = \sum_{i = 1}^n \frac{1}{n} = 1.$$


question:
 Let $A[1..n]$ be an array of $n$ distinct numbers. If $i < j$ and $A[i] > A[j]$, then the pair $(i, j)$ is called an **_inversion_** of $A$. (See Problem 2-4 for more on inversions.) Suppose that the elements of $A$ form a uniform random permutation of $\langle 1, 2, \ldots, n \rangle$. Use indicator random variables to compute the expected number of inversions.

answer:
Let $X_{i, j}$ for $i < j$ be the indicator of $A[i] > A[j]$. We have that the expected number of inversions

$$
\begin{aligned}
\text E\Bigg[\sum_{i < j} X_{i, j}\Bigg]
    & = \sum_{i < j} E[X_{i, j}] \\\\
    & = \sum_{i = 1}^{n - 1}\sum_{j = i + 1}^n \Pr\\{A[i] > A[j]\\} \\\\
    & = \frac{1}{2} \sum_{i = 1}^{n - 1} n - i \\\\
    & = \frac{n(n - 1)}{2} - \frac{n(n - 1)}{4} \\\\
    & = \frac{n(n - 1)}{4}.
\end{aligned}
$$


question: Write pseudocode for $\text{RIGHT-ROTATE}$.
answer:
```cpp
RIGHT-ROTATE(T, y)
    x = y.left
    y.left = x.right
    if x.right != T.nil
        x.right.p = y
    x.p = y.p
    if y.p == T.nil
        T.root = x
    else if y == y.p.right
        y.p.right = x
    else y.p.left = x
    x.right = y
    y.p = x
```


question: Argue that in every $n$-node binary search tree, there are exactly $n - 1$ possible rotations.
answer:
Every node can rotate with its parent, only the root does not have a parent, therefore there are $n - 1$ possible rotations.


answer: - $a$: increase by $1$. - $b$: unchanged. - $c$: decrease by $1$.
question:
 Let $a$, $b$, and $c$ be arbitrary nodes in subtrees $\alpha$, $\beta$, and $\gamma$, respectively, in the left tree of Figure 13.2. How do the depths of $a$, $b$, and $c$ change when a left rotation is performed on node $x$ in the figure?


question:
 Show that any arbitrary $n$-node binary search tree can be transformed into any other arbitrary $n$-node binary search tree using $O(n)$ rotations. ($\textit{Hint:}$ First show that at most $n - 1$ right rotations suffice to transform the tree into a right-going chain.)

answer:
Consider transforming an arbitrary $n$-node binary tree into a right-going chain as follows:

Let the root and all successive right children of the root be the elements of the chain initial chain. For any node $x$ which is a left child of a node on the chain, a single right rotation on the parent of $x$ will add that node to the chain and not remove any elements from the chain. Thus, we can convert any binary search tree to a right chain with at most $n − 1$ right rotations.

Let $r_1, r_2, \dots, r_k$ be the sequence of rotations required to convert some binary search tree $T_1$ into a right-going chain, and let $s_1, s_2, \dots, s_m$ be the sequence of rotations required to convert some other binary search tree $T_2$ to a right-going chain. Then $k < n$ and $m < n$, and we can convert $T_1$ to $T_2$ by performing the sequence $r_1, r_2, \dots, r_k, s_m', s_{m - 1}', \dots, s_1'$ where $s_i'$ is the opposite rotation of $s_i$. Since $k + m < 2n$, the number of rotations required is $O(n)$.


question:
 We say that a binary search tree $T_1$ can be **_right-converted_** to binary search tree $T_2$ if it is possible to obtain $T_2$ from $T_1$ via a series of calls to $\text{RIGHT-ROTATE}$. Give an example of two trees $T_1$ and $T_2$ such that $T_1$ cannot be right-converted to $T_2$. Then, show that if a tree $T_1$ can be right-converted to $T_2$, it can be right-converted using $O(n^2)$ calls to $\text{RIGHT-ROTATE}$.

answer:
We can use $O(n)$ calls to rotate the node which is the root in $T_2$ to $T_1$'s root, then use the same operation in the two subtrees. There are $n$ nodes, therefore the upper bound is $O(n^2)$.


question: Argue that after executing $\text{RB-DELETE-FIXUP}$, the root of the tree must be black.
answer:
- **Case 1:** transform to 2, 3, 4.
- **Case 2:** if terminates, the root of the subtree (the new $x$) is set to black.
- **Case 3:** transform to 4.
- **Case 4:** the root (the new $x$) is set to black.


question:
 Argue that if in $\text{RB-DELETE}$ both $x$ and $x.p$ are red, then property 4 is restored by the call to $\text{RB-DELETE-FIXUP}(T, x)$.

answer:
Suppose that both $x$ and $x.p$ are red in $\text{RB-DELETE}$. This can only happen
in the else-case of line 9. Since we are deleting from a red-black tree, the other child of y.p which becomes $x$'s sibling in the call to $\text{RB-TRANSPLANT}$ on line 14 must be black, so $x$ is the only child of $x.p$ which is red. The while-loop condition of $\text{RB-DELETE-FIXUP}(T, x)$ is immediately violated so we simply set $x.color = black$, restoring property 4.


question:
 In Exercise 13.3-2, you found the red-black tree that results from successively inserting the keys $41, 38, 31, 12, 19, 8$ into an initially empty tree. Now show the red-black trees that result from the successive deletion of the keys in the order $8, 12, 19, 31, 38, 41$.

answer:
- initial:

    ![](../img/13.4-3-1.png)

- delete $8$:

    ![](../img/13.4-3-2.png)

- delete $12$:

    ![](../img/13.4-3-3.png)

- delete $19$:

    ![](../img/13.4-3-4.png)

- delete $31$:

    ![](../img/13.4-3-5.png)

- delete $38$:

    ![](../img/13.4-3-6.png)

- delete $41$:

    ![](../img/13.4-3-7.png)


question:
 In which lines of the code for $\text{RB-DELETE-FIXUP}$ might we examine or modify the sentinel $T.nil$?

answer:
When the node $y$ in $\text{RB-DELETE}$ has no children, the node $x = T.nil$, so we'll examine the line 2 of $\text{RB-DELETE-FIXUP}$.

When the root node is deleted, $x = T.nil$ and the root at this time is $x$, so the line 23 of $\text{RB-DELETE-FIXUP}$ will draw $x$ to black.


question:
 In each of the cases of Figure 13.7, give the count of black nodes from the root of the subtree shown to each of the subtrees $\alpha, \beta, \ldots, \zeta$, and verify that each count remains the same after the transformation. When a node has a $color$ attribute $c$ or $c'$, use the notation $\text{count}\(c\)$ or $\text{count}(c')$ symbolically in your count.

answer:
Our count will include the root (if it is black).

- **Case 1:** For each subtree, it is $2$ both before and after.

- **Case 2:**

    - For $\alpha$ and $\beta$, it is $1 + \text{count}\(c\)$ in both cases.
    - For the rest of the subtrees, it is from $2 + \text{count}\(c\)$ to $1 + \text{count}\(c\)$.

    This decrease in the count for the other subtreese is handled by then having $x$ represent an additional black.

- **Case 3:**

    - For $\epsilon$ and $\zeta$, it is $2+\text{count}\(c\)$ both before and after.
    - For all the other subtrees, it is $1+\text{count}\(c\)$ both before and after.

- **Case 4:**

    - For $\alpha$ and $\beta$, it is from $1 + \text{count}\(c\)$ to $2 + \text{count}\(c\)$.
    - For $\gamma$ and $\delta$, it is $1 + \text{count}\(c\) + \text{count}(c')$ both before and after.
    - For $\epsilon$ and $\zeta$, it is $1 + \text{count}\(c\)$ both before and after.

This increase in the count for $\alpha$ and $\beta$ is because $x$ before indicated an extra black.


question:
 Professors Skelton and Baron are concerned that at the start of case 1 of $\text{RB-DELETE-FIXUP}$, the node $x.p$ might not be black. If the professors are correct, then lines 5–6 are wrong. Show that $x.p$ must be black at the start of case 1, so that the professors have nothing to worry about.

answer:
At the start of case 1 we have set $w$ to be the sibling of $x$. We check on line 4 that $w.color == red$, which means that the parent of $x$ and $w$ cannot be red. Otherwise property 4 is violated. Thus, their concerns are unfounded.


question:
 Suppose that a node $x$ is inserted into a red-black tree with $\text{RB-INSERT}$ and then is immediately deleted with $\text{RB-DELETE}$. Is the resulting red-black tree the same as the initial red-black tree? Justify your answer.

answer:
No, the red-black tree will not necessarily be the same.

- Example 1:

    - initial:

        ![](../img/13.4-7-1.png)

    - insert $1$:

        ![](../img/13.4-7-2.png)

    - delete $1$:

        ![](../img/13.4-7-3.png)

- Example 2:

    - initial:

        ![](../img/13.4-7-4.png)

    - insert $1$:

        ![](../img/13.4-7-5.png)

    - delete $1$:

        ![](../img/13.4-7-6.png)


question:
 In the style of Figure 13.1(a), draw the complete binary search tree of height $3$ on the keys $\\{1, 2, \ldots, 15\\}$. Add the $\text{NIL}$ leaves and color the nodes in three different ways such that the black-heights of the resulting red-black trees are $2$, $3$, and $4$.

answer:
- Complete binary tree of $height = 3$:

  ![](../img/13.1-1-1.png)

- Red-black tree of $black\text-heights = 2$:

  ![](../img/13.1-1-2.png)

- Red-black tree of $black\text-heights = 3$:

  ![](../img/13.1-1-3.png)

- Red-black tree of $black\text-heights = 4$:

  ![](../img/13.1-1-4.png)


question:
 Draw the red-black tree that results after $\text{TREE-INSERT}$ is called on the tree in Figure 13.1 with key $36$. If the inserted node is colored red, is the resulting tree a red-black tree? What if it is colored black?

answer:
- If the inserted node is colored red, the tree doesn't satisfy property 4 because $35$ will be the parent of $36$, which is also colored red.
- If the inserted node is colored black, the tree doesn't satisfy property 5 because there will be two paths from node $38$ to $T.nil$ which contain different numbers of black nodes.

We don't draw the _wrong_ red-black tree; however, we draw the adjusted correct tree:

![](../img/13.1-2-1.png)


question:
 Let us define a **_relaxed red-black tree_** as a binary search tree that satisfies red-black properties 1, 3, 4, and 5. In other words, the root may be either red or black. Consider a relaxed red-black tree $T$ whose root is red. If we color the root of $T$ black but make no other changes to $T$, is the resulting tree a red-black tree?

answer:
Yes, it is.

- Property 1 is trivially satisfied since only one node is changed and it is not changed to some mysterious third color.
- Property 3 is trivially satisfied since no new leaves are introduced.
- Property 4 is satisfied since there was no red node introduced, and root is in every path from the root to the leaves, but no others.
- Property 5 is satisfied since the only paths we will be changing the number of black nodes in are those coming from the root. All of these will increase by $1$, and so will all be equal.


question:
 Suppose that we "absorb" every red node in a red-black tree into its black parent, so that the children of the red node become children of the black parent. (Ignore what happens to the keys.) What are the possible degrees of a black node after all its red children are absorbed? What can you say about the depths of the leaves of the resulting tree?

answer:
The degree of a node in a rooted tree is the number of its children (see Section B.5.2). Given this definition, the possible degrees are $0$, $2$, $3$ and $4$, based on whether the black node had zero, one or two red children, each with either zero or two black children. The depths could shrink by at most a factor of $1 / 2$.


question:
 Show that the longest simple path from a node $x$ in a red-black tree to a descendant leaf has length at most twice that of the shortest simple path from node $x$ to a descendant leaf.

answer:
Suppose we have the longest simple path $(a_1, a_2, \dots, a_s)$ and the shortest simple path $(b_1, b_2, \dots, b_t)$. Then, by property 5 we know they have equal numbers of black nodes. By property 4, we know that neither contains a repeated red node. This tells us that at most $\left\lfloor \frac{s - 1}{2} \right\rfloor$ of the nodes in the longest path are red. This means that at least $\left\lceil \frac{s + 1}{2} \right\rceil$ are black, so, $t \ge \left\lceil \frac{s + 1}{2} \right\rceil$. Therefore, if, by way of contradiction, we had that $s > t \cdot 2$, then $t \ge \left\lceil \frac{s + 1}{2} \right\rceil \ge \left\lceil \frac{2t + 2}{2} \right\rceil = t + 1$ results a contradiction.


question:
 What is the largest possible number of internal nodes in a red-black tree with black-height $k$? What is the smallest possible number?

answer:
- The largest is a path with half black nodes and half red nodes, which has $2^{2k} - 1$ internal nodes.
- The smallest is a path with all black nodes, which has $2^k - 1$ internal nodes.


answer: - The largest ratio is $2$, each black node has two red children. - The smallest ratio is $0$.
question:
 Describe a red-black tree on $n$ keys that realizes the largest possible ratio of red internal nodes to black internal nodes. What is this ratio? What tree has the smallest possible ratio, and what is the ratio?


question:
 In line 16 of $\text{RB-INSERT}$, we set the color of the newly inserted node $z$ to red. Observe that if we had chosen to set $z$'s color to black, then property 4 of a red-black tree would not be violated. Why didn't we choose to set $z$'s color to black?

answer:
If we chose to set the color of $z$ to black then we would be violating property 5 of being a red-black tree. Because any path from the root to a leaf under $z$ would have one more black node than the paths to the other leaves.


question:
 Show the red-black trees that result after successively inserting the keys $41, 38, 31, 12, 19, 8$ into an initially empty red-black tree.

answer:
- insert $41$:

  ![](../img/13.3-2-1.png)

- insert $38$:

  ![](../img/13.3-2-2.png)

- insert $31$:

  ![](../img/13.3-2-3.png)

- insert $12$:

  ![](../img/13.3-2-4.png)

- insert $19$:

  ![](../img/13.3-2-5.png)

- insert $8$:

  ![](../img/13.3-2-6.png)


answer: (Removed)
question:
 Suppose that the black-height of each of the subtrees $\alpha, \beta, \gamma, \delta, \epsilon$ in Figures 13.5 and 13.6 is $k$. Label each node in each figure with its black-height to verify that the indicated transformation preserves property 5.


question:
 Professor Teach is concerned that $\text{RB-INSERT-FIXUP}$ might set $T.nil.color$ to $\text{RED}$, in which case the test in line 1 would not cause the loop to terminate when $z$ is the root. Show that the professor's concern is unfounded by arguing that $\text{RB-INSERT-FIXUP}$ never sets $T.nil.color$ to $\text{RED}$.

answer:
First observe that $\text{RB-INSERT-FIXUP}$ only modifies the child of a node if it is already $\text{RED}$, so we will never modify a child which is set to $T.nil$. We just need to check that the parent of the root is never set to $\text{RED}$.

Since the root and the parent of the root are automatically black, if $z$ is at depth less than $2$, the **while** loop will be broken. We only modify colors of nodes at most two levels above $z$, so the only case we need to worry about is when $z$ is at depth $2$. In this case we risk modifying the root to be $\text{RED}$, but this is handled in line 16. When $z$ is updated, it will be either the root or the child of the root. Either way, the root and the parent of the root are still $\text{BLACK}$, so the **while** condition is violated, making it impossibly to modify $T.nil$ to be $\text{RED}$.


question:
 Consider a red-black tree formed by inserting $n$ nodes with $\text{RB-INSERT}$. Argue that if $n > 1$, the tree has at least one red node.

answer:
- **Case 1:** $z$ and $z.p.p$ are $\text{RED}$, if the loop terminates, then $z$ could not be the root, thus $z$ is $\text{RED}$ after the fix up.
- **Case 2:** $z$ and $z.p$ are $\text{RED}$, and after the rotation $z.p$ could not be the root, thus $z.p$ is $\text{RED}$ after the fix up.
- **Case 3:** $z$ is $\text{RED}$ and $z$ could not be the root, thus $z$ is $\text{RED}$ after the fix up.

Therefore, there is always at least one red node.


question:
 Suggest how to implement $\text{RB-INSERT}$ efficiently if the representation for red-black trees includes no storage for parent pointers.

answer:
Use stack to record the path to the inserted node, then parent is the top element in the stack.

- **Case 1:** we pop $z.p$ and $z.p.p$.
- **Case 2:** we pop $z.p$ and $z.p.p$, then push $z.p.p$ and $z$.
- **Case 3:** we pop $z.p$, $z.p.p$ and $z.p.p.p$, then push $z.p$.


question:
 Write pseudocode for $\text{LEFT-ROTATE}$ that operates on nodes in an interval tree and updates the $max$ attributes in $O(1)$ time.

answer:
Add 2 lines in $\text{LEFT-ROTATE}$ in 13.2

```cpp
    y.max = x.max
    x.max = max(x.high, x.left.max, x.right.max)
```


question:
 Rewrite the code for $\text{INTERVAL-SEARCH}$ so that it works properly when all intervals are open.

answer:
```cpp
INTERVAL-SEARCH(T, i)
    x = T.root
    while x != T.nil and i does not overlap x.int
        if x.left != T.nil and x.left.max > i.low
            x = x.left
        else x = x.right
    return x
```


question:
 Describe an efficient algorithm that, given an interval $i$ , returns an interval overlapping $i$ that has the minimum low endpoint, or $T.nil$ if no such interval exists.

answer:
Consider the usual interval search given, but, instead of breaking out of the loop as soon as we have an overlap, we just keep track of the overlap that has the minimum low endpoint, and continue the loop. After the loop terminates, we return the overlap stored.


question:
 Given an interval tree $T$ and an interval $i$, describe how to list all intervals in $T$ that overlap $i$ in $O(\min(n, k \lg n))$ time, where $k$ is the number of intervals in the output list. ($\textit{Hint:}$ One simple method makes several queries, modifying the tree between queries. A slightly more complicated method does not modify the tree.)

answer:
```cpp
INTERVALS-SEARCH(T, x, i)
    let list be an empty array
    if i overlaps x.int
        list.APPEND(x)
    if x.left != T.nil and x.left.max > i.low
        list = list.APPEND(INTERVALS-SEARCH(T, x.left, i))
    if x.right != T.nil and x.int.low ≤ i.high and x.right.max ≥ i.low
        list = list.APPEND(INTERVALS-SEARCH(T, x.right, i))
    return list
```


question:
 Suggest modifications to the interval-tree procedures to support the new operation $\text{INTERVAL-SEARCH-EXACTLY}(T, i)$, where $T$ is an interval tree and $i$ is an interval. The operation should return a pointer to a node $x$ in $T$ such that $x.int.low = i.low$ and $x.int.high = i.high$, or $T.nil$ if $T$ contains no such node. All operations, including $\text{INTERVAL-SEARCH-EXACTLY}$, should run in $O(\lg n)$ time on an $n$-node interval tree.

answer:
Search for nodes which has exactly the same low value.

```cpp
INTERVAL-SEARCH-EXACTLY(T, i)
    x = T.root
    while x != T.nil and i not exactly overlap x
        if i.high > x.max
            x = T.nil
        else if i.low < x.low
            x = x.left
        else if i.low > x.low
            x = x.right
        else x = T.nil
    return x
```


question:
 Show how to maintain a dynamic set $Q$ of numbers that supports the operation $\text{MIN-GAP}$, which gives the magnitude of the difference of the two closest numbers in $Q$. For example, if $Q = \\{1, 5, 9, 15, 18, 22 \\}$, then $\text{MIN-GAP}(Q)$ returns $18 - 15 = 3$, since $15$ and $18$ are the two closest numbers in $Q$. Make the operations $\text{INSERT}$, $\text{DELETE}$, $\text{SEARCH}$, and $\text{MIN-GAP}$ as efficient as possible, and analyze their running times.

answer:
Store the elements in a red-black tree, where the key value is the value of each number itself. The auxiliary attribute stored at a node $x$ will be the min gap between elements in the subtree rooted at $x$, the maximum value contained in subtree rooted at $x$, and the minimum value contained in the subtree rooted at $x$. The min gap at a leaf will be $\infty$. Since we can determine the attributes of a node $x$ using only the information about the key at $x$, and the attributes in $x.left$ and $x.right$, Theorem 14.1 implies that we can maintain the values in all nodes of the tree during insertion and deletion without asymptotically affecting their $O(\lg n)$ performance. For $\text{MIN-GAP}$, just check the min gap at the root, in constant time.


question:
 VLSI databases commonly represent an integrated circuit as a list of rectangles. Assume that each rectangle is rectilinearly oriented (sides parallel to the $x$- and $y$-axes), so that we represent a rectangle by its minimum and maximum $x$ and $y$-coordinates. Give an $O(n\lg n)$-time algorithm to decide whether or not a set of $n$ rectangles so represented contains two rectangles that overlap. Your algorithm need not report all intersecting pairs, but it must report that an overlap exists if one rectangle entirely covers another, even if the boundary lines do not intersect. ($\textit{Hint:}$ Move a "sweep" line across the set of rectangles.)

answer:
Let $L$ be the set of left coordinates of rectangles. Let $R$ be the set of right coordinates of rectangles. Sort both of these sets in $O(n\lg n)$ time. Then, we will have a pointer to $L$ and a pointer to $R$. If the pointer to $L$ is smaller, call interval search on $T$ for the up-down interval corresponding to this left hand side. If it contains something that intersects the up-down bounds of this rectangle, there is an intersection, so stop.

Otherwise add this interval to $T$ and increment the pointer to $L$. If $R$ is the smaller one, remove the up-down interval that that right hand side corresponds to and increment the pointer to $R$. Since all the interval tree operations used run in time $O(\lg n)$ and we only call them at most $3n$ times, we have that the runtime is $O(n\lg n)$.


question:
 Show, by adding pointers to the nodes, how to support each of the dynamic-set queries $\text{MINIMUM}$, $\text{MAXIMUM}$, $\text{SUCCESSOR}$, and $\text{PREDECESSOR}$ in $O(1)$worstcase time on an augmented order-statistic tree. The asymptotic performance of other operations on order-statistic trees should not be affected.

answer:
- **MINIMUM:** A pointer points to the minimum node, if the node is being deleted, move the pointer to its successor.
- **MAXIMUM:** Similar to $\text{MINIMUM}$.
- **SUCCESSOR:** Every node records its successor, the insertion and deletion is similar to that in linked list.
- **PREDECESSOR:** Similar to $\text{MAXIMUM}$.


question:
 Can we maintain the black-heights of nodes in a red-black tree as attributes in the nodes of the tree without affecting the asymptotic performance of any of the redblack tree operations? Show how, or argue why not. How about maintaining the depths of nodes?

answer:
Since the black height of a node depends only on the black height and color of its children, Theorem 14.1 implies that we can maintain the attribute without affecting the asymptotic performance of the other red-black tree operations. The same is not true for maintaining the depths of nodes. If we delete the root of a tree we could potentially have to update the depths of $O(n)$ nodes, making the $\text{DELETE}$ operation asymptotically slower than before.


answer: $x.f = x.left.f \otimes x.a \otimes x.right.f$.
question:
 Let $\otimes$ be an associative binary operator, and let $a$ be an attribute maintained in each node of a red-black tree. Suppose that we want to include in each node $x$ an additional attribute $f$ such that $x.f = x_1.a \otimes x_2.a \otimes \cdots \otimes x_m.a$, where $x_1, x_2, \ldots ,x_m$ is the inorder listing of nodes in the subtree rooted at $x$. Show how to update the $f$ attributes in $O(1)$ time after a rotation. Modify your argument slightly to apply it to the $size$ attributes in order-statistic trees.


question:
 We wish to augment red-black trees with an operation $\text{RB-ENUMERATE}(x, a, b)$ that outputs all the keys $k$ such that $a \le k \le b$ in a red-black tree rooted at $x$. Describe how to implement $\text{RB-ENUMERATE}$ in $\Theta(m+\lg n)$ time, where $m$ is the number of keys that are output and $n$ is the number of internal nodes in the tree. ($\textit{Hint:}$ You do not need to add new attributes to the red-black tree.)

answer:
- $\Theta(\lg n)$: Find the smallest key that larger than or equal to $a$.
- $\Theta(m)$: Based on Exercise 14.2-1, find the $m$ successor.


question: Show how $\text{OS-SELECT}(T.root, 10)$ operates on the red-black tree $T$ of Figure 14.1.
answer:
- $26: r = 13, i = 10$, go left.
- $17: r =  8, i = 10$, go right.
- $21: r =  3, i =  2$, go left.
- $19: r =  1, i =  2$, go right.
- $20: r =  1, i =  1$, choose $20$.


answer: - $35: r = 1$. - $38: r = 1$. - $30: r = r + 2 = 3$. - $41: r = 3$. - $26: r = r + 13 = 16$.
question:
 Show how $\text{OS-RANK}(T, x)$ operates on the red-black tree $T$ of Figure 14.1 and the node $x$ with $x.key = 35$.


question: Write a nonrecursive version of $\text{OS-SELECT}$.
answer:
```cpp
OS-SELECT(x, i)
    r = x.left.size + 1
    while r != i
        if i < r
            x = x.left
        else x = x.right
            i = i - r
        r = x.left.size + 1
    return x
```


question:
 Write a recursive procedure $\text{OS-KEY-RANK}(T, k)$ that takes as input an order-statistic tree $T$ and a key $k$ and returns the rank of $k$ in the dynamic set represented by $T$. Assume that the keys of $T$ are distinct.

answer:
```cpp
OS-KEY-RANK(T, k)
    if k == T.root.key
        return T.root.left.size + 1
    else if T.root.key > k
        return OS-KEY-RANK(T.left, k)
    else return T.root.left.size + 1 + OS-KEY-RANK(T.right, k)
```


question:
 Given an element $x$ in an $n$-node order-statistic tree and a natural number $i$, how can we determine the $i$th successor of $x$ in the linear order of the tree in $O(\lg n)$ time?

answer:
The desired result is $\text{OS-SELECT}(T, \text{OS-RANK}(T, x) + i)$. This has runtime $O(h)$, which by the properties of red black trees, is $O(\lg n)$.


question:
 Observe that whenever we reference the size attribute of a node in either $\text{OS-SELECT}$ or $\text{OS-RANK}$, we use it only to compute a rank. Accordingly, suppose we store in each node its rank in the subtree of which it is the root. Show how to maintain this information during insertion and deletion. (Remember that these two operations can cause rotations.)

answer:
First perform the usual BST insertion procedure on $z$, the node to be inserted. Then add $1$ to the rank of every node on the path from the root to $z$ such that $z$ is in the left subtree of that node. Since the added node is a leaf, it will have no subtrees so its rank will always be $1$.

When a left rotation is performed on $x$, its rank within its subtree will remain the same. The rank of $x.right$ will be increased by the rank of $x$, plus one. If we perform a right rotation on a node $y$, its rank will decrement by $y.left.rank + 1$. The rank of $y.left$ will remain unchanged.

For deletion of $z$, decrement the rank of every node on the path from $z$ to the root such that $z$ is in the left subtree of that node. For any rotations, use the same rules as before.


question:
 Show how to use an order-statistic tree to count the number of inversions (see Problem 2-4) in an array of size $n$ in time $O(n\lg n)$.

answer:
The runtime to build a red-black tree is $O(n\lg n)$, so we need to calculate inversions while building trees.

Every time $\text{INSERT}$, we can use $\text{OS-RANK}$ to calculate the rank of the node, thus calculating inversions.


question:
 Consider $n$ chords on a circle, each defined by its endpoints. Describe an $O(n\lg n)$-time algorithm to determine the number of pairs of chords that intersect inside the circle. (For example, if the $n$ chords are all diameters that meet at the center, then the correct answer is $\binom{n}{2}$.) Assume that no two chords share an endpoint.

answer:
Sort the vertices in clock-wise order, and assign a unique value to each vertex. For each chord its two vertices are $u_i$, $v_i$ and $u_i < v_i$. Add the vertices one by one in clock-wise order, if we meet a $u_i$, we add it to the order-statistic tree, if we meet a $v_i$, we calculate how many nodes are larger than $u_i$ (which is the number of intersects with chord $i$), and remove $u_i$.


answer: (Omit!)
question:
 Show how $\text{ITERATIVE-FFT}$ computes the $\text{DFT}$ of the input vector $(0, 2, 3, -1, 4, 5, 7, 9)$.


answer: (Omit!)
question:
 Show how to implement an $\text{FFT}$ algorithm with the bit-reversal permutation occurring at the end, rather than at the beginning, of the computation. ($\textit{Hint:}$ Consider the inverse $\text{DFT}$.)


answer: (Omit!)
question:
 How many times does $\text{ITERATIVE-FFT}$ compute twiddle factors in each stage? Rewrite $\text{ITERATIVE-FFT}$ to compute twiddle factors only $2^{s - 1}$ times in stage $s$.


answer: (Omit!)
question:
 Suppose that the adders within the butterfly operations of the $\text{FFT}$ circuit sometimes fail in such a manner that they always produce a zero output, independent of their inputs. Suppose that exactly one adder has failed, but that you don't know which one. Describe how you can identify the failed adder by supplying inputs to the overall $\text{FFT}$ circuit and observing the outputs. How efficient is your method?


answer: (Omit!)
question:
 Describe the generalization of the $\text{FFT}$ procedure to the case in which $n$ is a power of $3$. Give a recurrence for the running time, and solve the recurrence.


answer: (Omit!)
question:
 Suppose that instead of performing an $n$-element $\text{FFT}$ over the field of complex numbers (where $n$ is even), we use the ring $\mathbb Z_m$ of integers modulo $m$, where $m = 2^{tn / 2} + 1$ and $t$ is an arbitrary positive integer. Use $\omega = 2^t$ instead of $\omega_n$ as a principal nth root of unity, modulo $m$. Prove that the $\text{DFT}$ and the inverse $\text{DFT}$ are well defined in this system.


answer: (Omit!)
question:
 Given a list of values $z_0, z_1, \dots, z_{n - 1}$ (possibly with repetitions), show how to find the coefficients of a polynomial $P(x)$ of degree-bound $n + 1$ that has zeros only at $z_0, z_1, \dots, z_{n - 1}$ (possibly with repetitions). Your procedure should run in time $O(n\lg^2 n)$. ($\textit{Hint:}$ The polynomial $P(x)$ has a zero at $z_j$ if and only if $P(x)$ is a multiple of $(x - z_j)$.)


answer: (Omit!)
question:
 The **_chirp transform_** of a vector $a = (a_0, a_1, \dots, a_{n - 1})$ is the vector $y = (y_0, y_1, \dots, y_{n - 1})$, where $y_k = \sum_{j = 0}^{n - 1} a_jz^{kj}$ and $z$ is any complex number. The $\text{DFT}$ is therefore a special case of the chirp transform, obtained by taking $z = \omega_n$. Show how to evaluate the chirp transform in time $O(n\lg n)$ for any complex number $z$. ($\textit{Hint:}$ Use the equation
>
> $$y_k = z^{k^2 / 2} \sum_{j = 0}^{n - 1} \Big(a_jz^{j^2 / 2}\Big) \Big(z^{-(k - j)^2 / 2}\Big)$$
>
> to view the chirp transform as a convolution.)


question:
 Multiply the polynomials $A(x) = 7x^3 - x^2 + x - 10$ and $B(x) = 8x^3 - 6x + 3$ using equations $\text{(30.1)}$ and $\text{(30.2)}$.

answer:
$$
\begin{array}{rl}
  & 56x^6 - 8x^5 + (8 - 42)x^4 + (-80 + 6 + 21)x^3 + (-3 - 6)x^2 + (60 + 3)x - 30 \\\\
= & 56x^6 - 8x^5 - 34x^4 - 53x^3 - 9x^2 + 63x - 30.
\end{array}
$$


question:
 Another way to evaluate a polynomial $A(x)$ of degree-bound $n$ at a given point $x_0$ is to divide $A(x)$ by the polynomial $(x - x_0)$, obtaining a quotient polynomial $q(x)$ of degree-bound $n - 1$ and a remainder $r$, such that
>
> $$A(x) = q(x)(x - x_0) + r.$$
>
> Clearly, $A(x_0) = r$. Show how to compute the remainder $r$ and the coefficients of $q(x)$ in time $\Theta(n)$ from $x_0$ and the coefficients of $A$.

answer:
Let $A$ be the matrix with $1$'s on the diagonal, $−x_0$'s on the super diagonal, and $0$'s everywhere else. Let $q$ be the vector $(r, q_0, q_1, \dots, x_{n − 2})$. If $a = (a_0, a_1, \dots, a_{n − 1})$ then we need to solve the matrix equation $Aq = a$ to compute the remainder and coefficients. Since $A$ is tridiagonal, Problem 28-1 (e) tells us how to solve this equation in linear time.


question:
 Derive a point-value representation for $A^\text{rev}(x) = \sum_{j = 0}^{n - 1} a_{n - 1 - j}x^j$ from a point-value representation for $A(x) = \sum_{j = 0}^{n - 1} a_jx^j$, assuming that none of the points is $0$.

answer:
For each pair of points, $(p, A(p))$, we can compute the pair $(\frac{1}{p}, A^{rev}(\frac{1}{p}))$. To do this, we note that

$$
\begin{aligned}
A^{rev}(\frac{1}{p})
    & = \sum_{j = 0}^{n - 1} a_{n - 1 - j} (\frac{1}{p})^j \\\\
    & = \sum_{j = 0}^{n - 1} a_j(\frac{1}{p})^{n - 1 - j} \\\\
    & = p^{1 - n} \sum_{j = 0}^{n - 1} a_jp^j \\\\
    & = p^{1 - n}A(p),
\end{aligned}
$$

since we know what $A(p)$ is, we can compute $A^{rev}(\frac{1}{p})$ of course, we are using the fact that $p \ne 0$ because we are dividing by it. Also, we know that each of these points are distinct, because $\frac{1}{p} = \frac{1}{p'}$ implies that $p = p'$ by cross multiplication. So, since all the $x$ values were distinct in the point value representation of $A$, they will be distinct in this point value representation of $A^{rev}$ that we have made.


question:
 Prove that $n$ distinct point-value pairs are necessary to uniquely specify a polynomial of degree-bound $n$, that is, if fewer than $n$ distinct point-value pairs are given, they fail to specify a unique polynomial of degree-bound $n$. ($\textit{Hint:}$ Using Theorem 30.1, what can you say about a set of $n - 1$ point-value pairs to which you add one more arbitrarily chosen point-value pair?)

answer:
Suppose that just $n − 1$ point-value pairs uniquely determine a polynomial $P$ which satisfies them. Append the point value pair $(x_{n - 1}, y_{n − 1})$ to them, and let $P'$ be the unique polynomial which agrees with the $n$ pairs, given by Theorem 30.1. Now append instead $(x\_{n - 1}, y'\_{n − 1})$ where $y_{n − 1} \ne y'_{n - 1}$, and let $P''$ be the polynomial obtained from these points via Theorem 30.1. Since polynomials coming from $n$ pairs are unique, $P' \ne P''$. However, $P'$ and $P''$ agree on the original $n − 1$ point-value pairs, contradicting the fact that $P$ was determined uniquely.


question:
 Show how to use equation $\text{(30.5)}$ to interpolate in time $\Theta(n^2)$. ($\textit{Hint:}$ First compute the coefficient representation of the polynomial $\prod_j (x - x_j)$ and then divide by $(x - x_k)$ as necessary for the numerator of each term; see Exercise 30.1-2. You can compute each of the $n$ denominators in time $O(n)$.)

answer:
First, we show that we can compute the coefficient representation of $\prod_j (x - x_j)$ in time $\Theta(n^2)$. We will do it by recursion, showing that multiplying $\prod_{j < k} (x − x_j)$ by $(x − x_k)$ only takes time $O(n)$, since this only needs to be done $n$ times, this gets is total runtime of $O(n)$. Suppose that $\sum_{i = 0}^{k - 1} k_ix^i$ is a coefficient representation of $\prod_{j < k} (x − x_j)$. To multiply this by $(x − x_k)$, we just set $(k + 1)\_i = k\_{i − 1} - x_kk_i$ for $i = 1, \dots, k$ and $(k + 1)_0 = −x_k \cdot k_0$. Each of these coefficients can be computed in constant time, since there are only linearly many coefficients, then, the time to compute the next partial product is just $O(n)$.

Now that we have a coefficient representation of $\prod_j (x − x_j)$, we need to compute, for each $k \prod_{j − k} (x − x_j)$, each of which can be computed in time $\Theta(n)$ by problem 30.1-2. Since the polynomial is defined as a product of things containing the thing we are dividing by, we have that the remainder in each case is equal to $0$. Lets call these polynomials $f_k$. Then, we need only compute the sum $\sum_k y_k \frac{f_k(x)}{f_k(x_k)}$. That is, we compute $f(x_k)$ each in time $\Theta(n)$, so all told, only $\Theta(n^2)$ time is spent computing all the $f(x_k)$ values. For each of the terms in the sum, dividing the polynomial $f_k(x)$ by the number $f_k(x_k)$ and multiplying by $y_k$ only takes time $\Theta(n)$, so total it takes time $\Theta(n^2)$. Lastly, we are adding up $n$ polynomials, each of degree bound $n − 1$, so the total time taken there is $\Theta(n^2)$.


question:
 Explain what is wrong with the "obvious" approach to polynomial division using a point-value representation, i.e., dividing the corresponding $y$ values. Discuss separately the case in which the division comes out exactly and the case in which it doesn't.

answer:
If we wish to compute $P / Q$ but $Q$ takes on the value zero at some of these points, then we can't carry out the "obvious" method. However, as long as all point value pairs $(x_i, y_i)$ we choose for $Q$ are such that $y_i \ne 0$, then the approach comes out exactly as we would like.


question:
 Consider two sets $A$ and $B$, each having $n$ integers in the range from $0$ to $10n$. We wish to compute the **_Cartesian sum_** of $A$ and $B$, defined by
>
> $$C = \\{x + y: x \in A \text{ and } y \in B\\}.$$
>
> Note that the integers in $C$ are in the range from $0$ to $20n$. We want to find the elements of $C$ and the number of times each element of $C$ is realized as a sum of elements in $A$ and $B$. Show how to solve the problem in $O(n\lg n)$ time. ($\textit{Hint:}$ Represent $A$ and $B$ as polynomials of degree at most $10n$.)

answer:
For the set $A$, we define the polynomial $f_A$ to have a coefficient representation that has $a_i$ equal zero if $i \notin A$ and equal to $1$ if $i \in A$. Similarly define $f_B$. Then, we claim that looking at $f_C := f_A \cdot f_B$ in coefficient form, we have that the $i$th coefficient, $c_i$ is exactly equal to the number of times that $i$ is realized as a sum of elements from $A$ and $B$. Since we can perform the polynomial multiplication in time $O(n \lg n)$ by the methods of this chapter, we can get the final answer in time $O(n \lg n)$. To see that $f_C$ has the nice property described, we'll look at the ways that we could end up having a term of $x^i$ appear. Each contribution to that coefficient must come from there being some $k$ so that $a_k \ne 0$ and $b_{i − k} \ne 0$, because the powers of $x$ attached to each are additive when we multiply. Since each of these contributions is only ever $1$, the final coefficient is counting the total number of such contributions, therefore counting the number of $k \in A$ such that $−k \in B$, which is exactly what we claimed $f_C$ was counting.


question:
 Suppose that $\text{CONNECTED-COMPONENTS}$ is run on the undirected graph $G = (V, E)$, where $V = \\{a, b, c, d, e, f, g, h, i, j, k\\}$ and the edges of $E$ are processed in the order $(d, i)$, $(f, k)$, $(g, i)$, $(b, g)$, $(a, h)$, $(i, j)$, $(d, k)$, $(b, j)$, $(d, f)$, $(g, j)$, $(a, e)$. List the vertices in each connected component after each iteration of lines 3–5.

answer:
$$
\begin{array}{c|lllllllllll}
\text{Edge processed} & \\\\
\hline
initial & \\{a\\}       & \\{b\\}                   & \\{c\\} & \\{d\\}       & \\{e\\} & \\{f\\}    & \\{g\\} & \\{h\\} & \\{i\\} & \\{j\\} & \\{k\\} \\\\
(d, i)  & \\{a\\}       & \\{b\\}                   & \\{c\\} & \\{d, i\\}    & \\{e\\} & \\{f\\}    & \\{g\\} & \\{h\\} &         & \\{j\\} & \\{k\\} \\\\
(f, k)  & \\{a\\}       & \\{b\\}                   & \\{c\\} & \\{d, i\\}    & \\{e\\} & \\{f, k\\} & \\{g\\} & \\{h\\} &         & \\{j\\} &         \\\\
(g, i)  & \\{a\\}       & \\{b\\}                   & \\{c\\} & \\{d, i, g\\} & \\{e\\} & \\{f, k\\} &         & \\{h\\} &         & \\{j\\} &         \\\\
(b, g)  & \\{a\\}       & \\{b, d, i, g\\}          & \\{c\\} &               & \\{e\\} & \\{f, k\\} &         & \\{h\\} &         & \\{j\\} &         \\\\
(a, h)  & \\{a, h\\}    & \\{b, d, i, g\\}          & \\{c\\} &               & \\{e\\} & \\{f, k\\} &         &         &         & \\{j\\} &         \\\\
(i, j)  & \\{a, h\\}    & \\{b, d, i, g, j\\}       & \\{c\\} &               & \\{e\\} & \\{f, k\\} &         &         &         &         &         \\\\
(d, k)  & \\{a, h\\}    & \\{b, d, i, g, j, f, k\\} & \\{c\\} &               & \\{e\\} &            &         &         &         &         &         \\\\
(b, j)  & \\{a, h\\}    & \\{b, d, i, g, j, f, k\\} & \\{c\\} &               & \\{e\\} &            &         &         &         &         &         \\\\
(d, f)  & \\{a, h\\}    & \\{b, d, i, g, j, f, k\\} & \\{c\\} &               & \\{e\\} &            &         &         &         &         &         \\\\
(g, j)  & \\{a, h\\}    & \\{b, d, i, g, j, f, k\\} & \\{c\\} &               & \\{e\\} &            &         &         &         &         &         \\\\
(a, e)  & \\{a, h, e\\} & \\{b, d, i, g, j, f, k\\} & \\{c\\} &               &         &            &         &         &         &         &
\end{array}
$$

So, the connected components that we are left with are $\\{a, h, e\\}$, $\\{b, d, i, g, j, f, k\\}$, and $\\{c\\}$.


question:
 Show that after all edges are processed by $\text{CONNECTED-COMPONENTS}$, two vertices are in the same connected component if and only if they are in the same set.

answer:
First suppose that two vertices are in the same connected component. Then there exists a path of edges connecting them. If two vertices are connected by a single edge, then they are put into the same set when that edge is processed. At some point during the algorithm every edge of the path will be processed, so all vertices on the path will be in the same set, including the endpoints. Now suppose two vertices $u$ and $v$ wind up in the same set. Since every vertex starts off in its own set, some sequence of edges in $G$ must have resulted in eventually combining the sets containing $u$ and $v$. From among these, there must be a path of edges from $u$ to $v$, implying that $u$ and $v$ are in the same connected component.


question:
 During the execution of $\text{CONNECTED-COMPONENTS}$ on an undirected graph $G = (V, E)$ with $k$ connected components, how many times is $\text{FIND-SET}$ called? How many times is $\text{UNION}$ called? Express your answers in terms of $|V|$, $|E|$, and $k$.

answer:
Find set is called twice on line 4, this is run once per edge in the graph, so, we have that find set is run $2|E|$ times. Since we start with $|V|$ sets, at the end only have $k$, and each call to $\text{UNION}$ reduces the number of sets by one, we have that we have to made $|V| - k$ calls to $\text{UNION}$.


question: Redo Exercise 21.2-2 using a disjoint-set forest with union by rank and path compression.
answer:
```
        1
 /    /    \       \
2    3      5       9
     |     / \     / \      \
     4    6   7   10 11     13
              |      |      / \
              8      12    14 15
                              |
                              16
```


question: Write a nonrecursive version of $\text{FIND-SET}$ with path compression.
answer:
To implement $\text{FIND-SET}$ nonrecursively, let $x$ be the element we call the function on. Create a linked list $A$ which contains a pointer to $x$. Each time we most one element up the tree, insert a pointer to that element into $A$. Once the root $r$ has been found, use the linked list to find each node on the path from the root to $x$ and update its parent to $r$.


question:
 Give a sequence of $m$ $\text{MAKE-SET}$, $\text{UNION}$, and $\text{FIND-SET}$ operations, $n$ of which are $\text{MAKE-SET}$ operations, that takes $\Omega(m\lg n)$ time when we use union by rank only.

answer:
Suppose that $n' = 2k$ is the smallest power of two less than $n$. To see that this sequences of operations does take the required amount of time, we'll first note that after each iteration of the **for** loop indexed by $j$, we have that the elements $x_1, \dots, x_{n'}$ are in trees of depth $i$. So, after we finish the outer **for** loop, we have that $x_1, \dots, x_{n'}$ all lie in the same set, but are represented by a tree of depth $k \in \Omega(\lg n)$. Then, since we repeatedly call $\text{FIND-SET}$ on an item that is $\lg n$ away from its set representative, we have that each one takes time $\lg n$. So, the last **for** loop alltogther takes time $\Omega(m \lg n)$.

```cpp
    for i = 1 to n
        MAKE-SET(x[i])
    for i = 1 to k
        for j = 1..n' - 2^{i = 1} by 2^i
            UNION(x[i], x[i + 2^{j - 1}])
    for i = 1 to m
        FIND-SET(x[1])
```


question:
 Suppose that we wish to add the operation $\text{PRINT-SET}(x)$, which is given a node $x$ and prints all the members of $x$'s set, in any order. Show how we can add just a single attribute to each node in a disjoint-set forest so that $\text{PRINT-SET}(x)$ takes time linear in the number of members of $x$'s set and the asymptotic running times of the other operations are unchanged. Assume that we can print each member of the set in $O(1)$ time.

answer:
In addition to each tree, we'll store a linked list (whose set object contains a single tail pointer) with which keeps track of all the names of elements in the tree. The only additional information we'll store in each node is a pointer $x.l$ to that element's position in the list.

- When we call $\text{MAKE-SET}(x)$, we'll also create a new linked list, insert the label of $x$ into the list, and set $x.l$ to a pointer to that label. This is all done in $O(1)$.
- $\text{FIND-SET}$ will remain unchanged.
- $\text{UNION}(x, y)$ will work as usual, with the additional requirement that we union the linked lists of $x$ and $y$, since we don't need to update pointers to the head, we can link up the lists in constant time, thus preserving the runtime of $\text{UNION}$.
- Finally, $\text{PRINT-SET}(x)$ works as follows: first, set $s = \text{FIND-SET}(x)$. Then print the elements in the linked list, starting with the element pointed to by $x$. (This will be the first element in the list). Since the list contains the same number of elements as the set and printing takes $O(1)$, this operation takes linear time in the number of set members.


question:
 Show that any sequence of $m$ $\text{MAKE-SET}$, $\text{FIND-SET}$, and $\text{LINK}$ operations, where all the $\text{LINK}$ operations appear before any of the $\text{FIND-SET}$ operations, takes only $O(m)$ time if we use both path compression and union by rank. What happens in the same situation if we use only the path-compression heuristic?

answer:
Clearly each $\text{MAKE-SET}$ and $\text{LINK}$ operation only takes time $O(1)$, so, supposing that $n$ is the number of $\text{FIND-SET}$ operations occuring after the making and linking, we need to show that all the $\text{FIND-SET}$ operations only take time $O(n)$.

To do this, we will ammortize some of the cost of the $\text{FIND-SET}$ operations into the cost of the $\text{MAKE-SET}$ operations. Imagine paying some constant amount extra for each $\text{MAKE-SET}$ operation. Then, when doing a $\text{FIND-SET}(x)$ operation, we have three possibilities:

- First, we could have that $x$ is the representative of its own set. In this case, it clearly only takes constant time to run.
- Second, we could have that the path from $x$ to its set's representative is already compressed, so it only takes a single step to find the set representative. In this case also, the time required is constant.
- Third, we could have that $x$ is not the representative and it's path has not been compressed. Then, suppose that there are k nodes between $x$ and its representative. The time of this $\text{FIND-SET}$ operation is $O(k)$, but it also ends up compressing the paths of $k$ nodes, so we use that extra amount that we paid during the $\text{MAKE-SET}$ operations for these $k$ nodes whose paths were compressed. Any subsequent call to find set for these nodes will take only a constant amount of time, so we would never try to use the work that amortization amount twice for a given node.


question: Prove Lemma 21.4.
answer:
The lemma states:

> For all nodes $x$, we have $x.rank \le x.p.rank$, with strict inequality if $x \ne x.p$. The value of $x.rank$ is initially $0$ and increases through time until $x \ne x.p$; from then on, $x.rank$ does not change. The value of $x.p.rank$ monotonically increases over time.

The initial value of $x.rank$ is $0$, as it is initialized in line 2 of the $\text{MAKE-SET}(x)$ procedure. When we run $\text{LINK}(x, y)$, whichever one has the larger rank is placed as the parent of the other, and if there is a tie, the parent's rank is incremented. This means that after any $\text{LINK}(y, x)$, the two nodes being linked satisfy this strict inequality of ranks.

Also, if we have that $x \ne x.p$, then, we have that $x$ is not its own set representative, so, any linking together of sets that would occur would not involve $x$, but that's the only way for ranks to increase, so, we have that $x.rank$ must remain constant after that point.


question: Prove that every node has rank at most $\lfloor \lg n \rfloor$.
answer:
We'll prove the claim by strong induction on the number of nodes. If $n = 1$, then that node has rank equal to $0 = \lfloor \lg 1 \rfloor$. Now suppose that the claim holds for $1, 2, \ldots, n$ nodes.

Given $n + 1$ nodes, suppose we perform a $\text{UNION}$ operation on two disjoint sets with $a$ and $b$ nodes respectively, where $a, b \le n$. Then the root of the first set has rank at most $\lfloor \lg a \rfloor$ and the root of the second set has rank at most $\lfloor \lg b\rfloor$.

If the ranks are unequal, then the $\text{UNION}$ operation preserves rank and we are done, so suppose the ranks are equal. Then the rank of the union increases by $1$, and the resulting set has rank $\lfloor\lg a\rfloor + 1 \le\lfloor\lg(n + 1) / 2\rfloor + 1 = \lfloor\lg(n + 1)\rfloor$.


question: In light of Exercise 21.4-2, how many bits are necessary to store $x.rank$ for each node $x$?
answer:
Since their value is at most $\lfloor \lg n \rfloor$, we can represent them using $\Theta(\lg(\lg(n)))$ bits, and may need to use that many bits to represent a number that can take that many values.


question:
 Using Exercise 21.4-2, give a simple proof that operations on a disjoint-set forest with union by rank but without path compression run in $O(m\lg n)$ time.

answer:
$\text{MAKE-SET}$ takes constant time and both $\text{FIND-SET}$ and $\text{UNION}$ are bounded by the largest rank among all the sets. Exercise 21.4-2 bounds this from about by $\lceil \lg n \rceil$, so the actual cost of each operation is $O(\lg n)$. Therefore the actual cost of $m$ operations is $O(m\lg n)$.


question:
 Professor Dante reasons that because node ranks increase strictly along a simple path to the root, node levels must monotonically increase along the path. In other words, if $x.rank > 0$ and $x.p$ is not a root, then $\text{level}(x) \le \text{level}(x.p)$. Is the professor correct?

answer:
Professor Dante is not correct.

Suppose that we had that $x.p.rank > A_2(x.rank)$ but that $x.p.p.rank = 1 + x.p.rank$, then we would have that $\text{level}(x.p) = 0$, but $\text{level}(x) \ge 2$. So, we don't have that $\text{level}(x) \le \text{level}(x.p)$ even though we have that the ranks are monotonically increasing as we go up in the tree. Put another way, even though the ranks are monotonically increasing, the rate at which they are increasing (roughly captured by the level values) doesn't have to be increasing.


question:
 Consider the function $\alpha'(n) = \min \\{k: A_k(1) \ge \lg(n + 1)\\}$. Show that $\alpha'(n) \le 3$ for all practical values of $n$ and, using Exercise 21.4-2, show how to modify the potential-function argument to prove that we can perform a sequence of $m$ $\text{MAKE-SET}$, $\text{UNION}$, and $\text{FIND-SET}$ operations, $n$ of which are $\text{MAKE-SET}$ operations, on a disjoint-set forest with union by rank and path compression in worst-case time $O(m \alpha'(n))$.

answer:
First, observe that by a change of variables, $\alpha'(2^{n − 1}) = \alpha(n)$. Earlier in the section we saw that $\alpha(n) \le 3$ for $0 \le n \le 2047$. This means that $\alpha'(n) \le 2$ for $0 \le n \le 2^{2046}$, which is larger than the estimated number of atoms in the observable universe.

To prove the improved bound $O(m\alpha'(n))$ on the operations, the general structure will be essentially the same as that given in the section.

First, modify bound 21.2 by observing that $A_{\alpha'(n)}(x.rank) \ge A_{\alpha'(n)}(1) \ge \lg(n + 1) > x.p.rank$ which implies $\text{level}(x) \le \alpha'(n)$.

Next, redefine the potential replacing $\alpha(n)$ by $\alpha'(n)$. Lemma 21.8 now goes through just as before. All subsequent lemmas rely on these previous observations, and their proofs go through exactly as in the section, yielding the bound.


question:
 Write pseudocode for $\text{MAKE-SET}$, $\text{FIND-SET}$, and $\text{UNION}$ using the linked-list representation and the weighted-union heuristic. Make sure to specify the attributes that you assume for set objects and list objects.

answer:
The three algorithms follow the english description and are provided here. There are alternate versions using the weighted union heuristic, suffixed with $\text{WU}$.

```cpp
MAKE-SET(x)
// Assume x is a pointer to a node contains .key .set .next
    Create a node S contains .head .tail .size
    x.set = S
    x.next = NIL
    S.head = x
    S.tail = x
    S.size = 1
    return S
```

```cpp
FIND-SET(x)
    return x.set.head
```

```cpp
UNION(x, y)
    S1 = x.set
    S2 = y.set
    if S1.size >= S2.size
        S1.tail.next = S2.head
        z = S2.head
        while z != NIL // z.next is incorrect, so S2.tail node should not be updated
            z.set = S1
            z = z.next
        S1.tail = S2.tail
        S1.size = S1.size + S2.size // Update the size of set
        return S1
    else
        same procedure as above
        change x to y
        change S1 to S2
```


question:
 Show the data structure that results and the answers returned by the $\text{FIND-SET}$ operations in the following program. Use the linked-list representation with the weighted-union heuristic.
>
> ```cpp
>     for i = 1 to 16
>         MAKE-SET(x[i])
>     for i = 1 to 15 by 2
>         UNION(x[i], x[i + 1])
>     for i = 1 to 13 by 4
>        UNION(x[i], x[i + 2])
>     UNION(x[1], x[5])
>     UNION(x[11], x[13])
>     UNION(x[1], x[10])
>     FIND-SET(x[2])
>     FIND-SET(x[9])
> ```
>
> Assume that if the sets containing $x_i$ and $x_j$ have the same size, then the operation $\text{UNION}(x_i, x_j)$ appends $x_j$'s list onto $x_i$'s list.

answer:
Originally we have $16$ sets, each containing $x_i$. In the following, we'll replace $x_i$ by $i$. After the **for** loop in line 3 we have:

$$\\{1,2\\}, \\{3, 4\\}, \\{5, 6\\}, \\{7, 8\\}, \\{9, 10\\}, \\{11, 12\\}, \\{13, 14\\}, \\{15, 16\\}.$$

After the **for** loop on line 5 we have

$$\\{1, 2, 3, 4\\}, \\{5, 6, 7, 8\\}, \\{9, 10, 11, 12\\}, \\{13, 14, 15, 16\\}.$$

Line 7 results in:

$$\\{1, 2, 3, 4, 5, 6, 7, 8\\}, \\{9, 10, 11, 12\\}, \\{13, 14, 15, 16\\}.$$

Line 8 results in:

$$\\{1, 2, 3, 4, 5, 6, 7, 8\\}, \\{9, 10, 11, 12, 13, 14, 15, 16\\}.$$

Line 9 results in:

$$\\{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16\\}.$$

$\text{FIND-SET}(x_2)$ and $\text{FIND-SET}(x_9)$ each return pointers to $x_1$.

```cpp
MAKE-SET-WU(x)
    L = MAKE-SET(x)
    L.size = 1
    return L
```

```cpp
UNION-WU(x, y)
    L1 = x.set
    L2 = y.set
    if L1.size ≥ L2.size
        L = UNION(x, y)
    else L = UNION(y, x)
    L.size = L1.size + L2.size
    return L
```


question:
 Adapt the aggregate proof of Theorem 21.1 to obtain amortized time bounds of $O(1)$ for $\text{MAKE-SET}$ and $\text{FIND-SET}$ and $O(\lg n)$ for $\text{UNION}$ using the linked-list representation and the weighted-union heuristic.

answer:
During the proof of theorem 21.1, we concluded that the time for the $n$ $\text{UNION}$ operations to run was at most $O(n \lg n)$. This means that each of them took an amortized time of at most $O(\lg n)$. Also, since there is only a constant actual amount of work in performing $\text{MAKE-SET}$ and $\text{FIND-SET}$ operations, and none of that ease is used to offset costs of $\text{UNION}$ operations, they both have $O(1)$ runtime.


question:
 Give a tight asymptotic bound on the running time of the sequence of operations in Figure 21.3 assuming the linked-list representation and the weighted-union heuristic.

answer:
We call $\text{MAKE-SET}$ $n$ times, which contributes $\Theta(n)$. In each union, the smaller set is of size $1$, so each of these takes $\Theta(1)$ time. Since we union $n - 1$ times, the runtime is $\Theta(n)$.


question:
 Professor Gompers suspects that it might be possible to keep just one pointer in each set object, rather than two ($head$ and $tail$), while keeping the number of pointers in each list element at two. Show that the professor's suspicion is well founded by describing how to represent each set by a linked list such that each operation has the same running time as the operations described in this section. Describe also how the operations work. Your scheme should allow for the weighted-union heuristic, with the same effect as described in this section. ($\textit{Hint:}$ Use the tail of a linked list as its set's representative.)

answer:
For each member of the set, we will make its first field which used to point back to the set object point instead to the last element of the linked list. Then, given any set, we can find its last element by going ot the head and following the pointer that that object maintains to the last element of the linked list. This only requires following exactly two pointers, so it takes a constant amount of time. Some care must be taken when unioning these modified sets. Since the set representative is the last element in the set, when we combine two linked lists, we place the smaller of the two sets before the larger, since we need to update their set representative pointers, unlike the original situation, where we update the representative of the objects that are placed on to the end of the linked list.


question:
 Suggest a simple change to the $\text{UNION}$ procedure for the linked-list representation that removes the need to keep the $tail$ pointer to the last object in each list. Whether or not the weighted-union heuristic is used, your change should not change the asymptotic running time of the $\text{UNION}$ procedure. ($\textit{Hint:}$ Rather than appending one list to another, splice them together.)

answer:
Instead of appending the second list to the end of the first, we can imagine splicing it into the first list, in between the head and the elements. Store a pointer to the first element in $S_1$. Then for each element $x$ in $S_2$, set $x.head = S_1.head$. When the last element of $S_2$ is reached, set its next pointer to the first element of $S_1$. If we always let $S_2$ play the role of the smaller set, this works well with the weighted-union heuristic and don't affect the asymptotic running time of $\text{UNION}$.


question:
 Run the Ford-Fulkerson algorithm on the flow network in Figure 26.8 \(c\) and show the residual network after each flow augmentation. Number the vertices in $L$ top to bottom from $1$ to $5$ and in $R$ top to bottom from $6$ to $9$. For each iteration, pick the augmenting path that is lexicographically smallest.

answer:
First, we pick an augmenting path that passes through vertices 1 and 6. Then, we pick the path going through 2 and 8. Then, we pick the path going through 3 and 7. Then, the resulting residual graph has no path from $s$ to $t$. So, we know that we are done, and that we are pairing up vertices $(1, 6)$, $(2, 8)$, and $(3, 7)$. This number of unit augmenting paths agrees with the value of the cut where you cut the edges $(s, 3)$, $(6, t)$, and $(7, t)$.


question: Prove Theorem 26.10.
answer:
We proceed by induction on the number of iterations of the while loop of Ford-Fulkerson. After the first iteration, since $c$ only takes on integer values and $(u, v).f$ is set to $0$, $c_f$ only takes on integer values. Thus, lines 7 and 8 of Ford-Fulkerson only assign integer values to $(u, v).f$. Assume that $(u, v).f \in \mathbb Z$ for all $(u, v)$ after the $n$th iteration. On the $(n + 1)$th iteration $c_f(p)$ is set to the minimum of $c_f(u, v)$ which is an integer by the induction hypothesis. Lines 7 and 8 compute $(u, v).f$ or $(v, u).f$. Either way, these the the sum or difference of integers by assumption, so after the $(n + 1)$th iteration we have that $(u, v).f$ is an integer for all $(u, v) \in E$. Since the value of the flow is a sum of flows of edges, we must have $|f| \in \mathbb Z$ as well.


answer: (Removed)
question:
 Let $G = (V, E)$ be a bipartite graph with vertex partition $V = L \cup R$, and let $G'$ be its corresponding flow network. Give a good upper bound on the length of any augmenting path found in $G'$ during the execution of $\text{FORD-FULKERSON}$.


question:
 A **_perfect matching_** is a matching in which every vertex is matched. Let $G = (V, E)$ be an undirected bipartite graph with vertex partition $V = L \cup R$, where $|L| = |R|$. For any $X \subseteq V$, define the **_neighborhood_** of $X$ as
>
> $$N(X) = \\{y \in V: (x, y) \in E \text{ for some } x \in X\\},$$
>
> that is, the set of vertices adjacent to some member of $X$. Prove **_Hall's theorem_**: there exists a perfect matching in $G$ if and only if $|A| \le |N(A)|$ for every subset $A \subseteq L$.

answer:
First suppose there exists a perfect matching in $G$. Then for any subset $A \subseteq L$, each vertex of $A$ is matched with a neighbor in $R$, and since it is a matching, no two such vertices are matched with the same vertex in $R$. Thus, there are at least $|A|$ vertices in the neighborhood of $A$.

Now suppose that $|A| \le |N(A)|$ for all $A \subseteq L$. Run Ford-Fulkerson on the corresponding flow network. The flow is increased by $1$ each time an augmenting path is found, so it will suffice to show that this happens $|L|$ times. Suppose the while loop has run fewer than $L$ times, but there is no augmenting path. Then fewer than $L$ edges from $L$ to $R$ have flow $1$.

Let $v_1 \in L$ be such that no edge from $v_1$ to a vertex in $R$ has nonzero flow. By assumption, $v_1$ has at least one neighbor $v_1' \in R$. If any of $v_1$'s neighbors are connected to $t$ in $G_f$ then there is a path, so assume this is not the case. Thus, there must be some edge $(v_2, v_1)$ with flow $1$. By assumption, $N(\\{v_1, v_2\\}) \ge 2$, so there must exist $v_2' \ne v_1'$ such that $v_2'\in N(\\{v_1, v_2 \\})$. If $(v_2', t)$ is an edge in the residual network we're done since $v_2'$ must be a neighbor of $v_2$, so $s$, $v_1$, $v_1'$, $v_2$, $v_2'$, and $t$ is a path in $G_f$. Otherwise $v_2'$ must have a neighbor $v_3 \in L$ such that $(v_3, v_2')$ is in $G_f$. Specifically, $v_3 \ne v_1$ since $(v_3, v_2')$ has flow $1$, and $v_3 \ne v_2$ since $(v_2, v_1')$ has flow $1$, so no more flow can leave $v_2$ without violating conservation of flow. Again by our hypothesis, $N(\\{v_1, v_2, v_3\\}) \ge 3$, so there is another neighbor $v_3' \in R$.

Continuing in this fashion, we keep building up the neighborhood $v_i'$, expanding each time we find that $(v_i', t)$ is not an edge in $G_f$. This cannot happen $L$ times, since we have run the Ford-Fulkerson while-loop fewer than $|L|$ times, so there still exist edges into $t$ in $G_f$. Thus, the process must stop at some vertex $v_k'$, and we obtain an augmenting path

$$s, v_1, v_1', v_2, v_2', v_3, \ldots, v_k, v_k', t,$$

contradicting our assumption that there was no such path. Therefore the while loop runs at least $|L|$ times. By Corollary 26.3 the flow strictly increases each time by $f_p$. By Theorem 26.10 $f_p$ is an integer. In particular, it is equal to $1$. This implies that $|f| \ge |L|$. It is clear that $|f| \le |L|$, so we must have $|f| = |L|$. By Corollary 26.11 this is the cardinality of a maximum matching. Since $|L| = |R|$, any maximum matching must be a perfect matching.


question:
 We say that a bipartite graph $G = (V, E)$, where $V = L \cup R$, is **_$d$-regular_** if every vertex $v \in V$ has degree exactly $d$. Every $d$-regular bipartite graph has $|L| = |R|$. Prove that every $d$-regular bipartite graph has a matching of cardinality $|L|$ by arguing that a minimum cut of the corresponding flow network has capacity $|L|$.

answer:
We convert the bipartite graph into a flow problem by making a new vertex for the source which has an edge of unit capacity going to each of the vertices in $L$, and a new vertex for the sink that has an edge from each of the vertices in $R$, each with unit capacity. We want to show that the number of edge between the two parts of the cut is at least $L$, this would get us by the max-flow-min-cut theorem that there is a flow of value at least $|L|$. The, we can apply the integrality theorem that all of the flow values are integers, meaning that we are selecting $|L|$ disjoint edges between $L$ and $R$.

To see that every cut must have capacity at lest $|L|$, let $S_1$ be the side of the cut containing the source and let $S_2$ be the side of the cut containing the sink. Then, look at $L \cap S_1$. The source has an edge going to each of $L \cap (S_1)^c$, and there is an edge from $R \cap S_1$ to the sink that will be cut. This means that we need that there are at least $|L \cap S_1| - |R \cap S_1|$ many edges going from $L \cap S_1$ to $R \cap S_2$. If we look at the set of all neighbors of $L \cap S_1$, we get that there must be at least the same number of neighbors in $R$, because otherwise we could sum up the degrees going from $L \cap S_1$ to $R$ on both sides, and get that some of the vertices in $R$ would need to have a degree higher than $d$. This means that the number of neighbors of $L \cap S_1$ is at least $L \cap S_1$, but we have that they are in $S_1$, but there are only $|R \cap S_1|$ of those, so, we have that the size of the set of neighbors of $L \cap S_1$ that are in $S_2$ is at least $|L \cap S_1| - |R \cap S_1|$. Since each of these neighbors has an edge crossing the cut, we have that the total number of edges that the cut breaks is at least

$$(|L| - |L \cap S_1|) + (|L \cap S_1| - |R \cap S_1|) + |R \cap S_1| = |L|.$$

Since each of these edges is unit valued, the value of the cut is at least $|L|$.


answer: (Removed)
question:
 Prove that, after the procedure $\text{INITIALIZE-PREFLOW}(G, S)$ terminates, we have $s.e \le -|f^\*|$, where $f^\*$ is a maximum flow for $G$.


question:
 Show how to implement the generic push-relabel algorithm using $O(V)$ time per relabel operation, $O(1)$ time per push, and $O(1)$ time to select an applicable operation, for a total time of $O(V^2E)$.

answer:
We must select an appropriate data structure to store all the information which will allow us to select a valid operation in constant time. To do this, we will need to maintain a list of overflowing vertices. By Lemma 26.14, a push or a relabel operation always applies to an overflowing vertex. To determine which operation to perform, we need to determine whether $u.h = v.h + 1$ for some $v \in N(u)$. We'll do this by maintaining a list $u.high$ of all neighbors of $u$ in $G_f$ which have height greater than or equal to $u$. We'll update these attributes in the $\text{PUSH}$ and $\text{RELABEL}$ functions. It is clear from the pseudocode given for $\text{PUSH}$ that we can execute it in constant time, provided we have maintain the attributes $\delta_f(u, v)$, $u.e$, $c_f(u, v)$, $(u, v).f$ and $u.h$. Each time we call $\text{PUSH}(u, v)$ the result is that $u$ is no longer overflowing, so we must remove it from the list.

Maintain a pointer $u.overflow$ to $u$'s position in the overflow list. If a vertex $u$ is not overflowing, set $u.overflow = \text{NIL}$. Next, check if $v$ became overflowing. If so, set $v.overflow$ equal to the head of the overflow list. Since we can update the pointer in constant time and delete from a linked list given a pointer to the element to be deleted in constant time, we can maintain the list in $O(1)$.

The $\text{RELABEL}$ operation takes $O(V)$ because we need to compute the minimum $v.h$ from among all $(u, v) \in E_f$, and there could be $|V| - 1$ many such $v$. We will also need to update $u.high$ during $\text{RELABEL}$. When $\text{RELABEL}(u)$ is called, set $u.high$ equal to the empty list and for each vertex $v$ which is adjacent to $u$, if $v.h = u.h + 1$, add $u$ to the list $v.high$. Since this takes constant time per adjacent vertex we can maintain the attribute in $O(V)$ per call to relabel.


answer: (Removed)
question:
 Prove that the generic push-relabel algorithm spends a total of only $O(VE)$ time in performing all the $O(V^2)$ relabel operations.


answer: (Removed)
question:
 Suppose that we have found a maximum flow in a flow network $G = (V, E)$ using a push-relabel algorithm. Give a fast algorithm to find a minimum cut in $G$.


question:
 Give an efficient push-relabel algorithm to find a maximum matching in a bipartite graph. Analyze your algorithm.

answer:
First, construct the flow network for the bipartite graph as in the previous section. Then, we relabel everything in $L$. Then, we push from every vertex in $L$ to a vertex in $R$, so long as it is possible.

Keeping track of those that vertices of $L$ that are still overflowing can be done by a simple bit vector. Then, we relabel everything in R and push to the last vertex. Once these operations have been done, The only possible valid operations are to relabel the vertices of $L$ that weren't able to find an edge that they could push their flow along, so could possibly have to get a push back from $R$ to $L$. This continues until there are no more operations to do. This takes time of $O(V(E + V))$.


question:
 Suppose that all edge capacities in a flow network $G = (V, E)$ are in the set $\\{1, 2, \ldots, k\\}$. Analyze the running time of the generic push-relabel algorithm in terms of $|V|$, $|E|$, and $k$. ($\textit{Hint:}$ How many times can each edge support a nonsaturating push before it becomes saturated?)

answer:
The number of relabel operations and saturating pushes is the same as before. An edge can handle at most $k$ nonsaturating pushes before it becomes saturated, so the number of nonsaturating pushes is at most $2k|V||E|$. Thus, the total number of basic operations is at most $2|V|^2 + 2|V||E| + 2k|V||E| = O(kVE)$.


answer: (Removed)
question:
 Show that we could change line 6 of $\text{INITIALIZE-PREFLOW}$ to
>
> ```cpp
>  6 s.h = |G.V| - 2
> ```
>
> without affecting the correctness or asymptotic performance of the generic pushrelabel algorithm.


question:
 Let $\delta_f(u, v)$ be the distance (number of edges) from $u$ to $v$ in the residual network $G_f$. Show that the $\text{GENERIC-PUSH-RELABEL}$ procedure maintains the properties that $u.h < |V|$ implies $u.h \le \delta_f(u, t)$ and that $u.h \ge |V|$ implies $u.h - |V| \le \delta_f(u, s)$.

answer:
We'll prove the claim by induction on the number of push and relabel operations. Initially, we have $u.h = |V|$ if $u = s$ and $0$ otherwise. We have $s.h - |V| = 0 \le \delta_f(s, s) = 0$ and $u.h = 0 \le \delta_f(u, t)$ for all $u \ne s$, so the claim holds prior to the first iteration of the while loop on line 2 of the $\text{GENERIC-PUSH-RELABEL}$ algorithm.

Suppose that the properties have been maintained thus far. If the next iteration is a nonsaturating push then the properties are maintained because the heights and existence of edges in the residual network are preserved. If it is a saturating push then edge $(u, v)$ is removed from the residual network, which increases both $\delta_f(u, t)$ and $\delta_f(u, s)$, so the properties are maintained regardless of the height of $u$.

Now suppose that the next iteration causes a relabel of vertex $u$. For all $v$ such that $(u, v) \in E_f$ we must have $u.h \le v.h$. Let $v' = \min\\{v.h \mid (u,v) \in E_f\\}$. There are two cases to consider.

- First, suppose that $v.h < |V|$. Then after relabeling we have

    $$u.h = 1 + v'.h \le 1 + \min_{(u, v)} \in E_f \delta_f(v, t) = \delta_f(u, t).$$

- Second, suppose that $v'.h \ge |V|$. Then after relabeling we have

    $$u.h = 1 + v'.h \le 1 + |V| + \min_{(u, v)} \in E_f \delta_f(v, s) = \delta_f(u, s) + |V|,$$

    which implies that $u.h - |V| \le \delta_f(u, s)$.

Therefore, the $\text{GENERIC-PUSH-RELABEL}$ procedure maintains the desired properties.


question:
 As in the previous exercise, let $\delta_f(u, v)$ be the distance from $u$ to $v$ in the residual network $G_f$. Show how to modify the generic push-relabel algorithm to maintain the property that $u.h < |V|$ implies $u.h = \delta_f(u, t)$ and that $u.h \ge |V|$ implies $u.h - |V| = \delta_f(u, s)$. The total time that your implementation dedicates to maintaining this property should be $O(VE)$.

answer:
What we should do is to, for successive backwards neighborhoods of $t$, relabel everything in that neighborhood. This will only take at most $O(VE)$ time (see 26.4-3). This also has the upshot of making it so that once we are done with it, every vertex's height is equal to the quantity $\delta_f(u, t)$. Then, since we begin with equality, after doing this, the inductive step we had in the solution to the previous exercise shows that this equality is preserved.


question:
 Show that the number of nonsaturating pushes executed by the $\text{GENERIC-PUSH-RELABEL}$ procedure on a flow network $G = (V, E)$ is at most $4|V|^2|E|$ for $|V| \ge 4$.

answer:
Each vertex has maximum height $2|V| - 1$. Since heights don't decrease, and there are $|V| - 2$ vertices which can be overflowing, the maximum contribution of relabels to $\Phi$ over all vertices is $(2|V| - 1)(|V| - 2)$. A saturating push from $u$ to $v$ increases $\Phi$ by at most $v.h \le 2|V| - 1$, and there are at most $2|V||E|$ saturating pushes, so the total contribution over all saturating pushes to $\Phi$ is at most $(2|V| - 1)(2|V||E|)$. Since each nonsaturating push decrements $\Phi$ by at least on and $\Phi$ must equal zero upon termination, we must have that the number of nonsaturating pushes is at most

$$(2|V| - 1)(|V| - 2) + (2|V| - 1)(2|V||E|) = 4|V|^2|E| + 2|V|^2 - 5|V| + 3 - 2|V||E|.$$

Using the fact that $|E| \ge |V| - 1$ and $|V| \ge 4$ we can bound the number of
saturating pushes by $4|V|^2|E|$.


question:
 Illustrate the execution of $\text{RELABEL-TO-FRONT}$ in the manner of Figure 26.10 for the flow network in Figure 26.1(a). Assume that the initial ordering of vertices in $L$ is $\langle v_1, v_2, v_3, v_4 \rangle$ and that the neighbor lists are
>
> $$
> \begin{aligned}
> v_1.N & = \langle s, v_2, v_3 \rangle, \\\\
> v_2.N & = \langle s, v_1, v_3, v_4 \rangle, \\\\
> v_3.N & = \langle v_1, v_2, v_4, t \rangle, \\\\
> v_4.N & = \langle v_2, v_3, t \rangle.
> \end{aligned}
> $$

answer:
When we initialize the preflow, we have $29$ units of flow leaving $s$. Then, we consider $v_1$ since it is the first element in the $L$ list. When we discharge it, we increase it's height to $1$ so that it can dump $12$ of it's excess along its edge to vertex $v_3$, to discharge the rest of it, it has to increase it's height to $|V| + 1$ to discharge it back to $s$. It was already at the front, so, we consider $v_2$. We increase its height to $1$. Then, we send all of its excess along its edge to $v_4$. We move it to the front, which means we next consider $v_1$, and do nothing because it is not overflowing. Up next is vertex $v_3$. After increasing its height to $1$, it can send all of its excess to $t$. This puts $v_3$ at the front, and we consider the non-overflowing vertices $v_2$ and $v_1$. Then, we consider $v_4$, it increases its height to $1$, then sends $4$ units to $t$. Since it still has an excess of $9$ units, it increases its height once again. Then it becomes valid for it to send flow back to $v_2$ or to $v_3$. It considers $v_2$ first because of the ordering of its neighbor list. This means that $9$ units of flow are pushed back to $v_2$. Since $v_4.h$ increased, it moves to the front of the list Then, we consider $v_2$ since it is the only still overflowing vertex. We increase its height to $3$. Then, it is overflowing by $9$ so it increases its height to $3$ to send $9$ units to $v_4$. It's height increased so it goes to the of the list. Then, we consider $v_4$, which is overflowing. It pushes $7$ units to $v_3$. Since it is still overflowing by $2$, it increases its height to $4$ and pushes the rest back to $v_2$ and goes to the front of the list. Up next is $v_2$, which increases its height by $2$ to sends its overflow to $v_4$. The excess flow keeps bobbing around the four vertices, each time requiring them to increase their height a bit to discharge to a neighbor only to have that neighbor increase to discharge it back until $v_2$ has increased in height enough to send all of it's excess back to s. Last but not least, $v_3$ pushes its overflow of $7$ units to $t$, and gives us a maximum flow of $23$.


question:
 We would like to implement a push-relabel algorithm in which we maintain a firstin, first-out queue of overflowing vertices. The algorithm repeatedly discharges the vertex at the head of the queue, and any vertices that were not overflowing before the discharge but are overflowing afterward are placed at the end of the queue. After the vertex at the head of the queue is discharged, it is removed. When the queue is empty, the algorithm terminates. Show how to implement this algorithm to compute a maximum flow in $O(V^3)$ time.

answer:
Initially, the vertices adjacent to $s$ are the only ones which are overflowing. The implementation is as follows:

```cpp
PUSH-RELABEL-QUEUE(G, s)
    INITIALIZE-PREFLOW(G, s)
    let q be a new empty queue
    for v ∈ G.Adj[s]
        PUSH(q, v)
    while q.head != NIL
        DISCHARGE(q.head)
        POP(q)
```

Note that we need to modify the $\text{DISCHARGE}$ algorithm to push vertices $v$ onto the queue if $v$ was not overflowing before a discharge but is overflowing after one.

Between lines 7 and 8 of $\text{DISCHARGE}(u)$, add the line "if $v.e > 0$, $\text{PUSH}(q, v)$." This is an implementation of the generic push-relabel algorithm, so we know it is correct. The analysis of runtime is almost identical to that of Theorem 26.30. We just need to verify that there are at most $|V|$ calls to $\text{DISCHARGE}$ between two consecutive relabel operations. Observe that after calling $\text{PUSH}(u, v)$, Corollary 26.28 tells us that no admissible edges are entering $v$. Thus, once $v$ is put into the queue because of the push, it can't be added again until it has been relabeled. Thus, at most $|V|$ vertices are added to the queue between relabel operations.


question:
 Show that the generic algorithm still works if $\text{RELABEL}$ updates $u.h$ by simply computing $u.h = u.h + 1$. How would this change affect the analysis of $\text{RELABEL-TO-FRONT}$?

answer:
If we change relabel to just increment the value of $u$, we will not be ruining the correctness of the Algorithm. This is because since it only applies when $u.h \le v.h$, we won't be every creating a graph where $h$ ceases to be a height function, since $u.h$ will only ever be increasing by exactly $1$ whenever relabel is called, ensuring that $u.h + 1 \le v.h$. This means that Lemmatae 26.15 and 26.16 will still hold. Even Corollary 26.21 holds since all it counts on is that relabel causes some vertex's $h$ value to increase by at least $1$, it will still work when we have all of the operations causing it to increase by exactly $1$. However, Lemma 26.28 will no longer hold. That is, it may require more than a single relabel operation to cause an admissible edge to appear, if for example, $u.h$ was strictly less than the $h$ values of all its neighbors. However, this lemma is not used in the proof of Exercise 26.4-3, which bounds the number of relabel operations. Since the number of relabel operations still have the same bound, and we know that we can simulate the old relabel operation by doing (possibly many) of these new relabel operations, we have the same bound as in the original algorithm with this different relabel operation.


question:
 Show that if we always discharge a highest overflowing vertex, we can make the push-relabel method run in $O(V^3)$ time.

answer:
We'll keep track of the heights of the overflowing vertices using an array and a series of doubly linked lists. In particular, let $A$ be an array of size $|V|$, and let $A[i]$ store a list of the elements of height $i$. Now we create another list $L$, which is a list of lists. The head points to the list containing the vertices of highest height. The next pointer of this list points to the next nonempty list stored in $A$, and so on. This allows for constant time insertion of a vertex into $A$, and also constant time access to an element of largest height, and because all lists are doubly linked, we can add and delete elements in constant time. Essentially, we are implementing the algorithm of Exercise 26.5-2, but with the queue replaced by a priority queue with constant time operations. As before, it will suffice to show that there are at most $|V|$ calls to discharge between consecutive relabel operations.

Consider what happens when a vertex $v$ is put into the priority queue. There must exist a vertex $u$ for which we have called $\text{PUSH}(u, v)$. After this, no ad- missible edge is entering $v$, so it can't be added to the priority queue again until after a relabel operation has occurred on $v$. Moreover, every call to $\text{DISCHARGE}$ terminates with a $\text{PUSH}$, so for every call to $\text{DISCHARGE}$ there is another vertex which can't be added until a relabel operation occurs. After $|V|$ $\text{DISCHARGE}$ operations and no relabel operations, there are no remaining valid $\text{PUSH}$ operations, so either the algorithm terminates, or there is a valid relabel operation which is performed. Thus, there are $O(V^3)$ calls to $\text{DISCHARGE}$. By carrying out the rest of the analysis of Theorem 26.30, we conclude that the runtime is $O(V^3)$.


question:
 Suppose that at some point in the execution of a push-relabel algorithm, there exists an integer $0 < k \le |V| - 1$ for which no vertex has $v.h = k$. Show that all vertices with $v.h > k$ are on the source side of a minimum cut. If such a $k$ exists, the **_gap heuristic_** updates every vertex $v \in V - \\{s\\}$ for which $v.h > k$, to set $v.h = \max(v.h, |V| + 1)$. Show that the resulting attribute $h$ is a height function. (The gap heuristic is crucial in making implementations of the push-relabel method perform well in practice.)

answer:
Suppose to try and obtain a contradiction that there were some minimum cut for which a vertex that had $v.h > k$ were on the sink side of that cut. For that minimum cut, there is a residual flow network for which that cut is saturated. Then, if there were any vertices that were also on the sink side of the cut which had an edge going to $v$ in this residual flow network, since it's $h$ value cannot be equal to $k$, we know that it must be greater than $k$ since it could be only at most one less than $v$. We can continue in this way to let $S$ be the set of vertices on the sink side of the graph which have an $h$ value greater than $k$. Suppose that there were some simple path from a vertex in $S$ to $s$. Then, at each of these steps, the height could only decrease by at most $1$, since it cannot get from above $k$ to $0$ without going through $k$, we know that there is no path in the residual flow network going from a vertex in $S$ to $s$. Since a minimal cut corresponds to disconnected parts of the residual graph for a maximum flow, and we know there is no path from $S$ to $s$, there is a minimum cut for which $S$ lies entirely on the source side of the cut. This was a contradiction to how we selected $v$, and so have shown the first claim.

Now we show that after updating the $h$ values as suggested, we are still left with a height function. Suppose we had an edge $(u, v)$ in the residual graph. We knew from before that $u.h \le v.h + 1$. However, this means that if $u.h > k$, so must be $v.h$. So, if both were above $k$, we would be making them equal, causing the inequality to still hold. Also, if just $v.k$ were above $k$, then we have not decreased it's $h$ value, meaning that the inequality also still must hold. Since we have not changed the value of $s.h$, and $t.h$, we have all the required properties to have a height function after modifying the $h$ values as described.


answer: (Removed)
question:
 Prove that the summations in equation $\text{(26.6)}$ equal the summations in equation $\text{(26.7)}$.


question:
 In Figure $\text{26.1}$(b), what is the flow across the cut $(\\{s, v_2, v_4\\}, \\{v_1, v_3, t\\})$? What is the capacity of this cut?

answer:
$$
\begin{aligned}
f(S, T) & = f(s, v_1) + f(v_2, v_1) + f(v_4, v_3) + f(v_4, t) - f(v_3, v_2) = 11 + 1 + 7 + 4 - 4 = 19, \\\\
c(S, T) & = c(s, v_1) + c(v_2, v_1) + c(v_4, v_3) + c(v_4, t) = 16 + 4 + 7 + 4 = 31.
\end{aligned}
$$


question: Show the execution of the Edmonds-Karp algorithm on the flow network of Figure 26.1(a).
answer:
If we perform a breadth first search where we consider the neighbors of a vertex as they appear in the ordering $\\{s, v_1, v_2, v_3, v_4, t\\}$, the first path that we will find is $s, v_1, v_3, t$. The min capacity of this augmenting path is $12$, so we send $12$ units along it. We perform a $\text{BFS}$ on the resulting residual network. This gets us the path $s, v_2, v_4, t$. The min capacity along this path is $4$, so we send $4$ units along it. Then, the only path remaining in the residual network is $\\{s, v_2, v_4, v_3, t\\}$ which has a min capacity of $7$, since that's all that's left, we find it in our $\text{BFS}$. Putting it all together, the total flow that we have found has a value of $23$.


question:
 In the example of Figure 26.6, what is the minimum cut corresponding to the maximum flow shown? Of the augmenting paths appearing in the example, which one cancels flow?

answer:
A minimum cut corresponding to the maximum flow is $S = \\{s, v_1, v_2, v_4\\}$ and $T = \\{v_3, t\\}$. The augmenting path in part \(c\) cancels flow on edge $(v_3, v_2)$.


question:
 Recall that the construction in Section 26.1 that converts a flow network with multiple sources and sinks into a single-source, single-sink network adds edges with infinite capacity. Prove that any flow in the resulting network has a finite value if the edges of the original network with multiple sources and sinks have finite capacity.

answer:
Since the only edges that have infinite value are those going from the supersource or to the supersink, as long as we pick a cut that has the supersource and all the original sources on one side, and the other side has the supersink as well as all the original sinks, then it will only cut through edges of finite capacity. Then, by Corollary 26.5, we have that the value of the flow is bounded above by the value of any of these types of cuts, which is finite.


answer: $c(s, s_i) = p_i$, $c(t_j, t) = q_j$.
question:
 Suppose that each source $s_i$ in a flow network with multiple sources and sinks produces exactly $p_i$ units of flow, so that $\sum_{v \in V} f(s_i, v) = p_i$. Suppose also that each sink $t_j$ consumes exactly $q_j$ units, so that $\sum_{v \in V} f(v, t_j) = q_j$, where $\sum_i p_i = \sum_j q_j$. Show how to convert the problem of finding a flow $f$ that obeys these additional constraints into the problem of finding a maximum flow in a single-source, single-sink flow network.


question: Prove Lemma 26.2.
answer:
To check that $f_p$ is a flow, we make sure that it satisfies both the capacity constraints and the flow constraints. First, the capacity constraints. To see this, we recall our definition of $c_f(p)$, that is, it is the smallest residual capacity of any of the edges along the path $p$. Since we have that the residual capacity is always less than or equal to the initial capacity, we have that each value of the flow is less than the capacity. Second, we check the flow constraints, Since the only edges that are given any flow are along a path, we have that at each vertex interior to the path, the flow in from one edge is immediately canceled by the flow out to the next vertex in the path. Lastly, we can check that its value is equal to $c_f(p)$ because, while $s$ may show up at spots later on in the path, it will be canceled out as it leaves to go to the next vertex. So, the only net flow from s is the initial edge along the path, since it (along with all the other edges) is given flow $c_f(p)$, that is the value of the flow $f_p$.


answer: (Removed)
question:
 Suppose that we redefine the residual network to disallow edges into $s$. Argue that the procedure $\text{FORD-FULKERSON}$ still correctly computes a maximum flow.


answer: (Removed)
question:
 Suppose that both $f$ and $f'$ are flows in a network $G$ and we compute flow $f \uparrow f'$. Does the augmented flow satisfy the flow conservation property? Does it satisfy the capacity constraint?


question:
 Show how to find a maximum flow in a network $G = (V, E)$ by a sequence of at most $|E|$ augmenting paths. ($\textit{Hint:}$ Determine the paths after finding the maximum flow.)

answer:
Suppose we already have a maximum flow $f$. Consider a new graph $G$ where we set the capacity of edge $(u, v)$ to $f(u, v)$. Run Ford-Fulkerson, with the modification that we remove an edge if its flow reaches its capacity. In other words, if $f(u, v) = c(u, v)$ then there should be no reverse edge appearing in residual network. This will still produce correct output in our case because we never exceed the actual maximum flow through an edge, so it is never advantageous to cancel flow. The augmenting paths chosen in this modified version of Ford-Fulkerson are precisely the ones we want. There are at most $|E|$ because every augmenting path produces at least one edge whose flow is equal to its capacity, which we set to be the actual flow for the edge in a maximum flow, and our modification prevents us from ever destroying this progress.


question:
 The **_edge connectivity_** of an undirected graph is the minimum number $k$ of edges that must be removed to disconnect the graph. For example, the edge connectivity of a tree is $1$, and the edge connectivity of a cyclic chain of vertices is $2$. Show how to determine the edge connectivity of an undirected graph $G = (V, E)$ by running a maximum-flow algorithm on at most $|V|$ flow networks, each having $O(V)$ vertices and $O(E)$ edges.

answer:
Create an directed version of the graph. Then create a flow network out of it, resolving all antiparallel edges. All edges' capacities are set to $1$. Pick any vertex that wasn't created for antiparallel workaround as the sink and run maximum-flow algorithm with all vertexes that aren't for antipararrel workaround (except the sink) as sources. Find the minimum value out of all $|V| - 1$ maximum flow values.


answer: (Removed)
question:
 Suppose that you are given a flow network $G$, and $G$ has edges entering the source $s$. Let $f$ be a flow in $G$ in which one of the edges $(v, s)$ entering the source has $f(v, s) = 1$. Prove that there must exist another flow $f'$ with $f'(v, s) = 0$ such that $|f| = |f'|$. Give an $O(E)$-time algorithm to compute $f'$, given $f$, and assuming that all edge capacities are integers.


answer: (Removed)
question:
 Suppose that you wish to find, among all minimum cuts in a flow network $G$ with integral capacities, one that contains the smallest number of edges. Show how to modify the capacities of $G$ to create a new flow network $G'$ in which any minimum cut in $G'$ is a minimum cut with the smallest number of edges in $G$.


question:
 Show that splitting an edge in a flow network yields an equivalent network. More formally, suppose that flow network $G$ contains edge $(u, v)$, and we create a new flow network $G'$ by creating a new vertex $x$ and replacing $(u, v)$ by new edges $(u, x)$ and $(x, v)$ with $c(u, x) = c(x, v) = c(u, v)$. Show that a maximum flow in $G'$ has the same value as a maximum flow in $G$.

answer:
Suppose the maximum flow of a graph $G = (V, E)$ with source $s$ and destination $t$ is $|f| = \sum{f(s, v)}$, where $v \in V$ are vertices in the maximum flow between $s$ and $t$.

We know every vertex $v \in V$ must obey the Flow conservation rule. Therefore, if we can add or delete some vertices between $s$ and $t$ without changing $|f|$ or violating the Flow conversation rule, then the new graph $G' = (V', E')$ will have the same maximum flow as the original graph $G$, and that's why we can replace edge $(u, v)$ by new edges $(u, x)$ and $(x, v)$ with $c(u, x) = c(x, v) = c(u, v)$.

After doing so, vertex $v_1$ and $v_2$ still obey the Flow conservation rule since the values flow in to or flow out of $v_1$ and $v_2$ do not change at all.
Meanwhile, the value $|f| = \sum{f(s, v)}$ remains the same.

In fact, we can split any edges in this way, even if two vertex $u$ and $v$ doesn't have any connection between them, we can still add a vertex $y$ and make $c(u, y) = c(y, v) = 0$.

To conclude, we can transform any graph with or without antiparallel edges into an equivalent graph without antiparallel edges and have the same maximum flow value.


question:
 Extend the flow properties and definitions to the multiple-source, multiple-sink problem. Show that any flow in a multiple-source, multiple-sink flow network corresponds to a flow of identical value in the single-source, single-sink network obtained by adding a supersource and a supersink, and vice versa.

answer:
Capacity constraint: for all $u, v \in V$, we require $0 \le f(u, v) \le c(u, v)$.

Flow conservation: for all $u \in V - S - T$, we require $\sum_{v \in V} f(v, u) = \sum_{v \in V} f(u, v)$.


answer: (Removed)
question:
 Suppose that a flow network $G = (V, E)$ violates the assumption that the network contains a path $s \leadsto v \leadsto t$ for all vertices $v \in V$. Let $u$ be a vertex for which there is no path $s \leadsto u \leadsto t$. Show that there must exist a maximum flow $f$ in $G$ such that $f(u, v) = f(v, u) = 0$ for all vertices $v \in V$.


answer: (Removed)
question:
 Let $f$ be a flow in a network, and let $\alpha$ be a real number. The **_scalar flow product_**, denoted $\alpha f$, is a function from $V \times V$ to $\mathbb{R}$ defined by
>
> $$(\alpha f)(u, v) = \alpha \cdot f(u, v).$$
>
> Prove that the flows in a network form a **_convex set_**. That is, show that if $f_1$ and $f_2$ are flows, then so is $\alpha f_1 + (1 - \alpha) f_2$ for all $\alpha$ in the range $0 \le \alpha \le 1$.


question: State the maximum-flow problem as a linear-programming problem.
answer:
$$
\begin{array}{ll}
\max & \sum\limits_{v \in V} f(s, v) - \sum\limits_{v \in V} f(v, s) \\\\
s.t. & 0 \le f(u, v) \le c(u, v) \\\\
     & \sum\limits_{v \in V} f(v, u) - \sum\limits_{v \in V} f(u, v) = 0
\end{array}
$$


answer: (Removed)
question:
 Professor Adam has two children who, unfortunately, dislike each other. The problem is so severe that not only do they refuse to walk to school together, but in fact each one refuses to walk on any block that the other child has stepped on that day. The children have no problem with their paths crossing at a corner. Fortunately both the professor's house and the school are on corners, but beyond that he is not sure if it is going to be possible to send both of his children to the same school. The professor has a map of his town. Show how to formulate the problem of determining whether both his children can go to the same school as a maximum-flow problem.


answer: (Removed)
question:
 Suppose that, in addition to edge capacities, a flow network has **_vertex capacities_**. That is each vertex $v$ has a limit $l(v)$ on how much flow can pass though $v$. Show how to transform a flow network $G = (V, E)$ with vertex capacities into an equivalent flow network $G' = (V', E')$ without vertex capacities, such that a maximum flow in $G'$ has the same value as a maximum flow in $G$. How many vertices and edges does $G'$ have?


