Divide and Conquer

A problem solving technique

Divide and conquer algorithms

• D & C algorithms divide a problem into two or more smaller problems

• The smaller problems are usually the same problem, but smaller than the original problem

• We keep dividing the problem into smaller instances until the solution of the problem is obvious

D & C is a top down approach

• The solution to a top-level problem is found by going down and finding solutions to smaller instances of the same problem

• This is the same method used by recursive routines

• When you are writing recursion, you should think at the problem solving level, and let the system handle the details

• Sometimes then it can be rewritten more efficiently iteratively

– This is often the case if there is tail recursion (no operations are done after the recursion)

Top down solution to binary search

• Known: a sorted array with indices from low to high

• Find: the location of an item x in the array; return 0 if not found

• Ignoring the details for now, write in pseudocode a recursive algorithm to solve this problem

Analyzing the worst case time complexity

• What we are looking for here is a way to find the
Big O time complexity, from the way the algorithm runs

• We assume that the algorithm is implemented as efficiently as possible

• Since a comparison is the most costly operation, we are interested in the # of comparisons

• For each recursive call,

– We make a comparison to see if we have found the item

– Then we divide the list in half

• A mathematical way to express this is:

T(n) = T(n/2) + 1 where T(n) is the running time with input size n

• This is called a recurrence relation

Recurrence relations

• Recurrence relations are used to find the time complexity of recursive function

• A recursive function always has two parts

– The stopping place (when the recursion ends)

– The rule for making the problem smaller

• With binary search, the stopping place is
T(1) = 1; when n is 1, there is one comparison

– Actually, this is a small simplification

• The rule for making the problem smaller is
T(n) = T(n/2) + 1

• We must solve this recurrence relation to find the time complexity

Solving the recurrence relation T(n) = T(n/2) + 1

• Solving a recurrence relation means to find T(n) independently of previous terms

• There are several ways to do this, depending on the type of recurrence relation

– You will spend more time doing this in CS 222

• To solve our recurrence relation let’s substitute a number for n and see if we can see a pattern emerging

• We know that T(1) = 1

• Let n = 2; T(2) = T(2/2) + 1

• n =4; T(4) = T(4/2) + 1

• n=8; T(8) = T(8/2) + 1

• n=16; T(16) = T(16/2) + 1

• At some point, we should be able to deduce what T(n) is in terms of n, not in terms of the previous term

Analyzing Merge sort

• With merge sort, we divide the list in half, then recursively merge sort each half; then we have to put the list back together

• So T(n) = T(n/2) + T(n/2) + n

– Collecting terms this is T(n) = 2T(n/2) + n

• T(1) = 1;

• N = 2; T(2) = 2T(1) + 2

• N = 4; T(4) = 2T(2) + 4

• N = 8; T(8) = 2T(4) + 8

Multiplying large integers

• Multiplication of very large integers is used in encryption schemes

– They must multiply two primes of about 200 digits

• The number of bits to represent these integers cannot be handled directly by the ALU of a single processor

• The usual algorithm used to multiple integers is O(n²) since each digit in X is multiplied by each digit in Y

• A divide and conquer algorithm can do better than O(n²)

A concrete example of multiplying large integers

• Suppose X = 61,438,521 and Y=94,736,407

• Then XY=5,820,464,730,934,047

• Break X and Y into two halves, consisting of the most significant and least significant digits.

– X_hi=6,143 , X_low = 8,521 Y_hi=9,473 Y_low=6,407

• It is also true that X = X_hi10⁴ + X_low and Y = Y_hi10⁴ + Y_low

Multiplying big integers

• We have X_h=6,143 , X_l = 8,521
Y_h=9,473 Y_l=6,407

• And X = X_h10⁴ + X_l and
Y = Y_h10⁴ + Y_l

_• So XY = X_hY_h10⁸ + (X_hY_l + X_lY_h)10⁴ + X_lY_l

• What do we do if the number of digits in the large integer is not even

– Let n be the number of digits

– Ceiling n/2 is the number of high digits

– Floor n/2 is the number of low order digits

– Floor n/2 is the exponent

Pseudocode to multiply large integers

• public static large prod(large x, large y)
large x_hi, x_lo, y_hi, y_lo;
int n, m;
n = max(#0f digits in x and y)
if (x == 0) || y==0) return 0;
else if (n<= threshold)
return x * y obtained in usual way
else
{ m = floor n/2;
x_hi = x divide 10^m; x_lo = x rem 10^m;
y_hi = y divide 10^m; y_lo = y rem 10^m;
return (prod x_hi, y_hi) * 10^2m +
(prod(x_hi, y_lo) + prod(y_hi, x_lo))* 10^m +
prod x_lo, y_lo);

Analyzing the multiplication

• The time comsuming part of the algorithm is the recursive calls

• return (prod x_hi, y_hi) * 10^2m +
(prod(x_hi, y_lo) + prod(y_hi, x_lo))* 10^m +
prod x_lo, y_lo);

• Notice that this equation has four multiplications, each half the size of the original problem

– Each multiplication is a recursive call to prod

• The addition, subtraction, and divide10^m or rem 10^m can be done in linear O(n) time

– Here, n is the number of digits

• So, what is the recurrence equation T(n) = ?

• T(n) = 4T(n/2) + cn; this is an O(n²) algorithm

Improving the time complexity of the algorithm

• To achieve a sub quadratic algorithm, we must use fewer than four recursive calls

– Each multiplication results in a recursive call

_• XY = X_hY_h10⁸ + (X_hY_l + X_lY_h)10⁴ + X_lY_l

• We can reduce this to three recursive calls by rewriting the middle term

_• Note that the middle term:
X_hY_l + X_lY_h = (X_h+X_l) (Y_l+Y_h ) - X_hY_h - X_lY_l

• To see this, multiply out (X_h+X_l) (Y_l+Y_h ); of the four terms, two are the first and last terms in the equation above

Improved pseudo code

• public static large prod2(large x, large y)
large x_hi, x_lo, y_hi, y_lo;
int n, m;
n = max(#0f digits in x and y)
if (x == 0) || y==0) return 0;
else if (n<= threshold)
return x * y obtained in usual way
else
{ m = floor n/2;
x_hi = x divide 10^m; x_lo = x rem 10^m;
y_hi = y divide 10^m; y_lo = y rem 10^m;
r = prod2(x_hi + x_lo, y_hi + y_lo);
p = prod2(x_hi, y_hi);
q = prod2(x_lo, y_lo);
return p * 10^2m + (r – p – q) * 10^m + q;
}

Analyzing the new algorithm

• How many recursive calls do we have here?

• What is the running time?

– T(n) = 3T(n/2)

• What is the time complexity?

– T(n) O(n^log3)