Divide and Conquer

2.1 Binary search

2.2 Merge sort

2.3 The D & C approach

2.4 Quick sort (skip)

2.5 Strassen’ matrix multiplication

2.6 Multiplying large integers

2.7 Determining thresholds

2.8 When not to use D & C

Multiplying matrices example

• Given: two matrices A and B the product C will have the same number of rows and columns

• A = a b B = w x C = aw+by ax+bz
c d y z cw+dy cx+dz

• Multiply rows in A by columns in B

• The [1][1]term in C is (a*w + b*y)
The [1][2]term in C is (a*x + b*z)

• The [2][1]term in C is (c*w + d*y)
The [2][2]term in C is (c*x + d*z)

Find: the product matrix C

• A = 2 4 B = 7 5
6 8 1 3

• C= 18 22
50 54

• Question: Is A * B equal to B * A?

• If it is, it is called commutativity of multiplication.

• Usually multiplication is commutative, but not in the case of multiplying matrices

Matrix multiplication pseudocode

• public static void matrixMult(int n, number [ ][ ] A, number [ ][ ] B, number [ ][ ] C)
{ index I, j, k;
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
C[i][j] = 0;
for(k=1; k<=n; k++)
C[i][j]= C[i][j] + A[i][k] * C[k][j];
}

Time complexity of matrix multiplication

• Finding the product of two n X n matrices involves eight multiplications and four additions/subtractions

^• Since the algorithm has three nested loops, it is T(n) = n³

^• The time complexity of the additions is
T(n) = n³ – n²

• So the overall time complexity is O(n³)

Improving on the algorithm

• In 1969 Strassen published an algorithm that is more efficient than this

• Instead of eight multiplications, it makes seven

– But it requires 18 additions/subtractions

• It is not clear how Strassen discovered the submatrix products that make this algorithm work.

Strassen’s matrix Multiplication

_• c₁₁ c₁₂ = a₁₁ a₁₂ * b₁₁ b₁₂
c₂₁ c₂₂ a₂₁ a₂₂ b₂₁ b₂₂

• m₁ = (a₁₁ + a₂₂)(b₁₁+b₂₂)

_• m₂ (a₂₁+a₂₂)b₁₁

• m₃ = a₁₁(b₁₂- b₂₂)

• m₄ = a₂₂(b₂₁- b₁₁)

_• m₅ = (a₁₁ + a₁₂) b₂₂

• m₆ = (a₂₁- a₁₁)(b₁₁ + b₁₂)

• m₇ = (a₁₂ – a₂₂)(b₂₁ + b₂₂)

_• C then is equal to:
m₁+ m₄ – m₅ + m₇ m₃+m₅
m₂ + m₄ m₁+m₃-m₂+m₆

Analyzing Strassen’s method

• The standard method to multiply matrices takes 8 multiplications and 4 add/sub

• Strassen’s method takes 7 multiplications and 18 add/sub

• In a 2 X 2 matrix, this is not worthwhile, but it can be used on larger matrices that are divided into four submatrices

– This works only because commutativity of multiplication is not used

• When the matrices have been divided to a point where they reach a threshold, standard multiplication is used

– since the overhead on a small matrix is not worthwhile

Time Complexity of Strassen’s

• Multiplications

– The basic operations is one simple multiplication

– The input size n is the number of rows and columns

– We analyze the case where we have divided the matrices down in to two 1 X 1 matrices, and multiply them; T(n) =1

– When n = 1, exactly one multiplication is done

– When we have two n X n matrices with n >1, the algorithm is called seven times with an n/2 X n/2 matrix passed to it each time

– So the recurrence relation is T(n) = 7T(n/2)

– Solving this recurrence we get T(n) = n^log7 ; this is an element of Θ(n^2.81)

If n is not a power of two

• There are several ways to solve this

• The simplest is just to add rows or columns until n is a power of 2

• Another way is to just add one extra row and one extra column in the recursive callif n is odd

Determining Thresholds

• Recursion requires quite a bit of overhead

• At some point in these D & C algorithms there comes a point at which it is more efficient to stop dividing, and use a more straight-forward algorithm

• The question often is, where is that point.

• The optimal threshold point depends on several things

– The D&C algorithm

– The alternative algorithm

– The computer on which they are implemented

• This threshold point can not always be determined exactly, but an approximate can be determined

– This point is usually determined empirically rather than theoretically.

– Your text gives a good example that I will not go over here.

• When it is determined, in the algorithm, the recursive calls are stopped, and an alternative algorithm is used

When not to use D& C

• If after n is divided, each new n is almost the same size as the original n

– This can happen if a bad way of partitioning the list is used in Quicksort

• If the size n is divided into almost n instances of size n/c, where c is a constant