The Knapsack Problem

Section 4.5

Two forms of the problem

• The 0-1 Knapsack problem

• The Fractional Knapsack problem

• We should look at least two ways to solve these problems

– Dynamic approach

– Greedy approach

• Often a greedy solution will be simpler than a dynamic programming solution

• But a greedy solution may not be optimal

The 0-1 knapsack problem

• Suppose a hiker is going on a trip and knows she can carry only W weight in the knapsack

• Among the items she may take, she attaches a value to each item

– Items might be things like tent, folding chair, water purifier, camp stove etc

• Basically, items that cannot be divided

• The problem is to maximize the value the value while minimizing the weight

– She wants to fit the highest value she can into the pack

– The highest value possible with weight less than W is the optimal solution

The 0-1 problem

• There are n items; Let
S= {item₁, item₂, …item_n} the set of items
w_i= weight of item_i
p_i = profit of item_i
W = maximum weight the knapsack can hold

• We want to find a subset A of S such that

– the sum of the profit of the items in A is maximized and

– the sum of the weights of the items in A is <= W

A greedy approach to the 0-1 problem
(3 approaches)

1. Take the items with the greatest profit first

• Example:W=30;
(item₁: w₁=25, p_i=10)(item₂: w₂=10, p₂=9) (item₃: w₃=10, p₃=9)

2. Take the items with the least weight first

– Will this always give an optimal solution?

• Any better solutions?

3. Take the items with the highest profit/weight ratio first

• Example:W=30; (item₁: w₁=5, p_i=50)
(item₂: w₂=10, p₂=60) (item₃: w₃=20, p₃=140)

• w₁/p_i=10 w₂/p₂=6 w₃/p₃=7

• Does this give an optimal solution?

• The facts are that a greedy algorithm cannot find the optimal solution.

A dynamic programming approach to the 0-1 problem

• We will be able to find a solution this way if the principle of optimality applies

– The principal of optimality applies if an optimal solution to an instance of a problem always contains optimal solutions to all sub-problems

• Does it apply?

– Let A be an optimal subset of the n items

– Either A contains item_n or it does not

– If A does not contain item_n, A is equal to an optimal subset of the first n-1 items

– If A does contain item_n, the
total profit = p_n + optimal profit from n-1 items where
the W is not exceeded

Solving the 0-1 problem

• To store our data we use a 2-D array P

– n is the total number of items;
W the max weight

– The rows are numbered from 0 to n

– The columns are numbered from 0 to W

– Initialize: P[0][w] = 0; P[i][0] = 0

– We compute row 1 from row 0, row 2 from row 1, etc

• P[n][W] is the maximum total profit for the entire knapsack with w <= W

• Let P[i][w] be the maximum total profit of the first i items in a subset having total weight w

• Use the following formula to fill in the table
p[i][w] = max(P[i-1][w], P_i + P[i-1][w-w_i] if w_i <= w or
p[i][w] = P[i-1][w] if w_i > w

• Example:W=30;
(item₁: w₁=25, p_i=10)(item₂: w₂=10, p₂=9) (item₃: w₃=10, p₃=9)

Solving the 0-1 problem (cont)

• Example:W=30;
(item₁: w₁=25, p_i=10)(item₂: w₂=10, p₂=9) (item₃: w₃=10, p₃=9)

• Improving on this algorithm

• Since what we really want is the total maximum profit P[n][W], we only do those calculations needed to find it

• In this example we need P[3][30]

– To find this, we need P[2][30] and P[2][10]

• To find P[2][30] we need P[1][30] and P[1][20]

• To find P[2][10] we need P[1][10 and P[1][20]

• So in the first row, we need three entries

• In the second row, we need two entries

• In the third row, we need only one entry

Analyzing the solution

• Solving the 0-1 knapsack problem using dynamic programming can be very expensive.

• What do you think the running time depends on?
The number of items?
The weight?

• If n = 20 and W = 20! The original algorithm will take thousands of years to run on a modern-day computer

– When W is extremely large compared to n, the original algorithm is worse than the brute-force algorithm that considers all the subsets

• With the improvement we made by only calculating the needed values, it is never worse than the brute force method, and often is much better

• But the worst case time complexity is still very bad
O(min(2ⁿ, nW))

Back to the fractional knapsack problem

• Look at the 0-1 problem as having gold and silver bars to put into the knapsack, and the fractional problem having bags of gold dusk to put into knapsack; you can take part of a bag, but not part of a gold bar.

• Look at one of the problems we looked at for the 0-1 problem

• Example:W=30;
(item₁: w₁=5, p_i=50) (item₂: w₂=10, p₂=60) (item₃: w₃=20, p₃=140)

• w₁/p_i=10 w₂/p₂=6 w₃/p₃=7

• Can this be solved by the greedy solution if the problem is fractional instead of 0-1