Chapter 7: Time Complexity

For Your Enjoyment

Alan Turing's 1936 paper, On Computable Numbers, with an Application to the Entscheidungsproblem

Chapter 7.1, Measuring Complexity

Time Complexity

Definition: Let M be a deterministic Turing Machine that halts on all inputs. The time complexity (or running time) of M is the function f: ℕ → ℕ, where f(n) is the maximum number of steps that M uses on any input of length n. If f(n) is the running time of M, we say that M runs in time f(n) and that M is an f(n) time Turing Machine. Customarily, we use n to represent the length of the input.
Note: The definition above assumes we are performing a worst-case analysis. Sometimes it is useful to perform average-case analysis or best-case analysis.

Big-O Definition

Let f and g be functions f, g: ℕ → ℝ⁺. Say that f(n) = O(g(n)) if positive integers c and n₀ exist such that for every integer n ≥ n₀,

                  f(n) ≤ c * g(n)

When f(n) = O(g(n)), we say that g(n) is an upper bound for f(n), or more precisely, that g(n) is an asymptotic upper bound for f(n), to emphasize that we are suppressing constant factors.

Small-O Definition

Let f and g be functions f, g: ℕ → ℝ⁺. Say that f(n) = o(g(n)) if

          lim_{n → ∞} (f(n) / g(n)) = 0

In other words, f(n) = o(g(n)) means that for any real number c > 0, a number n₀ exists, where f(n) < c * g(n) for all n ≥ n₀.

Time Complexity Measures

O indicates an upper bound. It should ideally be a good upper bound but it could also be a bad upper bound.
o indicates a bad upper bound.
Ω indicates a lower bound. It is analogous to O. How can it be defined?
ω indicates a bad lower bound. It is analogous to o. How can it be defined?
Θ indicates a perfect bound. How can it be defined?

Time Complexity Class Definition

Let t: ℕ → ℝ⁺ be a function. Define the time complexity class, TIME(t(n)), to be the collection of all languages that are decidable by an O(t(n)) Turing Machine.

Application

Analyze the algorithm for single-tape Turing Machine M₁ on page 279. This algorithm decides language A = {O^k1^k | k ≥ 0}.
Analyze the algorithm for single-tape TM M₂ on page 280. This algorithm also decides language A.
Analyze the algorithm for two-tape TM M₃ on page 281. This algorithm also decides language A.

Complexity Relations Among Models

Let t(n) be a function, where t(n) ≥ n. Then every t(n) time multitape Turing Machine has an equivalent O(t²(n)) time single-tape TM.
Let t(n) be a function, where t(n) ≥ n. Then every t(n) time nondeterministic single-tape TM has an equivalent 2^O(t(n)) time deterministic single-tape TM.

Test Your Comprehension

Exercise 7.1, parts b and d. Justify your answer by using the definition of Big-O to identify values for c and n₀.
Exercise 7.2, parts b, c and e. Justify your answer by showing the limit mentioned in the definition of Small-O.
Exercise 7.5.