Chapter 4: Theoretical Foundations of GAs

Section 4.1: Schemas and the Two-Armed Bandit Problem

Two-Armed Bandit

• A₁: arm with higher payoff, μ₁
• A₂: arm with lower payoff, μ₂
• A_H(N, N-n): observed arm with higher payoff on N-n trials
• A_L(N, n): observed arm with lower payoff on n trials
• q: Pr(A_L(N, n) = A₁)
• L(N-n, n), the losses over N trials, (1 - q) * n * (μ₁ - μ₂) + q * (N - n) * (μ₁ - μ₂)
• The goal is to find n = n^* to minimize L(N-n, n)
• After much mathematical work, N - n^* ≈ e^cn*

Relationship to Schemas

• The GA plays a 3^L armed schema bandit, where L is the length of a chromosome (actually, a 2^k armed bandit in each order-k schema partition)
• GAs are appropriate for online performance in the following domains: control, learning, prediction, satisficing, finding promising regions of a search space
• Schema Theorem: A simple GA increases the number of low-order, short-defining-length, high observed fitness schemas via the multi-armed bandit strategy
• Building Block Hypothesis: The above schema serve as building blocks that are combined via crossover into candidate solutions with increasingly higher order and higher observed fitness
• Static Building Block Hypothesis (SBBH) (assumed by many GA practitioners, but not necessarily correct): the building blocks with the best observed fitness will be found

Section 4.2: Royal Roads

Definition

• R₁(x) = Σ c_i δ_i where δ_i = 1 if x ∈ s_i
• Figure 4.1 shows a sample R₁(x)

Sigma Truncation Selection

• F_i: fitness of an individual
• Ḟ: average fitness of the population
• min(1.5, 1 + (F - Ḟ) / 2σ )

Other Weak Methods (page 129)

• Steepest Ascent Hill Climbing, SAHC
• Next Ascent Hill Climbing, NAHC
• Random Mutation Hill Climbing, RMHC

Table 4.1 shows the number of comparisons for SAHC, NAHC, RMHC and a GA

Random Mutation Hill Climbing Analysis

• N: number of adjacent blocks
• K: number of 1s per block
• E(K,1): expected time to find one block of K 1s
• E(K,2) = E(K,1) + E(K,1) * [KN / (KN - K)] = E(K,1) * N * (1/N + 1/(N-1))
• E(K,3) = E(K,2) + E(K,1) * [KN / (KN - 2K)]
• etc.
• E(K,N) = E(K,N-1) + E(K,1)*[KN / K] = E(K,1) * N * (1/N + etc. + 1) ≈ E(K,1) * N * ln N
• E(K,1) ≈ 2^K (actually, slightly larger)
• Figure 4.2 shows the percent of the population with schema i at a given generation. This illustrates hitchhiking where a bad solution component rides along with a good one.