Thursday, September 23

Proof by Induction.

We went over the homework exercise that was given to prove the lemma:

Let G be a context free grammar with rules of the form:

S --> 0
S --> 1
S --> empty string
S --> 0S0
S --> 1S1

Then every string w in L(G) is a palindrome. 

We did the proof taking special care to highlight each step of an induction proof and why those steps were important.

The first thing to do was to state clearly that this was to be an induction proof and what the induction was to be over.  In this case we chose to do the induction on the length of derivations in G that led to strings in L(G).

We then had to prove a base case that showed that for all derivations of length 1 in G leading to a string w in L(G), w was a palindrome.

Next, we had to state an induction hypothesis.  Our induction hypothesis stated that for every derivation of length i in G leading to a string w in L(G), w was a palindrome, where 1 <= i <= k. 

Finally, we had to prove the induction step.  That is, we had to prove that every derivation of length k+1 leading to a string w in L(G) was a palindrome.  If this proof really does require induction, then the induction step must have a statement somewhere in it similar to "by the induction hypothesis we know that for all derivations of length k of a string w in L(G), w is a palindrome," and use this fact to prove that the same result holds true for derivations of length k+1.  In our case we broke an arbitrary k+1 step derivation into the two cases:

           1               k
      S ===> 0S0 ===> 0x0

           1               k
      S ===> 1S1 ===> 1x1

and noted that by the induction hypotheses, since x is derived from S in both cases by a k-step derivation we know by the induction hypothesis that x is a palindrome, and therefore that the k+1 step derivation leading either to 0x0 or 1x1 must also be a palindrome.

Problem Reductions

We spent the rest of the class discussing how one would prove that a problem is unsolvable, that is, that there exists no Turing machine that is a decider for that problem. 

The first thing is to recognize that undecidable problems must exist, because the set of all Turing machines is countable whereas the set of all decision problems (i.e., languages) is uncountable.  That is, there are more problems than there are Turing machines, which can be shown by a diagonalization proof.

The second thing to do is to actually demonstrate an undecidable problem.  This must be done from first principles.  That is, one must somehow select a problem and then show that no Turing machine can exist to solve this problem.  In Sipser, the first undecidable problem demonstrated is A_TM.  This is done in a fashion similar to diagonalization.

The third thing is to recognize that new problems that are suspected to be unsolvable can now be shown to be unsolvable by reducing a known undecidable problem to this new problem.

We spent some time understanding what reducibility means. It is a process by which one problem A can be solved by using a solution to a second problem, B.  Somehow, we can modify or otherwise embed the solution to B into a new solution (Turing machine) in such a way that A can be solved.  Therefore, to show that a new problem B is undecidable, we assume that it is decidable and then show that we can use an assumed Turing machine to decide B to buld a new Turing machine to decide a known undecidable problem A.  Every step in our construction of this hypothetical machine must be doable and correct.  Thus, when the contradiction is reached that we have just constructed a new Turing machine that decides a known undecidable problem, the only thing that can be wrong is that our assumption that a Turing machine exists that solves B was wrong: no such Turing machine exists.