CS70 - Lecture 6 - Jan 31, 2011 - 10 Evans

We are increasing sections sizes, waitlist down to 2.

Section 105 (T 3-4) still has 4 slots. 

Our next goal is to use induction to analyze algorithms.

In other classes you learned about recursion.

Here our point is that recursion and induction are two

sides of the same coin: recursion is how an algorithm

works and induction is how we analyze it to either prove

it is correct or figure out how long it takes to execute.

EG: Fibonacci numbers: Let F(0)=0, F(1)=1 and F(n)=F(n-1)+F(n-2).

So F(0,1,2,…) = 0,1,1,2,3,5,8,13,21,...

Theorem: Let x_+ = (1+sqrt(5))/2 ~ 1.6 and x_- = (1-sqrt(5))/2 ~ -.6

 Then F(n) = ( x_+^n - x_-^n )/sqrt(5) ~ 1.6^n / sqrt(5)

  grows exponentially fast

Proof by induction:

  P(n) = " F(n) = ( x_+^n - x_-^n )/sqrt(5)"

  Bases case(s): Check P(0) and P(1) are true,

   i.e. that the formula yields F(0) = 0 and F(1) = 1 as desired

  Induction step: show that P(n-2) and P(n-1) -> P(n): 

    P(n-2) and P(n-1) ->

      F(n-2)+F(n-1) = ( x_+^(n-2) - x_-^(n-2) )/sqrt(5) + (x_+^(n-1) - x_-^(n-1))/sqrt(5)

                               = … = ( x_+^n - x_-^n)/sqrt(5)

                               = F(n)   ->   P(n)

Consider following 2 algorithms for computing F(n):

   func F1(n)

       if n=0 return 0

       elseif n = 1 return 1

       else 

            x = 0, y = 1, 

            for i= 2 to n

                tmp = y,  y = x+y, x=tmp  

(Note: bug in class notes; what does the function there compute instead?)

  func F2(n)

      if n=0 return 0

      else if n= 1 return 1

      else return F2(n-1) + F2(n-2)

Which algorithm is faster? More simply: how many additions does each one perform?

   Let A1(n) = #additions_in_F1(n) = ?

   Let A2(n) = #additions_in_F2(n) = 1 + A2(n-1) + A2(n-2)

   So A2(0,1,2,…) = 0, 0, 1, 2, 4, 7, 12, 20,...

   What is the relationship between A2(n) and F(n)?

   Looks like A2(n) = F(n+1)-1.

   Proof by induction:  A2(n) = 1 + A2(n-1) + A2(n-2)

                                                  = 1 + (F(n)-1) + (F(n-1)-1)     … by induction

                                                  = F(n) + F(n-1) -1

                                                  = F(n+1) -1   … as desired!

Looks like F1(n) is *much* faster than F2(n)

Ex: Assume your compute takes 1 nanosecond to add two numbers.

How long does it take to evaluate F1(129)? About  128 nanoseconds

How long does it take to evaluate F2(129)? About  

       F(130) nanoseconds ~ 10^27 nanoseconds ~ 34 billion years

How old is the universe?

Now suppose you had one computer for each atom in the universe,

about 10^80 of them, all running in parallel to help you run program F2.

How long would it take to compute F2(512)? About

   F(513) / 10^80 nanoseconds ~ 10^107/1e^80 nanoseconds ~ 23 billion years 

How would you ever guess the formula for F(n)? Exactly: guess!

Try F(n) = x^n and see what x has to be for this to be true:

    F(n) = x^n = F(n-1) + F(n-2) = x^(n-1) + x^(n-2)

or            x^n = x^(n-1) + x^(n-2)

or            x^2 = x + 1

or            x = (1+sqrt(5))/2 = x_+   or   x = (1-sqrt(5))/2 = x_-

So both F(n) = x_+^n and F(n) = x_-^n  satisfy F(n) = F(n-1)+F(n-2).

But neither satisfies F(0)=0 and F(1) = 1: what to do?

Note that for any constants r and s, F(n) = r*x_+^n + s*x_-^n

also satisfies F(n) = F(n-1) + F(n-2), so we can pick the 2 constants r and s

to satisfy the 2 constraints F(0)=0 and F(1) = 1 (or F(0)=7 and F(1) = -pi, whatever we like).

The same "guessing" procedure works for similar recurrences, like

    G(n) = 2*G(n-1) - 7*G(n-2), G(0) = 2, G(1) = -3

Try plugging in G(n) = x^n, solve a quadratic for two values of x, etc.

What do you think happens with

    H(n) = 3*H(n-1) + 2*H(n-2) - H(n-3)?

Such "linear recurrences" occur commonly, eg in analyzing signal processing.

Next example of using induction to analyze and algorithm, this time for sorting.

One of the fastest algorithms is called quicksort, and it works like this.

 function quicksort(n, A)

    … input is  array A of n numbers

    … output is array of these n numbers sorted in increasing order

    if (n=0 or n=1) 

       return

    else

       pick a random number 1 <= i <= n

       reorder the entries of A so that

           the initial entries are all <= A(i) (say there are m of them)

           the next entry = A(i)

           the remaining entries are > A(i)

        return S = [quicksort(m,A),A(i),quicksort(n-m-1,A(m+1:n))]

    end

The function quicksort is recursive, and we will prove it correctly

sorts by using induction on the length of the array being sorted

P(n) = "quicksort correctly sorts an input array of length n"

Base cases: P(0) and P(1) work because the algorithm doesn't have to do anything

Induction step: We assume P(0) and P(1) and … and P(n) and prove P(n+1):

after reordering array A of length n+1, it is partitioned into 3 subsets:

(1)    entries <= A(i), except A(i) itself

(2)    A(i)

(3)    entries > A(i)

Obviously, if we correctly sort subsets (1) and (3), the whole array will be sorted.

Quicksort correctly sorts these subsets because their lengths are at most n,

since they don't contain A(i).