CS70 - Lecture 10 - Feb 9, 2011 - 10 Evans
Goal for Note 5:
Modular arithmetic
How computers do integer arithmetic
computing gcds
Theorem (division algorithm) given integers a, d>0 (divisor), there is a
unique q (quotient) and r (remainder) such that 0<=r<d, a=q*d+r
DEF if a and d>0 are integers as above, then a mod d = r,
remainder after dividing a by d
Application of Division Algorithm: Modular Arithmetic
DEF We say "a is congruent to b mod d" (or "a == b mod d") if d|(a-b)
Otherwise we say "a !== b mod d"
EX: 3 == 17 mod 7 because 7 | (17-3)
Thm 1: a == b mod d if and only if there is an integer k such that
a = b + k*d
proof (=>) a == b mod d -> d|(a-b) -> exists k: a-b=k*d -> a=b+k*d
proof (<=) a=b+k*d -> a-b = k*d -> d|(a-b) -> a == b mod d
EX: 3 == 17 mod 7 <=> 7 | (17-3) <=> 17=3+2*7
Thm 2: a == b mod d if and only if a mod d = b mod d:
proof (<=) a mod d = b mod d -> a=qa*d+r, b=qb*d+r, ->
(a-b)=(qa-qb)*d, -> d|(a-b) -> a == b mod d
proof (=>) a == b mod d -> a = b + k*d by Thm 1, so when dividing
a = qa*d+ra, b=qb*d+rb, with 0 <= ra,rb < d, we get
ra-rb = (a-qa*d)-(b-qb*d) = (a-b)+qb*d-qa*d = (k+qb-qa)*d
now -d < ra-rb < d and ra-rb is also a multiple of d,
so ra-rb=0
EX: 3 == 17 mod 7 <=> 3 mod 7 = 17 mod 7
ASK&WAIT: is 111 == 63 mod 3? is 123 == 6789 mod 2?
Thm 3: a==b mod d and c==e mod d => a+c == b+e mod d
proof: a = qa*d+r1 and b = qb*d+r1 and c = qc*d+r2 and e = qe*d+r2 ->
(a+c)=(qa+qc)*d+r1+r2 and b+e=(qb+qe)*d+r1+r2 ->
(a+c)-(b+e) = d*(qa+qc-qb-qe) -> d|(a+c-b-e) -> a+c == b+e mod d
EX: 3==17 mod 7 and 11==4 mod 7 => 3+11 == 17+4 mod 7 (7|(21-14), i.e. 7|7)
ASK&WAIT: Is 112+227 == 31+65 mod 3?
Thm 4: a==b mod d and c==e mod d => a*c == b*e mod d
(try to prove this yourself)
EX: 3==17 mod 7 and 11==4 mod 7 => 3*11 == 17*4 mod 7 (7|(68-33), i.e. 7|35)
ASK&WAIT: Is 112*227 == 31*65 mod 3?
DEF: "arithmetic modulo d" or "modular arithmetic with modulus d" means
doing integer arithmetic (+,-,*) where any two a,b satisfying
a == b mod d are considered the same, because we only care what
the answer equals mod d
Thm 3 means that if we want to add and subtract numbers mod d, we can
take any number or intermediate result and add a multiple of d to it
(replace it by its value mod d) without changing the final answer.
Thm 4 says the same thing about multiplication
Ex: (13+15)*(2+8) mod 8 can be computed the following equivalent ways:
(1) ((13+15)*(2+8)) mod 8 = 280 mod 8 = 0 mod 8
(2) 13+15==28 mod 8 == 4 mod 8 and 2+8==10 mod 8 ==2 mod 8 so
((13+15)*(2+8)) mod 8 == 4*2 mod 8 == 8 mod 8 == 0 mod 8
ASK&WAIT: Any other ways?
Here are several useful applications of modular arithmetic:
Thm: Let x = d(n-1)d(n-2)...d(0) be an n-digit decimal integer.
Then 3|x if and only if 3 | d(n-1)+d(n-2)+...+d(0), i.e. the
sum of x's decimal digits.
Proof. We want to show that x mod 3 = 0 if and only if
d(n-1)+...+d(0) mod 3 = 0.
We will show more, namely that x == d(n-1)+...+d(0) mod 3:
x == sum_{i=0 to n-1} d(i)*10^i mod 3 ... by def of decimal number
== sum_{i=0 to n-1} d(i)*(10^i mod 3) mod 3
== sum_{i=0 to n-1} d(i)*(10 mod 3)^i mod 3
== sum_{i=0 to n-1} d(i)*(1)^i mod 3
== sum_{i=0 to n-1} d(i) mod 3
as desired
Thm: Let x = d(n-1)d(n-2)...d(0) be an n-digit decimal integer.
Then 9|x if and only if 9| d(n-1)+d(n-2)+...+d(0), i.e. the
sum of x's decimal digits.
Proof: the same as above, substituting 9 for 3
ASK&WAIT: What about a rule for deciding if 11 | x?
ASK&WAIT: what about a rule for deciding if 7 | x?
EX: Simplest way to implement "arithmetic modulo 8" means
only use numbers in set S={0,1,2,3,4,5,6,7};
after every operation (like 3*4) take result modulo 8 to get
number (4) in S
EX: Take a disk with d=8 equispaced points on circumference, labelled
0 (at top), 1 (to right) and around to d-1 = 7. Take another similar
disk with same center. to add a+b, align 0 of second disk
with a of first disk, and see where b of second disk hits
first disk, namely at a=b mod d ("circular slide rule").
Multiplication can be thought of as repeated addition.
(picture on board)
EX: Computer Implementation of arithmetic mod 8:
"Unsigned Integers" in C, C++
represent numbers in base 2 (with 3 binary digits, or bits):
Addition is done as in elementary school: add bits from right
to left, where you get a "carry" from one column of bits to the
next if the sum in the column is at least 2 (10 in decimal), as
indicated below. If there is a carry out of the last column, we
ignore it, since there is no place to "carry it to".
This discarded carry represents an 8, so the final sum is 8 smaller
than the true value, eg 12-8=4 instead of 12, which is the
answer mod 8.
carries: 000 100 110 000
001_2 = 1 010_2 = 2 101_2 = 5 100_2 = 4
+ 010_2 = 2 + 011_2 = 3 + 111_2 = 7 + 101_2 = 5
----------- ----------- ----------- -----------
011_2 = 3 101_2 = 5 100_2 = 4 001_2 = 1
= 12 mod 8 = 9 mod 8
EX: 2's complement arithmetic: how computers do integer arithmetic
with positive and negative integers.
Suppose computer words had just 3 bits, representing 2^3=8 numbers.
(A real computer would use 32 bits, representing 2^32 numbers, but
it is the same idea.). Then instead of doing modular arithmetic on
the set S={0,1,2,3,4,5,6,7}, we use the set S={-4,-3,-2,-1,0,1,2,3}.
I.e. after each operation, we add a multiple of 8 (or 2^32) to the
result to get an answer in S.
The way you tell positive from negative numbers among the 8 bit
patterns on our 3-bit computer is to look at the leftmost bit,
the "sign bit": it is 0 for nonnegative numbers, and 1 for negative.
bit pattern unsigned integer 2's complement integer
----------- ---------------- ----------------------
000 0 0
001 1 1
010 2 2
011 3 3
100 4 -4 = 4-8
101 5 -3 = 5-8
110 6 -2 = 6-8
111 7 -1 = 7-8
Rule to interpret bit pattern as 2's complement number:
if leading bit ("sign bit") is 0
the number is positive, with the same value as the unsigned integer
else if the sign bit is 1
the number is negative, with the value of unsigned integer minus 8
There are "circular slide rules" for unsigned (draw on board) and
2s complement (draw on board) integer arithmetic:
With 3 bits, numbers range from -2^2 = 100_2 = -4
to 2^2-1 = 011_2 = 3
With 32 bits, numbers range from -2^31 = 10...0_2 = -2147483648
to 2^31 - 1 = 011..1_2 = 2147483647
Arithmetic is done the same way, whether unsigned or 2's complement:
carries: 000 100 110 000
001_2 = 1 010_2 = 2 101_2 = -3 100_2 = -4
+ 010_2 = 2 + 011_2 = 3 + 111_2 = -1 + 101_2 = -3
----------- ----------- ----------- -----------
011_2 = 3 101_2 = -3 100_2 = -4 001_2 = 1
ASK: int a,b,c,d,e,f on 3 bit machine
a = 3
b = a+1 ... what is b?
c = a+2 ... what is c?
d = 2
e = 2*d ... what is e?
f = 2*e ... what is f?
ASK: int a,b,c,d,e,f on 32 bit machine (like most)
a = (2^30-1) + 2^30 ... what is a?
b = a+1 ... what is b?
c = a+2 ... what is c?
d = 2^30 ... what is d?
e = 2*d ... what is e?
f = 2*e ... what is f?
Try running above program on your own machine
What does division mean in modular arithmetic?
With real numbers: x = 2/3 same as solving 3*x=2 for x
With modular arithmetic (mod 7, say), we solve 3*x == 2 mod 7
Start with "reciprocals", or "multiplicative inverses"
solve 3*y == 1 mod 7 (if we can)
multiply by 2 to get 3*(2*y) == 2*1 == 2 mod 7, so x = 2*y
Can we always compute multiplicative inverses?
Ex: 3*x == 1 mod 7 satisfied by x=5
Ex: 12*x == 1 mod 15 satisfied by no x!
Proof: if it were, we would have 15 | (12*x-1)
or 15*d = 12*x - 1 or 15*d - 12*x = 1
or 3*(5*d - 4*x) = 1 or 3 divides 1 - contradiction!
Def: gcd(m,n) = greatest common division of m and n
= largest positive integer d such that d|m and d|n
Theorem: We can solve m*x == 1 mod n iff gcd(m,n) = 1
Proof: (=>) Proof by contrapositive: If gcd(m,n) = q > 1,
then q | m*x + y*n for any y so q | (m*x mod n),
so m*x mod n neq 1 = 1 mod n, i.e. m*x !== 1 mod n for any x.
(<=) Consider the sequence of numbers {0,m,2*m,3*m,…,(n-1)*m}.
We claim these are all distinct modulo n; we prove this
by contradiction: If they were not all distinct, that is if
i*m == j*m mod n for some 0 <= i < j < n, then n | (j-i)*m.
Since gcd(n,m) = 1, n can't have any common factors with m,
so it must divide j-i. But 0 <= j-i < n so n can't divide j-i either,
contradiction. And since they are all distinct modulo n, they
must range over all n possible values modulo n: {0,1,2,…,n-1},
so for some x, m*x == 1 mod n.
How to compute gcd quickly, using Euclidean algorithm
Computing gcd(m,n) by factoring m and n into products of primes and
taking all the common prime factors is very expensive (cryptography
depends on this!) so we want a faster algorithm:
Euclidean algorithm:
func gcd(m,n) …. assume m >= n >= 0
if n=0 return m else return gcd(n, m mod n))
Thm: The Euclidean algorithm computes gcd(m,n) in O(#bits in m) iterations (m is the larger argument).
Proof: d|m and d|n implies d|(m-q*n) for any q, so in particular d|(m mod n).
Conversely, d|n and d|(m-q*n) implies d|(m-q*n+q*n) or d|m. Thus
(m,n) have the same set of common divisors as (n, m mod n) and so gcd(m,n)=gcd(n, m mod n).
Furthermore, gcd(m,0) = m, so if the algorithm terminates, it must return the right answer.
To prove it must terminate, we show that the number of bits in the first (larger) argument
must decrease by at least 1 every 2 steps; this means the algorithm must stop after
at most 2*(#bits in m) steps. There are two cases:
Case 1: if n <= m/2, then n is at least one bit shorter than m after 1 step.
Case 2: if n > m/2, then the next call is to gcd(n, m mod n) = gcd(n,m-n), and the one after that
is to gcd(m-n,gcd(n,m-n)). Again, m-n < m/2 is at least one bit shorter than m.
Ex: gcd(15,12) => gcd(12,3) => gcd(3,0) = 3
gcd(15,11) => gcd(11,4) => gcd(4,3) => gcd(3,1) => gcd(1,0) = 1
How to compute multiplicative inverses quickly, using extended Euclidean algorithm:
Suppose given (m,n), we could find integers x and y so that d = gcd(m,n) = m*x + n*y.
In particular, suppose gcd(m,n)=1, so that x and y satisfies 1 = m*x + n*y;
what would this tell us? That m*x == 1 mod n, i.e. x is the multiplicative inverse of m.
We can extend the Euclidean algorithm to return (x,y) as follows:
func [d, x, y] = extended_gcd(m,n) … return d,x,y satisfying gcd(m,n)=d=m*x+n*y
if n = 0 return [m,1,0]
else [d, x, y] = extended_gcd(n, m mod n), return [d, y, x-floor(m/n)*y]
(Here floor(m/n) = quotient when dividing m by n
= m/n rounded down to the nearest integer.)
Thm: The extended Euclidean Algorithm returns d = gcd(m,n) and integers x,y s.t. d = m*x + n*y
Proof: The fact that the algorithm terminates with d = gcd(m,n) is the same as before.
It remains to show, by induction, that d = m*x + n*y.
The base case is when n=0, in which case x=1, y=0, and m = m*1 + 0*0 as desired.
For the induction step, suppose that d = gcd(n, m mod n) = n*x + (m mod n)*y.
Recall that m mod n = m - floor(m,n)*n. Substituting we get
d = n*x + (m - floor(m,n)*n)*y = m*y + n*(x - floor(m,n)*y)
which shows the algorithm returns the correct coefficients y and x-floor(m,n)*y.
Ex: gcd(15,12) calls gcd(12,3) calls gcd(3,0)
returns 3 = 3*1+0*0 returns 3 = 12*0 + 3*(1-floor(12/3)*0) = 12*0 + 3*1
returns 3 = 15*1 + 12*(0-floor(15/12)*1) = 15*1 + 12*(-1)
Ex: gcd(15,11) calls gcd(11,4) calls gcd(4,3) calls gcd(3,1) calls gcd(1,0)
returns 1 = 1*1 + 0*0
returns 1 = 3*0 + 1*(1-floor(3/1)*0) = 3*0 + 1*1
returns 1 = 4*1 + 3*(0-floor(4/3)*1) = 4*1 + 3*(-1)
returns 1 = 11*(-1) + 4*(1-floor(11/4)*(-1)) = 11*(-1) + 4*3
returns 1 = 15*3 + 11*(-1-floor(15/11)*3) = 15*3 + 11*(-4)