Definitions:

A PROBLEM is relation from finite input to acceptable output
An algorithm A SOLVES a problem if on every input A produces an
acceptable output in finite time

A DECISION PROBLEM is a problem where the output is yes or no

A LANGUAGE is the collection of instances for a decision problem where the answer is yes

A COMPLEXITY CLASS is a collection of languages. Usually we are interested complexity classes corresponding to languages (decision problems) computable on a particular type of machine.


History and Background:


1900: Hilbert's 23 problems for the next century 
http://en.wikipedia.org/wiki/Hilbert%27s_problems
http://www.ams.org/journals/bull/1902-08-10/S0002-9904-1902-00923-3/S0002-9904-1902-00923-3.pdf


2nd Problem: Prove that the axioms of arithmetic are consistent.


6th 	Problem: Axiomatize all of physics

10th problem as stated in 1902: Find a process by which in can be determined in a finite number of operations whether a given polynomial Diophantine equation with integer coefficients has an integer solution.

10th Problem (current terminology): Find an algorithm to determine whether a given polynomial Diophantine equation with integer coefficients has an integer solution.


How to prove that there is no algorithm for the 10th problem?

How to formalize Computation?

TRY 1:

Assumption: Every computational device only has finitely many states or configurations

Definition: Configuration

Theorem: No finite state device can count, say determine if the number of zeros and the number of ones in a streaming input are the same
Proof: Consider strings of the form 0^i. Then there must exist
an i and a j that are not equal, and 0^i and 0^j end up in the same state.
Then the strings 0^i1^i and 0^j1^i end up in the same state, and
give the same answer, which is a contradiction to correctness.


TRY 2:

Possible definition of computable: A problem is computable if for every
input size, there is a circuit that computes the output
for inputs of that size.

What is wrong with this definition of computable?
Answer: How does one find the circuit? More info later.

TRY 3:

Definition: Turing Machine


Example: Change 0's to 1's on input tape

Church-Turing Thesis: If some physical device in our universe can
solve some problem then a Turing machine can solve that problem.


How does one gain confidence that the Church-Turing Thesis is true?
The main support for this thesis is that all reasonable
models of computation that have been proposed are provably 
no more powerful than a Turing machine. There are many models
of computation that are as powerful as Turing machines.
Proof technique: Simulation


News Headline: The universe is really governed by the laws 
of "loopy string theory". How does that affect the 
Church-Turing thesis? How could you show that the Church-Turing
thesis is unaffected? How could you show that the 
Church-Turing thesis is not true?

Definition: The halting problem is given a Turing machine M and
an input x to decide if M halts on x. The halting language H
is the set of inputs where the answer is yes.


Theorem (Church, Turing 1936/1937) There is no Turing machine to solve the halting problem 
Proof: Diagonalization

Consider the two dimensional table defined by T(M,I)=1 if M halts on I and 0 otherwise

Consider the string S[M] = the opposite of T(M,M)

S[M] can not be a row in T because it differs from row r in column r

If you had a program to compute T(M,M) then you could write a program that
on input M produced output S(M), contradition.


Corollary: There is no Java program, C program .... to solve the halting problem
Proof: Simulation results between different computational languages/systems/machines


If the Church-Turing thesis is true, then that is no way to solve the halting
problem by any computational system.

Possible definition of computable: A problem is computable if for every
input size, there is a circuit that computes the output
for inputs of that size.

What is wrong with this definition of computable?

Consider the problem: 
Input: string of length n
Ouput: 1 if the nth Turing machine halts on input n, and 0 otherwise

This problem is computable using this new definition, but probably
not computable in the real universe since there is no way to
determine whether the circuit for each length should
be either the ALWAYS ACCEPT or ALWAYS REJECT machine.





Definition: A (Turing) reduction from a problem A to a problem B
is an algorithm that solves problem A lacking only a procedure
for problem B

Definition: Let A and B be languages or decision problems. A many-to-one
reduction from A to be is a program for A that looks like:
	Program for A
	Read input I
	Return Program_for_B(f(I)) 
Here f is some function.

Theorem: The problem of determining whether a TM 
M halts on the empty string is not recursive

Proof: Show that the halting problem many-to-one reduces to this problem.

Algorithm for halting problem (i.e the reduction):
	Read TM N, and input x
	Create a new TM M where x is a local variable instead of a parameter
	Then call procedure to see if M halts with no input parameter



Definition: Rewriting Problem
	Input: Strings A, B and pairs of strings of the form (c_i, d_i)
	Question: Can you derive B from A? The rules are you can replace
		any string of the form c_i by the string d_i

Theorem: The rewriting problem is undecidable

General proof:
By many-to-one reduction from halting problem. Let M, x be input to halting problem.
Let N be machine derived from M such that M(x)=N(x) and if N accepts
it always leaves the tape empty, ends in a unique accept state, and
has the tape head on the leftmost cell.

Let Q be states of N, Sigma be tape symbols, delta be transition function,
s be initial state, t be accept state

Let A= >s^x<   (that is left end of tape symbol, start state, new up arrow
		symbol, the input string, new right end of tape symbol)

Let B= >t^<     

For each transition delta(q, b)= (p, w, right) add rewrite rule
	q^b can be replaced by wp^

For each transition delta(q, b)= (p, w, left) and all m in Sigma add rewrite rule
	mq^b can be replaced by p^mw
	
Add rewrite rule 
	< can be replaced by _< (where _ is the blank character)


Then the claim is that you can derive A from B iff N halts on x

Proof by example:
Consider the Turing Machine
states: p=start, q, h=halt
input symbols:0,1,space
(p,0)->(q,0,right)
(p,1)->(p,_,right)
(q,0)->(p,0,left)
(q,1)->(r,1,left)
(r,0)->(p,_,right)
(p,_)->(h,_,stay)
(q,_)->(h,_,stay)

Input: 0110

>p011<
>0q11<
>r011<
>_p11<
>__p1<
>___p<
>___p_<
>___h_<



QUESTION: How would you show that there is not algorithm for  Hilbert's
10th Problem: Find an algorithm to determine whether a given polynomial 
Diophantine equation with integer coefficients has an integer solution?

ANSWER: Many to one reduction from Halting. Finished in 1970 by Matiyasevich following a long
line of development