Definitions:

A PROBLEM is relation from finite input to acceptable output
An algorithm A SOLVES a problem if on every input A produces an
acceptable output in finite time

A DECISION PROBLEM is a problem where the output is yes or no

A LANGUAGE is the collection of instances for a decision problem where the answer is yes

A COMPLEXITY CLASS is a collection of languages. Usually we are interested complexity classes corresponding to languages (decision problems) computable on a particular type of machine.


History and Background:


1900: Hilbert's 23 problems for the next century 
http://en.wikipedia.org/wiki/Hilbert%27s_problems
http://www.ams.org/journals/bull/1902-08-10/S0002-9904-1902-00923-3/S0002-9904-1902-00923-3.pdf


2nd Problem: Prove that the axioms of arithmetic are consistent.


6th 	Problem: Axiomatize all of physics

10th problem as stated in 1902: Find a process by which in can be determined in a finite number of operations whether a given polynomial Diophantine equation with integer coefficients has an integer solution.

10th Problem (current terminology): Find an algorithm to determine whether a given polynomial Diophantine equation with integer coefficients has an integer solution.


Finding an algorithm did not require formalization of of the concept of algorithm/computation, e.g. 
Euclid's algorithm: gcd(x, y) = gcd(x, y-x). 


How to prove that there is no algorithm for the 10th problem?

How to formalize Computation?

TRY 1:

Assumption: Every computational device only has finitely many states or configurations

Definition: Configuration

Theorem: No finite state device can count, say determine if the number of zeros and the number of ones in a streaming input are the same
Proof: Consider strings of the form 0^i. Then there must exist
an i and a j that are not equal, and 0^i and 0^j end up in the same state.
Then the strings 0^i1^i and 0^j1^i end up in the same state, and
give the same answer, which is a contradiction to correctness.


TRY 2:

Possible definition of computable: A problem is computable if for every
input size, there is a circuit that computes the output
for inputs of that size.

What is wrong with this definition of computable?
Answer: How does one find the circuit? More info later.

TRY 3:

Definition: Turing Machine


Example: Change 0's to 1's on input tape

Church-Turing Thesis: If some physical device in our universe can
solve some problem then a Turing machine can solve that problem.


How does one gain confidence that the Church-Turing Thesis is true?
The main support for this thesis is that all reasonable
models of computation that have been proposed are provably 
no more powerful than a Turing machine. There are many models
of computation that are as powerful as Turing machines.
Proof technique: Simulation


News Headline: The universe is really governed by the laws 
of "loopy string theory" not Newtonian mechanics. 
*How does that affect the Church-Turing thesis? 
*How could you show that the Church-Turing thesis is unaffected? 
*How could you show that the Church-Turing thesis is not true?

Definition: The halting problem is given a Turing machine M and
an input x to decide if M halts on x. The halting language H
is the set of inputs where the answer is yes.


Theorem (Church, Turing 1936/1937) There is no Turing machine to solve the halting problem 
Proof: Diagonalization

Consider the two dimensional table defined by T(M,I)=1 if M halts on I and 0 otherwise

Consider the string S[M] = the opposite of T(M,M)

S[M] can not be a row in T because it differs from row r in column r

If you had a program to compute T(M,M) then you could write a program that
on input M produced output S(M), contradition.


Corollary: There is no Java program, C program .... to solve the halting problem
Proof: Simulation results between different computational languages/systems/machines


If the Church-Turing thesis is true, then that is no way to solve the halting
problem by any computational system.



Definition: A (Turing) reduction from a problem A to a problem B
is an algorithm that solves problem A lacking only a procedure
for problem B

Definition: Let A and B be languages or decision problems. A many-to-one
reduction from A to be is a program for A that looks like:
	Program for A
	Read input I
	Return Program_for_B(f(I)) 
Here f is some function.

Theorem: The problem of determining whether a TM 
M halts on the empty string is not recursive

Proof: Show that the halting problem many-to-one reduces to this problem.

Algorithm for halting problem (i.e the reduction):
	Read TM N, and input x
	Create a new TM M where x is a local variable instead of a parameter
	Then call procedure to see if M halts with no input parameter



Definition: Rewriting Problem
	Input: Strings A, B and pairs of strings of the form (c_i, d_i)
	Question: Can you derive B from A? The rules are you can replace
		any string of the form c_i by the string d_i

Theorem: The rewriting problem is undecidable

General proof:
By many-to-one reduction from halting problem. Let M, x be input to halting problem.
Let N be machine derived from M such that M(x)=N(x) and if N accepts
it always leaves the tape empty, ends in a unique accept state, and
has the tape head on the leftmost cell.

Let Q be states of N, Sigma be tape symbols, delta be transition function,
s be initial state, t be accept state

Let A= >s^x<   (that is left end of tape symbol, start state, new up arrow
		symbol, the input string, new right end of tape symbol)

Let B= >t^<     

For each transition delta(q, b)= (p, w, right) add rewrite rule
	q^b can be replaced by wp^

For each transition delta(q, b)= (p, w, left) and all m in Sigma add rewrite rule
	mq^b can be replaced by p^mw
	
Add rewrite rule 
	< can be replaced by _< (where _ is the blank character)


Then the claim is that you can derive A from B iff N halts on x

Proof by example:
Consider the Turing Machine
states: p=start, q, h=halt
input symbols:0,1,space
(p,0)->(q,0,right)
(p,1)->(p,_,right)
(q,0)->(p,0,left)
(q,1)->(r,1,left)
(r,0)->(p,_,right)
(p,_)->(h,_,stay)
(q,_)->(h,_,stay)

Input: 0110

>p011<
>0q11<
>r011<
>_p11<
>__p1<
>___p<
>___p_<
>___h_<



QUESTION: How would you show that there is no algorithm for  Hilbert's
10th Problem: Find an algorithm to determine whether a given polynomial 
Diophantine equation with integer coefficients has an integer solution?

ANSWER: Many to one reduction from Halting. Finished in 1970 by Matiyasevich following a long
line of development