Day 40

Final Day of Class. It's been a good ride.

Assignment 4 has been graded and solutions on the resource page. Assignment 5 is due December 13. Because of the limited time for grading there will be no extensions and a 1 point/hour late penalty. If you have any other uncompleted assignments they will also be due December 13 for partial credit.

We covered a Kolmogorov complexity version of Robin Moser's proof of the Lovász local lemma.

• A constructive proof of the Lovász local lemma by Robin Moser
• A Kolmogorov Complexity Proof of the Lovász Local Lemma (My blog post)

Hope you enjoyed the class and may all your lives be complete!

Day 39

Prefix-free complexity and the universal measure

• Sections 5 and 6 from my Kolmogorov Complexity survey
• Lecture notes with more details.

Day 38

Assignment 5 has been posted and due December 13.

Assignment 4 is still being graded but solutions have been posted on the T-Square resources page.

Introduction to Kolmogorov complexity with basic results, applications to primes and mathematical logic.

• Kolmogorov Complexity by Lance Fortnow

Day 37

Guest lecture by DeVon Ingram on the Sensitivity Conjecture and GKS games.

• A New Approach to the Sensitivity Conjecture by Justin Gilmer, Michal Koucký and Michael Saks

Day 36

The following are polynomial related for any function f:{0,1}ⁿ→{0,1}.

• Decision Tree Complexity (deterministic query complexity)
• Certificate Complexity
• Block Sensitivity
• Degree of approximate polynomial
• Degree of exact polynomial
• Quantum query complexity

Links:

• Quantum lower bounds by polynomials by Robert Beals, Harry Buhrman, Richard Cleve, Michele Mosca and Ronald de Wolf
• On the degree of boolean functions as real polynomials by Noam Nisan and Mario Szegedy

Day 35

Shor's Algorithm - polynomial-time quantum algorithm for factoring

• Algorithms for quantum computation: discrete logarithms and factoring by Peter Shor

Day 34

More quantum: Deutsch-Jozsa, Grover and Simon

• Rapid solution of problems by quantum computation by David Deutsch and Richard Jozsa
• A fast quantum mechanical algorithm for database search by Lov Grover
• On the Power of Quantum Computation by Dan Simon

Day 33

Introduction to Quantum Computing. Gave the matrix definition of quantum machines (see my paper below) and described the Hadamard matrix and its properties.

• One complexity theorist's view of quantum computing by Lance Fortnow

Day 32

In this class we showed how MIP = NEXP implies lower bounds on approximating the clique problem and how this led to the PCP theorem. The PCP theorem states that every language in NP has  probabilistically checkable proof where the verifier uses O(log n) random bits and makes only a constant number of queries to a proof that just returns bit. Here is a strong version.

For any constant α > 7/8, there is a polynomial-time computable function f mapping 3-CNFs to 3-CNFs such that for all formula φ,

• If φ is satisfiable then f(φ) is satisfiable.
• If φ is not satisfiable then every assignment to f(φ) satisfies at most an α-fraction of the clauses.

The proof is an assignment, O(log n) random coins are used to pick a clause and only the 3-queries to the variables of that clause are needed.

We also talked about the Unique Games conjecture. 

Links:

• Interactive proofs and the hardness of approximating cliques by Uriel Feige, Shafi Goldwasser, Laszlo Lovász, Shmuel Safra and Mario Szegedy
• Proof verification and the hardness of approximation problems by Sanjeev Arora, Carsten Lund, Rajeev Motwani, Madhu Sudan and Mario Szegedy. First proof of the PCP theorem.
• The PCP theorem by gap amplification by Irit Dinur. A simpler more combinatorial proof of the PCP theorem.
• Some optimal inapproximability results by Johan Håstad. Gets the tight result for 3-SAT above.
• On the power of unique 2-prover 1-round games by Subhash Khot. This paper introduces unique games.
• Optimal Inapproximability Results for MAX‐CUT and Other 2‐Variable CSPs? by Subhash Khot, Guy Kindler, Elchanan Mossel, and Ryan O’Donnell
• Compendium of NP Optimization Problems. Good list of problems and their best known approximations and limitations.

Day 31

Assignment 3 has been graded and solutions posted on the T-Square resource page. Assignment 4 has been posted and is due November 20.

We showed that MIP (in PCP form) was equal to NEXP.

• Non-deterministic exponential time has two-prover interactive protocols by László Babai, Lance Fortnow and Carsten Lund
• Robust Characterizations of Polynomials with Applications to Program Testing by Ronitt Rubinfeld and Madhu Sudan. This is a more direct and robust low-degree test.

Day 30

We showed that IP = PSPACE via the Shen proof and introduced the MIP model.

• IP = PSPACE by Adi Shamir
• IP =PSPACE: Simplified proof by Alexander Shen
• On the power of multi-prover interactive protocols by Lance Fortnow, John Rompel and Michael Sipser

Day 29

We gave the proof that co-NP and P^#P have interactive protocols (Arthur-Merlin games with unbounded rounds). We will generalize to PSPACE on Monday.

• Algebraic methods for interactive proof systems by Carsten Lund, Lance Fortnow, Howard Karloff and Noam Nisan

Day 28

We proved Toda's theorem that PH is in P^#P.
Along the way we needed:

• NP in BPP^⊕P (follows from Valiant-Vazirani discussed last class)
• ⊕P^⊕P = ⊕P (Papadimitrious Zachos)
• NP in BPP implies PH in BPP (Assignment 3)

Links

• PP is as Hard as the Polynomial-Time Hierarchy by Seinsuke Toda
• A Simple Proof of Toda's Theorem by Lance Fortnow
• Two remarks on the power of counting by Christos Papadimitriou and Stathis Zachos

Day 27

We discussed closure properties of #P functions, some basic counting classes and the proof of Valiant Vazirani.

Let f(x,y) be a #P function and r(x) a polynomial-time computable function bounded by a polynomial in |x|.

• The sum of f(x,y) for all y with |y| = r(x) is in #P.
• The product of f(x,y) for all y with 1 ≤ y ≤ r(x) is in #P.
• (f(x,0) choose r(x)) is in #P.

L is in PP if there is a #P function f and a polynomial-time computable r such that

• x in L implies f(x) ≥ r(x)
• x not in L implies f(x) < r(x).

L is in ⊕P if there is a #P function f  such that

• x in L implies f(x) is odd
• x not in L implies f(x) is even

L is in UP if there is a #P function f  such that

• x in L implies f(x) = 1
• x not in L implies f(x) = 0

We also proved the Valiant-Vazirani theorem: There is a polynomial q and a probabilistic polynomial-time function f mapping CNF formulas to CNF formulas such that

• If φ is not satisfiable then f(φ) is never satisfiable.
• If φ is satisfiable then with probability at least 1/q(n), f(φ) has a unique satisfying assignment.

Link:

• NP is as easy as detecting unique solutions by Les Valiant and Vijay Vazirani. We gave Noam Ta-Shma's proof of Mulmuley-Vazirani-Vazirani isolation lemma. See also my blog post.

I also wanted to mention two upcoming talks by Joel Spencer, a great speaker and master of the probabilistic method. 

Day 26

Avi Wigderson, one of the giants of computational complexity, has released a PDF draft of his new book Mathematics and Computation. The book gives a philosophical and technical overview though a number of topics in computational complexity. Definitely worth downloading and reading if you are interested in the material in this course.

Today we proved Valiant's theorem that the permanent of a matrix is #P-complete.

•  The complexity of computing the permanent by Les Valiant
• #P-Completeness Via Many-One Reductions by Viktória Zankó. This paper gives a direct reduction from permanent with exponentially large weights to the permanent of a 0-1 matrix.

Day 25

Showed that AM in Π₂^P and MA in Σ₂^P∩Π₂^P and if AM is in co-NP then PH collapses to Σ₂^P.

This implies that if Graph Isomorphism is NP-complete then PH collapses to Σ₂^P.

Introduced counting complexity, #P and the Permanent.

A function f:{0,1}^*→ℕ is in #P if there is an NP machine M such that f(x) = the number of accepting paths of M(x).

The permanent of a matrix A is the sum over all permutations s of a_1s(1)...a_ns(n).

Day 24

Assignment 2 solutions have been posted on the T-Square resources page. A reminder that Assignment 3 is now available and is due November 1.

We gave the public-coin protocol for Graph Non-Isomorphism and showed that MA is contained in AM.

• Public-coin protocol write up by Jonathan Katz

Day 23

Assignment 3 has been posted on T-Square and due November 1. Assignment 2 grades have been released. Solutions to assignment 2 will be posted in the T-Square resources shortly.

Started our unit on Arthur-Merlin games.

Defined AM and gave the private-coin protocol for graph non-isomorphism and informally described the public coin protocol. We'll do a detailed public-coin protocol on Friday.

• Arthur-Merlin on Wikipedia
• Arthur-Merlin games: A randomized proof system, and a hierarchy of complexity classes by László Babai and Shlomo Moran.
• Proofs that yield nothing but their validity and a methodology of cryptographic protocol design by Oded Goldreich, Silvio Micali and Avi Wigderson. Among other things this paper gives the first Arthur-Merlin protocol for graph isomorphism. 
• Private coins versus public coins in interactive proof systems by Shafi Goldwasser and Michael Sipser
• Graph Isomorphism in Quasipolynomial Time by László Babai

Day 22

Today we did the Schwartz-Zippel lemma and used it to give a co-RP algorithm for polynomial identity-testing.

We also showed that BPP is in P/poly and in Σ^P₂∪Π^P₂.

Links:

• Fast Probabilistic Algorithms for Verification of Polynomial Identities by Jack Schwartz
• Probabilistic algorithms for sparse polynomials by Richard Zippel
• Two theorems on random polynomial time by Len Adleman. Shows that BPP is in P/poly.
• A complexity theoretic approach to randomness by Michael Sipser and BPP and the polynomial hierarchy by Clemens Lautemann. Show BPP in  Σ^P₂∪Π^P₂
• Wikipedia articles (with good definitions and proofs) for BPP, Schwartz-Zippel, Polynomial Identity-Testing and BPP in  Σ^P₂∪Π^P₂.

Day 21

We did the proof of the Chernoff Bounds. Here is a detailed write-up.

We also discussed the polynomial identity testing question: Given a polynomial p(x1,...,xn) described by a circuit of product and sum gates, is that polynomial exactly zero?

To give a probabilistic algorithm for polynomial identity testing we looked at the Schwartz-Zippel lemma and gave part of the proof.

Day 20

We introduced probabilistic Turing machines and talked about the classes BPP, RP and ZPP.

A language is in BPP if there is a probabilistic polynomial-time Turing machine such that for all x,

• If x is in L then the probability that M(x) accepts is at least 2/3.
• If x is not in L then the probability that M(x) accepts is at most 1/3.

A language is in RP if there is a probabilistic polynomial-time Turing machine such that for all x,

• If x is in L then the probability that M(x) accepts is at least 1/2.
• If x is not in L then the probability that M(x) accepts is 0.

A language is in ZPP if there is a probabilistic expected polynomial-time Turing machine such that for all x,

• If x is in L then the probability that M(x) accepts is 1.
• If x is not in L then the probability that M(x) accepts is 0.

We sketched why Composite testing is in RP and used Chernoff bounds to reduce the error in probabilistic algorithms to exponentially small. The exact statement of Chernoff bounds I gave in class was incorrect--we'll revisit on Friday.

Links:

• Computational Complexity of Probabilistic Turing Machines by John Gill. Defined the basic complexity classes. 
• Miller-Rabin primality test - An RP algorithm for composite testing works even when you have Carmichael numbers.
• Primes in P by Agrawal, Kayal and Saxena

Day 19

Finished proof of Razborov-Smolensky. Links in Day 18 post.

Kannan's non-constructive proof that Σ^p₂ does not have n^k-size circuit for any constant k.

Links:

• Circuit-size lower bounds and non-reducibility to sparse sets by Ravi Kannan
• Blog post on Kannan's result

Day 18

First half of Razborov-Smolensky proof that depth d circuits of Mod₃, AND, OR and NOT gates computing parity must have an exponential number of gates.

Links:

• We've been following the proof in Boppana-Sipser
• Algebraic methods in the theory of lower bounds for Boolean circuit complexity by Roman Smolensky
• Lower bounds on the size of bounded depth circuits over a complete basis with logical addition by Alexander Razborov

Day 17

P-complete sets including Monotone Circuit Value: Given a circuit of just AND/OR gates with variables set to the constants TRUE and FALSE, will the circuit evaluate to TRUE?

The major results in circuit complexity:

• Parity requires exponential-size circuits. 
• For distinct primes p and q, there is no polynomial-size constant-depth circuit with AND, OR, NOT and Mod_p gates that can compute the Mod_q function.
• Clique and Matching do not have polynomial-size monotone circuits even with arbitrary depth.
• *NEXP does not have constant-depth circuits with Mod_m gates for any fixed m, prime or composite. 

Links:

• The Complexity of Finite Functions by Ravi Boppana and Michael Sipser. A wonderful survey that covers definitions, results, proofs and references of the major circuit functions (except * which came after this survey).
• Nonuniform ACC Circuit Lower Bounds by Ryan Williams. This is the paper proving *.

Day 16

Karp-Lipton result that if SAT has a polynomial-size circuit family then PH = Σ^p₂.

Definitions of circuit complexity classes (NCⁱ, ACⁱ). NCⁱ is the set of languages accepted by poly-size O(logⁱ n) depth with bounded fan-in. ACⁱ circuits are the same with no restriction on fan-in.

Links:

• Some connections between nonuniform and uniform complexity classes by Richard Karp and Richard Lipton. 
• Wikipedia article on Karp-Lipton with proof. 
• Wikipedia article on NC

Day 15

Assignment 2 now available on the T-Square site. Due October 11.

Defined circuits, size and depth. Showed that a language has a polynomial-size circuit family if and only if L is in P/poly.

Blog post on P/poly.

Day 14

Defined quantified Boolean formula and showed that TQBF, the set of true quantified Boolean formula, is PSPACE-complete.

Assignment 1 has been graded. Average score is 62. Solutions are posted to the resources section of the T-Square site. Assignment 2 will be handed out Wednesday.

Links:

• Alternation by Chandra, Kozen and Stockmeyer. This is the paper that showed the equivalence of alternating polynomial time and PSPACE and my other relationships.
• Nice write-up of proof that TQBF is PSPACE-complete via Joan Feigenbaum. 

Day 13

We talked about the three definitions of the polynomial-time hierarchy, by oracles, alternation and quantification, and the basic properties.

• The equivalence problem for regular expressions with squaring requires exponential space by Albert Meyer and Larry Stockmeyer. This is the paper that first defined the polynomial-time hierarchy. 
• The wikipedia article on the polynomial-time hierarchy goes through all three definitions and the basic relationships.
• Completeness in the Polynomial-Time Hierarchy by Marcus Schaefer and Chris Umans. A compendium of problems complete for various higher levels of the hierarchy. 

Day 12

Guest lecture from Venkat.

Defined polynomial-time hierarchy in terns of oracle TMs and stated some simple facts about the hierarchy. Defined alternating Turing machines. Started the proof of the characterization of polynomial-time hierarchy using alternating Turing machines.

Details and links will come in next post.

Day 11

Finished proof of Immerman-Szelepcsényi. Links in last post.

Defined oracle Turing machines and the Baker-Gill-Solovay result that there exist oracles A and B such that

• P^A = NP^A
• P^B ≠ NP^B

Links:

Relativizations of the P =? NP Question by Baker, Gill and Solovay

You can simply take A to be any PSPACE-complete problem. For B, here is a good write-up. 

Day 10

Today we covered the complements of languages and co-classes and did most of the proof of the Immerman-Szelepcsényi theorem that non-deterministic space is closed under complement. In particular we focused on the proof that NL = co-NL

Definitions:

• The complement of a language L, denoted L, is the set of strings not in L.
• For a complexity class C, we define co-C as the set of languages L for L in C.

Links:

• My writeup of Immerman-Szelepcsényi
• Nondeterministic Space is Closed under Complementation by Neil Immerman
• The method of forced enumeration for nondeterministic automata by Róbert Szelepcsényi

Day 9

We looked at deterministic and nondeterministic space complexity and showed the basic relationships:

DTIME(f(n)) ⊆ NTIME(f(n)) ⊆ DSPACE(f(n)) ⊆ NSPACE(f(n)) ⊆ ∪_c DTIME(c^f(n))

We also proved Savitch's theorem that NSPACE(s(n)) is contained in DSPACE(s²(n)).

Links:

• Relationships between nondeterministic and deterministic tape complexities by Walter Savitch
• My writeups of Space Complexity and Savitch's Theorem

Day 8

Assignment 1, due Monday, September 18, has been posted on T-Square. I highly recommend you start thinking about the problems immediately.

The Berman-Hartmanis Isomorphism Conjecture states that all NP-complete sets are polynomial-time isomorphic to each other. The conjecture is still open.

Let Σⁿ be the set of all strings of length n and Σ^≤n be the set of all strings of length at most n.

We say a set S is dense if for some ε>0, |S∩Σ^≤n| ≥ 2^nε for all n. A set S is sparse if for some c, |S∩Σⁿ|≤ n^c. Every finite set is sparse and no dense set is sparse.

We showed in class that if the Berman-Hartmanis conjecture is true then every NP-complete set is dense. In paticular under this assumption there are no sparse or finite NP-complete sets and P ≠ NP.

The Berman-Hartmanis conjecture remains open but Mahaney proved the following: If there is a sparse NP-complete set then P = NP. In class we gave the proof of Ogihara and Watanabe which I also wrote up in my blog.

Day 7

The Cantor-Bernstein Theorem states that for any two arbitrary sets A and B if there are 1-1 functions from A to B and B to A then there is a 1-1 and onto function from A to B. How does this carry into complexity?

Two sets of strings A and B are polynomial-time isomorphic if there is a function f:Σ^*→Σ^*

1. f is computable in polynomial time
2. The inverse of f is computable in polynomial time
3. f reduces A to B (x in A iff f(x) in B)
4. f is 1-1
5. f is onto

We say f is a length-increasing invertible reduction if there is a f:Σ^*→Σ^*

1. f is computable in polynomial time
2. The inverse of f is computable in polynomial time
3. f is 1-1
4. f is length-increasing, i.e., |f(x)| > |x| for all strings x.

We define A to be paddable if there is a function h:Σ^*×Σ^*→Σ^*

1. h is computable in polynomial time
2. For all x,y, x is in A iff h(x,y) is in A
3. For all y, |h(x,y)| > |y|
4. For all x, y and y' with y≠y', h(x,y)≠h(x,y')
5. There is a polynomial-time computable function g s.t. for all x,y, g(h(x,y))=y

We showed (see Berman-Hartmanis) that

1. Every A in NP reduces to Satisfiability via a 1-1 length-increasing reduction.
2. If B reduces to a paddable set A then there is a reduction from B to A that is length-increasing and invertible. 
3. If A and B reduce to each other via length-increasing invertible reductions then they are isomorphic.

Essentially all the commonly known NP-complete problems are paddable and thus isomorphic to Satisfiability and each other.

Day 6

We proved Ladner's Theorem that there exists a set A such that A is in NP, A is not in P and A is not NP-complete. Here is a write-up.

Enjoy the Labor Day weekend and see you on Wednesday.

Day 5

Finished proof of Cook's theorem. Talked about potential NP-incomplete problems such as Graph Isomorphism, recently shown in quasipolynomial time by Laszlo Babai.

Day 4

Cook's proof that CNF-Satisfiability is NP-complete

Day 3

Reducibility and NP-Completeness

Papers:

• Steve Cook, The Complexity of Theorem-Proving Procedure
• Defined the P v NP problem and showed SAT is NP-complete
• Richard Karp, Reducibility Among Combinatorial Problems
• Shows 21 combinatorial problems NP-complete
• Michael Garey and David Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness
• The reference for NP-completeness. Every computer scientist should own this book.

Day 2

More introduction to computation, Turing Machines and time complexity. Defined P and briefly NP.

Papers mentioned:
Alan Turing, On Computable Numbers, with an Application to the Entscheidungsproblem
Where he defines the Turing machine. Section 9 is worth reading where he basically defends his thesis that this model defines computability.
Juris Hartmanis and Richard Stearns, On the Computational Complexity of Algorithms
This paper started the field of computational complexity. It's major insight was to consider resources like time and memory as a function of the input size.
Jack Edmonds, Paths, Trees and Flowers
This paper gives a polynomial-time algorithm for graph matching and argues that polynomial-time is a reasonable stand in for efficient computation.
Alan Cobham, The Intrinsic Computational Difficulty of Functions
Cobham gave a similar argument for polynomial time.

Day 1

Guest lecture by Venkat.

Covered basics of Turing Machines, Complexity Classes and Polynomial Time.

Reminder that Assignment 0 is due Friday. Download and submit on T-Square.

Assignment 0

A short assignment is posted on T-Square and due on Friday.

Course Overview

CS 6520 Computational Complexity

Fall 2017

Instructor: Lance Fortnow

Time: MWF 9:05-9:55

Room: Whitaker Biomedical Engineering 1103

Description: This course will give an overview of advanced topics in computational complexity including the P versus NP problem, the structure of NP, the polynomial-time hierarchy, circuit complexity, randomness, counting, interactive and probabilistically checkable proofs.

Detailed list of topics

Prerequisites: A strong mathematical background and a basic knowledge of Turing machines and the P v NP problem at the level of CS 4510. This is a course meant for advanced students with a strong interest in theoretical computer science--if you just want to fulfill the graduate theory requirement you should take CS 6505.

Textbook: None. Computational Complexity by Arora and Barak cover much of the material from the course but we won't be following them.

Coursework: Assignments roughly every other week. This is a theoretical course and assignments will be problem solving and will not involve any programming. There are no exams.

Course Website: gtcc17.blogspot.com