- New PhD program in Data Science at NYU: http://cds.nyu.edu/phd-program/
- New Moore-Sloan Data Science Fellow Position at NYU CDS: http://cds.nyu.edu/nyu-moore-sloan-data-science-fellows/
- New Results on the Resilience of the Littlewood-Offord Problem: https://arxiv.org/abs/1609.08136
- New Results on Optimality/Sub-optimality of PCA under Spike random matrix models: https://arxiv.org/abs/1609.05573

]]>

The semester is under way and, as I announced here, I am teaching a new version of the Mathematics of Data Science course I taught last fall. As I go through the material, and the open problems, I will announce here in the blog progress that has been done on the open problems since I last gave the class.

I am very happy to announce that Luis Daniel Abreu posted a proof for the complex version of the Conjecture regarding the Monotonicity of the average Singular Value of a Gaussian Matrix (see the Conjecture here: https://afonsobandeira.wordpress.com/2013/11/01/a-conjecture-on-the-singular-values-of-a-gaussian-matrix/ , it is Open Problem 1.2 of last years version of the class: http://www.cims.nyu.edu/~bandeira/Fall2015.18.S096.html )

Luis Daniel Abreu was a mentor of mine back in my undergraduate times, and was my first co-author! The solution is available in the arxiv at: http://arxiv.org/abs/1606.00494

]]>

]]>

Congratulations to the three!

]]>

I will defer the details to both the notes and Nicolas’ paper. In the language of Open Problem 10.1, Nicolas shows that the Projected Power Method (with a slight technical twist) converges to the desired solution when . The bottleneck of the analysis seems to be related to Open Problem 10.2. in the same notes (in a similar way to the connection here). I highly recommend taking a look at Nicolas’ paper.

Congratulations Nicolas!

]]>

I’ll refer the reader to both Section 9 of my notes and the papers above for a description of the context of the problem, but, in a nutshell, the Conjecture of Decelle et al. that stated that for communities it is information theoretically possible to recover the community structure better than chance below the Kesten-Stigum bound is now proved! This is particularly remarkable because the Kesten-Stigum bound is suspected to characterize the limit of efficient algorithms, which suggests the bizarre phenomenon that a computational gap is present for but not for !

In fact, Emmanuel Abbe & Colin Sandon shed extra light on the Kesten-Stigum bound by developping a new algorithm, Acyclic Belief Propagation, and proving that it reaches the Kesten-Stigum bound for all .

Jess Banks & Cristopher Moore also investigate the threshold at which is starts being impossible to distinguish the graph drawn from the stochastic block model from an Erdos-Renyi graph with the same average degree!

While we now know that the information theoretical bound cross the Kesten-Stigum bound for we still don’t know where exactly it lies, so the problem remains! A particularly interesting regime described in these papers is the case, which is tightly connected to coloring.

Congratulations to the four of them!

]]>

]]>

**Open Problem 10.1. (Angular Synchronization via Projected Power Method)**

Does the projected power method converge (with high probability) to the optimal solution of the angular synchronization problem with (small enough) gaussian noise?

**Open Problem 10.2. (Sharp tightness of the Angular Synchronization SDP)**

Is the SDP for angular synchronization tight (with high probability) for noise levels essentially until the solution of angular synchronization no longer correlates with the ground truth?

**Open Problem 10.3. (Tightness of the Multireference Alignment SDP)**

For which levels of noise is the SDP for Multireference Alignment tight?

**Open Problem 10.4. (Consistency and sample complexity of Multireference Alignment)**

- Is the Maximum likelihood for Multireference Alignment consistent? (after fixing the power spectrum)
- What is the sample complexity of the Multireference Alignment problem?

]]>

**Open Problem 6.1. (Random Partial Discrete Fourier Transform)**

Consider a obtained by sampling random rows of a Discrete Fourier Tranform. How large does need to be in order for, with high probability, to satisfy the -RIP?

**Open Problem 6.2. (Mutually Unbiased Bases)**

How many mutually unbiased bases are there in 6 dimensions?

**Open Problem 6.3. (Zauner’s Conjecture)**

Prove or disprove the SIC-POVM / Zauner’s conjecture: For any , there exists an Equiangular tight frame with vectors in dimensions. (or, there exist equiangular lines in ).

**Open Problem 6.4. (The Paley ETF Conjecture)**

Does the Paley Equiangular tight frame satisfy the Restricted Isometry Property pass the square root bottleneck? (even by logarithmic factors?).

**Open Problem 6.5. (Constructive Kadison-Singer)**

Give a (polynomial time) construction of the tight frame partition satisfying the properties required in the Kadison-Singer problem (or the related Weaver’s conjecture). These partitions were proven to exist (with a non-constructive proof) in the recent breakthrough of Marcus, Spielman, and Srivastava.

]]>

**Open Problem 8.1. (Unique Games Conjecture)**

Is the Unique Games conjecture true? In particular, can it be refuted by a constant degree Sum-of-squares relaxation?

**Open Problem 8.2. ( Sum of Squares approximation ratio for Max-Cut)**

What is the approximation ratio (or integrality gap) for the Sum-of-Squares (SOS) relaxation of degree 4 for the Max-Cut problem? What about other constant degrees?

**Open Problem 8.3. ( The Grothendieck Constant)**

What is the value of the (real) Grothendieck constant?

**Open Problem 8.4. ( The Paley Clique Problem)**

What is the clique number of the Paley graph? Can the the SOS degree 4 analogue of the theta number help upper bound it?

**Open Problem 8.5. ( Maximum and minimum bisections on random regular graphs)**

Given a -regular graph on nodes . Let and denote, respectively, the size of its largest and smallest bisection. Is it true that for every

,

where is a term that goes to zero as grows?

]]>