Books versus triangles at the extremal density

Abstract

A celebrated result of Mantel shows that every graph on n vertices with [n²/4] + 1 edges must contain a triangle. A robust version of this result, due to Rademacher, says that there must, in fact, be at least [n/2] triangles in any such graph. Another strengthening, due to the combined efforts of many authors starting with Erdős, says that any such graph must have an edge which is contained in at least n/6 triangles. Following Mubayi, we study the interplay between these two results, that is, between the number of triangles in such graphs and their book number, the largest number of triangles sharing an edge. Among other results, Mubayi showed that for any 1/6 ≤ β < 1/4 there is γ > 0 such that any graph on n vertices with at least [n²/4] +1 edges and book number at most βn contains at least (γ - o(1))n³ triangles. He also asked for a more precise estimate for γ in terms of β. We make a conjecture about this dependency and prove this conjecture for β = 1/6 and for 0.2495 ≤ β < 1/4, thereby answering Mubayi's question in these ranges.