This survey hinges on the interplay between regularity and approximation for linear and quasilinear fractional elliptic problems on Lipschitz domains. For the linear Dirichlet integral Laplacian, after briefly recalling Hölder regularity and applications, we discuss novel optimal shift theorems in Besov spaces and their Sobolev counterparts. These results extend to problems with finite horizon and are instrumental for the subsequent error analysis. Moreover, we dwell on extensions of Besov regularity to the fractional p-Laplacian and review the regularity of fractional minimal graphs and stickiness. We discretize these problems using continuous piecewise linear finite elements and derive global and local error estimates for linear problems, thereby improving some existing error estimates for both quasi-uniform and graded meshes. We also present a BPX preconditioner which turns out to be robust with respect to both the fractional order and the number of levels. We conclude with the discretization of fractional quasilinear problems and their error analysis. We illustrate the theory with several illuminating numerical experiments.