Solutions of nonlocal problem with critical exponent

This paper deals with the system with linearly coupled of nonlocal problem with critical exponent,

$ \begin{equation*} \begin{cases} (-\Delta)^\alpha u+\lambda_1u = |u|^{2_\alpha^*-2}u+\beta v , \quad x\in \Omega , \\ (-\Delta)^\alpha v+\lambda_2v = |v|^{2_\alpha^*-2}v+\beta u , \,\quad x\in \Omega , \\ u = v = 0,\ \ \qquad \qquad \qquad \quad \quad \qquad \,x\in \partial\Omega. \end{cases} \end{equation*} $

Here $ \Omega $ is a smooth bounded domain in $ {\mathbb{R}}^N(N>4\alpha) $, $ 0<\alpha<1 $, $ \lambda_1,\lambda_2>-\lambda_1(\Omega) $ are constants, $ \lambda_1(\Omega) $ is the first eigenvalue of fractional Laplacian with Dirichlet boundary, $ 2_\alpha^* = \frac{2N}{N-2\alpha} $ is the Sobolev critical exponent and $ \beta\in {\mathbb{R}} $ is a coupling parameter. By variational method, we prove that this system has a positive ground state solution for some $ \beta>0 $. Via a perturbation argument, by doing some delicate estimates for the nonlocal term, we overcome some difficulties and find a positive higher energy solution when $ |\beta| $ is small. Moreover, the asymptotic behaviors of the positive ground state and higher energy solutions as $ \beta\rightarrow 0 $ are analyzed.