1 Introduction

Photonuclear reactions can be studied in ultra-peripheral collisions (UPCs) of heavy ions where the two nuclei pass by with an impact parameter larger than the sum of their radii. Hadronic interactions are suppressed and the dominant electromagnetic interactions are mediated by photons of small virtualities. The intensity of the photon flux is growing with the squared nuclear charge of the colliding ion resulting in large cross sections for the photoproduction of vector mesons in heavy-ion collisions. The photoproduction process has a clear experimental signature: the decay products of vector mesons are the only signal in an otherwise empty detector.

The physics of vector meson photoproduction is described in [1,2,3,4]. Photoproduction of vector mesons in ion collisions can either be coherent, i.e. the photon interacts consistently with all nucleons in a nucleus, or incoherent, i.e. the photon interacts with a single nucleon. Experimentally, one can distinguish between these two production types through the typical transverse momentum of the produced vector mesons, which is inversely proportional to the transverse size of the target. While the coherent photoproduction is characterized by the production of mesons with low transverse momentum (\(\langle p_{\mathrm{T}} \rangle \sim \) 60 MeV/c), the incoherent is dominated by mesons with higher values (\(\langle p_{\mathrm{T}} \rangle \sim \) 500 MeV/c). In the first case, the nuclei usually do not dissociate, but the electromagnetic fields of ultrarelativistic heavy nuclei are strong enough to develop other independent soft electromagnetic interactions accompanying the coherent photoproduction process and resulting in the excitation of one or both of the nuclei. In the second case, the nucleus breaks up and usually emits neutrons close to the beam rapidities which can be measured in zero-degree calorimeters (ZDC) placed at long distances on both sides of the detector [5]. The incoherent photoproduction can also be accompanied by the excitation and dissociation of the target nucleon resulting in even higher transverse momenta of the produced vector mesons [6].

Coherent heavy vector meson photoproduction is of particular interest because of its connection with the gluon distribution functions (PDFs) in protons and nuclei [7]. At low Bjorken-x values, nuclear parton distribution functions are significantly suppressed in the nucleus with respect to free proton PDFs, a phenomenon known as parton shadowing [8]. Shadowing effects are usually attributed to multiple scattering and addressed in various phenomenological approaches based on elastic Glauber-like rescatterings of hadronic components of the photon, Glauber–Gribov inelastic rescatterings, and high-density QCD [9,10,11,12,13,14]. Besides, different parameterizations of nuclear partonic distributions based on fits to existing data are available [15,16,17,18], however these parameterizations are affected by large uncertainties at low Bjorken-x values due to the limited kinematic coverage of the available data samples.

Heavy vector meson photoproduction measurements provide a powerful tool to study poorly known gluon shadowing effects at low x. The scale of the four-momentum transfer of the interaction is related to the mass \(m_V\) of the vector meson as \(Q^{2}~\sim ~m^{2}_{V}/4\) corresponding to the perturbative regime in the case of heavy charmonium states. The rapidity of the coherently produced \({\mathrm{c}} {{\bar{\mathrm{c}}}}\) states is related to the Bjorken-x of the gluons as \(x~=~\left( m_V/\sqrt{s_{NN}}\right) \exp \left( \pm ~y\right) \), where the sign of the exponent reflects that each of the incoming lead nuclei may act as the photon source. The gluon shadowing factor \(R_g(x,Q^2)\), i.e. the ratio of the nuclear gluon density distribution to the gluon distribution in the proton, can be evaluated via the measurement of the nuclear suppression factor defined as the square root of the ratio of the coherent vector meson photoproduction cross section on nuclei to the photoproduction cross section in the impulse approximation that is based on the exclusive photoproduction measurements with the proton target [19, 20]. The square root in this definition is motivated by the fact that the coherent photoproduction cross section is expected to scale as the square of the gluon density in leading order pQCD.

The extraction of the nuclear suppression factor in UPC measurements is complicated by the fact that the measured vector meson cross section in UPCs is expressed as a sum of two contributions since either of the colliding ions can serve as a photon source. At forward rapidities one contribution corresponds to higher photon–nucleus energies while the other to lower energies resulting in ambiguities in the extraction of the nuclear suppression factor. The midrapidity region has the advantage that both contributions are the same and the suppression factor can be extracted unambiguously in this case. Since the contribution of the high and low energies as the rapidity changes varies across models, the measurement of the rapidity dependence of the cross section for coherent charmonium production may provide a new tool to constrain the evolution of the parton distribution in the different models.

Photoproduction cross section measurements for different quarkonium species provide an opportunity to probe gluon shadowing effects at different \(Q^2\) scales. On the other hand, the comparison of excited and ground states of charmonia can shed light on the modelling of the charmonium wave functions and help to disentangle perturbative from non-perturbative effects in the model calculations [21, 22].

Charmonium photoproduction in Pb–Pb UPCs was previously studied by the ALICE Collaboration at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV [23,24,25]. The coherent \(\mathrm{J}/\psi \) photoproduction cross section was measured both at midrapidity \(|y|<0.9\) and at forward rapidity \(-3.6< y < -2.6\). In addition, the CMS Collaboration studied the coherent \(\mathrm{J}/\psi \) photoproduction accompanied by neutron emission at semi-forward rapidity \(1.8< |y| < 2.3\) at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV [26]. The results were compared with various models and the best description was found amongst those introducing moderate gluon shadowing in the nucleus. The ALICE measurements were used in Ref. [19] to extract the nuclear gluon shadowing factor \(R_g\) yielding \(R_g(x \sim 10^{-3}) = 0.61^{+0.05}_{-0.04}\) and \(R_g(x \sim 10^{-2}) = 0.74^{+0.11}_{-0.12}\) at the scale of the charm quark mass. The ALICE measurement of \({\uppsi '}\) photoproduction at midrapidity also supports the moderate-shadowing scenario [25]. A complementary rapidity-differential measurement of the coherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) photoproduction at forward rapidity in Pb–Pb UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV by the ALICE Collaboration further underlines the importance of gluon shadowing effects [27]. The gluon shadowing factor \(R_g(x \sim 10^{-2}) \sim 0.8\) was obtained under assumption that the contribution from high photon-nucleus energies, i.e. low Bjorken \(x\sim 10^{-5}\), can be neglected in the measured cross sections.

In this publication, we present the first measurement of the coherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) photoproduction cross sections at the midrapidity range \(|y|<0.8\) in the Pb–Pb UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV, recorded by ALICE in 2018. The \(\mathrm{J}/\psi \) photoproduction cross section in this measurement is sensitive to \(x\in (0.3,1.4)\times 10^{-3}\), a factor 2 smaller than in the previous midrapidity measurement at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV [24]. This data sample is approximately 10 times larger than Pb–Pb sample at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV used for the ALICE results reported in Refs. [24, 25]. The larger data sample allows for a measurement of the \(\mathrm{J}/\psi \) cross section in three rapidity intervals (\(|y|<0.15\), \(0.15<|y|<0.35\), \(0.35<|y|<0.8\)) extending the previous rapidity-differential cross section measurement in the forward range at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV [27]. \(\mathrm{J}/\psi \) decays to \(\mu ^{+} \mu ^{-}\), \(e^{+} e^{-}\) and \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) and \({\uppsi '}\) decays to \(\mu ^{+} \mu ^{-} \pi ^{+} \pi ^{-}\), \(e^{+} e^{-} \pi ^{+} \pi ^{-}\) and \(l^{+} l^{-}\) are investigated. The coherent \(\mathrm{J}/\psi \) production in the \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) channel in UPCs is measured for the first time. The ratio of the \({\uppsi '}\) and \(\mathrm{J}/\psi \) cross sections is also measured and compared with earlier ALICE measurements [25, 27]. The measured cross sections are compared to models assuming no gluon shadowing as well as to predictions that employ moderate gluon shadowing. Shadowing models are based on a parametrization of previously available data, the leading twist approximation and several variations of the color dipole approach.

2 Detector description

The ALICE detector and its performance are described in [28, 29]. The main components of the ALICE detector are a central barrel placed in a large solenoid magnet (\(B=0.5\) T), covering the central pseudorapidity region, and a muon spectrometer at forward rapidity, covering the range \(-4.0<\eta <-2.5\). Three central barrel detectors, the Inner Tracking System (ITS), the Time Projection Chamber (TPC), and the Time-of-Flight detector (TOF), are used in this analysis.

The ITS is made of six silicon layers and is used for particle tracking and interaction vertex reconstruction [30]. The Silicon Pixel Detector (SPD) makes up the two innermost layers of the ITS with about \(10^7\) pixels covering the pseudorapidity intervals \(|\eta | < 2\) and \(|\eta | < 1.4\) for the inner (radius 3.9 cm) and outer (radius 7.6 cm) layers, respectively. The SPD is read out by 400 (800) chips in the inner (outer) layer with each of the readout chips also providing a trigger signal if at least one of its pixels is fired. When projected into the transverse plane, the chips are arranged in 20 (40) azimuthal regions in the inner (outer) layer allowing for a topological selection of events at the trigger level.

The TPC is used for tracking and for particle identification [31]. A 100 kV central electrode separates the two drift volumes, providing an electric field for electron drift. The two end-plates, at \(|z| = 250\) cm, are instrumented with Multi-Wire-Proportional-Chambers (MWPCs) with 560,000 readout pads, allowing high precision track measurements in the transverse plane. The z coordinate is given by the time of drift in the TPC electric field. The TPC acceptance covers the pseudorapidity region \(|\eta | <0.9\). Ionization measurements of individual track clusters are used for particle identification.

The TOF detector is a large cylindrical barrel of multigap resistive plate chambers with about 150,000 readout channels surrounding the TPC and providing very high precision timing measurement [32]. The TOF pseudorapidity coverage is \(|\eta | <0.8\). In combination with the tracking system, the TOF detector is used for charged particle identification up to a momentum of about 2.5 GeV/c for pions and kaons and up to 4 GeV/c for protons. The TOF readout channels are grouped into 1608 trigger channels (maxipads) arranged into 18 azimuthal regions and provide topological-trigger decisions.

The measurement also makes use of the three forward detectors. The V0 counters consist of two arrays of 32 scintillator tiles each, covering the interval \(2.8\mathrm<\eta <5.1\) (V0A) and \(-3.7\mathrm<\eta <-1.7\) (V0C) and positioned respectively at \(z=340\) cm and \(z=-90\) cm from the interaction point [33]. The ALICE Diffractive (AD) detector consists of two arrays of 8 scintillator tiles each arranged in two layers, covering the range \(4.9\mathrm<\eta <6.3\) (ADA) and \(-7.0\mathrm<\eta <-4.8\) (ADC) and positioned at \(z=17\) m and \(z=-19.5\) m from the interaction point, respectively [34]. Both V0 and AD can be used to veto hadronic interactions at the trigger level.

Finally, two zero-degree calorimeters ZNA and ZNC, located at \(\pm 112.5\) m from the interaction point, are used for the measurement of neutrons at beam rapidity. They have good efficiency (\(\approx 0.94\)) to detect neutrons with \(|\eta |>8.8\) and have a relative energy resolution of about 20% for single neutrons, which allows for a clear separation of events with either zero or a few neutrons at beam rapidity [5].

3 Data analysis

3.1 Event selection

The data analysis in this paper is based on the event sample recorded during the Pb–Pb  at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV data taking period in 2018. A dedicated central barrel UPC trigger consists of topological trigger formed by at least two and up to six TOF maxipads with at least one pair of maxipads having an opening angle in azimuth larger than 150 degrees and a topological trigger formed by at least four triggered SPD chips. The triggered SPD chips are required to form two pairs, each pair with two chips in different SPD layers falling in compatible azimuthal regions. The two pairs of chips are required to have an opening angle in azimuth larger than 153 degrees. It is further vetoed by any activity within the time windows for nominal beam–beam interactions on the V0 and AD detectors on both sides of the interaction point.

The used data sample corresponds to an integrated luminosity of \(233\, \mu \mathrm{b}^{-1}\), derived from the counts of two independent reference triggers, one was based on multiplicity selection in the V0 detector and another one based on neutron detection in the ZDC. The reference trigger cross sections were determined from van der Meer scans; this procedure has an uncertainty of 2.2% [35].The determination of the live-time of the UPC trigger has an additional uncertainty of 1.5%. The total relative systematic uncertainty of the integrated luminosity is thus 2.7%. Alternatively the reference cross section can be calibrated using Glauber calculation [27]. The difference between the two methods is smaller than the quoted systematic uncertainty. Alternatively the reference cross section can be calibrated using Glauber calculation [27]. The difference between the two methods is smaller than the quoted systematic uncertainty.

Additional offline vetoes are applied on the AD and V0 detector signals to ensure the exclusive production of the charmonia. The offline selection in these detectors is more precise than vetoes at the trigger level, because it relies on larger time windows than the trigger electronics and on a more refined algorithm to quantify the signal.

Online and offline V0 and AD veto requirements may result in significant inefficiencies (denoted as veto inefficiencies) in selecting signal events with exclusive charmonium production due to additional activity induced by hadronic or electromagnetic pile–up processes from independent Pb–Pb collisions accompanying the coherent charmonium photoproduction. The probability of hadronic pile–up in the collected sample does not exceed 0.2%, however there is a significant pile–up contribution from the electromagnetic electron-pair production process. The veto inefficiency induced by these pile–up effects in the V0 and AD detectors is estimated using events selected with an unbiased trigger based only on the timing of bunches crossing the interaction region. The average veto efficiency \(\varepsilon _{\mathrm{veto}}^{\mathrm{pileup}}=0.920 \pm 0.002\) is applied to raw charmonium yields to account for hadronic and electromagnetic pile-up processes. In the forward rapidity analysis a higher veto efficiency was obtained due to missing veto on V0C detector.

Signal events with exclusive charmonium production, accompanied by electromagnetic nuclear dissociation (EMD), can be rejected if, in addition to the forward neutrons, other particles, produced at large rapidities, leave a signal either in the AD or the V0 detectors. These extra particles may come from multifragmentation or pion production processes, and the corresponding cross sections are expected to be large [36]. The amount of good events with neutrons, which are lost due to AD and V0 vetoes, is estimated using control triggers without veto on AD and/or VZERO detectors. The fraction of losses for this category of events (EMD) amounts to \(26\% \pm 4\%\) for events with a signal either in ZNA or ZNC and reach \(43\% \pm 5\%\) for events with a signal in both ZNA and ZNC. The average event loss is computed using fractions of events with and without neutrons on either side. The average veto efficiency correction \(\varepsilon _{\mathrm{veto}}^{\mathrm{EMD}} = 0.92 \pm 0.02\) is applied to raw charmonium yields to account for the EMD process.

The selected events are required to have a reconstructed primary vertex determined using at least two reconstructed tracks and having a longitudinal position within 15 cm of either side of the nominal interaction point. The analysis is aimed at the reconstruction of \(\mathrm{J}/\psi \) decaying to \(\mu ^{+} \mu ^{-}\), \(e^{+} e^{-}\), \({\mathrm{p}} {\overline{{\mathrm{p}}}}\)  and of \({\uppsi '}\) decaying to \(l^{+} l^{-}\)and \(\mathrm{J}/\psi \) \(\pi ^+\pi ^-\) followed by \(\mathrm{J}/\psi \) \(\rightarrow l^+ l^-\). Therefore, events with two or four tracks in the central barrel are required.

Two types of tracks are considered in the analysis: global tracks and ITS standalone tracks. Global tracks are reconstructed using combined tracking in ITS and TPC detectors. Tracks are required to cross at least 70 (out of 159) TPC pad-rows and to have a cluster on each of the two layers of the SPD. Each track must have a distance of closest approach to the primary vertex of less than 2 cm in the direction of z-axis. ITS standalone tracks are reconstructed using ITS clusters not attached to any global track, requiring at least four clusters in the ITS, out of which two must be in the SPD.

The two-body decays are selected by looking for events with exactly two global tracks with opposite electric charge (unlike-sign). The probability to find extra global tracks not passing the standard track selection criteria or being reconstructed only in ITS is found to be negligible. The four-body decays of \({\uppsi '}\) are selected by looking for exactly four tracks with at least two being global tracks. The kinematics of the \(\psi ' \rightarrow l^+l^-\pi ^+\pi ^-\) decay is such that pions and leptons are well separated: leptons have high \(p_{\mathrm{T}} \approx 1\) GeV/c while pions are much softer with \(p_{\mathrm{T}} \approx 0.3\) GeV/c. This feature is used to identify the pion pair. Tracks are sorted according to their \(p_{\mathrm{T}}\) and the two with lowest \(p_{\mathrm{T}}\) are assumed to be pions, while the other two are assumed to be leptons. The tagged pions and lepton pairs are required to consist of opposite-sign tracks.

To separate the \(\mathrm{J}/\psi \) \(\rightarrow \) \(\mu ^{+} \mu ^{-}\), \(e^{+} e^{-}\) and \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) decays, the particle identification (PID) capabilities of the TPC and TOF detectors are used. The momenta of the tracks from \(\mathrm{J}/\psi \) decays are \(p \in (1.0, 2.0)\) GeV/c for the \(\mu ^{+} \mu ^{-}\) and \(e^{+} e^{-}\) channels and \( p \in (0.75, 1.75)\) GeV/c for the \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) channel. The PID resolution of the TPC allows for complete separation of electrons and muons in the momentum range mentioned above. Since the specific ionization energy loss (\({\mathrm{d}}E/{\mathrm{d}}x\)) of electron and proton become equal at momenta around 1 GeV/c, the TPC PID is not applicable for the identification of protons from coherently produced \(\mathrm{J}/\psi \). However, the PID capabilities of the TOF detector allow for the separation of protons from other particle species in the momentum range relevant for this analysis. For the \(\mathrm{J}/\psi \) \(\rightarrow \) \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) channel, at least one track is required to have valid TOF PID information. If no TOF PID is available for the second track, TPC PID is used. The \({\mathrm{d}}E/{\mathrm{d}}x\) in TPC or the Lorentz Beta factor (\(\beta = v/c\)) of each reconstructed track in TOF is measured in units of the standard deviation (\(\sigma \)) with respect to expected values for \(\mu ,e,{\mathrm{p}}\) at the given measured momentum. The track pair is accepted if \(n^{2}_{\sigma +} + n^{2}_{\sigma - } < 16\).

The charmonium photoproduction may be accompanied by pile-up from electromagnetic electron-pair production or by noise in the SPD resulting in extra fired SPD trigger chips satisfying the SPD trigger selection topology. In order to exclude contamination of events not triggered by the charmonium decay products, the fired SPD trigger chips are required to match SPD clusters corresponding to the selected tracks. It is found that 11% (7%) of the events with a \(\mathrm{J}/\psi \) candidate decaying into dimuons (di-electrons) with 4 SPD clusters cannot be matched to the fired trigger chips. The matching requirement has a much stronger effect for the 4-track decay channels of \({\uppsi '}\) removing 40% and 22% of the candidates in the \({\uppsi '}\) \(\rightarrow \) \(\mu ^{+} \mu ^{-} \pi ^{+} \pi ^{-}\) and \({\uppsi '}\) \(\rightarrow \) \(e^{+} e^{-} \pi ^{+} \pi ^{-}\)channel, respectively.

3.2 Acceptance and efficiency correction

The product of acceptance and efficiency of the \(\mathrm{J}/\psi \) and \({\uppsi '}\) reconstruction (\(\varepsilon \)) is evaluated using a large Monte Carlo (MC) sample of coherent and incoherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) events generated by STARlight 2.2.0 [37] with decay particles tracked in a model of the experimental apparatus implemented in GEANT 3.21 [38]. The model includes a realistic description of the detector status during data taking and its variation with time.

For this analysis, the primary \(\mathrm{J}/\psi \) and \({\uppsi '}\) vector mesons produced in UPCs are considered to be transversely polarized. This is consistent with expectations from helicity conservation in photo production and consistent with H1 and ZEUS measurements [39,40,41]. As observed in previous experiments, both \(\mathrm{J}/\psi \) and the two pions from \({\uppsi '}\) decay are in the S-wave state resulting into the full transfer of the \({\uppsi '}\) polarisation to the \(\mathrm{J}/\psi \)[42]. The expected polarization states of primary \(\mathrm{J}/\psi \) and \({\uppsi '}\) as well as of secondary \(\mathrm{J}/\psi \) from \({\uppsi '}\) decays are properly taken into account in the MC simulations used in this analysis. These MC simulations are also used in the evaluation of the feed-down contribution to the two-body decay channels from the \(\psi ' \rightarrow {\mathrm{J}}/\psi + \pi ^{+} \pi ^{-}\) and \(\psi ' \rightarrow {\mathrm{J}}/\psi + \pi ^{0} \pi ^{0}\) decays and for modeling the signal shape and different background contributions.

The efficiency of the SPD trigger chips is measured with a data-driven approach using a minimum bias trigger. Tracks selected without requiring hits in both SPD layers are matched to the trigger chips they cross. The obtained efficiency maps are introduced on an event-by-event basis to the MC simulations. The single chip efficiency is about 92%. As the trigger requires 4 chips, the overall effect corresponds to an efficiency of about 0.72 ± 0.01.

The TOF trigger efficiency is also estimated with a data-driven approach and is taken into account in the MC simulations. The average coverage of the active TOF trigger channels is approximately 90%. The average trigger efficiency of the active channels is defined as the probability to find signals in maxipads crossed by extrapolated TPC tracks from minimum bias events and is found to be 97–98%, depending on track arrival times. More than 93% of the active channels are almost 100% efficient. The low efficiency in some channels is caused by timing alignment issues and partially disconnected or broken equipment.

3.3 Signal extraction

The extraction of coherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) yields in all decay channels is performed in the rapidity interval \(|y|<0.8\). In addition, the \(\mathrm{J}/\psi \) measurements in the dielectron and dimuon channels are performed in three rapidity intervals: \(|y| < 0.15\), \(0.15< |y| < 0.35\), and \(0.35< |y| < 0.8\) where the y ranges were chosen to have approximately the same number of candidates per range. An enriched sample of coherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) candidates is obtained by selecting the reconstructed candidates with transverse momentum \(p_{\mathrm{T}} < 0.2\) GeV/c.

The invariant mass distributions for dimuon and dielectron pairs reconstructed in the full rapidity range are shown in Fig. 1, left. The inclusive \(\mathrm{J}/\psi \) yields are obtained by fitting the invariant mass distributions with an exponential function describing the underlying continuum and two Crystal Ball functions to describe the \(\mathrm{J}/\psi \) and \({\uppsi '}\) signals. The \(\mathrm{J}/\psi \) pole mass and width were left free, while the tail parameters (\(\alpha \) and n) in the Crystal Ball function were fixed to the values obtained in MC simulations in order to gain higher stability of the fits. In the case of the \({\uppsi '}\) signal, all the Crystal Ball parameters were fixed to the values obtained in MC simulations.

The raw inclusive \(\mathrm{J}/\psi \) yields obtained from invariant mass fits contain contributions from the coherent and incoherent \(\mathrm{J}/\psi \) photoproduction that can be separated via the analysis of the transverse momentum spectra. The inclusive \(p_{\mathrm{T}}\)  distributions for \(\mu ^{+} \mu ^{-}\) and \(e^{+} e^{-}\) candidates around the \(\mathrm{J}/\psi \) mass are shown in the right panels of Fig. 1. These distributions are fitted with MC templates produced using STARlight, followed by full detector simulation and reconstruction, corresponding to different production mechanisms: coherent and incoherent \(\mathrm{J}/\psi \), feed-down \(\mathrm{J}/\psi \) from decays of coherent and incoherent \({\uppsi '}\) and the dilepton continuum from the \(\gamma \gamma \rightarrow l^{+}l^{-}\) process. Incoherent \(\mathrm{J}/\psi \) production with nucleon dissociation (or dissociative \(\mathrm{J}/\psi \)) is also taken into account to describe the high-\(p_{\mathrm{T}}\)  tail with the template based on the H1 parametrization [39]. Normalization of feed-down \(\mathrm{J}/\psi \) from coherent and incoherent \({\uppsi '}\) decays is constrained to the normalization of primary \(\mathrm{J}/\psi \) templates according to the feed-down fractions extracted as described below. The normalization of the dilepton continuum from the \(\gamma \gamma \rightarrow l^{+}l^{-}\) process is fixed by the results for the background description of the invariant mass fits. The combinatorial background, estimated by considering the distribution of like-sign candidates, is found to be negligible in the \(\mathrm{J}/\psi \) mass region.

The templates are fitted to the data leaving the normalization free for coherent \(\mathrm{J}/\psi \), incoherent \(\mathrm{J}/\psi \) and dissociative \(\mathrm{J}/\psi \) production. The extracted incoherent \(\mathrm{J}/\psi \) fraction \(f_{\mathrm{I}} = \frac{N^{\mathrm{incoh}}}{N^{\mathrm{coh}}}\) for \(p_{\mathrm{T}} <0.2\) GeV/c is \(4.7 \pm 0.3\%\) \((5.0 \pm 0.5)\%\) for the \(\mu ^{+} \mu ^{-}\)(\(e^{+} e^{-}\)) decay channel. The quoted fractions include the contribution of incoherent \(\mathrm{J}/\psi \) with nucleon dissociation.

The invariant mass and the \(p_{\mathrm{T}}\)  distributions for the \({\mathrm{J}}/\psi \rightarrow {\mathrm{p}}{\bar{\mathrm{p}}}\) decay channel are shown in Fig. 2. The data sample obtained in this channel is too small to fit the \(p_{\mathrm{T}}\)  distribution with MC templates. However, since the difference in resolution of \(p_{\mathrm{T}}\)  shapes of the coherent or incoherent MC samples for the \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) and \(\mu ^{+} \mu ^{-}\) channels is negligible, one can expect the \(f_{\mathrm{I}}\) fraction to be the same. This is due to the fact that neither the \(\mu ^{+} \mu ^{-}\) nor the \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) channels suffer from bremsstrahlung. This is not the case for dielectrons where bremsstrahlung induces the large difference in the mass and momentum resolution which affect the templates and consequently the \(f_{\mathrm{I}}\) fraction.

Fig. 1
figure 1

Left: Invariant mass distribution of \(l^{+} l^{-}\) pairs. The dashed green line corresponds to the background. The solid magenta and red lines correspond to Crystal Ball functions representing the \(\mathrm{J}/\psi \) and \({\uppsi '}\) signal, respectively. The solid blue line corresponds to the sum of background and signal functions. Right: Transverse momentum distribution of \(\mathrm{J}/\psi \) candidates in the range quoted in the figure (around the \(\mathrm{J}/\psi \) nominal mass)

Full size image
Fig. 2
figure 2

Left: Invariant mass distribution of \({\mathrm{p}} {\overline{{\mathrm{p}}}}\) pairs. The dashed green line corresponds to the background description. The solid magenta and red lines correspond to Crystal Ball functions representing the \(\mathrm{J}/\psi \) and \({\uppsi '}\) signals, respectively. The solid blue line corresponds to the sum of background and signal functions. Right: Transverse momentum distribution of \(\mathrm{J}/\psi \) candidates in the range quoted in the figure (around the \(\mathrm{J}/\psi \) nominal mass)

Full size image
Fig. 3
figure 3

Left: Invariant mass distribution of \(l^{+} l^{-}\) pairs. The dashed green line corresponds to the background description. The solid magenta and red line correspond to Crystal Ball functions representing the \(\mathrm{J}/\psi \) and \({\uppsi '}\) signals, respectively. The solid blue line corresponds to the sum of background and signal functions. Right: Transverse momentum distribution of \({\uppsi '}\) candidates in the mass range quoted in the figure (around the \({\uppsi '}\) mass)

Full size image
Fig. 4
figure 4

Left: Invariant mass distribution for \({\uppsi '}\) \(\rightarrow \) \(\mu ^{+} \mu ^{-} \pi ^{+} \pi ^{-}\) (upper panel) and \({\uppsi '}\) \(\rightarrow \) \(e^{+} e^{-} \pi ^{+} \pi ^{-}\) (bottom panel). The green line shows the wrong sign four-track events. The red line shows the \({\uppsi '}\) signal as described in the text. Right: Transverse momentum distribution of \({\uppsi '}\) candidates in the mass range quoted in the figure (around the \({\uppsi '}\) nominal mass)

Full size image

As one can see in Fig. 1, the \({\uppsi '}\) yields in the \(\mu ^{+} \mu ^{-}\) and \(e^{+} e^{-}\) channels are small and lying on top of a significant background. In order to increase the significance of the \({\uppsi '}\) signal and to reduce the statistical uncertainty, the \({\uppsi '}\) yield is extracted from the merged \(l^{+} l^{-}\) sample with significance \(\approx 3\). Figure 3 shows the merged dilepton mass spectrum together with the \(p_{\mathrm{T}}\) distribution of the dilepton candidates in the invariant mass range under the \({\uppsi '}\) mass peak. The fit to the invariant mass distribution is performed in the same way as described before.

Table 1 Raw \(\mathrm{J}/\psi \) yields, \(\varepsilon \), \(f_{\mathrm{D}}\) and \(f_{\mathrm{I}}\) fractions and coherent \(\mathrm{J}/\psi \) cross sections per decay channel
Full size table
Table 2 Raw \({\uppsi '}\) yields, \(\varepsilon \), \(f_{\mathrm{I}}\) fractions and coherent \({\uppsi '}\) cross sections per decay channel
Full size table

Figure 4 shows the invariant mass (left) and the \(p_{\mathrm{T}}\)  distribution (right) for \({\uppsi '}\) \(\rightarrow \) \(\mu ^{+} \mu ^{-} \pi ^{+} \pi ^{-}\) and \({\uppsi '}\) \(\rightarrow \) \(e^{+} e^{-} \pi ^{+} \pi ^{-}\) quadruplets. Distributions include combinatorial background. A coherent peak is clearly visible at low \(p_{\mathrm{T}}\). The signal extraction in the \(\mu ^{+} \mu ^{-} \pi ^{+} \pi ^{-}\) and \(e^{+} e^{-} \pi ^{+} \pi ^{-}\) channel is straight-forward since the signal is very clean. The number of candidates is extracted by summing the bin contents in the mass interval \(3.6<{\mathrm{m}}_{\mu \mu \pi \pi }<3.8\) GeV/\(c^2\) and \( 3.4<{\mathrm{m}}_{{\mathrm{ee}}\pi \pi }<3.8\) GeV/\(c^2\). The number of candidates with wrong-sign combinations in the same mass interval, representing the level of background, is subtracted afterwards.

The incoherent contamination of the \({\uppsi '}\) sample is estimated as follows. The incoherent-to-coherent photoproduction cross section ratio is expected to be similar for 1S and 2S charmonium states [37, 43]. Due to lack of model calculations for the incoherent \({\uppsi '}\) cross section in UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV, predicted incoherent-to-coherent cross section ratios for \(\mathrm{J}/\psi \) from Refs. [6, 14, 37] are used as an estimate of the incoherent-to-coherent cross section ratio for \({\uppsi '}\). The factor \(f_{\mathrm{I}} = \frac{N^{\mathrm{incoh}}}{N^{\mathrm{coh}}} \approx 6\%\) is extracted from the predicted cross section ratios, corrected for acceptance and efficiency of coherent and incoherent \({\uppsi '}\) states. The difference in the predicted incoherent-to-coherent cross section ratios is used as an estimate of the systematic uncertainty.

The raw \(\mathrm{J}/\psi \) yields contain a significant feed-down contribution originating from decays \(\psi '\ \rightarrow {\mathrm{J}}/\psi + {\mathrm{anything}}\), dominated by the \({\uppsi '}\) \(\rightarrow \) \(\mathrm{J}/\psi \) + \(\pi ^{+} \pi ^{-}\) and \({\uppsi '}\) \(\rightarrow \) \(\mathrm{J}/\psi \)+ \(\pi ^{0} \pi ^{0}\) decay channels. The feed-down fraction \(f_{\mathrm{D}} = \frac{N(\mathrm{FeedDown})}{N({\mathrm{primary}})}\) can be extracted from the ratio of raw \(\mathrm{J}/\psi \) and \({\uppsi '}\) yields:

$$\begin{aligned} R_N = \frac{N_{\psi '}}{N_{J/\psi }} = 0.0170 \pm 0.0024 (0.0184 \pm 0.0030), \end{aligned}$$
(1)

for \(\mu ^{+} \mu ^{-}\)(\(e^{+} e^{-}\)). The raw \({\uppsi '}\) and \(\mathrm{J}/\psi \) yields in this ratio contain contributions both from coherent and incoherent photoproduction. However, according to the \(p_{\mathrm{T}}\) fits, the fraction \(f_{\mathrm{I}}\) does not exceed 6% and, according to STARlight, the fraction of the incoherent contributions is expected to be similar in the \({\uppsi '}\) and \(\mathrm{J}/\psi \) yields. The \(R_N\) ratio can therefore be considered as a good estimate of the ratio of coherent \(\mathrm{J}/\psi \) and \({\uppsi '}\) yields, since the incoherent fractions largely cancel in the ratio. The \(f_{\mathrm{D}}\) ratio can be expressed via the measured \(R_N\) ratio:

$$\begin{aligned} \left( \frac{1}{f_{\mathrm{D}}}+1 \right) ^{-1}&= \frac{N_{{\mathrm{J}}/\psi }^{\mathrm{feed-down}}}{N_{{\mathrm{J}}/\psi }} = \frac{ ({\mathrm{BR}} \times \varepsilon )_{\psi '\rightarrow {\mathrm{J}}/\psi \pi ^+\pi ^-\rightarrow l^+l^-\pi ^+\pi ^-}+ ({\mathrm{BR}} \times \varepsilon )_{\psi '\rightarrow {\mathrm{J}}/\psi \pi ^0\pi ^0\rightarrow l^+l^-\pi ^0\pi ^0} }{({\mathrm{BR}}\times \varepsilon )_{\psi ' \rightarrow l^+l^-}} \times R_{N} , \end{aligned}$$
(2)

where \(({\mathrm{BR}} \times \varepsilon )\) in the corresponding channels denote products of world-average branching ratios [44] and the product of acceptance and efficiency of measuring exactly two leptons. The \(f_\mathrm{D}\) fractions of \(3.5\% \pm 0.5\%\) and \(4.3\% \pm 0.7\%\) are obtained in the \(\mu ^{+} \mu ^{-}\)and in \(e^{+} e^{-}\) channel respectively, with the uncertainty being the quadratic sum of statistical and systematic uncertainties, where the statistical uncertainty dominates. The systematic uncertainty includes contributions from the \(\mathrm{J}/\psi \) and \({\uppsi '}\) signal extraction and the branching ratios.

4 Results

The coherent vector meson differential cross section is given by:

$$\begin{aligned} \frac{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{\mathrm{VM}}}{\mathrm{d y}} = \frac{N^{\mathrm{coh}}_{\mathrm{VM}}}{\varepsilon _{\mathrm{VM}} ~\times ~ \varepsilon _{\mathrm{veto}}^{\mathrm{pileup}} ~\times ~ \varepsilon _{\mathrm{veto}}^{\mathrm{EMD}} ~\times ~ \mathrm{BR} ~\times ~ {\mathscr {L}}_{\mathrm{int}} ~\times ~ \Delta y} \end{aligned}$$
(3)

where

$$\begin{aligned} N^{\mathrm{coh}}_{J/\psi } = \frac{N_{\mathrm{yield}}}{1+f_{\mathrm{I}}+f_\mathrm{D}} \end{aligned}$$
(4)

and

$$\begin{aligned} N^{\mathrm{coh}}_{\psi '} = \frac{N_{\mathrm{yield}}}{1+f_{\mathrm{I}}}. \end{aligned}$$
(5)

The raw \(\mathrm{J}/\psi \) and \({\uppsi '}\) yield values, reconstruction efficiencies \(\varepsilon \), fractions corresponding to incoherent \(f_{\mathrm{I}}\) and feed-down \(f_{\mathrm{D}}\) contamination as well as coherent cross sections with statistical and systematic uncertainties are summarized in Tables 1 and 2. Further components in the formula include integrated luminosity \({\mathscr {L}}_{\mathrm{int}}\), branching ratio of the decay BR, veto efficiencies corresponding to the pile-up \(\varepsilon _{\mathrm{veto}}^{\mathrm{pileup}}\) and electromagnetic dissociation \(\varepsilon _{\mathrm{veto}}^{\mathrm{EMD}}\) and the rapidity interval \(\Delta \mathrm y\). The associated systematic uncertainties are briefly described in the following.

Table 3 Sources of systematic uncertainty for the coherent \(\mathrm{J}/\psi \) cross section measurements per decay channel in percent
Full size table
Table 4 Sources of systematic uncertainty for the coherent \({\uppsi '}\) cross section measurements per decay channel in percent
Full size table

The offline AD and V0 vetoes decreases the \(\mathrm{J}/\psi \) (\({\uppsi '}\)) yield by 8% (16%) and also results in a lower \(\varepsilon _{{\mathrm{veto}}}^{{\mathrm{pileup}}}\) efficiency. These two effects do not cancel-out completely. A residual discrepancy of 3% (10%) in the cross section of \(\mathrm{J}/\psi \) (\({\uppsi '}\)) is included in the systematic uncertainty.

The uncertainty related to the evaluation of the incoherent contamination comes from the shape of the continuum template. By default the \(\gamma \gamma \rightarrow l^+ l^-\) template from the STARlight MC simulation is used. Alternative way would be to use a data-driven template generated from the side bands of the invariant mass distribution of the \(\mathrm{J}/\psi \). These events are, in contrast to the pure MC, supposed to include the same background as under the \(\mathrm{J}/\psi \) peak. By comparing the \(f_{\mathrm{I}}\) fraction obtained with the side bands method and the STARlight template, a 0.8 (0.5%) uncertainty of the cross section for the \(\mu ^{+} \mu ^{-}\) (\(e^{+} e^{-}\)) channel is found.

A systematic uncertainty of the tracking efficiency of 2% per track is estimated by comparing, in data and in MC, the ITS (TPC) hit matching efficiency to the tracks reconstructed with TPC (ITS) hits only. This leads to a 2.8% (4%) systematic uncertainty for two-track (four-track) channels.

For the signal extraction in the \(\mathrm{J}/\psi \) analysis the goodness of the description of the \(\mathrm{J}/\psi \) signal by the Crystal Ball function is checked. The yield from the fit is compared to the number of events computed by bin counting in the peak region with the background contribution subtracted using the exponential background shape from the fit. Half differences of 0.3%, 2.4%, 0.6% were assigned as the systematic uncertainty in the muon, electron and proton channel, respectively.

Fig. 5
figure 5

Measured differential cross section of coherent \(\mathrm{J}/\psi \) (left) and \({\uppsi '}\) (right) photoproduction in Pb–Pb UPCs in \(|y|<0.8\). The points show the measurements for different decay channels. The error bars (boxes) represent the statistical (decay channel uncorrelated systematic) uncertainty. The gray box shows the average value (dashed line) and correlated systematic uncertainty

Full size image

Another contribution to the signal extraction uncertainties is the difference between the function used in the fit and the true shape of the background. It is estimated by varying the fit range. A systematic uncertainty of 0.4% (0.3%) is determined for the \(\mu ^{+} \mu ^{-}\) (\(e^{+} e^{-}\)) decay channel of the \(\mathrm{J}/\psi \)meson. The uncertainty in the background subtraction rises rapidly as the signal-to-background ratio drops. A similar study for the \(l^{+} l^{-}\) decay of the \({\uppsi '}\) meson results in a 10% systematic uncertainty.

The uncertainty associated to the determination of the trigger efficiency of the SPD chips is obtained by changing the requirements on the probe tracks used in the data-driven method. Variations include the running conditions, the maximum amount of activity allowed in the event, and the definition of tracks accepted in the efficiency computation. This uncertainty amounts to 1%.

The uncertainty of the TOF trigger efficiency due to the spread of the arrival times of various particle species to TOF is evaluated as 0.5% per track (1% in total). The uncertainty in case of four track decays of \({\uppsi '}\) applies only for the lepton tracks since the low-momentum pions do not reach the TOF detector.

In the \({\mathrm{J}}/\psi \rightarrow {\mathrm{p}} \overline{\mathrm{p}}\) analysis, at least one track is required to have proton PID from the TOF. Comparing the efficiency of the track matching to TOF in data and MC samples, a \(10\%\) disagreement is found. The matching efficiency from MC is used and a half-difference of 5% as an additional systematic uncertainty for the \({\mathrm{p}} {\overline{\mathrm{p}}}\) channel is assigned.

Tables 3 and 4 show the uncertainties for each source and channel separately as well as quadratic sums of the channel-correlated and uncorrelated sources.

The signal extraction, incoherent contamination and branching ratio are considered as channel-uncorrelated sources of systematic uncertainties. The other sources are fully correlated and are the same for all channels. In the case of the \({\uppsi '}\), the four track channels have an extra ITS-TPC matching uncertainty for the pion tracks which is not correlated with the \(l^{+} l^{-}\) channel, thus it is quoted separately.

The \(\mathrm{J}/\psi \) and \({\uppsi '}\) cross sections for various decay channels computed using Eq. (3) are shown in Fig. 5. The mean values of the \(\mathrm{J}/\psi \) and \({\uppsi '}\) cross sections are obtained as a weighted average of the cross sections per decay channel with weights corresponding to the inverse of the quadratic sum of statistical and channel-uncorrelated systematic uncertainties. The cross section value averaged over the three decay channels of the coherent \(\mathrm{J}/\psi \) photoproduction measurements is:

$$\begin{aligned} \frac{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{J/\psi }}{{\mathrm{d}} y} = 4.10~\pm 0.07{\mathrm{(stat.)}} \pm 0.23 \mathrm{\mathrm{(syst.)}} {\mathrm{mb}}. \end{aligned}$$
(6)

The cross section value averaged over the three channels of coherent \({\uppsi '}\) photoproduction measurements is:

$$\begin{aligned} \frac{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{\psi '}}{{\mathrm{d}} y} = 0.76~\pm 0.08{\mathrm{(stat.)}} \pm 0.09 {\mathrm{(syst.)}} {\mathrm{mb}}. \end{aligned}$$
(7)

The ratio of the 2S to 1S charmonium states is:

$$\begin{aligned} \frac{\frac{\sigma ^{\mathrm{coh}}_{\psi '}}{{\mathrm{d}} y}}{\frac{\sigma ^{\mathrm{coh}}_{J/\psi }}{{\mathrm{d}} \mathrm{y}}} = 0.18~\pm 0.0185{\mathrm{(stat.)}} \pm 0.028 {\mathrm{(syst.)}} \pm 0.005 {\mathrm{(BR)}}. \end{aligned}$$
(8)

Many systematic uncertainties of the \(\mathrm{J}/\psi \) and \({\uppsi '}\) cross section measurements are correlated and cancel in the cross section ratio. Since the analysis relies on the same data sample and on the same trigger, the systematic uncertainties of the luminosity evaluation, trigger efficiency, EMD correction and ITS-TPC matching of leptons were considered as fully correlated. The AD and V0 offline veto uncertainty is partially correlated, so the difference of the uncertainties for \({\uppsi '}\) and \(\mathrm{J}/\psi \) is taken into account in the uncertainty of the ratio. The systematic uncertainties connected to the signal extraction, incoherent contamination and the branching ratio are considered uncorrelated between the two measurements. The dominant uncertainty comes from the uncorrelated part of the AD and V0 veto uncertainty for \({\uppsi '}\).

5 Discussion

Figure 6 shows the rapidity-differential cross section of the coherent photoproduction of \(\mathrm{J}/\psi \) and \({\uppsi '}\) vector mesons in Pb–Pb UPCs including previous ALICE measurements of \(\mathrm{J}/\psi \) at forward rapidity [27]. At midrapidity, \(\mathrm{J}/\psi \) measurements performed in absolute rapidity ranges are shown at positive rapidities and reflected into negative rapidities. The ALICE measurements are compared to several models which are discussed in the following:

Fig. 6
figure 6

Measured differential cross section of the coherent \(\mathrm{J}/\psi \) (left) and \({\uppsi '}\) (right) photoproduction in Pb–Pb UPC events. The error bars (boxes) show the statistical (systematic) uncertainties. The theoretical calculations are also shown. The green band represents the uncertainties of the EPS09 LO calculation

Full size image

The impulse approximation, taken from STARlight [45], is based on data from exclusive \(\mathrm{J}/\psi \) photoproduction off protons and neglects all nuclear effects except for the coherence. The square root of the ratio of experimental cross sections to the impulse approximation is \(0.64 \pm 0.04\) for \(\mathrm{J}/\psi \) and \(0.66 \pm 0.06\) for \({\uppsi '}\), where statistical and systematic uncertainties of the ALICE measurements and a conservative 10% uncertainty on the impulse approximation are added in quadrature. The obtained nuclear suppression factor reflects the magnitude of the nuclear gluon shadowing factor at typical Bjorken-x values in the range \((0.3,1.4)\times 10^{-3}\) and is in good agreement with \(R_g(x \sim 10^{-3}) = 0.61^{+0.05}_{-0.04}\) obtained in Ref. [19] from the \(\mathrm{J}/\psi \) cross section measurement in UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV.

STARlight is based on the Vector Meson Dominance model and a parametrization of the existing data on \(\mathrm{J}/\psi \) photoproduction off protons [37]. A Glauber-like formalism is used to calculate the \(\mathrm{J}/\psi \) photoproduction cross section in Pb–Pb  UPCs accounting for multiple interactions within the nucleus but not accounting for the gluon shadowing corrections. The STARlight model overpredicts the data indicating that Glauber-like rescatterings alone are not enough to explain the observed suppression of the coherent \(\mathrm{J}/\psi \) cross section.

Guzey, Kryshen and Zhalov [46] provide two calculations (GKZ), one based on the EPS09 LO parametrization of the available nuclear shadowing data [15] and the other on the leading twist approximation (LTA) of nuclear shadowing based on the combination of the Gribov–Glauber theory and the diffractive PDFs from HERA [9]. Both the LTA model and the EPS09 curve, corresponding to the EPS09 LO central set (uncertainties of the EPS09 calculation are represented by the green band), are found to be in a good agreement with the \(\mathrm{J}/\psi \) and \({\uppsi '}\) cross sections measured at midrapidity. However, these models are in tension with the \(\mathrm{J}/\psi \) data at semi-forward rapidity in the range \(2.5<|y|<3.5\), indicating that the nuclear shadowing might have a smaller effect at Bjorken \(x\sim 10^{-2}\) or \(x\sim 5\times 10^{-5}\) corresponding to this rapidity range. It is worth noting that the GKZ predictions are based on gluon shadowing effects at a scale \(Q^2 = 3\, \mathrm{GeV^2}\) in contrast to the default value of \(2.4 \, \mathrm{GeV^2}\) which is used in other models and also in LTA predictions at lower energies [47]. The modified \(Q^2\) value was found to provide better description of the coherent \(\mathrm{J}/\psi \) production cross section in Pb–Pb UPC measured by ALICE in Run 1 as well as exclusive \(\mathrm{J}/\psi \) photoproduction off protons [48].

Calculations by Cepila, Contreras, Krelina and Tapia Takaki (CCK) are based on the colour dipole model with the structure of the nucleon in the transverse plane described by the so-called hot spots, regions of high gluonic density, whose number increases with the increasing energy [14, 49]. Nuclear effects are implemented along the ideas proposed in the energy-dependent hot-spot model with the standard Glauber–Gribov formalism (GG-HS) for the extension to the nuclear case. The GG-HS model agrees with the \(\mathrm{J}/\psi \) measurements at midrapidity and at most forward rapidities but underpredicts them at semi-forward rapidities. The \({\uppsi '}\) measurement at midrapidity is overpredicted by this model.

Calculations by Bendova, Cepila, Contreras, Matas (BCCM) are based on the color dipole approach coupled to the solutions of the impact-parameter dependent Balitsky–Kovchegov equation with initial conditions based on the Woods-Saxon shape of the Pb nucleus [10]. The model is in a reasonable agreement with the \(\mathrm{J}/\psi \) and \({\uppsi '}\) data at midrapidity.

Several theory groups provided predictions for \(\mathrm{J}/\psi \) within the color dipole approach coupled to the Color Glass Condensate (CGC) formalism with different assumptions on the dipole-proton scattering amplitude. Predictions by Gonçalves, Machado et al. (GM) [11, 50] based on the IIM and b-CGC models for the scattering amplitude agree with the \(\mathrm{J}/\psi \) data rather well at midrapidity but strongly underpredict the data at forward rapidities. Predictions by Lappi and Mäntysaari (LM) based on the IPsat model [12, 51] overpredict the ALICE measurements at midrapidity, but match them at forward rapidities. Recent predictions by Łuszczak and Schäfer (LS BGK-I) within the color-dipole formulation of the Glauber–Gribov theory [13] are in agreement with the \(\mathrm{J}/\psi \) data at semi-forward rapidities, \(2.5<|y|<3\), slightly underpredict the data at more forward rapidities \(3<|y|<4\) and overpredict the data at midrapidity.

The comparison of the current results and those at lower energies of Run 1 may provide interesting insights for model builders and further constraints on gluon shadowing effects. In particular, the ratio of the coherent \(\mathrm{J}/\psi \) production cross sections at central and forward rapidities at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV:

$$\begin{aligned} r(5.02\,{\mathrm{TeV}})&= \frac{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{J/\psi }/{\mathrm{d}}y (|y|<0.8, 5.02\, {\mathrm{TeV}})}{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{J/\psi }/{\mathrm{d}}y (-3.25<y<-3.0, 5.02\, {\mathrm{TeV}})} \nonumber \\&= 1.45^{+0.13}_{-0.12} \end{aligned}$$
(9)

is significantly smaller compared to a similar ratio at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV [24]:

$$\begin{aligned} r(2.76\,{\mathrm{TeV}})&= \frac{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{J/\psi }/{\mathrm{d}}y (|y|<0.9, 2.76\, {\mathrm{TeV}})}{{\mathrm{d}}\sigma ^{\mathrm{coh}}_{J/\psi }/{\mathrm{d}}y (-3.6<y<-2.6, 2.76\, {\mathrm{TeV}})} \nonumber \\&= 2.38^{+0.46}_{-0.38} \end{aligned}$$
(10)

where statistical and systematic uncertainties were added in quadrature. The observed flattening of the shape of the rapidity-differential cross section is qualitatively reproduced in the impulse approximation from STARlight where the central-to-forward ratio decreases from 2.66 at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV to 2.22 at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV mainly due to increased contribution of higher photon-nucleus energies to the measured cross section at forward rapidity. However, the magnitude and the energy dependence of the central-to-forward ratio are significantly modified compared to the impulse approximation due to gluon shadowing effects.

The measured ratio of the \({\uppsi '}\) to \(\mathrm{J}/\psi \) cross section is compatible with the previous ALICE measurement at forward rapidities \(R = 0.150 \pm 0.018 {\mathrm{(stat.)}} \pm 0.021 {\mathrm{(syst.)}} \pm 0.007 {\mathrm{(BR)}}\) [27], with the exclusive photoproduction cross section ratio \(R = 0.166 \pm 0.007 {\mathrm{(stat.)}} \pm 0.008 {\mathrm{(syst.)}} \pm 0.007 {\mathrm{(BR)}}\) measured by the H1 collaboration in ep collisions [40] and with the ratio \(R \approx 0.19\) measured by the LHCb collaboration in pp collisions [52]. The measured ratio also agrees with the ratio \(R \approx 0.20\) predicted in the leading twist approximation [46] for Pb–Pb UPCs at midrapidity. The \({\uppsi '}\) to \(\mathrm{J}/\psi \) coherent cross section ratio is expected to have a mild dependence on the collision energy [46]. Therefore, the measured ratio can be directly compared to the unexpectedly large \({\uppsi '}\) to \(\mathrm{J}/\psi \) coherent cross section ratio \(R = 0.34^{+0.08}_{-0.07}\), measured by ALICE at midrapidity in Pb–Pb UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV [25]. The previous measurement is about a factor two larger but is still compatible within 2 standard deviations with the present measurement, owing mainly to the large uncertainties of the previous result.

6 Conclusions

The first rapidity-differential measurement on the coherent photoproduction of \(\mathrm{J}/\psi \) at midrapidity \(|y|<0.8\) in Pb–Pb UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~5.02\) TeV has been presented and compared to the model calculations. This data complements the ALICE measurement of the coherent \(\mathrm{J}/\psi \) cross section at forward rapidity \(-4<y<-2.5\) allowing us to provide stringent constraints on nuclear gluon shadowing models.

The nuclear gluon shadowing factor of about 0.64 at Bjorken-x values \(x\in (0.3,1.4)\times 10^{-3}\) is estimated from the comparison of the measured coherent \(\mathrm{J}/\psi \) cross section with the impulse approximation at midrapidity. This result is in agreement with the gluon shadowing factor extracted from the previous ALICE measurement of the coherent \(\mathrm{J}/\psi \) cross section at midrapidity in Pb–Pb UPCs at \(\sqrt{s_{\mathrm {NN}}}~=~2.76\) TeV.

None of the models is able to fully describe the measured forward and central rapidity dependence of the coherent \(\mathrm{J}/\psi \) cross section. The \(\mathrm{J}/\psi \) measurements at central and most forward rapidities are found to be in agreement with the models based either on the leading twist approximation of nuclear shadowing, or the central value of the EPS09 parameterization as well as with the energy-dependent hot-spot model extended to the nuclear case by the standard Glauber–Gribov formalism and the color dipole approach coupled to the solutions of the impact-parameter dependent Balitsky–Kovchegov equation. However, these models appear to be in tension with the data at semi-forward rapidities in the range \(2.5<|y|<3.5\). The data might be better explained with a model where shadowing has a smaller effect at Bjorken \(x\sim 10^{-2}\) or \(x\sim 5\times 10^{-5}\) corresponding to this rapidity range. On the other hand, the models based on the color dipole approach coupled to the color glass condensate formalism describe either the forward or central measurements depending on the dipole scattering amplitude assumptions but they are not able to describe the measured cross section in the full rapidity range.

The ratio of the \({\uppsi '}\) to \(\mathrm{J}/\psi \) cross sections at midrapidity is in a reasonable agreement with the ratio of photoproduction cross sections off protons measured by the H1 and LHCb collaborations, with the leading twist approximation predictions for Pb–Pb UPCs as well as with the ALICE measurement at forward rapidities.