Search for tt̅H/A→tt̅tt̅ production in the multilepton final state in proton-proton collisions at √s = 13 TeV with the ATLAS detector

A search for a new heavy scalar or pseudo-scalar Higgs boson (H/A) produced in association with a pair of top quarks, with the Higgs boson decaying into a pair of top quarks (H/A→tt̅) is reported. The search targets a final state with exactly two leptons with same-sign electric charges or at least three leptons. The analysed dataset corresponds to an integrated luminosity of 139 fb⁻¹ of proton-proton collisions collected at a centre-of-mass energy of 13 TeV with the ATLAS detector at the LHC. Two multivariate classifiers are used to separate the signal from the background. No significant excess of events over the Standard Model expectation is observed. The results are interpreted in the context of a type-II two-Higgs-doublet model. The observed (expected) upper limits at 95% confidence level on the tt̅H/A production cross-section times the branching ratio of H/A→tt̅ range between 14 (10) fb and 6 (5) fb for a heavy Higgs boson with mass between 400 GeV and 1000 GeV, respectively. Assuming that only one particle, either the scalar H or the pseudo-scalar A, contributes to the tt̅tt̅ final state, values of tanβ below 1.2 or 0.5 are excluded for a mass of 400 GeV or 1000 GeV, respectively. These exclusion ranges increase to tanβ below 1.6 or 0.6 when both particles are considered.