Limits and Discoveries

In experimental physics, not every experiment ends with a discovery. Scientists can arrive at two fundamentally different kinds of results. One is a limit, when no particular effect is observed in the data, and we can quantitatively state that if such an effect exists, its magnitude cannot exceed a certain level. The other is a discovery, a statistically significant observation of an effect. A discovery does not necessarily mean something completely unexpected. Sometimes it is the confirmation of a theoretical prediction at a level where the probability of a random statistical fluctuation becomes extremely small.

In one case, we narrow the boundaries of what is possible. In the other, we establish the presence of a new signal. But are these two outcomes fundamentally different? And can the research strategy itself influence which conclusion we eventually reach?

The distinction between a limit and a discovery is formalized statistically. Statistical significance is usually discussed in terms of sigma, not only in particle physics but in experimental science more broadly. A deviation at the level of 5 sigma means that the probability of the observed effect arising purely from a statistical fluctuation is roughly one in several million. Such a result is conventionally considered a discovery.

Limits are formulated differently. Physicists often speak of a 95 percent confidence level. This does not mean that “we are 95 percent sure that the effect does not exist.” It means something more subtle. If the effect truly did not exist, then in repeated realizations of the same experiment, the observed data would be consistent with its absence in 95 percent of cases.

So a discovery is a positive statement about the presence of an effect, while a limit is a quantitative constraint on its possible magnitude. But behind these statistical definitions lies a deeper question. How does a researcher approach the problem in the first place, and what do they internally consider the real goal?

During my PhD work within the ATLAS Collaboration experiment at the CERN Large Hadron Collider, I studied processes involving the production of elementary particles in proton-proton collisions, in particular neutral bosons that mediate the electroweak interaction. To understand what exactly we were looking for, it is useful to think in terms of Feynman diagrams.

Within the Standard Model, such events can occur, for example, through photon radiation from one of the quarks. This is the standard process: a photon is emitted, a Z boson is produced, and the Z then decays into leptons. There is no direct interaction between neutral bosons here. Everything happens through radiation. arXiv:1604.05232

But one can imagine a different scenario. If a direct interaction between neutral bosons existed, something forbidden by the Standard Model, its contribution would appear as a deviation in the data, for example as an excess of signal events at high energies. This is why we were not simply looking for Zγ events themselves, but for deviations in kinematic distributions relative to theoretical predictions. In that sense, this was genuinely a search for discovery. Observing such a signal would not have meant a refinement of known parameters, but evidence for a new type of interaction. But no such signal appeared. Instead of a discovery, we set limits on the possible size of such deviations.

Years later, I often find myself thinking about the overall mindset behind that analysis. In reality, the goal was formulated quite pragmatically: to perform a careful statistical analysis and obtain a robust exclusion of part of the model parameter space, regardless of whether a signal would appear. A strong publication was valuable in itself. Today I understand much better how important it can be, at certain stages of a scientific career, to produce a solid paper and a reliable result. Limits are a fully legitimate and necessary scientific product. They clean up the space of hypotheses and make the theory more constrained.

And yet I remember a certain inner frustration. During those years, while working on my PhD, I had the opportunity to attend the CERN-Fermilab School, where one of the lecturers dedicated an entire lecture to searches for physics beyond the Standard Model. After the lecture, we had a conversation about possible deviations in neutral boson interactions, exactly the topic I was working on at the time. What I remember most is not the content of the arguments, but his emotional intensity. It was not an abstract academic discussion about parameters. It was genuine scientific excitement, the feeling that perhaps a signal could appear right here.

That feeling is contagious. It cannot be expressed in sigma or confidence levels, but it affects how you look at the data. In some sense, it was precisely that inspiration that helped me finish my PhD with real interest, rather than simply forcing myself to complete it. Because at that moment, the work was not about setting another limit. It was about the possibility of seeing something genuinely new.

Almost ten years have passed since then. Limits continue to become stronger, statistics continue to grow, and analysis techniques continue to improve. More universal approaches have also emerged, for example within effective field theory, where different possible deviations can be described within a common parameter framework. Such approaches make it possible to combine multiple production channels in proton-proton collisions, increasing sensitivity to new physics. Perhaps the gradual tightening of limits is the only realistic way we currently have to probe physics at energy scales that remain directly inaccessible.

Another example from my past work is the search for the decay of the Higgs boson into a pair of second-generation leptons: muons. This is a channel predicted by the Standard Model, but its branching ratio is extremely small. Experimentally, this means that the signal is almost completely buried under background, primarily from dimuon production through the Z boson.

The Higgs signal amplified by 100 (red color) appears as a tiny excess on top of the enormous peak formed by muons coming from Z boson decays (blue color). ATLAS-CONF-2019-028

The task here was fundamentally different. We were not looking for an unexpected effect, but trying to extract an extremely weak signal that was already expected to exist. All efforts were focused on maximizing sensitivity to this channel. At the time when I participated in this analysis, the observed significance was around 2 sigma, in other words, a hint but not an observation. At the same time, the existence of this decay mode itself was hardly in doubt. Our group within ATLAS Collaboration did not manage to reach even 3 sigma at that stage. However, the methods developed at that time, as far as I know, have now brought the result much closer to 5 sigma with larger datasets. This is the kind of measurement where improved statistics and analysis methods gradually reduce uncertainty and increase confidence in the observed process. It requires patience.

Instead of a conclusion

If we look more broadly, any researcher starting a project may internally lean toward one of two strategies: searching for a new effect, or refining the boundaries of an existing theory. The laws of nature do not change because of this. But the way we formulate the question can influence how we interpret the data. In complex analyses, where signals are weak and statistical fluctuations are subtle, expectations themselves can become a kind of hidden systematic in interpretation. Sometimes this leads people to see an effect where none exists. Sometimes it leads them to interpret everything as yet another tightening of limits.

In the fragile balance between data and noise, the mindset of the researcher can quietly shift the emphasis. Limits are essential because they protect us from self-deception and discipline our theories. But discoveries, even when they merely confirm theoretical predictions, change our picture of nature in a fundamentally different way.

That is why I believe it is important to pay attention to the mindset with which we approach data. At this stage of my career, when working with real experimental data, I find it more natural to maintain a certain skepticism toward theoretical expectations in order to preserve objectivity. This is especially important in my current field, the study of cosmic rays, where a fully consistent theoretical framework still does not exist.

Take, for example, the search for antimatter in space. The Alpha Magnetic Spectrometer experiment, where I currently work, was designed in part to search for extremely rare antiparticles among an overwhelming background of ordinary matter. In the simplest picture, the early universe should have produced matter and antimatter in roughly equal amounts. When they meet, they annihilate into radiation. This naturally leads to one of the deepest questions in modern physics: why does the universe we observe today consist almost entirely of matter? Does antimatter exist somewhere on astrophysical scales, or did this asymmetry emerge during the earliest stages of cosmic evolution?

Even in such a search, different scientific strategies are possible. One can focus on identifying a statistically significant signal, or on setting increasingly strict limits on its possible flux. If a truly significant effect appears in the data, both approaches will eventually recognize it. But at the early stages, when significance is only beginning to grow, statistical uncertainties are still large, and systematic effects are not yet fully understood, the mindset of the researcher may influence how the observed deviation is interpreted, and where exactly the boundary is drawn between a hint of discovery and yet another limit.

Thanks to the work of hundreds of people, our experiment continues to operate, and that means we still have a chance to collect enough data to answer one of the most fundamental questions in modern physics: how much antimatter actually exists in our universe.

Discover more from Main page

Subscribe now to keep reading and get access to the full archive.

Continue reading