I had been thinking a lot about the topic James and I had strongly disagreed on last time. Although I hadn’t changed my mind, James’s position had become clearer to me.
When we met again, I admitted that models can, at times, unnoticedly fill in gaps in our perception of the surrounding world.

“But it doesn’t happen that often,” I added.

“Quite often,” he objected. “They fill gaps not only in space, but also in time.”

“In the past, our agency conducted an experiment,” he continued. “You know, we regularly organize training sessions for agents. They work in groups — typically three people — for several weeks, each day of the week devoted to a specific activity. For example, on Tuesdays they’re trained to memorize secret addresses and passwords; on Wednesdays, they practice rapid orientation using geographical maps; on Thursdays, they encode and decode confidential reports, and so on.”

“After a few weeks of training, all three agents were given a harmless drug that caused them to sleep for about 30 hours, effectively skipping the entire Wednesday. They woke up Thursday morning, quickly realizing something was off. At first, they exhibited signs of anxiety, but soon they forced themselves to carry on as if everything was normal.”

“Did they try to share their suspicions with each other?” I asked.

“No,” James said. “Each of them pretended nothing had happened. They hid their unease and quickly became fully immersed in the regular Thursday activities. It was as if they collectively chose to ignore what had occurred.

And here’s the most interesting part: after the training was over, they each wrote detailed reports describing their activities — day by day — including a day that never actually existed. I must stress, these were highly trained and scrupulously honest individuals.”

“Amazing!” I said. “By the way, I didn’t know your agency was involved in basic psychological research.

“It is,” he nodded. “And for very practical reasons. Since that experiment, the agency no longer trusts a posteriori reports. Agents are now required to make brief, real-time (encoded) notes throughout a mission — accurately recording the current time as they go.”

Exploitative vs Explorative Strategies

The most peculiar detail in the story told by James was the fact that none of the participants in the experiment tried to share their doubts with others. Were they afraid of appearing completely insane? Maybe not. Perhaps they were somehow hypnotized by what is often called common opinion, even when it contradicted their own subjective experience.

Indeed, it is nearly impossible to question a well-established, widely accepted model dictating that Tuesday is followed by Wednesday. This model is very useful in 99.99 percent of cases. However, if you once clearly notice that Thursday comes after Tuesday, maybe it’s better to admit that fact honestly and explore what consequences might arise.

This suggests that your strategy should not be 100 percent exploitative (i.e., relying solely on established concepts), but partially explorative — actively checking whether factors not previously included in the model, or even completely unknown ones, might better explain the observations.

Let me sketch an example from structural biology. In that field, scientists collect thousands of TEM images of proteins. Each individual image has such weak contrast that it’s impossible to discern any structure. But by correlating many images taken from different orientations and averaging them, the underlying structure emerges quite clearly.

Let’s try to mimic this technique. Suppose we’ve got a number of TEM images that look like this:

Not very impressive, right? But if we average all these noisy images — omitting, for simplicity, the complex steps of projection, orientation, and alignment — some pattern begins to appear, resembling known reference structures that we call here “stars” and “hexagons”.

We might even guess that these structures are present in a 2:1 ratio.

So far, so good! Let’s fit each individual image to either the star or hexagon reference and count how many of each we have. Then, we sum up the resulting classes and compare the averages with the reference structures.

The averaged pattern for the star looks almost identical to the reference—though a careful examination reveals a few excessive features. Maybe our classification algorithm wasn’t perfect. But the average pattern for the hexagon is so different from the reference that it’s hard to ignore.

At this point, it’s probably time to switch from an exploitative to an explorative strategy. Suppose, that there are no hexagons at all. It’s not a strongly grounded suggestion, but let’s see where it leads. If that’s the case, then everything that doesn’t match the star pattern must belong to something else — a previously unidentified class.

After some hard work, we arrive at another picture: a mixture of stars and something new — triangles!

Now the observations are coherent. The star pattern is well reproduced and the new triangle patterns explain all the rest. Since the triangles appear in two mirrored variants, we conclude there is a 1:1 mixture of stars and triangles (See the script for details on how this quantification is performed).

Here’s the funny part: the new interpretation produces exactly the same average image as our initial conservative hypothesis.

Conclusion: We took the risk of exploring an unknown domain — and it paid off. A new protein structure is discovered!

The Python codes can be found in the pdf version of this document: Full Text with Codes.