The brain often blurs the senses, a fact that marketers often use in the design of food packaging. AIs appear to do the same.

What is the flavor of a pink sphere? And what is the sound of a Sauvignon Blanc? 

Such questions may sound ridiculous, but a huge body of literature shows us that the human brain naturally merges sensory experiences. We may not be conscious of the phenomenon, but we associate different colors, shapes, and sounds with different flavors in ways that can subtly shape our perceptual experience, for example.

The color of our glass, or music playing in the background of a bar, can determine how sweet or musky a wine tastes, for instance. "This cross-talk between the senses is happening almost on an ongoing basis all the time," explains Carlos Velasco at the BI Norwegian Business School in Oslo, Norway. In extreme cases, it can manifest in a blurred sensory experience for some people where words might trigger specific tastes or music produces a riot of color – something known as synaesthesia. 

While the idea that you can "taste" a color or sound may seem absurd enough, Velasco's latest research suggests that generative artificial intelligence systems may also do this. As with all AI algorithms, this largely reflects biases in the data they were trained on, so they are perhaps just highlighting how common these associations may be. But Velasco and his colleagues hope to use this fact to find many other ways to hack human senses.

First, a note on terminology. Scientists use the term "sensory modality" to describe the means that the body uses to encode information – through, for example, our taste buds, eardrums, the retina in our eyes, or the "tactile corpuscles" in our skin. The associations that we tend to form between different sensory qualities are therefore known as "cross-modal correspondences".

Experimental evidence for this phenomenon first emerged in the 1970s, with studies suggesting that red and pink hues are associated with sweetness, yellow or green with sourness, white with saltiness, and brown or black with bitterness. These general patterns have now been replicated many times since, using multiple experimental methods.

Participants may be asked for their subjective judgment of abstract questions such as: "On a scale from 1 to 10, with 10 being the sweetest, how sweet is the color red?" From this, you can see that, on average, each color has a unique flavor profile shared by large numbers of people across different cultures. A multinational collaboration, led by Xiaoang Wang at Tsinghua University in China, found similar cross-modal correspondences in Chinese, Indian, and Malaysian participants.

Alternatively, participants may be given a particular food or drink presented in multiple colors and asked to judge the taste of each one. Eriko Sugimori and Yayoi Kawasaki at Waseda University in Japan, for instance, have found that bitter chocolate tastes considerably sweeter when it is wrapped in pink, rather than black, packaging.

The shapes of food can have similar effects. We tend to associate rounder shapes with sweetness, while spikier shapes are considered to be more sour or bitter – with knock-on effects on people's perceptions of the foods. We eat with our eyes as well as our tongues. 

The origin of these associations is still a matter of debate. "The safest assumption is that we learn them all," says Charles Spence, the head of the cross-modal research laboratory at the University of Oxford. "They could be thought of as a kind of internalization of the statistics of the environment. In nature, fruits go from green, when they are sour, to redder and warmer hues, when they are sweeter. If we internalize that statistic, associating reddish hues with sweeter taste, we know which trees to climb for the for the fruit that will sustain us."

The associations between shape and taste are harder to explain. "It may be the emotions associated with or triggered by the stimuli," Spence says. We may associate sweetness with pleasure, for example, and we prefer round shapes since they are less likely to hurt us, compared to something sharp. As a result, we start to associate sweetness with curviness through this indirect association. Bitter substances, in contrast, are more likely to be poisonous – and so we might link them to sharp shapes that also have more potential to cause bodily harm.