Sensory Translation and the Impossibility of Total Documentation

Maurice Merleau-Ponty's phenomenology insists that perception is always embodied, always multi-sensory, always more than the sum of discrete sensory inputs. Yet urban documentation privileges vision almost exclusively. We photograph cities, map them, describe what we see. Sound becomes "ambient noise." Smell goes largely unrecorded. Touch registers only as obstacle - the cobblestone that trips you, the rain that forces retreat. Synaesthesia responds to this sensory hierarchy by attempting systematic documentation across multiple modalities: visual, sonic, textual, synthetic. The project doesn't claim to achieve total sensory documentation - that would be phenomenologically naive. Instead, it investigates what each sensory mode captures that others miss, and what gets lost in translation between modes.

Marshall McLuhan argued that media are not neutral channels but actively shape the content they transmit. "The medium is the message" applies with particular force to sensory documentation. A photograph of a Belgrade street market captures spatial arrangement, colour, light, composition - but misses entirely the acoustic signature that defines the experience for anyone actually present. The vendors' calls, the haggling voices, the traffic noise, the underlying hum of human density - none of this registers visually. Conversely, a field recording captures temporal flow and sonic texture but provides no spatial information about how the market organises itself architecturally. Each medium reveals and conceals simultaneously.

The project deploys multiple documentation streams deliberately. Field recordings made with the Teenage Engineering KO II sampler capture urban soundscapes in situ - morning traffic patterns, language rhythms, infrastructure noise, the acoustic signatures of different neighbourhoods at different times. These recordings function as primary evidence: this is what Banjica sounds like at 7am, this is the frequency pattern of rush hour, this is how sound changes between riverfront and interior zones.

Unlike visual documentation, sound doesn't lie about density, activity, or use patterns. You can stage a street photographically. You can't stage an authentic soundscape without actually generating the activity that produces it.

But Synaesthesia doesn't stop at documentary field recording. The synthesis work transforms captured sound into composition - processing Belgrade traffic through the KO II to create something that derives from actual urban sound but no longer functions as pure documentation. This synthetic layer introduces interpretation explicitly. The city becomes source material for understanding how urban sound can be both captured and transformed, documented and performed, analysed and aestheticised. The dual approach maintains WWTC's characteristic balance: rigorous documentation alongside frank acknowledgment that all documentation interprets.

Walter Benjamin's concept of mechanical reproduction becomes relevant here, updated for digital synthesis. Benjamin worried that photographic reproduction destroyed the "aura" of artworks - their unique presence in time and space. But field recording and synthesis operate differently. The original sound (traffic at 7am on a specific Tuesday) disappears the moment it's produced. The recording doesn't reproduce an artwork but captures an unrepeatable event. The synthesis then transforms this captured event into something that never existed in the world - a composed piece that references Belgrade but doesn't document it. The relationship between field recording and synthesis parallels the relationship between photography and painting: different epistemological claims, different relationships to truth.

Visual aphorisms constitute the third documentation stream. These minimal geometric diagrams distill complex concepts into essential visual form - "sustained residency vs. extended tourism" becomes two timelines of different lengths; "what the water sees" becomes converging river lines with observation markers. The aphorisms translate theoretical insights into visual language, testing whether concepts that work in prose maintain coherence when forced into diagrammatic constraint. Some translations succeed. Others reveal conceptual confusion that prose obscured through verbal elaboration. The visual medium demands clarity that written language can fake.

Written blog posts provide the fourth stream - analytical frameworks, theoretical context, narrative documentation of walking routes and observations. Text allows for complexity, qualification, citation, argument in ways that other media cannot. But text also distances. When you read about a walk, you're not walking. The words mediate rather than presenting immediate experience. This mediation isn't weakness but the condition of textual knowledge. Writing allows reflection, comparison, synthesis across multiple experiences. It trades immediacy for analytical capacity.

The crucial methodological move is translation between modes. How do you represent a soundscape as visual aphorism? What's the textual equivalent of a synthesis patch? Can you photograph the experience of reading about a neighbourhood? These translation attempts inevitably fail - but productively. The gap between a field recording and its textual description reveals what language can and cannot capture. The impossibility of perfectly translating a photograph into sound demonstrates that different senses access different aspects of the same phenomenon. The failures make explicit what usually remains implicit: that all documentation is partial, all representation is reduction, all media have both affordances and constraints.

Roland Barthes' Camera Lucida meditates on photography's peculiar relationship to reality - the photograph testifies that "this was" but tells you nothing about what it meant. A photograph of a Belgrade street documents that these buildings stood in this arrangement on this day under this light. But it doesn't capture the smell of roasting coffee, the texture of cobblestones underfoot, the temperature that made people walk quickly rather than lingering, the social atmosphere that determined whether strangers made eye contact. The photograph's evidentiary power is real but narrow. Synaesthesia surrounds the photograph with other documentation streams that capture what the image misses - not to achieve completeness but to make the gaps visible.

The project draws on ethnographic traditions of thick description - Clifford Geertz's insistence that cultural analysis requires not just noting what happened but interpreting significance, context, meaning. But Synaesthesia extends this: thick description across multiple sensory modalities, each providing different access to the same cultural phenomena.

The thickness comes not from exhaustive textual elaboration but from layering different media that triangulate toward phenomena no single medium adequately captures.

Sound theorist R. Murray Schafer coined the term "soundscape" by analogy with landscape - the sonic environment as something that can be composed, designed, studied. Synaesthesia treats urban environments as simultaneously soundscape, landscape, textscape (the language environment), and affectscape (the emotional atmospheres different zones generate). Each -scape operates according to different logics, reveals different patterns, requires different documentation strategies. The field recording captures soundscape. Photography captures landscape. Blog posts navigate textscape. The synthesis work creates new affectscapes that reference but don't reproduce the originals.

What the project refuses is the fantasy of total documentation - the idea that if you just deployed enough sensors, captured enough data, generated enough description, you could fully represent urban experience. Merleau-Ponty knew this was impossible. Perception is always perspectival, always incomplete, always structured by the body and its capacities. Adding multiple perspectives doesn't eliminate partiality. It multiplies partial views, creating what Donna Haraway calls "situated knowledges" - knowledge that's explicitly positional, explicitly limited, explicitly honest about its construction.

The translation failures matter more than the successes. When you try to describe a sound in words and the description feels inadequate, you're discovering something about the specificity of sonic knowledge that textual knowledge cannot capture. When you try to photograph what a neighbourhood sounds like and the photograph feels wrong, you're learning that vision and hearing access different aspects of spatial experience. These failures don't represent methodological problems to solve. They represent the actually existing condition of sensory knowledge: different senses tell you different things, and no translation between them preserves everything.

Synaesthesia ultimately demonstrates that urban investigation requires multiple documentation streams not to achieve completeness but to honestly acknowledge partiality. Each stream - field recording, synthesis, photography, visual aphorism, written analysis - captures something real while missing something else real. Together they don't add up to total knowledge. They reveal the complexity of what "documenting a city" actually entails when you try to do it rigorously across multiple sensory modes. The city sounds different than it looks. It reads different than it sounds. The gaps between these modes aren't failures - they're the phenomenon itself. Urban experience is irreducibly multi-sensory. Documentation that pretends otherwise isn't being rigorous. It's being naive about its own partiality.