You are here

Culture of Diagram


"Stress contours calculated on an incised human cornea during a radial keratotomy simulation." Computer-driven interface from Ian W. Hunter et al., "Ophthalmic Microsurgical Robot and Associated Virtual Environment,"
Photo Credit: 
Computers in Biology and Medicine, vol. 25 (1995). ©1995, with permission from Elsevier.

Trust in a Virtual World

We now live in what is being called the Information Age, because with the invention of the internet and digitization humans can access, exchange, and act through massive quantities of knowledge more easily than in any previous time in human history. A unique aspect of digital information, however, is that much of what we accept with certainty is nothing more than a virtual representation generated by a mathematical algorithm.

Take for example the modern operating room where the surgeon and patient implicitly trust a 3-D image of the patient’s retina that the surgeon uses to map her incision points. Every day, devices like TVs, mobile phones, and satellites present people with showers of digital imagery from which they glean information and knowledge.  But how did people come to believe what they see on the screen?

Stanford humanities professors Michael Marrinan and John Bender work together to understand modes of representation that came before the digital revolution and also includes it.  Bender, an English professor, analyzes how writers represent life with words, and Marrinan, an art history professor, investigates how artists depict reality through their paintings. Their combined areas of expertise bridge the components of a pervasive kind of visual representation called a diagram.

By combining elements of the written word and the artistic rendering, a diagram allows its creator to illustrate a process while enabling its viewer both to visualize knowledge and to use it to make new working objects. The professors suspected that by exploring this long-standing model of visual representation, they might find out how modern society arrived at an almost ubiquitous confidence in the digital reproduction. Their combined research effort resulted in a multi-disciplinary perspective on, as Bender and Marrinan explained, “why people trust the algorithms that create the information.”  The book they co-authored is called The Culture of Diagram (Stanford University Press, 2010).

Visualizing Knowledge with Diagrams

Marrinan and Bender--both scholars of eighteenth century European culture--surmised that our trust in the representation might feel innate because humans have had over almost three hundred years to get used to the idea. Denis Diderot, a French philosopher and writer during the European Enlightenment of the 18th century, was charged with editing and organizing a dictionary designed to catalogue all fields in the arts and sciences.

He and his collaborators, including co-editor Jean le Rond d’Alembert, created the Encyclopédie--a comprehensive and cross-referenced catalogue of human knowledge. Marrinan explained that by presenting information visually in Encyclopédie, Diderot ushered in a novel way of learning: "The plates, in particular, broke with most of the conventions of pictorial representation to afford users knowledge of things in finer grain and from unusual or even impossible vantage points that their physical bodies and senses could not provide.  We believe the Encyclopédie offered users something like an ‘out of body’ experience, and that its presentation of data resembles a ‘mash up’ more than a dictionary.  This struck us as both very modern and very pertinent to our questions about the digital turn in contemporary culture."

Diagrams, technical drawings that illustrate processes, proved to be valuable tools for Diderot, and he filled the 28-volume publication with over three thousand of them. The plates represented all manner of process from the workflow in a pastry shop to the human circulatory system. What differentiates diagrams from other visual representations such as paintings, is that they enable the viewer to correlate large swaths of knowledge in a way that allows them to extrapolate new findings from existing bits of information. 

According to Bender and Marrinan, diagrams mimic the way people explore things in the real world. “Traditional pictures are said to open windows onto the world. We wanted to show how diagrams broke the window, shattering visual knowledge into multiple scales and points of view.” Bender said. He also observed that: “Diagrams often are drawings like those engineers make in which each thing is shown at whatever size will make it clear to the viewer, but the user is the one who puts it all together into a machine or a building.  People often think of pictures as final objects but diagrams are not finished.  They are tools for users to work with as their minds encounter the world.”

Convergence of the Mathematical and the Digital

In the 19th century, diagrams became progressively more mathematical and less visual as they were used to illustrate physical processes invisible to the naked eye, and as they depicted increasingly sophisticated scientific discoveries. The simultaneous and growing adoption of mathematics, and more specifically statistics and calculus, further enlarged people’s ability accurately to conceptualize things that could not be illustrated on the printed page.

The Culture of Diagram argues that mathematical formulas, as used in virtual reality, can “define fields” and “specify spaces and movement” in ways that correct for the “body’s vagaries of perception.”  Such correction allows for minute precision in surgical operations on the eye and brain.  In other words, the precise nature of mathematical calculations insures that any two people who use a formula can act effectively and accurately. 

So, although disciplines like calculus and geometry describe seemingly invisible concepts, they allow users to make definite correlations that are “of practical value in the real world.” “We saw that images were at the heart of diagrammatic knowledge up through the eighteenth century.  But because the problems became more and more complex, statistics and the calculus began to take a central place when nineteenth-century scientists did experiments and presented them to the world as diagrams.” 

Bender also pointed out that during this period images remained but were less important:  “Now, in the realms of virtual reality, human beings use powerfully mobile and responsive images like those in robotic eye surgery.  These images are visual on the surface, but underlying mathematical algorithms generate them and turn them into powerful tools that let us use our sense to reach out of our senses.”

Expanding the Human “Sensorium” Electronically

Advanced technological devices developed in the 20th century have allowed scientists and laymen alike to “see” all sorts of data that previously eluded the human senses. Radio wavelengths, weather fronts, protein cells and heart rhythms were brought into the realm of reality because they could be seen.  The mathematical and visual combined to allow activities like robotic brain and eye surgery.

Marrinan and Bender suggest that human life has been enhanced and improved by these new “electronic appendages.” Marrinan said that this observation was an impetus guiding the trajectory of their research: “What struck us, and motivated our book, was the shift from instruments designed as finer-grained analog detectors patterned on human sensory systems to instruments that take discrete samples at very high rates and ‘smooth’ their results using algorithms of error-correction and probability.  The digital turn is basically calculus applied to perception.”

Marrinan gave the example of the difference between a phonograph--read by a stylus following a wavy groove molded in vinyl--and a compact disc to illustrate how digital reproductions inherently require a degree of trust: The CD is a massive compilation of samples that must be constantly corrected for reading errors and “smoothed out” by algorithms of probability in order to  render musical waves.  You can actually ‘hear’ a vinyl record by holding a pin against the spinning disk.  No such thing is possible with a CD:  you must have decoders and error-checkers to hear anything. 

So the leading question of our book was something like this:  how is it that humans have come to entrust their senses to this chain of mathematical slicing, splicing, and smoothing over?  Of course, it’s true that the end product from a CD sounds like Mozart to our ears, but the encoded media resembles it not at all.  This type of media disjunction operates today in almost every aspect of our lives. As they come to their final conclusions in Culture of Diagram, Bender and Marrinan assert that the foundation for trust in, and even reliance on, the man-machine interface that is so prevalent in today’s hi-tech world, was built long before anyone envisioned anything akin to an electronic appendage.

Approaching the topic from a historical perspective, Bender and Marrinan found a source of this uniquely human phenomenon in Diderot and d’Alembert’s innovative approach to organizing information. Professor Marrinan stated that like the Encyclopédie, the Internet encourages users to learn by making connections between seemingly disparate bits of information: “A common view holds the Encyclopédie to be an attempt to codify all human knowledge.  What we found was not a seamless body of knowledge, but something akin to ‘samples’ loosely strung together by an alphabetical order and criss-crossed by distractions, ruminations, parallel paths, and a whole host of open-ended question marks that invite a reader (or user, as we prefer to say) to find his or her path towards understanding.  It’s a bit like a web page with hyperlinks that branch out from one point in many directions.”

Diderot could not have predicted how the unique presentation of words and pictures in Encyclopédie would influence the ways people would process information for centuries to come. In the same vein, as virtual representations today invite users to suspend their disbelief and to accept alternate versions of reality, we cannot yet imagine all of the ways that visual knowledge will expand the boundaries of the human experience in the future.