Flong

Printed from www.flong.com/texts/essays/essay_mit_application/
Contents © 2020 Golan Levin and Collaborators

Golan Levin and Collaborators

Essays and Statements

Statement of Objectives, MIT Graduate School Application

Written in support of an Application for Admission to Masters' Studies in the Aesthetics and Computation Group, MIT Media Laboratory, Cambridge MA.
Golan Levin, 15 December 1997.

It was quite clear to me that painting was capable of developing powers of exactly the same order as those music possessed.
— Wassily Kandinsky, 1922

[The new medium is:] Dynamic, manipulable graphics with complex behavior.
— Bill Verplank, 1981

Systems for Abstract Creation and Communication

I have had, for as long as I can remember, a deep interest in abstract visual communication. My mother, who is an Abstract Expressionist painter, and my father, who is an engraver, exposed me continually and from the youngest age to imagery whose content was form itself. Oddly enough, however, my earliest recollection of experiencing the power of abstract form is a memory of an event that occurred in my family's synagogue when I was very small: I had just learned to read English, but it hadn't yet dawned on me that there could be other writing systems apart from the one I knew. One evening during services, I asked my father what the funny black squiggles were in the prayer books we were holding. "Sh!" he said: " — that is how we talk to God." I became riveted by the black squiggles, which no longer seemed quite so funny, and stared at them intently until they danced before my eyes; only later did I learn that these marks were Hebrew. Since that time, I have been preoccupied by the idea that abstract forms can connect us to a reality beyond language. Having much to communicate which is neither linear, segmented, nor divisible into minimally distinctive semantic units, I have come to regard with suspicion the wholesale reduction of human existence into verbal language, and have striven instead to create meaningful expressions that could only be conveyed through non-verbal media.

For the past year I have attempted to explore abstraction by creating prototype "visual instruments" that afford continuous, expressive handles into the real-time performance of dynamic, abstract animation. Much of this work has been inspired by, and performed in close collaboration with my colleague Scott Snibbe, to whom I am indebted for introducing me to the domain of performative abstraction, and with whom I presently share many of the motivations expressed in this statement. The intent behind our work in interactive visual instruments has been to deliver similar experiences of joy, surprise, whimsy, creation, and non-verbal communication as are afforded, for example, by traditional musical instruments. In fact, the systems we have built are highly analogous to musical instruments, but in the visual domain — allowing interactants to gesturally perform visual patterns whose formal language consists of geometry and color, instead of sound, over time.

Musical instruments provide an especially rewarding basis for analogies because they have offered, for thousands of years, what may be the best example of humans deriving gratifying interactions from machines. Although the systems I have recently developed do not produce "music" as such, the idea that these works could nevertheless be considered "instruments" for the performance of animated graphics also serves to distinguish them from other software systems which might, on the surface, appear similar. "Visual instruments" are distinct from, for example, screen savers (which depend on a passive interaction model), games (which are generally not concerned with the user's creative expression), visual interfaces to musical instruments (in which the creative expression is made in audio), or music visualization systems (in which, again, graphic expression is an accessory to music).

The broad goal of the work I propose here is to promote creativity and communication via computational visual media. In the remainder of this statement, I put forth one possible theoretical context within which this pursuit may be framed; some criteria by which I have come to evaluate the success of systems of this kind; and a list of some of the new questions and directions to which I'd like to open this endeavor in graduate study.

Hot and Cool

In my attempt to understand the design of great media for creative personal expression, I've been tremendously influenced by Marshall McLuhan's distinction between what he termed "hot" and "cool" media. To McLuhan, "hot" media are high-definition, high-resolution experiences that are "well-filled with data," while "cool" media are low-definition experiences that leave a great deal of information to be filled in by the mind of the viewer or listener. Photography and film are hot media, for example, while cartoons and telephony are cool. McLuhan's definitions establish a strongly inverse link between the "temperature" of a medium and the degree to which it invites or requires audience participation: hot media demand little completion by their audience, while cool media, "with their promise of depth involvement and integral expression," are highly participatory.

A quick glance at the tradeshow floor at SIGGRAPH is all that is necessary to observe that the trend in the computer graphics industry to date has been the development of high-resolution, high-bandwidth, mega-polygon experiences. The products of this focus — typically photorealistic three-dimensional virtual realities — have been dazzling and hypnotizing. But our relations to these spaces are rarely ever more than as spectators, and almost never as creators. The industry's rush to develop these hot experiences has left in its wake numerous fertile and untrammeled technologies for cooler, more participatory media. Scott Snibbe has pointed out that one such territory is the domain of synthetic two-dimensional computer graphics, which, largely neglected after the early Macintosh era, has only recently begun to be revisited in the roughly two years since consumer PC's became capable of real-time, full-screen dynamic interaction.

I seek to build sophisticated cool media for interactive communication and personal expression. In doing so, I interpret McLuhan's specification for cool media — that they demand "completion by a participant" — quite literally. The notable property of cool media, I believe, is that they blur the distinctions we make between subject and object, enabling the completion of each by the other. An example of such a subject/object distinction is that between author and authored, the blurring of which, according to psychologist Mihalyi Csikszentmihalyi, is critical to the Zen-like experience of creative flow. Another such distinction is that between sender and recipient, to whose dissolution, wrote philosopher Georges Bataille, we owe the delight of communication itself. My aspiration for graduate study is to build — and understand the mechanisms of — systems that successfully blur these boundaries, enabling the joyful flow and authentic communication that are possible people engage, through a medium, in a transparent, continuous and transformative dialogue with themselves and others. My foremost measure of success for any medium I design begins, therefore, not with the question "for how long can I suspend my disbelief in it?" but with the questions: "for how long can I feel it to be a seamless extension of myself?" and "to what depth can I feel connected to another person through it?"

Instantly Knowable, Infinitely Masterable

The design of seamless extensions to ourselves is "non-trivial", as is the design of structures that can afford deep intercommunications. In attempting to surmount these difficulties, nevertheless, I have become convinced of a somewhat more tractable design implication which follows from them: that, ideally, such systems should be instantly knowable, yet infinitely masterable and expressible.

By "instantly knowable," I mean that no instructions or explanations ought to be necessary for novice use: the mechanisms of control are laid bare to the intuition, and thus the operation of such a system is self-revealing. By "infinitely masterable and expressible," on the other hand, I mean that the system has an inexhaustible expressive range, which, like the finest instruments, requires a lifetime to master — and furthermore, that this expressive range is wide enough that different users can develop unique styles or creative "voices" in that medium. This remarkable combination of attributes is commonly found in real-world expressive instruments but rarely — if ever — in computational ones. Consider, for example, how a three-year-old child can sit before a piano or a pencil and almost immediately intuit its basic principles of operation. Yet we take for granted that pianos and pencils are also rich enough media that the same child could spend the remainder of her life mastering them.

I know of no software tool which matches a piano or a pencil by these criteria. Instead, instant knowability and infinite expressibilityare all too often traded for one another: we slog through thick software manuals written "For Dummies," only to reach the limits of our tools' capabilities in homogenized outputs that, processed by the same filters and plug-ins, look and sound like everybody else's. At best, computers have only inverted the cultural logic of tool use, making our most general software tools difficult to learn and our specialized tools easy. I believe that the remedy for this, in the case of expressive instruments for visual communication, is a double-pronged research effort focused not only on the development of more "intuitive" interfaces, but also on the development of "intelligent" graphics models which know as much as possible about both themselves and the user. In the section that follows, I detail some of the questions and approaches I intend to pursue on the path towards knowable and expressive, intuitive and intelligent graphical experiences.

New Directions

  • Whole-body interactions. Many artists and designers have traded the direct control afforded by physical media for the flexibility, precision, transmissability and undo-ability furnished by computational tools. But this trade has not been without a cost: for the most part, computer interfaces have physically disconnected us from direct expression, and haven't taken advantage of the many physical heuristics we use to understand the world. We have gone from using our whole bodies directly to using a single index finger abstractly — and what use to feel like play has come to feel more and more like labor. In short, we now spend far too much time hunched over clumsy keyboards, poking at cryptic symbols with inarticulate mice, when we ought to be playing in a spacious and malleable world that conforms to our intuitive physical understanding and all manner of direct manipulations.

    I strongly feel that creative expression should be something that we do with our whole bodies — just as it was for millenia in the pre-computational era. It is my hope that, by designing and developing interfaces which create more fluid connections between our computational and physical environments, I can work to dissolve the modern boundaries between artists, tools and artifacts which gave rise to the term "interface" in the first place. Although the introduction of devices like pen-based tablets has been a good, if small beginning, I anticipate that the next great advances in direct-manipulation interfaces will inherit from the development of unique physical bridges to computation, using technologies like force-sensitive resistors, accelerometers, electric field sensors, inclinometers, and so forth. I've had some experience connecting software to physical sensors at Interval Research, but I would like to do more, and I haven't yet had the opportunity to meld these technologies with expressive, synthetic graphical environments.

  • Alternative vehicles for software delivery. I love two-dimensional graphic displays, but I dislike fifty-pound monitors — especially since even the tiniest two-bit LCD is a microworld bursting with interesting opportunities for expressive interactive display. I want to bring dynamic graphics off of the desktop and into my own body-space, towards engaging software for such small-screen devices as portable hand-held game machines, Tamagotchi-like keychain computers, and palmtop computers like the U.S. Robotics Palm Pilot. I came to feel at home in a low-resolution universe after I spend years, as an undergraduate research assistant, developing icons on a 32-by-32 pixel grid; now I wish to make these small spaces breathe with expressive interactivity.

  • Design for multiple users. Most image-making activities, for practical and historical reasons, have been solitary ventures. Even the most up-to-date commercial software tools assume and perpetuate this lonely mode of artistic endeavor — but this need not be so. The computer has made it possible for people to share and experience simultaneous visual communication, and evidence from recent networked improvisation systems such as Scott Snibbe's MotionPhone suggests that people truly enjoy creating in this way. The challenge is that the idioms and mechanics of collaborative image-making remain poorly understood. What are the formalisms of image and animation which, like the Blues in music, can provide strangers with idiomatic structures in which to improvise together? What are the mechanisms by which participants can engage in a visual dialogue with each other, learning and responding to each other's unique signatures? I seek to design systems for collaborative visual improvisation that permit multiple users — however abstractly their expressions are communicated — to develop sophisticated understandings of, and relationships to, each other.

  • Instruments as cybernetic systems. When sophisticated instruments are performed expertly, the boundary between human and machine dissolves, and we perceive only a single expressive system. How can the instrument level itself to the user naturally and gracefully, continuously but unobtrusively suggesting what can be done next? Can the parts of the interaction language the users do know intrinsically suggest the parts of the language they don't? As users engage in a dialogue with a medium, how can both user and software grow and change together as a coupled system?

Conclusion

It's still an open question as to whether black squiggles, properly deployed, could allow us to communicate directly with supernatural forces. In the meantime, it's my hope that by developing systems for creative visual communication, we may at least have a means for connecting to each other in the reality beyond verbal language. The questions and issues I describe above are only some of the many interesting hurdles that lie in the path of creating such a connection.

The design space of interactive abstract visual communication remains fundamentally unexplored. My goal is to derive an understanding of this space as well as create systems which embody the design principles I discover. The lessons learned from this pursuit may be generalizable to traditional software applications and potentially to all areas of human-computer interaction, transforming the design of our already-prevalent dynamic information graphics in ways that have not yet been brought to bear.

The research I have conducted thus far, and which I propose to continue, cannot be understood as located wholly within the domain of Human-Computer Interaction (HCI) research, or the domain of Computer Graphics (CG) research. Instead, the work I propose resides in the connection between these two areas of endeavor — an interdisciplinary niche which must necessarily assume a hyphenated acronym at best. It is my hope that, as a graduate student at the MIT Media Laboratory, I will be able to delve deeply into this niche, toward the design of intuitive, articulate, whole-body instruments for multimedia play and communication.

Golan Levin, 15 December 1997