In Interaction Design Sketchbook, Bill Verplank brilliantly describes how computer interfaces have developed exactly in the opposite way than humans. Indeed we humans we start to interact with the world, when babies, only in an enactive way, unable to speak and to recognise images. Then we develop an iconic thinking, recognising images and finally we learn how to read and write. While digital systems have developed following exactly the opposite path.
Piaget described three stages of learning. We are born with ENACTIVE or kinesthetic knowledge; we know how to grasp and suck. At a certain age we pay more attention to how things look; our ICONIC thinking is mistaken for example by a tall glass as “more”. Only at a certain age do we understand conservation; then we are ready for SYMBOLIC thinking. […]
The development of human-computer interfaces has followed the opposite path. The first interactive computers used teletypes (TTY) and the style of interaction was a dialog of symbols; I type and the computer types back at me. With CRTs we first emulated the old style with “glass teletypes” but with the invention of mouse and bit-map display, the iconic graphical “direct manipulation” interface became the dominant style. This progression suggests that the next stage is enactive interfaces, more suited to expressive musical interaction than with pictures or symbols. One possibility is Ishii’s Tangible User Interfaces (TUI).
Bill Verplank, Interaction Design Sketchbook