iPad - The rise of the naturalistic user interface

It was fascinating to see Apple unveiling its new iPad hardware recently, and one of the things that caught my eye were the interfaces of the various apps that were demonstrated. They look different from apps on other platforms, and even from the equivalent apps on the iPhone. It seems to me as if there’s been a change to a more naturalistic style of user interfaces. Why is this, and what is it about the iPad that makes it suited to this kind of UI?

Form factor

The most obvious reason that this kind of interface works on the iPad is the form factor of the device itself. Think about it: an iPhone is a small (well, smallish) object which has no physical, real-world equivalent. It’s not supposed to “be” anything other than a phone. The iPad, on the other hand, is intended to mimic an actual object, or rather, a variety of objects: a book, a newspaper or an address book for example. The interface for each of these uses reflects this directly: the address book looks like a “little black book”, complete with tabs.

The iPhone uses some of the same metaphors - like the address book for contacts - but the representation is abstract, whereas the iPad UI is literal, using an actual image of a book. The abstraction is necessary on the iPhone in order for it to remain usable with a smaller screen. Compare the iPad and iPhone Contacts interfaces:

[caption id="attachment_682" width="300" caption="The realistic looking address book application on the iPad"][](http://www.ianvoyce.com/wp-content/uploads/2010/02/contacts_ipad_fullpage_small.png)[/caption]

[caption id="attachment_684" width="120" caption="iPhone contacts interface"][](http://www.ianvoyce.com/wp-content/uploads/2010/02/contacts_iphone_small.png)[/caption]

The abstract to realistic continuum has been explored extensively in the world of icons (explained brilliantly here), but less so in the interface as a whole. Obviously some apps don’t have any form of real world equivalent (think System Preferences.app), so they’re likely to stay in a more abstract form.

Display fidelity

Of course, it’s always been possible to represent real objects on the screen, but without a display capable of a decent resolution it probably won’t look real. So another enabler for naturalistic UI is display technology: the more pixels and colour depth the better when it comes to rendering physical things. Realism often relies on subtle lighting and rich textures to achieve the desired effect. Anyone who’s ever attempted to render wood grain using 8-bit colour will appreciate how much easier it is with a 32-bit 1024x768 display.

The need for display quality goes hand-in-hand with a need for more processing power to push the pixels. If the processing “oomph” required to render your interface with sufficient quality means that it’s not responsive enough, then you’ve failed by any measure. The iPad seems to have reached a sweet spot where display capabilities, raw power and form-factor meet to make naturalistic UIs a feasible approach.

Direct manipulation

Up until the relatively recent introduction of direct manipulation technologies like touch-senstive displays, there’s a been a forced physical separation between users and their computer in the form of a mouse and keyboard. A drag operation using a mouse involves disconnected, abstract operations like pointing and clicking, which bear no relation to the object being acted upon. As such, there’s little use in having an extremely realistic representation of something to interact with, because the suspension-of-disbelief has already been broken.

But in the world of touch-based devices, our interactions are much closer to the physical actions we’d make, so having a closer visual representation to go along with it is beneficial. For instance, turning a page in an eBook and actually seeing the page flick over seems consistent when using a natural gesture like flicking a finger across the screen. Otherwise if you had to click a corner of the page and drag it across in a poor imitation of the physical action the visual aspects can quickly start to feel superfluous.

Perhaps given the iPad’s larger interaction surface we can expect to see a wider and more expressive range of gestures supported?

History

Of course, naturalistic UI isn’t really a new thing. It’s in use in quite a few applications already, albeit mostly in desktop apps, where capable hardware is more readily available. In fact, there was much a-twittering (not all of it from Will Shipley) about the similarity between the demo’d iPad apps and the interface of the fantastic Mac application Delicious Library. I’m sure of course - how do they say it in the movie credits - the similarity is entirely coincidental…

[caption id="attachment_733" align="alignnone" width="293" caption="Delicious Library UI"][/caption]

[caption id="attachment_734" align="alignnone" width="150" caption="iBooks UI"][/caption]

One of the first widespread uses of naturalistic UI elements was in software synths. Guys like Propellerheads and Native Instruments set about creating exact software duplicates of well-known and well-loved bits of mid-80’s audio hardware. The UIs included virtual versions of a huge number of physical elements that had never been used before: including things like sliders, LEDs, toggle buttons, rotary knobs and rocker switches. These interface elements were probably first seen in software-based mixing desks, but the techniques used to make them feel real have gone on to be used in many different ways. Even the lowly push button in both Windows and MacOS got an upgrade by application of some of the 3D effects that they originated.

Where Now?

It will be interesting to see what other types of app are given the iPad realism makeover. It certainly looks like all of the standard Apple apps are being changed, including Notes, iCal, iBooks and Contacts, which were shown in the launch presentation, but how will 3rd-party developers manage? It undoubtedly takes extra effort to make something look convincing when it can be compared to the physical object itself.

Apple has previously managed to get a good level of consistency in its abstract UIs by providing the various interface elements ready-made, but will that approach scale to this kind of UI? No doubt the developers who take great pride in the interfaces now will continue to strive to create apps that sit comfortably alongside Apple’s own, but perhaps there’ll just be a more obvious visual distinction between those guys and the less polished application user interfaces than there has been in the past.