| boris müller: Life Blog Projects Teaching & Research

Observations on Interaction Design – Part I: Misconceptions

24. September 2014

A couple of weeks ago, I met up with friends and family for a beer. We had a chat about this and that. We even talked about physics and I got the opportunity to explain my naive understanding of Heisenberg’s Uncertainty Principle. We also talked about work. Some distant acquaintance asked me what I was actually doing. I started talking about the role of digital technology in our everyday life, the iPhone, data visualisation and Facebook. And I realised that it was easier to explain the Uncertainty Principle than to explain interaction design.

For someone who is working in the area of interaction design for some time now – and who is teaching interaction design at university – this is a bit frustrating.

Everyone seems to work on a computer nowadays. There are millions of smartphones out there. One seventh of the world population is on Facebook. So if you change an element on the Facebook webpage, one billion people have to deal with it. It is clear that interaction design has a huge impact on the way we use and perceive digital technology. So why the hell is it so hard to explain what we do?

I will try to address this question in a series of blog postings. Today: Misconceptions.

I think a major reason for the fact that interaction design is difficult to understand is that a lot of people have a completely wrong idea about design. Not only about interaction design but about design in general. And this is partially the fault of the design community.

There is an odd understanding of design in our society. Design is an integral part of the industrial age. Almost all consumer goods are designed. In that respect, both a luxury car and a plastic water bottle are design objects. Everyone knows this – but no one is really aware of it.

Here is a related, amusing anecdote about a friend and former colleague of mine: He is a leading designer in the area of transportation design. He has designed trams for dozens of cities. He know them inside out, knows about platforms, technology and the production constrains. He knows how people use trams, how they behave in them and how they fit visually and aesthetically into the urban landscape. So – in short – he is a real design expert on trams. At one point, a public transport authority rang him up. They told him that they would like to participate in an upcoming design festival so they wanted to give him the opportunity to come up with a »design tram«. It would be a great opportunity for him – if he wanted, he could go wild. His reply was very clear – but not fit for publication. He is still furious when he tells this story.

When there is talk of »design objects« in the media, it usually refers to extravagant luxury items. Geometric chairs, oddly shaped shelves, golden lamps in the shape of a machine gun. Design is considered to be »different«, exclusive, expensive and it’s main function is to convey a high social status. Design is synonymous with style.

I have an ambiguous relationship with style. In design, without style everything is lost. But style alone is fluff. For me, style is a vehicle. It is a container for concepts, ideas and solutions. It’s a positive Trojan Horse1. Design in general – not only interaction design – is a highly complex affair. The fact that the public likes to focus on style is highly problematic and leads to great misconceptions2. Design is about many things – it’s not only about style.

One of the things that really struck me while reading the recent biography of Steve Jobs by Walter Isaacson is that the author is completely unaware of interaction design. The way Isaacson talks about design is very much about Jony Ive and Apple’s product design achievements – which are undeniably great. But it is surprising that Isaacson does not even touch the surface of interaction design3. Obviously, for Isaacson, design is style. It is regrettable to find such a design understanding in a biography about the founder of a company who fundamentally defined the way we interact with digital technology.

It is very difficult to explain interaction design to someone who has such an approach to design – because interaction design is very different from »pure« style. Interaction design is understandable and inclusive. It’s inexpensive and it is mundane – bordering on the invisible. Interaction design is consistent, egalitarian and profound. And it is highly fascinating.

By principle, interaction design cannot be exclusive. There is no haute couture in interaction design4. This partly lies in the nature of software. If you can replicate something with no effort and no cost in any volume – how can it be exclusive?

There are exclusive hardware products like the Vertu phone range. But they are decidedly not about interaction design. You can buy a phone by Vertu for over 10.000 € – and you get an exclusive, ostentatious and »different« piece of hardware. But the funny thing is that the interaction design will be the same. That’s why Vertu only shows phones on their website with blank screens. You switch them on and – hey – it’s Android! Or – if you are lucky – Symbian5.

As John Gruber has pointed out in his posting about the Apple Watch, digital products are in a way quite egalitarian. The iPhone itself has quite a price tag – but for ten times the money you won’t get a better phone. There are no  better iPhones out there as the current models. However, Gruber points out that »Apple Watch changes this dynamic.«.

I don’t quite agree with that. I have not seen and tried the Apple Watch. But I am quite sure that the golden Apple Watch Edition is not going to be a better watch. It’s just going to be a more expensive one. The interaction design will be the same – no matter if the casing is aluminium, steel or gold. So the quality of use will be the same for all Apple Watches. This move might work out well for Apple. However, as an interaction designer I feel a bit uncomfortable about the fact that Apple is now playing the stupid style game.

I started with Heisenberg and I have arrived at the Apple Watch. Does this help you to understand what interaction designers are doing? Probably not. But it provides context for understanding interaction design better.

Interaction design is not a style-driven design discipline. And as long as the general perception of design is focused on style, it will be difficult to explain what interaction design actually is.  However, I will have a go at it in one of my next postings.

(To be continued)

  1. Style alone is then – logically – a Trojan Rabbit

  2. Misconceptions that we designers are often guilty of invoking.  

  3. The term »interaction« appears three times in the biography. It is only used once for the interaction with digital technology. There is one anecdote about the design of iDVD that touches on interaction design. That’s it. 

  4. I sometimes regret this. 

  5. Vertu has concierge service – which will probably change the user experience of the phone. But I have not tried it – for obvious reasons. 

A Map is not a Service

22. September 2014

When developing new apps or new software products, designers often refer to the notion of »services«. The idea is that the software should have a clear purpose and help the users to complete a specific task. Conceptual and technical complexities should be hidden in order to give the users a pleasant and frictionless user experience. This works especially well when the parameters and the possible variations, that are taken care of in the background, do not have a great influence on the quality of the user experience.

Consider taking a cab. You enter the taxi, state your destination and relax. At some point the cab reaches its destination, you pay and you leave the car. All the complexities – building a car, owning a car, driving a car, navigating through the city – are hidden from you. In most cities, taking a cab is a good service.

A service is like a black box. You specify your problem – and you get a solution.

There are many situations, where such a service-oriented approach in software design is absolutely preferable. Overwhelming options and dependencies can frustrate users and the notion of delivering clear and simple services through software is fine. But in order to make a service work, you have to trust it.

Maps and data visualisations have a completely different approach. They do not offer a service or a simple solution. They show the complexity of a situation or an issue – but they can enable the users to relate to this complexity. They contain much more information than the users currently need. But this additional information presents a context for understanding and thus provides the user with a scope of possible actions. The user has to generate the solution for him- or herself.

When you are using them, maps and visualisations are essentially about decision making. Using being the operative word here. There are many intriguing visualisations and many captivating maps out there. The National Maps of Switzerland are probably the most beautiful maps ever1. I could spend hours just looking at them, imagining the mountains, glaciers and the valleys, enjoying the sheer beauty of the maps. But it makes a difference if you are warm and comfy at home and enjoying the map – or if you are near the Cima di Gagnone, lost in the clouds and trying to figure out a way over the ridge 2. In a moment like this, a map becomes a vital instrument for decision making. Using it means that you literally decide your next steps based on the interpretation of the map.

The same is true for good data visualisation. There are many beautiful and intriguing data visualisations out there. I love to explore them, discover relationships, learn new things and just enjoy the playfulness of the interaction. But their real power enfolds when you have to figure something out and act on it.

The gold standard for data visualisation is informed decision making. To have an interactive »map« that enables you to judge the situation, that displays possible options and that allows you to create a plan for action.

It takes more effort to interpret and to understand data visualisations than services. Maps and data visualisations are not necessarily about reducing complexity. They make complexities readable and allow the users to relate to the data, generate insights and make decisions. Visualisations can be empowering as they leave the interpretation of the data to the user3.

These two different approaches are actually not totally divergent. Digital maps are visualisations, turn-by-turn navigation is a service. So both perspectives can be incorporated into one product. But conceptually they are very different.

The comparison of services and visualisations highlights a fundamental challenge in interaction- and interface design. When do you need a simple, uncomplicated solution that is easy to use but which internal decisions are opaque to the users? And when is it better to provide users with an interface or visualisation that is visually complex and contains a lot of options – but that is transparent and enables understanding?

As always, design is about trade-offs. But it should be a conscious decision. Not everything needs to be complex – but at the same time not everything needs to be simple.

  1. The maps are available online in all scales. 

  2. True story. 

  3. Deutungshoheit for my German readers. 

A Posting about Sulphur

18. September 2014

One of my all time favourite interactive museum installations is floating.numbers by Art+Com1. I remember being in the Jewish Museum in Berlin and playing with the installation for a long time. It was an absolutely delightful experience. The spacial appearance was impressive – a nine meter river of numbers. The visual structure was beautiful to behold and I loved the fact that I could not just select any number. You had to wait until a number swam to the surface and turned yellow. There was no pattern – it was up to chance which number would appear next. It was indeed like fishing. If you wanted to know the meaning of the number 13 you simply had to wait for it to appear. But in the meantime you could learn a lot about the significance of other numbers. It was quite addictive in a way that you were curious which number would come up next and what it would mean.

The amazing thing about floating.numbers is that it was decidedly not a computer terminal. Remember – this was 10 years ago. Back then (and sometimes still today), computers in museums were understood as a repository for all the things the curators could not get into the exhibition space. They were used as a tool for the exploration of the collection, enabling the visitors to delve deeper into the exhibition and into related materials. Which they usually didn’t.

There is nothing wrong with the »terminal approach« in a sense that it provides an opportunity to learn more about objects and themes of the exhibition. But it relies on the notion that you want to learn more about an object, that you know which object you want and what to ask it. It works for informed visitors who clearly know what they are doing. But in most cases it does not work very well.

Imagine visiting a museum just to find an empty room and a desk with a clerk who politely asks you what you want to see – he could get any object and tell you a lot about it. This would be an interesting but very different approach2 to the spatial display you usually encounter in museums. But I doubt it would work. Most visitors to museums don’t know what they want to see.

Floating.numbers turned the »terminal approach« upside down. Instead of providing the users with an interface that would allow them to select any number they liked and learn more about it, the visitors were presented with a flow of numbers of which occasionally a single number would disengage, become active and would – on touch – reveal its meaning. The users were confronted with a serendipitous machine that would present them with random information in a delightful way.

What has a ten year old interactive installation to do with current data visualisations? I believe one or two things.

Many current data visualisations are fancy »terminals«. They assume that you want to learn more about an subject, that you know which subject you want and what to ask it. Again – there is nothing wrong with this approach. But I believe it limits the potential of data visualisations. A lot of interactive visualisations are missing an incentive to dig deeper and immerse yourself in the data. When confronted with a thousand options – where should you start? If you start somewhere – how do you know it is interesting? How can you make sure to find something relevant?

There is an interesting space between full – but intimidating – »exploreability« of data and linear storytelling. Floating.numbers is neither a data visualisation nor a story. It is more a collection of facts, anecdotes and historical fragments. It does not go very deep – but the installation manages to pull you in. This has not only to do with the aesthetics but with the way the data is randomly organised into small, visually appealing units that are informative and entertaining. Imagine having a »deep« data visualisation that would use such small units to pull you into the visualisation and the data.

I very much like the concept of the Information Flaneur3. The comparison of a flaneur in a city and information seekers in digital information spaces is compelling. The flaneur is attracted to the events, signs and narrative fragments of the urban landscape. He or she uses these signs to decide what is important and entertaining and connected.

We need more signals in data visualisation that indicate interesting relationships and observations. These signals can be editorial nodes that point the users to possible starting points for exploring the data. The signals do not have to be a full story – they can be a surprising fact, a strange relationship or an editorial observation. Their main role is to create serendipity, invoke curiosity and to pull the user into the data.

Take trade data – a seemingly boring subject. But did you know that Austria is the worlds largest exporter of handguns? Followed by Germany and Brazil? Or that Canada is the main exporter of sulphur? And Zambia the main importer? These are curious facts and they immediately invoke a number of questions. Who builds handguns in Austria? Why are they so successful? Isn’t Austria supposed to be harmless? And what is Zambia doing with all the sulphur? (This took me a while to figure out. Sulphuric acid is used for copper mining. Copper is the main export of Zambia – over 72% of its exports. I assume the sulphur is used to create acid – with terrible consequences.)4  Data visualisation itself cannot answer all these questions – but it should raise them.

The idea of telling stories with data was discussed a lot in the last few months. I strongly believe that storytelling with data is an integral part of data visualisation. But it does not always have to be a long, conclusive narrative. Little anecdotes, fragments, questions and contradictions that attract attention and invoke the curiosity of the users are quite enough. Like little orange numbers that quickly disappear if you can’t catch them. Or like blog titles with »sulphur« in it.

  1. Team: Joachim Sauter, Dennis Paul and Patrick Kochlik; 2004 

  2. Actually – the Kupferstichkabinett (Museum of Prints and Drawings) in Berlin works a bit like that. 

  3. Marian Dörk, Sheelagh Carpendale, Carey Williamson. The information flaneur: a fresh look at information seeking. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011 

  4. OK – long footnote: my starting point for the explorations was the excellent Observatory of Economic Complexity by Alexander Simoes et al. at the MIT Media Lab. They even have a little serendipitous element – you always get a random product when you first select the product category. This is how I found out about the Handguns. The Observatory has two sulphurs: »sulphur« and »sulfur«. »Sulphur« is HS 2503  Sulphur (Crude or Unrefined) in Mineral Products, »sulfur« is HS 2802 Sulphur (Sublimed or Precipitated; Colloidal Sulphur) in Chemicals. In both categories, Canada is the main exporter. However, Zambia did not even show up as an importer in the Observatory. But I checked the data on the UN Comtrade Database website. Here, Zambia is named as main importer for HS 2802 Sulphur. It’s tricky to link to the database, so I copied the sorted table. The Observatory relies on UN Comtrade data – but also uses the BACI database – which is a harmonisation of the UN Comtrade data. All this stuff is really interesting – there will be a follow-up posting on this matter! 

Cultural Image-Making

17. September 2014

In the wonderful book Design for Information1 by Isabel Meirelles I came across a quote from Ben Shneiderman:

»Like Galileo’s telescope (1564–1642), Hooke’s microscope (1635–1703), or Roentgen’s x-rays (1845–1923), new information analysis tools are creating visualizations of never before seen structures. Jupiter’s moon, plant cells, and skeletons of living creatures were all revealed by previous technologies. Today, new network science concepts and analysis tools are making isolated groups, influential participants, and community structures visible in ways never before possible.«

I very much agree with this vision. As more and more data is collected and made available, the creation and the interpretation of  data visualisations is about to become an essential skill that will help to understand the complexities of the modern world. As Shneiderman says – these visualisations will allow us to see structures that would otherwise remain invisible.

While I agree with the vision, I am slightly uncomfortable with an underlying notion of the above quote. The comparison with telescopes, x-ray and microscopes implies that data visualisation is mainly a technical problem – a problem of resolution. Optical devices magnify small or distant objects. Lenses and mirrors direct and focus light waves and are thus revealing existing physical structures like plant cells or galaxies2. The process of creating optical magnifications is essentially deterministic3. The process of creating data visualisations is not.

This view is also expressed very well in the excellent posting »Worlds, not stories« by Moritz Stefaner. Moritz is referring to the similarities between data journalism and photo journalism:

»This is why I see data visualization as sort of a new photojournalism — a highly editorial activity using a deceivingly objective-looking apparatus.«

I would like to take this idea even further. Data visualisations are essentially cultural images. There is no definite way how to transform abstract data into an image. Even fairly obvious visual solutions like a bar chart are just one option how to represent data. The transformation of abstract data into a concrete form is a creative act. It’s design.

Take personal data – your bank balance, your emails, your travel itineraries, your school grades, your Instagram stream, your health data. If you were trying to create a visualisation of your digital self – how would it look? How would your public data image be different from your private one? We don’t know. We don’t know because we have not decided yet which questions we would like to ask the data. We don’t know because this visualisation does not yet exist. We have the data and the algorithms – but so far we don´t have a meaningful and adequate design.

I like the comparison of data visualisation with photography in terms of the role of the designer / visualizer. As Moritz puts it: »the most important issues today are not photographable«. So we need new image-makers that help scientists and journalists to convey complex information. However, I find the comparison problematic as it suggests that there is some kind of apparatus that allows us to take a snapshot of »reality«. As if we just have to find the right angle and the right spot in order to create an image.4

Data visualisation is about finding appropriate visual encodings that enable viewers to decode and interpret the generated image. The appropriateness of the visual encoding depends very strongly on the data that is depicted. Comparing this process to image-making with optical devices like microscopes, telescopes or photo cameras suggests that we are mainly dealing with a technical problem. It suggests that we just need better virtual lenses in order to see data structures more clearly.

Don’t get me wrong – it is true that the ability to record, store and process large amounts of data is only possible due to the great progress in computing that we are witnessing right now. Furthermore, the development of new visualisation algorithms provides us with new ways to manage the shape and relationship of visual entities. All these elements are great achievements in themselves. But you cannot derive meaning from technology.

In a way, data is like the script of a play. The words are all written down – but it is up to the director to decide how the words are spoken, who speaks them and how the environment of the play is set.

The transformation of abstract data into an image requires creativity. It is essentially a design problem. And I believe it is one of the most interesting and most relevant design problems in the years to come.

  1. Meirelles, 2013. Design for Information. Rockport Publishers, p. 63 

  2. It’s interesting that the further we look out or look in – the less the notion of amplification works. The borders of our scientific knowledge defy the experiences of our everyday life. 

  3. That is the magnification of form – not of colour. It is quite surprising that in many scientific images the colours do not represent the real appearance. 

  4. I have a lot of friends who are professional photographers and I will probably get a thrashing for this paragraph. But I believe there is a widespread naive understanding of photography. Here, I am referring to this understanding. The naive view of photography might be the subject for another posting. 

A Blog.

16. September 2014

So. A blog. 10 years late.1

Ten years ago blogging was already quite established – but still young enough to be considered cool. I remember that 2002 – during my time at the Interaction Design Institute Ivrea – blogs were still quite a thing. And when I started teaching in Potsdam, it was kind of expected of a young Interaction Design professor. But I didn’t.

So what has changed in the last 10 years? Quite a bit. When the Interface Design programme in Potsdam was launched in 2003, it was a challenge to explain what interface and interaction design really are. The iPod was released in 2001 – and it was a hallmark of good interaction design. There were no iPhones, no Tablets and no Google Glass. Mac OS X was quite new and iOS pretty far away. Microsoft dominated the market, young people still used email, social networking was based on SMS and not on Facebook and twitter. I hated family parties where I was constantly asked what I was actually doing.2

In 2013 things are different. Digital technology has permeated our society. There are hardly any jobs or workflows that are not influenced by digital technology. One seventh of the world population is on Facebook. There are billions of small computers in the world that for some reason we still call »phones«. We play, shop, date, work, talk, read (etc.) using digital technology. This does not mean that interaction design is widely understood and recognised as one of the most important design disciplines of our time. But it is widely acknowledged that digital technology has a dramatic cultural presence in our global society. However, the idea that the interaction with digital technology can be understood and shaped is not very prevalent. It is a bit like acknowledging that human beings trade, work, pay taxes and acquire wealth – without realising that there is a discipline called »economics«.

So the point of this blog is to reflect on my activities as an interaction designer. Being a practitioner, a researcher and a teacher I talk a lot about design – but so far I have written very little. So I would like to thank Microsoft Research for inviting me over to Cambridge and give me the opportunity to actually sit down and write.

The blog posts are not scientific – and I don’t intend them to be. They are short essays that deal with observations and reflections on design. 10 years ago, a blog seemed like a good format for pinning down ideas and publish them. It still is.

  1. Now that blogs are officially dead, I am starting one. But I am a 40-something with kids – so I think that’s OK. 

  2. I still don’t like family parties. But hopefully this blog will clear things up. 

« Older postings | | Recent postings »

RSS Feed | Twitter @borism | Impressum