Freely connected

How will we live if virtual and physical reality are inextricably linked with one another? If we can connect all the objects in our home with one another with an app and give them new functions as interfaces? Product designer Valentin Heun addresses precisely these questions. A research associate in the “Fluid Interfaces Group” in the Media Lab at Massachusetts Institute of Technology (MIT), he and his colleagues are working on transferring what is possible in the digital sphere to the real physical world. Heun is the inventor of Reality Editor, an app that makes it possible to digitally link the functionalities of everyday objects with one another or give them totally different functions. Using the Reality Editor, furniture and home accessories such as lamps and chairs become switches and sensors – for as long as and only for as long as the user wishes. Stefan Carsten spoke to Heun about the limits of human influence on living environments that are becoming ever “smarter” and what the future will be like if we can control the physical space around us just as easily as if it were part of the Internet.

Stefan Carsten: What has your Reality Editor got to do with the smart home of the future?

Valentin Heun: Reality Editor opens a door to a future in which we can link up what we are able to do digitally with physical reality. Today we still differentiate between physical objects on the one hand and virtualization on the other. But we’ve now reached a point in the technological development process where it is becoming more and more difficult to maintain this division. I’m convinced that in the next 10 to 20 years this division will completely disappear in our own four walls as well.

What does your vision actually involve?

Valentin Heun: Like a sort of digital screwdriver, Reality Editor allows you to control the behavior of products and appliances, and not the other way round. You can then allocate the rooms and objects in your own house or apartment behavior patterns. That is almost exactly the opposite of what is the case now, because at the moment we are still lacking user interfaces that enable extensive and self-confident operation of the Internet of Things. For this reason, right now more and more devices are being launched that learn through our behavior and can then automatically make decisions for us. My vision, on the other hand, is for the users themselves to be able to decide how they use and operate devices. But this flexible approach is not yet possible today because we just don’t have the technology for it.

Digital screwdriver: Reality Editor links objects and gives them new properties. For example, tables and doors become light switches. Photo © MIT Fluid Interfaces Group

In future will I have to be part of the digital elite to use your digital screwdriver?

Valentin Heun: No, my vision is based on the basic principles of the World Wide Web. Tim Berners-Lee developed an interface that is so simple that everyone can link up with one another. Moreover, it’s an open standard, so everyone can use it. Nowadays we can still open first-generation websites in our Web browser. From a computer point of view that’s almost a miracle. Reality Editor is similar. It enables all an object’s functions to be broken down into their individual components. This way these functions can then be printed out as simple numbers. Simple numbers are the most stable standard we can create for data communication. This means that in 50 or 100 years things will still be able to be made that are compatible with things that have already been around for 50 or 100 years.

What will tomorrow’s house or apartment be like?

Valentin Heun: In tomorrow’s house I see more computers than you can possibly imagine today, but on the other hand considerably few computers that people can see. There will be more interaction with the immediate environment again. But it will always be supported by computers, which will be new in shape and have nothing in common with our present-day ideas of a desktop computer or smartphone. They will be integrated, cordless and out of sight, in our lamps, switches, floors, doors and so on, supporting or linking these real things’ functions. I don’t think the current trend of fitting touchscreens everywhere is a particularly good solution. A touchscreen is actually a very inadequate interface for everyday interaction needs. By way of example: In my research I defined three interaction modes for getting to know objects or operating things. The first is that we want to learn what sort of object we’re actually dealing with. The second is that it is adapted to one’s own needs. And the third is that we just want to operate it. A digital environment is particularly suitable for the first two modes, videos and interactive texts, for example, can aid learning, and functions are far easier to set with a screen and added to at a later point in time. But when it is a matter of operating devices, with a toaster all you want to do is press the button, switch the light on with a light switch, turn the radio up or turn on the ignition switch in a car. For these things, their being digital and operated via a touchscreen makes no sense.

So this means the spaces in which I live, work, and consume things will continue to be defined by physical objects, but also by the digital connections between them. Might that not all sometimes be very complex?

Valentin Heun: That’s an important question and one which has yet to be solved. Just imagine that while you are away someone changes the digital connections between specific objects that you defined, such that you lose control over the space and can no longer understand the connections between the devices. Incidentally, this is a problem with all augmented reality and Internet of Things applications. Our built environments are static, actually even set in stone and almost always correspond with our cultural ideas of these spaces. When these are interwoven with dynamic, virtual realities, problems can arise. In our communication with these devices we need feedback loops, so that the mixer (with my finger in it) doesn’t switch on when somebody turns the light on. That will account for a considerable part of my continued research work.

Thanks to the freely accessible programming code, the app can be adapted to all kinds of users and products. Photo © MIT Fluid Interfaces Group

Why should I actually want all this? Will it make our everyday life more pleasant?

Valentin Heun: It will reduce the complexity of the digital environment. It is primarily about increasing efficiency, and comfort. To this end I can define sequences for activities. A quarter of an hour before the alarm clock rings the house is brought to life and the room temperature increased. When I then get up, the sensor in the bed registers it. The water for my shower is heated, the espresso machine increases pressure, the toaster switches on and in winter the car warms itself up. The user can define and set this sort of scenario. And if that’s not the way things should be, the user can just cut the connections or redefine them.

Through the developments on the Internet of Things all this is now within reach.

Valentin Heun: Well, the current trends on the so-called “Internet of Things” actually have nothing to do with the Internet, but are just direct data lines from A to B, where B is a company, on which you become increasingly dependent. Instead of the Internet of Things one ought to talk of the “cloud service of things”. This is a very important difference for the physical world. If, for example, I don’t like a site such as Facebook, I just don’t visit it anymore. But if a heating system, a locking system or a sprinkler system are controlled via it, I would have to give my entire house a makeover if I no longer liked the provider. I am highly dependent on this system provider and might possibly have to pay for services that were previously free or accept changes without having a say in the matter. For consumers the future we are moving towards is very inadequate.

What is your answer to this dystopic trend?

Valentin Heun: The basis of Reality Editor is an open-source system called “Open Hybrid”. It has open-source licenses, so that it will always be open source. All data are based on the most-used open Internet standards and the software code is easy to change and can be viewed at any time. The data are never sent to an external service, rather all the data for interaction with the object are stored in the object itself. The objects contact other objects directly, without data being transferred half way round the world. If you buy things they are your own private things, you want to create a world for yourself in which you live, and you don’t want to be controlled by an external force. That’s the principle of the system.

The smartphone networks in the background, but the central operating element remains the button or switch, i.e. physical objects. Photo © MIT Fluid Interfaces Group
Every link can be playfully reversed or changed at any time. Photo © MIT Fluid Interfaces Group

Are you critical of current activities in the smart home field?

Valentin Heun: No, in fact I’m pretty excited by the things we’re already using. With my criticism, however, I would like to help to consciously shape our future such that we have sufficient space to continue realizing our ideas. Tim Berners-Lee opened up a space like that with the World Wide Web, and we need something similar for the Internet of Things. I would like more companies to pursue basic thoughts such as these. In the past two years I attended several conferences on the topic in San Francisco. The number of lectures on the opportunities offered by closed platforms and systems is astonishing. But at the same time there are extremely few ideas about what we can do with these platforms, because since they are closed no one bothers about the things that really benefit a client. I would like these systems to be opened and tools to emerge that enable products’ users and designers to participate. At the moment Reality Editor only works in a private network and not via the Internet. But open systems for the Internet of Things that are not centrally organized are already being developed. These will help us to design a better digital future of things. Then the functions of Reality Editor will also be able to be connected via the Internet of Things, without the client having to supply data.

And how does the business model work?

Valentin Heun: If you look at the prevailing business model for the Internet of Things for private consumers it’s an attempt to transport a model that works on the Internet to the real world. But in the real world products are developed, produced, and sold. That gives the company a particular value. On the Internet it is valued for placing advertising or for data that are sold. I don’t think a model financed by advertising is the right one. I find working with companies that produce real products and goods much more interesting when it comes to developing new business models for new markets. To this end I have developed an open platform with the accompanying tools to enable product designers in these companies to create value added for their products. This value added results from connecting physical and virtual worlds. Let’s say a switch in this example costs 10 euros. But if the switch becomes an integral part of a networked world, there is a network effect. That means that the sum total of all the components supports the value of an individual one. As such the same switch can represent a value of 50 euros, with genuine, real value added.

Video © Valentin Heun