Microsoft’s Mike Pell on Smart Information & Data, Plus Blowing up the HCD Status Quo
Interview with Mike Pell
We’re talking design and data with Microsoft Garage’s Envisoneer and Design Lead, Mike Pell.
Following on from his first book, ‘Envisioning Holograms’, Pell has just released his second book called ‘The Age of Smart Information’ that envisions how AI and XR will fundamentally change the nature of information and our relationship with it.
We discuss everything from ditching the over-complicated ’36-step’ human centred design process, forgoing the faceless word of ‘user’, to how information will evolve from dead pixels on a page to a new way to learn, be inspired and informed by data.
For more information on Mike Pell’s The Age of Smart Information visit https://theageofsmartinformation.com/
Douglas Nicol: Welcome to this Smart dust podcast from South by Southwest in Austin, Texas. As always, I’m joined by my colleague, Mr. Nick Abraham’s. Nick.
Nick Abrahams: Hello, everyone.
Douglas Nicol: Today, we’re very excited to be joined by M. Pell from Microsoft, author of the Age of Smart Information, you’re very welcome to Smart Dust. Having read your book, seen you present, and seeing some of your work, I feel like you’re an artist, and a programmer, and a Microsoft guy.
Mike Pell: Yeah. I don’t think about it that carefully. But no, it’s true. I mean, I’ve always loved design and art, I went to school to study fine art, discovered programming in my first semester, figured I can make more money doing that and stuck to it. But I’ve always been able to do both and for me, that was both my joy, and the happiness, but also my superpower, is being able to be like left brain, right brain, go back and forth. And so I think as a designer, creative person, the ability to understand technology is a huge advantage, not because you necessarily have to code yourself, but if you get it, if you get your medium that you’re working in, you’re going to make better decisions, you’re going to know what’s possible, what’s not possible, not be at the mercy of other people to execute your vision.
Mike Pell: And on the flip side, people who are more technical or customer focused, knowing a little bit about design thinking and how designers and creatives go about figuring out how to go forward is hugely helpful. I mean, some of the most creative people I’ve ever met in my career, were working as testers as engineers, and you don’t think of engineers as like creative, but they’re super creative, they’re problem solvers, that’s what they do all day long. It’s just a different kind. And so I found that they’ve been hugely interested in the design thinking, and once you give them a little bit of exposure to it, they want to know everything about it, because it helps them with their job.
Douglas Nicol: Yeah. And you’ve got some quite interesting points of view on things like human centered design, and the many processes that are put forward as best practice [would 00:02:06] and in a sense, you’re saying that a lot of those processes are broken.
Mike Pell: Yeah, throw them all out.
Douglas Nicol: Talk to me about how it should be you’ve come up with your own methodology.
Mike Pell: Yeah. So I developed fast design in the garage a few years back, because we bring lots of teams in to just get out of their environment, do different things. And we love to talk about, you know, people on design thinking, I always tell people, please don’t ever use the word user around me, because there’s no such thing as a user, right? User is a faceless, you know, not even a person, like just call people, people, right? There’s a person behind all of this. And so once you start with that mindset that we’re designing for people, the most important thing is not to focus on the preciousness of your design methodology that you learned in school, or the way that your agency goes about doing it, or the way that your firm, you know, has plotted out this very careful 36 step process, throw all that out in your mind first. And imagine the optimal outcome of something, like the absolute best way that it could turn out and start there, and try to figure out step by step, how can we get there.
Mike Pell: And as you start doing that, you’ll find that there’s a big leap that you need to make somehow, we need a different set of people to do this, we have to get some new technology, we have to invent some new technology, we have to undo our process, we have to undo our thinking like whatever it is. I found that just trying to get from A to B, as quickly as possible is the best way to make those leaps forward. And just like let all the constraints go, let all the the doubting and the well, we’ve never done this before, you can’t do that. Just forget it, right? Just forget all about your process, and what school has taught you or even prior experience, and just think of the optimal, and then try to get there. And invariably, you won’t get there perfectly, but you’ll get close if you try hard enough.
Nick Abrahams: If we think about what that would look like for an organization. So we’ve got, you know, you’re challenging people that are designed thinking, think of it from the human point of view. Are you a fan of sort of laying minimum viable product type, you know, let’s do the minimum amount, see if it works, learn from that, iterate, iterate, iterate, is that part of the process?
Mike Pell: Yeah, I think in general, lean and agile like that, that’s fine. I don’t like the dogma associated with it, everybody has to stand up in a stairwell for five minutes like that. That’s just ridiculous. But the thought is a good thought like, let’s go fast, let’s figure out the right way, let’s pay attention to our customers, let’s test it, sure iterate, but I’m more a fan of moving forward quickly, right? And like not moving forward in a reckless way. But moving forward in a very focused way to get to that optimal solution, so that you’re listening to the people who are going to use your product or service or, in your case, perhaps the end result, like some kind of artifact you may produce. But you know, a lot of it … You know, process I’m not real big on process but I love being able to do things that get you from A to B quickly.
Douglas Nicol: Yeah. What do you think stands in the way for a lot of larger organizations who claim to be working in an agile way? What are the things that get in the way of truly being fast and getting to really great solutions quickly?
Mike Pell: I think it’s the same old story, right? It’s the DNA of whatever organization prevents them from thinking in a new way, because of short term profit, management didn’t set it up that way, the founder would have never done that, we can’t do that, we don’t have permission. You’re not going to be working at that place very long if you still have that attitude, like we all have to change, like none of us can be the same person we were five years ago, right? Like, you just couldn’t do what you’re doing. And the same thing as with organizations, right? Whether it’s school or company, or startup doesn’t matter, the organization is only in place to support the work of the people to get to that optimal outcome.
Mike Pell: And so if you have to throw out the way that you act and think, to go faster to get … You know, because you’re going to keep doing the path of least resistance, I suppose, right? And so we’re all guilty of that in corporations and medium sized companies. But that’s not going to help because the world has changed. And the kids coming out of school are like 50 times smarter than you, and they are all interested in changing things for the better. And so we can’t be sort of clinging to the old way of doing things or thinking when we have to integrate that workforce.
Nick Abrahams: So you here at South by Southwest to launch your latest book, the Age of Smart Information, congratulations, it is a fantastic read.
Mike Pell: Thank you.
Nick Abrahams: And a very well received real thought leadership stuff, it’s a sort of bold vision, as to our future and a very change relationship with data and information. So really the book is about how AI and XR will change the fundamental nature of information and our relationship with it. And really you’re talking about a revolution.
Mike Pell: Well put.
Nick Abrahams: A revolution away from the status quo, which is a very sort of Excel based very sort of manual one dimensional view of data and information. I guess my first question is, why change it? What’s wrong with the status quo in terms of how we currently find information and insight?
Mike Pell: It’s a good point, there’s nothing wrong with the way people do things now, we’ve done it this way, in some cases, for centuries, like books are essentially the way they’ve always been. Information is … My definition of information is very broad, right? Now, it could be a book, it could be a tweet, it can be a film, it can be a piece of music. To me, that’s all information, because it’s conveying something that we’re understanding. So there’s nothing wrong with it. But what I understood, you know, in the insight that I had, when I thought a little bit more about the impact of AI, and spatial computing on information, is we have all these mental models, and we have all these levels of detail, that are sort of … They are ephemeral, right? Like we know them because we’re intelligent, and we think about things the way that we need to. But a book has many different levels of detail that we can choose to look at it in. So you have an excellent summary of this new book Age of Smart Information, right? A very concise one sentence description. That’s the summary level of detail for me, a physical book sitting on a table is the common form, that’s the one that we’re all familiar with.
Mike Pell: But in our minds, we could very easily generate five keywords associated with that book, if we had to describe it to someone or, you know, do like a search for it. And so you would type in AI, design, spatial, right? So that is the meta level of detail, but there’s also as you sort of go down through the levels of detail it’s super, you know, interesting data, connections, associations, that’s contained within everything. So it doesn’t have to be a book, but every piece of information has all of these different levels of detail that we know exist, and we can summon them on demand. But our machines, right? You know, the systems that we use, don’t create them automatically, until now, and they will start to create all these different levels for us automatically. And that’s going to free us to do more, because maybe we’ll be able to author something in one form, and many other forms will automatically be generated.
Mike Pell: My example is, if you wrote a medium article about creativity, that’s great, you’re done, you maybe just finished, hit submit, and off it goes to medium, but you didn’t have the keywords auto generated for you necessarily, you didn’t have a one sentence summary that you could post on Twitter, you didn’t have a breakdown of all the concepts that are included in that article automatically generated for you, that can actually attach to the supporting information. The machines, and the authoring systems that we’re using now are more than capable of doing that, it just takes the introduction of artificial intelligence, machine learning, reinforcement learning, in some cases, to actually help it with the generation and the playback of those things. So that’s sort of the central concept to the book is everything around us has all of these invisible unseen properties that now are going to become artifacts.
Douglas Nicol: I’m not an engineer and I was able to understand what you were talking about so it was sort of that perfect sort of book for the plane trip and so forth.
Mike Pell: That was intentional.
Douglas Nicol: Oh good.
Mike Pell: So thank you. No, I set out to write what I call an airplane book.
Douglas Nicol: Okay.
Mike Pell: Yeah, so like an airport book is one that a traveling exec manager, curious person, you know, who wants to sort of be in the know, perhaps would grab as they’re running through C-Tech, or Austin, or like wherever they happen to be, and get something important out of it. And so I purposely tried to do that, just like in the future, I’m sure in all of our work, it’s going to be more important to create bite size summaries of things, because we’re all so busy, we just can’t get to the things we even want to get to.
Douglas Nicol: Yeah, and it’s certainly achieves that goal and also has a lot of pictures, which is helpful for me.
Mike Pell: That’s my favorite part.
Douglas Nicol: It is very good. But I mean, do you imagine a world where there’s data or it’s generally smart information, which is continually up grading and so forth. And I think one of your central concerns appears to be that the data we’re acting on is dead, if you like, and it’s not real time. Is that sort of where you headed it, the whole … You know, it’ll be spatial, it’ll be visual, it will be around us to be accessed?
Mike Pell: Yeah, absolutely. Yeah, that was the premise, really, that all the electronic artifacts we’ve created over the last 40 years are dead pixels, and just markings for the most part. There are some things that are connected and active and even learning but in the future, and coming very quickly, the things that we generate, whether it’s a medium article, or you know, a new contract, or a book, or a document, or a tweet, all these things have the ability to be injected with enough machine learning or artificial intelligence to learn a little bit more about you. And they will know how you like to consume information and adjust themselves, you know, the piece of information will adjust itself to sort of convey its essence, or its key points to you in the way that makes sense for this time.
Mike Pell: Like, so I talk a lot about context, so contextual design you know very well, is really taking what’s happening in the moment in trying to prepare something in the right way, so it makes sense, and it’s appropriate. So we’ll take the book example, again, if I was reading the book, and let’s say like, I had a physical book, and somehow I had the electronic version too, and I was trying to catch up as I was heading into the Austin Convention Center for South by, if I had an AirPod like an earbud in, why not listen to part of the book as I’m heading in, if I was sitting down to watch a session, and I just happened to open my laptop, I can see the visual form of that. If I’m out like, sitting maybe in my hotel where I have a big screen available to me, why can I enjoy it that way, or see it in mixed reality?
Mike Pell: So this is more than just content following you two different devices, it’s the content actually, knowing the context where are you, when are you, who’s around you, what do I know that helps me prepare something, so you better understand it, sort of can internalize it, and then do whatever you need to do with it, which may be nothing. It may just be like, okay, great, thanks for telling me, or it could be take action.
Nick Abrahams: And I guess in your presentation, and this is I think verging on the philosophical underpinnings of where you seem to be headed, you had gradations of information and there was insight and then ultimately, there was wisdom. And so is it … Do you think that as the machines get smarter, and help us to access information in realtime and interact with it that will actually help to unlock wisdom for us is that where we’re headed?
Mike Pell: First of all, I think wisdom is a very human thing, I don’t know that a machine is right now capable of recognizing that it is truly wise, because we imbue wisdom to people, there’s something that’s sort of like a line that you cross. Machines and AI systems can be very insightful. They can generate a tremendous amount of knowledge, and they can certainly tell stories and narratives. You know, we’ve seen that, can they be wise? Are they truly wise perhaps? Perhaps, that’s where we’re going, we’ll see.
Nick Abrahams: One of the enablers for the Age of Smart Information is probably mainstream adoption of concepts like augmented reality. I mean, there’s endless labels for this virtual reality, mixed reality, XR. When do you see there being a tipping point where actually will be able to access this Age of Smart Information with more widespread use of augmented reality, and what will be the device that unlocks that?
Mike Pell: I think that we are already living in the age of augmented reality through our phones, so people don’t consider it that way but Snapchat filters, right? And all these like filters that kids like to play with, that’s all augmented reality, it’s just through a phone. So we’re already there, as far as smart information. The prediction in the book is that within five years, it will be very widespread, the devices really don’t matter to me, that’s not the point. The point is that the systems, and the tools, and the playback mechanisms become more ubiquitous, so that we’re capable of doing this. And then I think it will happen very quickly, very exponentially, once that happens. But I don’t think that there’s any particular instance, like there won’t be a particular device that unlocks it, it won’t be a particular time. But it’s sort of everything that’s happening now that you can see all of ourselves by happening all together.
Nick Abrahams: I often say that data is nothing more than a description of a human being. And what I loved about one of the sort of attributes of this Age of Smart Information was emotional engagement, between you know, humans and data. Can you talk a bit about that, because I find that really, really good?
Mike Pell: It’s so interesting to me, that we don’t talk about emotion enough in the things that we do. So as designers we’re trained to look at emotion, understand it, generate it you know when we can. We know that we’re successful as designers, when we create an emotional connection to something. Do you have an emotional connection to your spreadsheet, maybe not. Unless it’s somehow personal, or like, you know, you put a lot of work into it [crosstalk 00:17:23]
Douglas Nicol: Or unless you loose it.
Mike Pell: Yeah, and there’s a different emotion for that, and a different set of vocabulary for that. It just seems to me that we have made so many advances like so there’s something called the Microsoft Cognitive Services toolkit, which is an AI powered service available to developers, that will help recognize human emotion, whether it’s through micro actions, or the tone of your voice, or some other factors. But that helps us understand context. And so trying to get to the point where we understand your emotional state, when you’re trying to absorb something like whether it’s your spreadsheet, or a book, or a talk, whatever it happens to be, we can tailor the information to your emotion and do a better job. But the flip side is, once these systems that we’re building start to get good at what they’re doing, they may actually start, as I say, in the book, to fret about things, they may have their own thoughts.
Mike Pell: And it’s not that like, you know, I’ll say my tweet is now like, very emotional and concerned that I didn’t phrase something well, but you can see that it could go there, right? As the systems we build start to learn more about us, they may have their own thoughts, and their own emotions. And so it’s not to say that our information is going to get away from us, or our data is going to turn on us necessarily but I do think that as designers, and engineers, and as people who understand what people are looking for, giving personality to something is what we’re always striving for. Like, how many times have you talked to your Alexa, right? And just for that instant, you know, when Siri or Alexa or Cortana, or any your assistance has a little bit of a snarky answer or says something funny, you immediately sort of imbue that with emotion, and the ability to be funny or entertaining.
Mike Pell: This is the same thing with information, because it’ll feel like that, it’ll feel like, you’re not just consuming a piece of information, you’re having a conversation with it, which is what I also talked about on the book. And that’s where the emotional part really comes in. It’s not so much that your spreadsheet is being finicky today, it’s that when you interact with it, you may imbue it that some type of emotional state that you wouldn’t normally.
Nick Abrahams: I think one of the interesting things is sometimes the data analysis part of the presentation is often perceived as the dull bit of the presentation and in your book, another of the enabling factors is what you call stunning presentation. And I’m just trying to picture in my mind, if you’re presenting data back to a client, what would that future experience be in the Age of Smart Information?
Mike Pell: Well, in the first book, I wrote envisioning holograms, I talked a lot about the impact that room scale data has. So if you’ve ever get a chance to look at some type of data visualization at room scale, meaning, you know, as big as room, you know, larger than a tabletop presentation, there’s, again, an emotional connection that happens, there’s something that unlocks in your brain. And in the way that you feel, as long as you understand the data, you know, if it’s foreign data, you may get a superficial impression of oh, that’s interesting, and whatever, but if it’s your data, and you get it, you are completely in it. And you start to do calculations, and you start to go like, wow, what if we did this? And that immersion, and that connection to data that’s all around you, is pretty powerful, and it’s great …
Mike Pell: And so I do spend some time talking about smart data, which is more on … It’s actually, you know, words and numbers and insights that we need to unlock, and seeing that and being able to experience it in, I think a more cinematic way, is really something that we’re just scratching the surface of and will continue to be super interesting for all of us, whether it’s in law, or science or art.
Douglas Nicol: And what’s the most exciting thing for you, what are you working on, or what are you most excited about for the next couple of years?
Mike Pell: Well, this is a pretty exciting concept. And I’m really happy that people are recognizing that now is the time to go work on this and talk about it. I’m starting to get it, so now that I’ve finished this book, I have a third book in me, like it’ll be a trilogy. And so the next book is really going to be about visualization, you know, for businesses and for organizations, and what can this new type of smart data visualization do for your organization? So that’s really starting to excite me, but I will talk about smart information all day with you if you want.
Douglas Nicol: Fantastic. So we have … Lord of the Rings is a great trilogy and now smart information is a great trilogy.
Mike Pell: Yeah, I got one more in me I think.
Nick Abrahams: Now, if people want to read the book, it’s available online, it’s available in bookstores, it’s called theageofsmartinformation.com. And also, you’ve written some really interesting articles in the area of Human Centered Design and design thinking, and that is on the microsoft.com blog.
Mike Pell: Yeah. There’s a garage blog, and Microsoft blog, there’s also some posts that I’ve done on medium that you can find pretty easily about some of these thoughts.
Nick Abrahams: And Pell, thank you so much for joining us today on Smart Dust. It’s been absolutely fascinating. And I think what’s exciting about your vision is it will, I think, once again, provoke curiosity in the minds of so many people who may be have lost the ability to be curious. And I think the age of smart information, I think will do a lot of people real good because it will provoke that curiosity that maybe they’ve lost. So thank you very inspiring vision.
Mike Pell: Thank you Doug, thank you Nick, for having me. Really I would love to talk all day. We should do this again. You both did an excellent job of understanding the points I was trying to get across. So thank you.
Douglas Nicol: That’s very kind. Thank you. And can I just say that idea of people creating their own job is very profound. So thank you for that as well as your other insights.
Mike Pell: [crosstalk 00:23:25]
Douglas Nicol: And Pell, good luck with the book. And we look forward to catching up.