Voices in AI – Episode 86: A Conversation with Amir Husain

[voices_in_ai_byline]

About this Episode

Episode 86 of Voices in AI features Byron speaking with fellow author Amir Husain about the nature of Artificial Intelligence and Amir’s book The Sentient Machine.

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

Transcript Excerpt

Byron Reese: This is Voices in AI brought to you by GigaOm, and I’m Byron Reese. Today my guest is Amir Husain. He is the founder and CEO of SparkCognition Inc., and he’s the author of The Sentient Machine, a fine book about artificial intelligence. In addition to that, he is a member of the AI task force with the Center for New American Security. He is a member of the board of advisors at UT Austin’s Department of Computer Science. He’s a member of the Council on Foreign Relations. In short, he is a very busy guy, but has found 30 minutes to join us today. Welcome to the show, Amir.

Amir Husain: Thank you very much for having me Byron. It’s my pleasure.

You and I had a cup of coffee a while ago and you gave me a copy of your book and I’ve read it and really enjoyed it. Why don’t we start with the book. Talk about that a little bit and then we’ll talk about SparkCognition Inc. Why did you write The Sentient Machine: The Coming Age of Artificial Intelligence?

Byron, I wrote this book because I thought that there was a lot of writing on artificial intelligence—what it could be. There’s a lot of sci fi that has visions of artificial intelligence and there’s a lot of very technical material around where artificial intelligence is as a science and as a practice today. So there’s a lot of that literature out there. But what I also saw was there was a lot of angst back in 2015, 2014. I actually had a personal experience in that realm where outside of my South by Southwest talks there was an anti-AI protest.

So just watching those protesters and seeing what their concerns were, I felt that a lot of the sort of philosophical questions, existential questions around the advent of AI, if AI indeed ends up being like Commander Data, it has sentience, it becomes artificial general intelligence, then it will be able to do the jobs better than we can and it will be more capable in let’s say the ‘art of war’ than we are and therefore does this mean that we will lose our jobs. We will be meaningless and our lives will be lacking in meaning and maybe the AI will kill us?

These are the kinds of concerns that people have had around AI and I wanted to sort of reflect on notions of man’s ability to create—the aspects around that that are embedded in our historical and religious tradition and what our conception of Man vs. he who can create, our creator—what those are and how that influences how we see this age of AI where man might be empowered to create something which can in turn create, which can in turn think.

There’s a lot of folks also that feel that this is far away, and I am an AI practitioner and I agree I don’t think that artificial general intelligence is around the corner. It’s not going to happen next May, even though I suppose some group could surprise us, but the likely outcome is that we are going to wait a few decades. I think waiting a few decades isn’t a big deal because in the grand scheme of things, in the history of the human race, what is a few decades? So ultimately the questions are still valid and this book was written to address some of those existential questions lurking in elements of philosophy, as well as science, as well as the reality of where AI stands at the moment.

So talk about those philosophical questions just broadly. What are those kinds of questions that will affect what happens with artificial intelligence?

Well I mean one question is a very simple one of self-worth. We tend to define ourselves by our capabilities and the jobs that we do. Many of our last names in many cultures are literally indicative of our profession. You know goldsmiths as an example, farmer as an example. And this is not just a European thing. Across the world you see this phenomenon of last names just reflecting the profession of a woman or a man. And it is to this extent that we internalize the jobs that we do as essentially being our identity, literally to the point where we take it on as a name.

So now when you de-link a man or a woman’s ability to produce or to engage in that particular labor that is a part of their identity, then what’s left? Are you still, the human that you were with that skill? Are you less of a human being? Is humanity in any way linked to your ability to conduct this kind of economic labor? And this is one question that I explored in the book because I don’t know whether people really contemplate this issue so directly and think about it in philosophical terms, but I do know that subjectively people get depressed when they’re confronted with the idea that they might not be able to do the job that they are comfortable doing or have been comfortable doing for decades. So at some level obviously it’s having an impact.

And the question then is: is our ability to perform a certain class of economic labor in any way intrinsically connected to identity? Is it part of humanity? And I sort of explore this concept and I say “OK well, let’s sort of take this away and let’s cut this away let’s take away all of the extra frills, let’s take away all of what is not absolutely fundamentally uniquely human.” And that was an interesting exercise for me. The conclusions that I came to—I don’t know whether I should spoil the book by sharing it here—but in a nutshell—this is no surprise—that our cognitive function, our higher order thinking, our creativity, these are the things which make us absolutely unique amongst the known creation. And it is that which makes us unique and different. So this is one question of self worth in the age of AI, and another one is…

Just to put a pin in that for a moment, in the United States the workforce participation rate is only about 50% to begin with, so only about 50% of people work because you’ve got adults that are retired, you have people who are unable to work, you have people that are independently wealthy… I mean we already had like half of adults not working. Does it does it really rise to the level of a philosophical question when it’s already something we have thousands of years of history with? Like what are the really needy things that AI gets at? For instance, do you think a machine can be creative?

Absolutely I think the machine can be creative.

You think people are machines?

I do think people are machines.

So then if that’s the case, how do you explain things like the mind? How do you think about consciousness? We don’t just measure temperature, we can feel warmth, we have a first person experience of the universe. How can a machine experience the world?

Well you know look there’s this age old discussion about qualia and there’s this discussion about the subjective experience, and obviously that’s linked to consciousness because that kind of subjective experience requires you to first know of your own existence and then apply the feeling of that experience to you in your mind. Essentially you are simulating not only the world but you also have a model of yourself. And ultimately in my view consciousness is an emergent phenomenon.

You know the very famous Marvin Minsky hypothesis of The Society of Mind. And in all of its details I don’t know that I agree with every last bit of it, but the basic concept is that there are a large number of processes that are specialized in different things that are running in the mind, the software being the mind, and the hardware being the brain, and that the complex interactions of a lot of these things result in something that looks very different from any one of these processes independently. This in general is a phenomenon that’s called emergence. It exists in nature and it also exists in computers.

One of the first few graphical programs that I wrote as a child in basic [coding] was drawing straight lines, and yet on a CRT display, what I actually saw were curves. I’d never drawn curves but it turns out that when you light a large number of pixels with a certain gap in the middle and it’s on a CRT display there there are all sorts of effects and interactions like the Moire effect and so on and so forth where what you thought you were drawing was lines, and it shows up if you look at it from an angle, as curves.

So I mean the process of drawing a line is nothing like drawing a curve, there was no active intent or design to produce a curve, the curve just shows up. It’s a very simple example of a kid writing a few lines of basic can do this experiment and look at this but there are obviously more complex examples of emergence as well. And so consciousness to me is an emergent property, it’s an emergent phenomenon. It’s not about the one thing.

I don’t think there is a consciousness gland. I think that there are a large number of processes that interact to produce this consciousness. And what does that require? It requires for example a complex simulation capability which the human brain has, the ability to think about time, to think about objects, model them and to also apply your knowledge of physical forces and other phenomena within your brain to try and figure out where things are going.

So that simulation capability is very important, and then the other capability that’s important is the ability to model yourself. So when you model yourself and you put yourself in a simulator and you see all these different things happening, there is not the real pain that you experience when you simulate for example being struck by an arrow, but there might be some fear and a why is that fear emanating? It’s because you watch your own model in your imagination, in your simulation suffer some sort of a problem. And now that is a very internal. Right? None of this has happened in the external world but you’re conscious of this happening, so to me at the end of the day it has some fundamental requirements. I believe simulation and self modeling are two of those requirements, but ultimately it’s an emergent property.

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

[voices_in_ai_link_back]

Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

Voices in AI – Episode 85: A Conversation with Ilya Sutskever

[voices_in_ai_byline]

About this Episode

Episode 85 of Voices in AI features host Byron Reese and Ilya Sutskever of Open AI talk about the future of general intelligence and the ramifications of building a computer smarter than us.

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

Transcript Excerpt

Byron Reese: This is Voices in AI brought to you by GigaOm and I’m Byron Reese. Today my guest is Ilya Sutskever. He is the co-founder and the chief scientist at OpenAI, one of the most fascinating institutions on the face of this planet. Welcome to the show Ilya.

Ilya Sutskever: Great to be here.

Just to bring the listeners up to speed, talk a little bit about what OpenAI is, what its mission is, and kind of where it’s at. Set the scene for us of what OpenAI does.

Great, for sure. The best way to describe OpenAI is this: so at OpenAI we take the long term view that eventually computers will become as smart or smarter than humans in every single way. We don’t know when it’s going to happen, — some number of years, something [like] tens of years, it’s unknown. And the goal of OpenAI is to make sure that when this does happen, when computers which are smarter than humans are built, when AGI is built, then its benefits will be widely distributed. We want it to be a beneficial event, and that’s the goal of OpenAI.

And so we were founded three years ago, and since then we’ve been doing a lot of work in three different areas. We’ve done a lot of work in AI capabilities and over the past three years we’ve done a lot of work we are very proud of. Some of the notable highlights are: our Dota results where we had the first and very convincing demonstration of an agent playing a real time strategy game, trained the reinforcement learning with no human data. We’ve trained robots to record, robot hands to re-orientate the block. This was really cool, it was cool to see it transfer.

And recently we’ve released the GPT-2 — a very large language model which can generate very realistic text as well as solve lots of different energy problems [with] a very high level of accuracy. And so this has been our working capabilities.

Another thrust to the work that we are doing is AI safety, which at [its] core is the problem of finding ways of communicating a very complicated reward function to an agent so that the agent that we build, can achieve goals and great competence. It will do so while taking human values and preferences into account. And so we’ve done some significant amount of work there as well.

And the third line of work we’re doing is AI policy, where we basically have a number of really good people thinking hard about what kind of policies should be designed and how should governments and other institutions respond to the fact that AI is improving pretty rapidly. But overall our goal, eventually the end game of the field, is that AGI will be built. The goal of OpenAI is to make sure that the development of AGI will be a positive event and that its benefits are widely distributed.

So 99.9% of all the money that goes into AI is working on specific narrow AI projects. I tried to get an idea of how many people are actually working on AGI and I find that to be an incredibly tiny number. There’s you guys, maybe you would say Carnegie Mellon, maybe Google, there’s a handful, but is my sense of that wrong? Or do you think there are lots of groups of people who are actually explicitly trying to build a general intelligence?

So explicitly. OK, a great question. So it’s an explicitly… most people, most research labs are indeed not having this as their goal, but I think that many people, the work of many people indirectly contributes to this. Where for example the fact is that much better learning algorithms, better network architecture, better optimization methods, all tools which are classically categorized as conventional machine learning, they also are likely to be directly contributing to those…

Well let’s stop there for a second, because I noticed you changed your word there to “likely.” Do you still think it’s an open question whether narrow AI, whatever technologies we have that do that, is it an open question whether that has anything to do with general intelligence, or is it still the case that a general intelligence might have absolutely nothing to do with that propagation, neural nets and machine learning?

So I think it’s very highly unlikely. Sorry. I want to make it clear, I think that the tools, that is the field of machine learning that is developing today, such as deep networks, backpropagation, — I think those are immensely powerful tools, and I think that it is likely that they will stay with us, with the field, for a long time all the way until we build very true general intelligence. At the same time I also believe, I want to emphasize that, important missing pieces exist and we haven’t figured out everything. But I think that the deep learning has proven itself to be so versatile and so powerful and it’s basically been exceeding our expectations in every turn. And so for these reasons I feel that deep learning is going to stay with us.

Well let’s talk about that though, because one could summarize the techniques we have right now as: let’s take a lot of data about the past, let’s look for patterns in that data and let’s make predictions about the future, which isn’t all that exciting when you say it like that. It’s just that we’ve gotten very good at it.

But why do you believe that method is the solution to things like creativity, intuition, emotion and all of these kind of human abilities? It seems to be at an intuitive level that if you want to teach a machine to play Dota or Go or whatever, yeah that works great. But really when you come down to human level intelligence with its versatility, with transferred learning with all the things we do effortlessly, it’s not even… it doesn’t seem at first glance to be a match. So why do you suspect that it is?

Well I mean I can tell you how I look at it. So for example you mentioned intuition is one thing which – so you used the certain phrase to describe the current tools where you kind of look for patterns in the past data and you use that to make predictions about the future and therefore it sounds not exciting. But I don’t know if I’d agree with that statement. And on the question of intuition, I can tell you a story about about AlphaGo. So… if you look at how AlphaGo works, there is a convolutional neural network.

OK actually let me give you a better analogy – so I believe there is a book by Malcolm Gladwell where he talks about experts, and one of the things that he has to say about experts is that an expert as a result of all their practice. They can look at a very complicated situation and then they can instantly tell like the three most important things in this situation. And then they think really hard about which of those things is really important. And apparently the same thing happens with Go players, where a Go player might look at the board and then instantly see the most important moves and then do a little bit of thinking about those moves. And like I said, instantly seeing what those moves are, — this is their intuition. And so I think that it’s basically unquestionable with the neural network that’s inside AlphaGo calculates a  solution very well. So I think I think it’s not correct to say that intuition cannot be captured.

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

[voices_in_ai_link_back]

Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

How Intel’s Newest Product Enhancements Could Redefine the Future of Infrastructure Design

On the 2nd of April, at their Data Centric Innovation Day, Intel announced a slew of new products, including brand spanking new ones as well as updates to existing product line-ups. There was some news for everybody – whether your specific interests are in datacenter, or edge, or infrastructures.  Among other things, what impressed me the most was the new Optane DC Persistent Memory DIMM, and even more so its implications when coupled with the new (56-core) Intel Xeon Platinium 9200 CPU.

Datacenter Optanization!

Up until yesterday, Optane was already considered a great technology, although to some extent it was seen as a niche product. Although more and more vendors are adopting it as an adequate substitute of NVDIMMs as a tier 0 or cache for their storage systems,  it was still harder to foresee a broader adoption of the technology. Yes, it is cheaper than RAM and faster than a standard NAND-based device, but that’s about it.

Maybe it was part of Intel’s strategy. In fact, the first generation of Intel Optane products was developed with Micron, and with the first generation of products perhaps they didn’t want to be too aggressive — something which has most likely radically changed after their divorce. The introduction of Optane DC Persistent Memory DIMM actually offers a good idea of the real potential and benefits of this technology and demonstrates how this could change the way infrastructures will be designed in the future, especially for all data-intensive workloads. In practice, in a very simplistic way, Optane DC Persistent Memory DIMMs work in conjunction with standard DDR4 RAM DIMMs. They are bigger (up to 512GB each) and slightly slower than DDR4 DIMMs, but they allow configuration of servers with several TBs of memory at a very low cost.  Optane is slower than RAM, but caching techniques and the fact that these DIMMs sit next to the CPU make the solution a good compromise. At the end of the day, you are trading some performance for a huge amount of capacity, avoiding data starvation for the CPU that otherwise should access data on SSD or, even worse, HDDs or network.

How Does It Work?

There are two operation modes for Optane DIMMs, persistent and non-persistent. I know it could sound confusing, but it’s actually very straight forward.

The way the DIMM operates is selected at the beginning of the bootstrap process. When non-persistent mode is selected, the DIMMs look like RAM, and the real RAM is used as a cache. This means that you don’t have to re-write your app, and practically any application can benefit from the increased memory capacity. On the other hand, when Optane DIMMs operate in persistent mode, it is the application that stores data directly in the Optane and manages RAM and Optane as it fits and, as the name suggests, your data is still there after a reboot. SAP HANA, for example, has already demonstrated this mode and there are several other applications that will follow suit. Take a look at the video below, recorded at the Tech Field Day event that followed the main presentation, its a good deep dive on the product.

There Are Several Benefits (And a Few Trade-Offs)

All the performance tests shown during the event demonstrated the benefits of this solution. Long story short, and crystal clear, every application that relays on large amounts of RAM to work will see a huge benefit. This is mostly because the latency to reach active data is lower than any other solution on the market, but it is also because it will do so at a cost that is a fraction of what you’d get from a 100% configuration. This is not possible today due to the size of the DIMMs and the cost. All of this translates into faster results, fewer servers to obtain them, better efficiency and, of course, lower infrastructure costs.

Furthermore, with more and more applications taking advantage of the persistent memory mode, we will see interesting applications with Optane DIMMs that could replace traditional, and expensive, NVDIMMs in many scenarios like, for example, storage controllers. What are the trade-offs then? Actually, these are not real trade-offs, but the consequences of the introduction of this innovative technology. In fact, Optane DIMMs work only on new servers based on the latest Intel CPUs, those announced alongside the DIMMs. The reason for this is that the memory controller is in the CPU, and older CPUs wouldn’t be able to understand the Optane DIMM nor manage the interaction between RAM and Optane.

Maintaining the Balance

As mentioned earlier, Intel announced many other products alongside Optane Persistent Memory DIMMs. All of them are quite impressive and, at the same time, necessary to justify each other; meaning that it would be useless to have multi-TB RAM systems without a CPU strong enough to get the work done. The same goes for the network, which can quickly become a bottleneck if you don’t provide the necessary bandwidth to get data back and forth from the server.

From my point of view, it’s really important to understand that we are talking about data-crunching monsters here, with the focus on applications like HPC, Big Data, AI/ML and the like. These are not your average servers for VMs, not today at least. On the other hand, it is also true that this technology opens up many additional options for enterprise end users too, including the possibility to create larger VMs or consolidate more workloads on a single machine (with all its pros and cons).

Another feature which I thought noteworthy is the new set of instructions added to the new Xeon Platinum 9200 CPU for deep learning. We are far from having general purpose CPUs competing against GPUs, but the benchmark given during the presentations shows an incredible improvement in this area regarding Inference workloads (the process that is behind the day-to-day ML activity after the neural network is trained). Intel has done a great job, both on hardware and software. With more and more applications taking advantage of AI/ML to work, an increasing number of users will be able to benefit from it.

Closing the Circle

In this article, I’ve covered only a few aspects of these announcements, those that are more related to my job. There is much more to it, including IoT, edge computing, and security. I was  especially impressed because they express a vision that is broad, clear and without doubts, providing answers for all the most challenging aspects of modern IT.

Most of the products presented during this announcement are focused on ultimate performance and efficiency or, at least, in finding the best compromise to serve next-gen data-hungry and high demanding applications. Something which is beyond the reach of many enterprises today and more in the ballpark of web and hyper scalers. That said, even if on a smaller scale, all enterprises are beginning to face these kinds of challenges and, no matter if the solutions come from the cloud or their on-prem infrastructure, the technology to do it is now more accessible than ever.

Voices in AI – Episode 84: A Conversation with David Cox

[voices_in_ai_byline]

About this Episode

Episode 84 of Voices in AI features host Byron Reese and David Cox discuss classifications of AI, and how the research has been evolving and growing

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

Transcript Excerpt

Byron Reese: This is Voices in AI, brought to you by GigaOm and I’m Byron Reese. I’m so excited about today’s show. Today we have David Cox. He is the Director of the MIT IBM Watson AI Lab, which is part of IBM Research. Before that he spent 11 years teaching at Harvard, interestingly in the Life Sciences. He holds an AB degree from Harvard in Biology and Psychology, and he holds a PhD in Neuroscience from MIT. Welcome to the show David!

David Cox: Thanks. It’s a great pleasure to be here.

I always like to start with my Rorschach question which is, “What is intelligence and why is Artificial Intelligence artificial?” And you’re a neuroscientist and a psychologist and a biologist, so how do you think of intelligence?

That’s a great question. I think we don’t necessarily need to have just one definition. I think people get hung up on the words, but at the end of the day, what makes us intelligent, what makes other organisms on this planet intelligent is the ability to absorb information about the environment, to build models of what’s going to happen next, to predict and then to make actions that help achieve whatever goal you’re trying to achieve. And when you look at it that way that’s a pretty broad definition.

Some people are purists and they want to say this is AI, but this other thing is just statistics or regression or if-then-else loops. At the end of the day, what we’re about is we’re trying to make machines that can make decisions the way we do and sometimes our decisions are very complicated. Sometimes our decisions are less complicated, but it really is about how do we model the world, how do we take actions that really drive us forward?

It’s funny, the AI word too. I’m a recovering academic as you said. I was at Harvard for many years and I think as a field, we were really uncomfortable with the term ‘AI.’ so, we desperately wanted to call it anything else. In 2017 and before we wanted to call it ‘machine learning’ or we wanted to call it ‘deep learning’ [to] be more specific. But in 2018 for whatever reason, we all just gave up and we just embraced this term ‘AI.’ In some ways I think it’s healthy. But when I joined IBM I was actually really pleasantly surprised by some framing that the company had done.

IBM does this thing called the Global Technology Outlook or GTO which happens every year and the company tries to collectively figure out—research plays a very big part of this—we try to figure out ‘What does the future look like?’ And they came up with this framing that I really like for AI. They did something extremely simple. They just put some adjectives in front of AI and I think it clarifies the debate a lot.

So basically, what we have today like deep learning, machine learning, tremendously powerful technologies are going to disrupt a lot of things. We call those Narrow AI and I think that narrow framing really calls attention to the ways in which even if it’s powerful, it’s fundamentally limited. And then on the other end of the spectrum we have General AI.  This is a term that’s been around for a long time, this idea of systems that can decide what they want to do for themselves that are broadly autonomous and that’s fine. Those are really interesting discussions to have but we’re not there as a field yet.

In the middle and I think this is really where the interesting stroke is, there’s this notion we have a Broad AI and I think that’s really where the stakes are today. How do we have systems that are able to go beyond what we have that’s narrow without necessarily getting hung up on all these notions of what ‘General Intelligence’ might be. So things like having systems that are that are interpretable, having systems that can work with different kinds of data that can integrate knowledge from other sources, that’s sort of the domain of Broad AI. Broad Intelligence is really what the lab I lead is all about.

There’s a lot in there and I agree with you. I’m not really that interested in that low end and what’s the lowest bar in AI. What makes the question interesting to me is really the mechanism by which we are intelligent, whatever that is, and does that intelligence require a mechanistic reductionist view of the world? In other words, is that something that you believe we’re going to be able to duplicate either… in terms of its function, or are we going to be able to build machines that are as versatile as a human in intelligence, and creative and would have emotions and all of the rest, or is that an open question?

I have no doubt that we’re going to eventually, as a human race be able to figure out how to build intelligent systems that are just as intelligent as we are. I think in some of these things, we tend to think about how we’re different from other kinds of intelligences on Earth. We do things like… there was a period of time where we wanted to distinguish ourselves from the animals and we thought of reason, the ability to reason and do things like mathematics and abstract logic was what was uniquely human about us.

And then, computers came along and all of a sudden, computers can actually do some of those things better than we can even in arithmetic and solving complex logic problems or math problems. Then we move towards thinking that maybe it’s emotion. Maybe emotion is what makes us uniquely human and rational. It was a kind of narcissism I think to our own view which is understandable and justifiable. How are we special in this world?

But I think in many ways we’re going to end up having systems that do have something like emotion. Even you look at reinforcement learning—those systems have a notion of reward. I don’t think it’s such a far reach to think maybe we’ll even in a sci-fi world have machines that have senses of pleasure and hopes and ambitions and things like that.

At the end of day, our brains are computers. I think that’s sometimes a controversial statement but it’s one that I think is well-grounded. It’s a very sophisticated computer. It happens to be made out of biological materials. But at the end of the day, it’s a tremendously efficient, tremendously powerful, tremendously parallel nanoscale biological computer. These are like biological nanotechnology. And to the extent that it is a computer and to think to the extent that we can agree on that, Computer Science gives us equivalencies. We can build a computer with different hardware. We don’t have to emulate the hardware. We don’t have to slavishly copy the brain, but it is sort of a given that will eventually be able to do everything the brain does in a computer. Now of course all that’s all farther off, I think. Those are not the stakes—those aren’t the battlefronts that we’re working on today. But I think the sky’s the limit in terms of where AI can go.

You mentioned Narrow and General AI, and this classification you’re putting in between them is broad, and I have an opinion and I’m curious of what you think. At least with regards to Narrow and General they are not on a continuum. They’re actually unrelated technologies. Would you agree with that or not?

Would you say like that a narrow (AI) gets a little better then a little better, a little better, a little better, a little better, then, ta-da! One day it can compose a Hamilton, or do you think that they may be completely unrelated? That this model of, ‘Hey let’s take a lot of data about the past and let’s study it very carefully to learn to do one thing’ is very different than whatever General Intelligence is going to be.

There’s this idea that if you want to go to the moon, one way to go to the moon—to get closer to the moon—is to climb the mountain.

Right. Exactly.

And you’ll get closer, but you’re not on the right path. And, maybe you’d be better off on top of a building or a little rocket and maybe go as high as the tree or as high as the mountain, but it’ll get you where you need to go. I do think there is a strong flavor of that with today’s AI.

And in today’s AI, if we’re plain about things, is deep learning. This model… what’s really been successful in deep learning is supervised learning. We train a model to do every part of seeing based on classifying objects and you classify a lot – many images, you have lots of training data and you build a statistical model. And that’s everything the model has ever seen. It has to learn from those images and from that task.

And we’re starting to see that actually the solutions you get—again, they are tremendously useful, but they do have a little bit of that quality of climbing a tree or climbing a mountain. There’s a bunch of recent work suggesting… basically they’re looking at texture, so a lot of solution for supervision is looking at the rough texture.

There are also some wonderful examples where you take a captioning system—a system can take an image and produce a caption. You can produce wonderful captions in cases where the images look like the ones it was trained on, but you show it anything just a little bit weird like an airplane that’s about to crash or a family fleeing their home on a flooding beach and it’ll produce things like an airplane is on the tarmac at an airport or a family is standing on a beach. It’s like they kind of missed the point, like it was able to do something because it learned correlations between the inputs it was given and the outputs that we asked it for, but it didn’t have a deep understanding. And I think that’s the crux of what you’re getting at and I agree at least in part.

So with Broad, the way you’re thinking of it, it sounds to me just from the few words you said, it’s an incremental improvement over Narrow. It’s not a junior version of General AI. Would you agree with that? You’re basically taking techniques we have and just doing them bigger and more expansively and smarter and better, or is that not the case?

No. When we think about Broad AI, we really are thinking about a little bit ‘press the reset button, don’t throw away things that work.’ Deep learning is a set of tools which is tremendously powerful, and we’d be kind of foolish to throw them away. But when we think about Broad AI, what we’re really getting at is how do we start to make contact with that deep structure in the world… like commonsense.

We have all kinds of common sense. When I look at a scene I look at the desk in front of me, I didn’t learn to do tasks that have to do with the desk in front of me by lots and lots of labeled examples or even many, many trials in a reinforcement learning kind of setup. I know things about the world – simple things. And things we take for granted like I know that my desk is probably made of wood and I know that wood is a solid, and solids can’t pass through other solids. And I know that it’s probably flat, and if I put my hand out I would be able to orient it in a position that would be appropriate to hover above it…

There are all these affordances and all this super simple commonsense stuff that you don’t get when you just do brute force statistical learning. When we think about Broad AI, we’re really thinking about is ‘How do we infuse that knowledge, that understanding and that commonsense?’ And one area that we’re excited about and that we’re working on here at the MIT IBM Lab is this idea of neuro-symbolic hybrids.

So again, this is in the spirit of ‘don’t throw away neural-networks.’ They’re wonderful in extracting certain kinds of statistical structure from the world – convolutional neural network does wonderful job of extracting information from an image. LSDMs and recurrent neural networks do a wonderful job of extracting structure from natural language, but building in symbolic systems as first-class citizens in a hybrid system that combines those all together.

Some of the work we’re doing now is building systems where we use neural networks to extract structure from these noisy, messy inputs of vision and different modalities but then actually having symbolic AI systems. Symbolic AI systems have been around basically contemporaneous with neural networks. They’ve been ‘in the wings’ all this time. Neural networks deep learning is in any way… everyone knows this is a rebrand of the neural networks from the 1980s that are suddenly powerful again. They’re powerful for the first time because we have enough data and we have enough compute.

I think in many ways a lot of the symbolic ideas, sort of logical operations, planning, things like that. They’re also very powerful techniques, but they haven’t really been able to shine yet partly because they’ve been waiting for something—just the way that neural networks were waiting for compute and data to come along. I think in many ways some of these symbolic techniques have been waiting for neural networks to come along—because neural networks can kind of bridge that [gap] from the messiness of the signals coming in to this sort of symbolic regime where we can start to actually work. One of things we’re really excited about is building these systems that can bridge across that gap.

Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com

[voices_in_ai_link_back]

Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

The Future of Software Innovation? Hardware-Enabled AI & ML Innovation

Hardware innovation is a fickle beast.  It takes money, lots of money.  It takes time, a great team, and execution in more than twenty separate domains, many of which are often overlooked until it’s too late (certifications anyone?!).  But, I’ve got some good news. Hardware is back and it’s about to get really exciting.

You’re probably thinking right now,  “All I ever read about is how AI, ML, blockchain, and XR are ready to revolutionize the world” and that’s exactly the point.  There are some amazing software technologies coming out, but this “new” software has hardware in its DNA.  Until recently, smart technologies have largely been limited by their access points: computers, tablets, smartphones, etc.  Going forward, hardware innovations will become increasingly integral and valuable as the interface for tomorrow’s software. Hardware will capture the data through wearables, hearables, cameras and an increasing variety of sensors and will then be leveraged as the outputs to interact with the world as robots, drones, and the myriad of other IoT devices that are being developed.

As the ecosystem of devices, computation, connection, and data evolve, the platforms, tools and systems are naturally finding more synergy and lowering the barriers of integration.   The line between what is a hardware or software product will continue to blur.  The sensor technologies leading the way are cameras and microphones. If there is a camera, there’s a good chance there’s an AI stack behind it, with self-driving cars being the most prominent example.   On the microphone/speaker side, the Smart Home assistants Amazon Echo, Google Home, and others are obvious and ubiquitous.

The beauty of hardware enabled AI/ML is that it not only crosses the boundary between the physical and virtual, but also between analog and digital.  It’s particularly valuable when interacting with the world and dealing with its messy data.  The next generation of AI hardware startups will take all that messy analog data and transform it into productive and executable knowledge that provide better experiences all the way from shopping to cancer treatments that enable personalized health care at scale.

While the future is clear, the hurdles are as well.  Processing power, robustness, generalization and cost are all tradeoffs future hardware products will need to balance.  Unlike the on-demand and scalable cloud and other services software enjoys, each hardware product will have onboard processing, sensors, connectivity tech, and other requirements that all make their way into the product cost.  Sure a GPU can be thrown into the BOM, but can the market accept the price?  High performance computing at the edge is still in its nascent stages so real-time processing of images and other data can be expensive as well.

At the same time, specific tools are being developed to improve this integration.  We’re seeing a lot of edge computing such as NVIDIA’s Jetson line and Google’s Edge TPUs. TensorFlow is probably the most common AI framework, since it has such broad support for hardware deployment, including Raspberry Pi. ROS is still fairly popular despite being a jumble of mismatched and complicated software, and people have done ports to OpenAI’s Gym environments.

The future of hardware is bright and full of highly accessible, processed, happiness-inducing data.

Join 600 hardware innovators, entrepreneurs, disruptors and investors at HardwareCon 2019, the premier event for hardware innovation.  Plan to attend April 17-18 at the Computer History Museum in Mountain View, California.  Use promo code: GIGA-OM-IL for a special 20% discount on the ticket. Visit www.hardwarecon.com to redeem the discount.

Author Bio

Greg Fisher is all about hardware innovation. As founder/CEO of Berkeley Sourcing Group, Greg has spent the last 13 years working with over 1000 hardware startups to develop and manufacture innovative products. Living in China one third of that time, he worked with hardware startups and factories to help improve their designs for manufacturing, qualify and select factories, manage factory negotiations and relationships, and develop and implement quality control processes. With this history, Greg has a unique perspective and immense passion for what it takes for hardware startups to build the right foundation and scale their operations.

Seeing the need for more support for hardware startups to realize success, Greg started Hardware Massive, which is now the leading Global Community/Platform for Hardware Startup Innovation, and HardwareCon, the Bay Area’s premier hardware innovation conference. Their missions are to empower hardware startups to succeed through networking, events, education, and providing access to resources.

How IBM is Rethinking its Data Protection Line-Up

Following up on my take on the evolution of product and strategy of companies like Cohesity and NetApp, today I’d like to talk about IBM and its new data protection solution Spectrum Data Protect Plus.

IBM AND ITS DATA PROTECTION LINE-UP

In short, not too long ago IBM changed the names of all its storage products. I totally understand why they did it and it makes a lot of sense from a marketing point of view, but it is still confusing for people like myself that were familiar with the products before this change. Besides, with products now having similar names, it could be difficult to discern who does what.

In this particular case, data protection, you now have two products:

• IBM Spectrum Data Protect: the good, old, TSM. While this product is one of those that have written Backup’s history and supports a myriad of Operating Systems and applications as well as backup, it is complex to operate and designed for large environments. Furthermore, it was designed well before the advent of hypervisors and modern applications, making it really tough to protect this environment efficiently.
• IBM Data Protect Plus: a new product designed from the ground up for modern environments, including hypervisors, NoSQL DBs and more. It has a very modern snapshot-based design that pairs nicely with VMWARE CBT (change block tracking) for example. It’s easy to use and can be adopted by IT organizations of all sizes.

Videos from SFD18 can give you a good idea of the features and the potential of IBM Spectrum Data Protect Plus and there are a few aspects that I think are interesting to note:

IBM Spectrum Data Protect Plus might be a good companion for IBM TSM customers. Although the two don’t share anything, it is still an IBM product and from the procurement and budgetary standpoints, it could be much easier to adopt this solution instead of others.

Licensing is pretty flexible, making this product competitive from a cost standpoint on smaller infrastructures too. And this also makes it easier to place it in large infrastructures, aligning the cost with what is actually being protected.

Data Protect Plus is not at the level of features you can find on more mature products like Veeam, but the Data Protect Plus team is very committed and have a very aggressive release schedule.

This product has a good, scalable, architecture and the roadmap shows great potential for future releases, especially when it comes to sophisticated features around data reuse and management.

CLOSING THE CIRCLE

As I wrote above, Data Protect Plus might be a good option for IBM customers that already have TSM for their legacy infrastructure. What the IBM Spectrum Data Protect family is lacking the most for this type of customer at the moment, is a sort of unified GUI to allow SysAdmins to speed up operations and have better control of the backup infrastructure. But, as far as I can tell, it seems I’m not the first one to note this deficiency … the development team is already looking into it.

NetApp NDAS Integrate On-Premises & Cloud as One

At the end of February 2019, at Storage Field Day 18, NetApp presented another tool aimed at integrating its on-premises solutions with the cloud, NetApp Data Availability Service (NDAS). As already mentioned in a previous post, this tool might be somewhat immature, but it has huge potential if developed in the right way.

TWO WORDS ABOUT NDAS

Long story short, NDAS takes advantage of snap mirror functionalities available on NetApp Arrays and syncs volumes to the cloud. The cool part is that the content of the volumes is converted into objects (taking advantage of AWS S3). In the short term, it’s all about saving money because S3 is way cheaper than Elastic Block Storage (EBS), but the real deal comes from the fact that data stored in this format is much more re-usable for a major number of use cases, including index and search, ransomware protection, analytics and so on. Take a look at the videos recorded during their SDF18 session to have an idea of what I’m talking about.

NETAPP IS (ALSO) A CLOUD COMPANY

NetApp already had me a few years back when they presented their Data Fabric vision. But, as is the case for any other vision, it’s only good on paper until it gets executed properly. If I needed confirmation about how they are executing, NDAS, and especially the speech that Dave Hitz gave at the beginning of the session, were what sustained my excitement level about them. He gave clear examples of their cloud products, their approach with customers, of partnerships that are all-in with cloud, and of cloud-native end users — all while keeping an eye on traditional customers and how to support them in their journey to the cloud.

The risk they are taking is to cannibalize some of their on-premises sales or, better, their traditional product sales … but this is paying off, both in terms of mindshare as well as overall results. And if we look at company growth and financial results in the last three years, they seem nothing but positive.

CLOSING THE CIRCLE

Many storage vendors are now more cloud-focused than in the past. NetApp just started sooner and had the courage to disrupt its internal status quo, they listened to end users, hired people with a different mindset, designed new cloud-focused products and services; but they also opened their core products to better cloud integration. And this is paying off big time.

Usually, you can expect this kind of turnaround from smaller, younger, and nimbler companies but it’s always refreshing to see it happening to those of NetApp’s size.

Originally posted on Juku.it