Tech 2015: Deep Learning And Machine Intelligence Will Eat The World


Anthony Wing Kosner, Contributor

December 31, 2014

Despite what Stephen Hawking or Elon Musk say, hostile Artificial Intelligence is not going to destroy the world anytime soon. What is certain to happen, however, is the continued ascent of the practical applications of AI, namely deep learning and machine intelligence. The word is spreading in all corners of the tech industry that the biggest part of big data, the unstructured part, possesses learnable patterns that we now have the computing power and algorithmic leverage to discern—and in short order.

The effects of this technology will change the economics of virtually every industry. And although the market value of machine learning and data science talent is climbing rapidly, the value of most human labor will precipitously fall. This change marks a true disruption, and there are fortunes to be made. There are also tremendous social consequences to consider that require as much creativity and investment as the more immediately lucrative deep learning startups that are popping up all over (but particularly in San Francisco.)

Shivon Zilis, an investor at BloombergBETA in San Francisco, put together the graphic below to show what she calls the Machine Intelligence Landscape. The fund specifically focuses on “companies that change the world of work,” so these sorts of automation are a large area of concern. Zilis explains, “I created this landscape to start to put startups into context. I’m a thesis-oriented investor and it’s much easier to identify crowded areas and see white space once the landscape has some sort of taxonomy.”

What is striking in this landscape is how filled-in it is. At the top are core technologies that power the applications below. Big American companies like Google, IBM, Microsoft, Facebook and China’s Baidu are well-represented in the core technologies themselves. These companies, particularly Google, are also the prime bidders for core startups as well. Many of the companies that describe themselves as engaging in artificial intelligence, deep learning or machine learning have some claim to general algorithms that work across multiple types of applications. Others specialize in the areas of natural language processing, prediction, image recognition and speech recognition.

For the companies that are rethinking enterprise processes like sales, marketing, security or recruitment, or for others that are remaking industry verticals, the choices of technologies to license are dizzying. As Pete Warden, creator of the open source Data Science Toolkit, wrote in a recent post on deep learning, “I don’t see any reason why the tools we use to develop… and train networks, should be used to execute them in production.” Entering 2015 we see all of this research finding its way into actual applications that relatively ordinary humans will use. “I also think we’ll end up with small numbers of research-oriented folks who develop models,” Warden continues, “and a wider group of developers who apply them with less understanding of what’s going on inside the black box.”

These companies will need more people who can create, iterate and debug deep learning and other kinds of machine learning models. They will also need an even larger cohort of developers and designers who can create usable experiences on screens that make all of this intelligence actionable. Big companies are poised to be the big winners here. Obviously they have the resources to attract or acquihire this talent. Even more crucial, big companies have big data and ongoing relationships with large numbers of customers. In machine learning, it is most often the quality and quantity of data available that is the limiting factor, not the cleverness of the algorithms.

And what most concerns the big tech companies from Apple to Google to Microsoft and IBM? Yep, mobile, and as Zilis points out, “Winning mobile will require lots of machine intelligence.” Siri and Google Now are responses to the need for highly contextual voice interaction in mobile. Visual search like Amazon’s FireFly involves location-based pattern recognition to create a pleasing experience. The reason for the current great enthusiasm for deep learning is that these kinds of problems can be solved now in minutes or days instead of years.

One of the most enthusiastic proponents of deep learning is Jeremy Howard, now the founder and CEO of medical diagnostic startup Enlitic. Howard was previously President and Chief Scientist of Kaggle, an open platform for machine learning competitions. Tellingly, Howard claims no previous medical industry experience. One of the big selling points of deep learning is that it is a general technology that does not require extensive domain knowledge to create effective solutions. In his recent TEDxBrussels talk (see video below) Howard gives a history of machine learning. He also explains how deep learning is now becoming capable of providing the kinds of services that currently employ 80% of the developed world.

Lest you think Howard, and other deep learning practitioners are heartless nerds bent on world domination, there is a positive human side to this as well. One of the things that led him into medical diagnostics was the profound need for these services. The developing world currently has less than a tenth the number of trained doctors required to deliver adequate health care to the majority of the world’s population. Training the required number of doctors using current methods would take 300 years!

Machine intelligence, broadly speaking, is a set of technologies that will solve some problems and cause others. Among all of the enterprise processes and industry applications that are being developed it is not at all clear what the true “killer apps” will be. As Jeff Hawkins, inventor of the Palm Pilot and now CEO of the AI company Numenta told interviewer Derrick Harris recently in GigaOM, “history tells you that the obvious applications are not the killer apps.… The killer apps tend to be surprising. No one anticipates them.”

Numenta itself offers an instructive contrast to the enthusiasm for deep learning. Although deep learning uses neural networks, Hawkins claims that the Numenta approach is significantly more brain-like. The difference is that Numenta’s method uses Hierarchical Temporal Memory (HTM), which can natively discern patterns in time, as well as computational space. The first commercial product the company has made using this technology, Grok, detects anomalies in servers and applications running on Amazon Web Services (AWS).

When Hawkins talks about the brain, he is more precisely describing the neocortex, the center of higher-level human capabilities. And neural networks, which have been in development since the 1940s, are only loosely related to the way the brain actually works. There is a big gap between the pieces of cortical architecture that machine intelligence practitioners are codifying and the much messier evolutionary system of systems that is the human brain.

Machine intelligence does not need to resemble the human brain at all. As Jeremy Howard said in a recent AMA on Reddit, “The more interesting question is: what can machines do? Not ‘are they truly intelligent?’ Machine ‘intelligence’ is different enough from human intelligence that I don’t think it is a terribly useful analogy.” What neuroscience and cognitive psychology do inform is an understanding of what kinds of tasks are performed by which systems and circuits of systems in the brain.

Deep learning, for instance, is very well-suited to sorting and categorizing problems that humans do mostly without conscious effort. These are functions of what Danial Kahneman calls “system 1,” the fast part of thinking. The Numenta approach, with its emphasis on time-based anomalies, could be said to be more a function of the brain’s threat detection system, “system 2,” the slower thinking that questions the quick associations of the neocortex. But just as well, it could have a more cortical referent in the brain’s language system. Neuroscientist Daniel Levitin has identified Brodmann area 47, on the sides of the temples, as a place where we process our temporal expectations in language and music. As he writes in This is Your Brain on Music, much of the pleasure of music has to do with how it plays with these expectations, often using anomalies to delight or temporarily frustrate us.

It is important to emphasize that as impressive as some recent demonstrations of machine intelligence are, there is still a lot of work required to turn this technology into usable products for people. A lot of this has to do with understanding what can easily be done (the fast thinking) vs. what takes more time (the slow.) Deep learning has become even more accurate than humans at image classification (see Howard’s video above to see just how good) but certain language tasks are still a reach. Many prediction algorithms are pretty good now at general sentiment analysis but still stumble on irony and some types of negation and ambiguity.

Machine intelligence in general and deep learning in particular will have a significant impact on what happens in technology in the coming year. Large tech companies with vast data holdings will be particularly motivated to extract value from all of this data now that there appears to be a scalable way to do so. On the other end of the spectrum, app developers will be encouraged to ramp up the input factories they entice people to place on their smartphones now that the output has a clear value. Machine learning and data science talent will continue to move from academia to big companies like Google for access to all that data. This will not be the year that 80% of the developed world loses their jobs to intelligent machines, but it is not too soon to start figuring out what to do about that eventuality. These same machine technologies can help us redefine and redistribute human value, and we will need to use them for that as well.

Great ! Thanks for your subscription !

You will soon receive the first Content Loop Newsletter