Transhumanismus

The Brain Predicts Reward Like an AI, Says New DeepMind Research

Singularity HUB - 59 min zpět

The idea of reinforcement learning—or learning based on reward—has been around for so long it’s easy to forget we don’t really know how it works.

If DeepMind’s new bombshell paper in Nature is any indication, a common approach in AI, one that’s led to humanity’s defeat in the game of Go against machines, may have the answer.

We all subconsciously learn complex behaviors in response to positive and negative feedback, but how that works in the brain remains a century-long mystery. By examining a powerful variant of reinforcement learning, dubbed distributional reinforcement learning, that outperforms original methods, the team suggests that the brain may simultaneously represent multiple predicted futures in parallel. Each future is assigned a different probability, or chance of actually occurring, based on reward.

Here’s the kicker: the team didn’t leave it as an AI-inspired hypothesis. In a collaboration with a lab at Harvard University, they recording straight from a mouse’s brain, and found signs of their idea encoded in its reward-processing neurons.

“Neuroscience and AI have always inspired one another. Reinforcement learning is one of the central approaches in contemporary AI research and was originally motivated by thinking about how rewards could be used to reinforce certain behaviors,” said Dr. Demis Hassabis, co-founder and CEO of DeepMind. “In this new work, we find that distributional reinforcement learning…may also be at work in human biology.”

Dopamine Rush

You’ve heard of “reward-based learning.” Take Pavlov’s dogs, a famous experiment in the early 1900s. Russian physiologist Ivan Pavlov repeatedly rang a bell before feeding his dogs, and found the dogs learned to instinctively salivate as soon as they heard the bell, no food in sight. They learned that the bell—something not associated with food at all—predicted food, the reward.

Neuroscientists eventually figured out that dopamine, a chemical messenger active in reward circuits in the brain, is involved in processing reward signals. Although dopaminergic neurons—neurons that release dopamine—are often associated with a “rush” or “high,” that’s not to say they make us “happy” or “feel good” per se.

Rather, dopaminergic neurons are high-rollers at a betting game. They constantly make predictions about the chance of receiving a reward, and only change how much dopamine they release if the prediction is off. A spot-on bet—mom calls you to lunch, you get lunch—and the neurons stay quiet. If the reward is better or larger than predicted, however, they amp up their activity and shoot off packets of dopamine; if the result is worse or less than expected, the neurons lower their activity. With time, dopaminergic neurons learn to adjust their predictions to better match the real world. In other words, dopamine doesn’t equate to “happiness” or “high”; rather, it responds to predictions’ errors. We learn by trying to correct that error.

These ideas aren’t just hypotheses. Using electrodes and chemical probes inside living brains, neuroscientists can watch dopaminergic neurons in action as animals are challenged with tasks that eventually lead to rewards—food, water, sex, and drugs.

But a question remained: do dopaminergic neurons work as a unit, encoding for a single expected outcome? Or do they diverge in their predictions?

Meanwhile in AI Land…

While neuroscientists busily examined living brains, AI researchers directly tested their high-level ideas in machines.

Enter reinforcement learning. When challenged with a task, an AI algorithm starts out with random predictions. It then takes an action, observes whether it gets a reward, and adjusts its predictions based on reality. After millions of trials, the AI hopefully minimizes its prediction errors, meaning that it knows exactly how to solve the task. Step by step, it can then tackle extremely complex problems, such as beating a human champion at Go.

That’s the standard narrative. When AI researchers dug into the weeds, however, they immediately faced a head-scratcher: how do you represent the chance for reward? The traditional approach is to give it an average number—a general “gut feeling,” so to speak, based on classic reward-learning theory in neuroscience. But in the real world, the chance of getting a reward is never perfectly averaged into a fixed number.

“When playing the lottery, for example, people expect to either win big, or win nothing—no one is thinking about getting the average outcome,” said first author Dr. Will Dabney, a research scientist at DeepMind.

In 2017, DeepMind researchers decided to encode that randomness into reinforcement learning. Rather than affixing a single number for prediction error, they swapped it for a sophisticated distribution of probabilities. In distributional reinforcement learning, the AI algorithm predicts a full spectrum of future rewards: some are more optimistic and amplify their reward signals when the reward is larger than expected; others more pessimistic, lowering their reward signals when it’s smaller than predicted.

Sound familiar? The ties to the brain’s reward-based predictive powers are hard to ignore. This led the DeepMind team to ask: can we see a similar distribution in individual dopaminergic neural predictions as well?

Come Together

Partnering with Harvard, the teams tested out their idea in the brains of mice.

The mice were taught to work for a juice reward, but how much tasty juice—a drop or a gush—was highly unpredictable, basically relying on the roll of a seven-sided die. As the mice struggled to predict their next sugary “hit,” the team watched the electrical activity of individual dopaminergic neurons in their brains’ reward regions.

In contrast to neuroscience canon, the team said, dopaminergic neurons didn’t act as one. Rather than collectively encoding for a single expected outcome, they were each “tuned” to a different prediction, with some expecting a larger amount of juice reward, and others less hopeful, predicting smaller volumes. When the team mapped out the whole swath of predictions, the distribution closely matched reality—the distribution of actual rewards.

“We found that dopaminergic neurons in the brain were each tuned to different levels of pessimism or optimism. If they were a choir, they wouldn’t all be singing the same note, but harmonizing,” the authors wrote.

In other words, they seemed to operate on very similar principles to distributed reinforcement learning, a powerful method in AI.

That’s great news for both AI and neuroscience. Although AI researchers empirically already know that distributional reinforcement learning, combined with deep neural networks, is extremely powerful, the new data further validates it as a potential path towards AI that learns in a manner more similar to human brains—the only example of high-level intelligence we know of.

“When we’re able to demonstrate that the brain employs algorithms like those we are using in our AI work, it bolsters our confidence that those algorithms will be useful in the long run—that they will scale well to complex real-world problems, and interface well with other computational processes. There’s a kind of validation involved: if the brain is doing it, it’s probably a good idea,” said senior author Dr. Matt Botvinick, Director of Neuroscience Research at DeepMind.

But neuroscience also has much to gain. If the brain follows a similar reward “program,” what happens if it preferentially listens to optimistic rather than pessimistic dopaminergic neurons? Does it cause impulsivity or depression? What mechanisms in the brain’s reward circuits, or single dopaminergic neurons, can shift reward prediction? Do these changes lead to drug addiction, gambling, and other unhealthy behaviors? And finally, by borrowing “back” ideas from AI, can we finally understand what motivates us, why we take risks, and what makes us tick?

“As the fields of AI and neuroscience try to understand the nature of intelligence, we continually borrow methods and inspiration from each other. I feel extremely lucky to have been able to add to this ongoing conversation, as we try to piece together the fundamental algorithms behind learning and intelligence, be it biological or artificial,” said Dabney.

Image Credit: Image by Raman Oza from Pixabay

Kategorie: Transhumanismus

Wise Practitioner – Predictive Analytics Interview Series: Bas Geerdink at Aizonic

Home AI - 2 hodiny 30 min zpět

Roles that guide the process and practice of machine learning will become more common, such as the Analytics Translator and AIOps Engineer.


Link to Full Article: Read Here

Kategorie: Transhumanismus

AI can help find illegal opioid sellers online. And wildlife traffickers. And counterfeits.

Home AI - 2 hodiny 1 min zpět

That's where data — lots of it — and artificial intelligence come in. Last year, Mackey and his team used a type of artificial intelligence called deep …


Link to Full Article: Read Here

Kategorie: Transhumanismus

How to Manage the Human-Machine Workforce

Home AI - 2 hodiny 1 min zpět

Today's managers and executives need to oversee humans and machines in this age of AI … Chatbots are everywhere and so is machine learning.


Link to Full Article: Read Here

Kategorie: Transhumanismus

Stellar Cyber Supercharges Firewalls With Traffic Analysis App

Home AI - 3 hodiny 3 min zpět

New App Applies Machine Learning to Firewall Data to Spot Anomalies. Security provider Stellar Cyber, with the first Open-XDR platform, today …


Link to Full Article: Read Here

Kategorie: Transhumanismus

How Data Science is Shaping Fintech?

Home AI - 3 hodiny 14 min zpět

Fintech companies majorly depend on AI, machine learning, predictive analytics and data science to provide superior services. Let us see how fintech …


Link to Full Article: Read Here

Kategorie: Transhumanismus

GSK bets on tech to help boost lagging R&D as it looks to start 23andMe trial

Home AI - 3 hodiny 36 min zpět

The latent power of artificial intelligence, machine learning and the financial might of Google are all trendy right now, but there's still a lot to prove; we'll …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Department of Homeland Security Funds Synapse Technology to Develop Artificial Intelligence …

Home AI - 3 hodiny 48 min zpět

Synapse Technology Corporation received a contract from DHS's Silicon Valley Innovation Program to develop artificial intelligence technology for …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Artificial Intelligence Predicted Trend for “Dead Artists Tours”

Home AI - 3 hodiny 59 min zpět

Already in June 2019, the Signum.ai trend-forecast artificial intelligence predicted an increased global interest in hologram tours. As we see, the …


Link to Full Article: Read Here

Kategorie: Transhumanismus

JetBrains to reimagine IntelliJ as text editor, add machine learning

Home AI - 3 hodiny 59 min zpět

JetBrains has added further destinations to the IntelliJ-based roadmap it sketched out last year, promising more localization, machine learning and Git …


Link to Full Article: Read Here

Kategorie: Transhumanismus

AI Can Do Great Things—If It Doesn’t Burn the Planet

Home AI - 3 hodiny 59 min zpět

“The concern is that machinelearning algorithms in general are consuming more and more energy, using more data, training for longer and longer,” …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Macworld speculates that Apple bought Xnor.ai to make Siri smarter

Home AI - 3 hodiny 59 min zpět

Its biggest focus is on machine learning and image recognition tools that can run on low-power devices, rather than relying on cloud infrastructure […].


Link to Full Article: Read Here

Kategorie: Transhumanismus

Global Deep Learning System Software Market Segmentation and Forecasts 2019 to 2025. By …

Home AI - 4 hodiny 10 min zpět

We, Global Marketers.biz, after comprehensive analysis, have introduced a new research study on Global Deep Learning System Software Market.


Link to Full Article: Read Here

Kategorie: Transhumanismus

Applying Occam’s Razor to Deep Learning

Home AI - 4 hodiny 1 min zpět

The surge in interest in using complex neural network architectures, i.e., deep learning due to their unprecedented success in certain tasks, pushes …


Link to Full Article: Read Here

Kategorie: Transhumanismus

“Edge AI”: encapsulating machine learning classifiers in lightweight, energy-efficient, airgapped chips

Home AI - 4 hodiny 1 min zpět

Writing in Wired, Boing Boing contributor Clive Thompson discusses the rise and rise of "Edge AI" startups that sell lightweight machinelearning …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Artificial intelligence developers need external regulation

Home AI - 4 hodiny 1 min zpět

Artificial intelligence (AI), we are told, will transform our lives. The technology is still in its infancy; we will not be seeing the all-knowing AI portrayed in …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Improved speckle-pattern analysis helps see round corners

Home AI - 4 hodiny 33 min zpět

The creation of an accurate noise model for the initial NLoS correlography operation is key, and deep-learning approaches have traditionally had to …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Automotive Artificial Intelligence Market: Intelligence Study on Transformation 2025

Home AI - 4 hodiny 44 min zpět

Research Report Inc. recently published an informational report entitled “Automotive Artificial Intelligence Market”. The report contains a focused …


Link to Full Article: Read Here

Kategorie: Transhumanismus

Artificial Valve Market (2020 To 2027) Global Industry Analysis with Top Players Colibri Heart …

Home AI - 4 hodiny 44 min zpět

The Report Titled on “Deep Learning Market” firstly presented the Deep Learning fundamentals: Scope, Definition, Classifications, Segmentation, …


Link to Full Article: Read Here

Kategorie: Transhumanismus

UK Invests in Revolutionary Artificial Intelligence Warships

Home AI - 4 hodiny 44 min zpět

s Ministry of Defense recently announced contracts to use AI-based (artificial intelligence) technology in warships. According to a source, Defense and …


Link to Full Article: Read Here

Kategorie: Transhumanismus
Syndikovat obsah