<\/p>\n
For a decade<\/span> now, many of the most impressive artificial intelligence systems have been taught using a huge inventory of labeled data. An image might be labeled \u201ctabby cat\u201d or \u201ctiger cat,\u201d for example, to \u201ctrain\u201d an artificial neural network to correctly distinguish a tabby from a tiger. The strategy has been both spectacularly successful and woefully deficient.<\/p>\n Such \u201csupervised\u201d training requires data laboriously labeled by humans, and the neural networks often take shortcuts, learning to associate the labels with minimal and sometimes superficial information. For example, a neural network might use the presence of grass to recognize a photo of a cow, because cows are typically photographed in fields.<\/p>\n \u201cWe are raising a generation of algorithms that are like undergrads [who] didn\u2019t come to class the whole semester and then the night before the final, they\u2019re cramming,\u201d said Alexei Efros, a computer scientist at the University of California, Berkeley. \u201cThey don\u2019t really learn the material, but they do well on the test.\u201d<\/p>\n For researchers interested in the intersection of animal and machine intelligence, moreover, this \u201csupervised learning\u201d might be limited in what it can reveal about biological brains. Animals\u2014including humans\u2014don\u2019t use labeled data sets to learn. For the most part, they explore the environment on their own, and in doing so, they gain a rich and robust understanding of the world.<\/p>\n Now some computational neuroscientists have begun to explore neural networks that have been trained with little or no human-labeled data. These \u201cself-supervised learning\u201d algorithms have proved enormously successful at modeling human language and, more recently, image recognition. In recent work, computational models of the mammalian visual and auditory systems built using self-supervised learning models have shown a closer correspondence to brain function than their supervised-learning counterparts. To some neuroscientists, it seems as if the artificial networks are beginning to reveal some of the actual methods our brains use to learn.<\/p>\n Flawed Supervision<\/p>\n Brain models inspired by artificial neural networks came of age about 10 years ago, around the same time that a neural network named AlexNet revolutionized the task of classifying unknown images. That network, like all neural networks, was made of layers of artificial neurons, computational units that form connections to one another that can vary in strength, or \u201cweight.\u201d If a neural network fails to classify an image correctly, the learning algorithm updates the weights of the connections between the neurons to make that misclassification less likely in the next round of training. The algorithm repeats this process many times with all the training images, tweaking weights, until the network\u2019s error rate is acceptably low.<\/p>\n Alexei Efros, a computer scientist at the University of California, Berkeley, thinks that most modern AI systems are too reliant on human-created labels. \u201cThey don\u2019t really learn the material,\u201d he said.<\/span>Courtesy of Alexei Efros<\/span><\/p>\n<\/figure>\n<\/div>\n Around the same time, neuroscientists developed the first computational models of the primate visual system, using neural networks like AlexNet and its successors. The union looked promising: When monkeys and artificial neural nets were shown the same images, for example, the activity of the real neurons and the artificial neurons showed an intriguing correspondence. Artificial models of hearing and odor detection followed.<\/p>\n But as the field progressed, researchers realized the limitations of supervised training. For instance, in 2017, Leon Gatys, a computer scientist then at the University of T\u00fcbingen in Germany, and his colleagues took an image of a Ford Model T, then overlaid a leopard skin pattern across the photo, generating a bizarre but easily recognizable image. A leading artificial neural network correctly classified the original image as a Model T, but considered the modified image a leopard. It had fixated on the texture and had no understanding of the shape of a car (or a leopard, for that matter).<\/p>\n Self-supervised learning strategies are designed to avoid such problems. In this approach, humans don\u2019t label the data. Rather, \u201cthe labels come from the data itself,\u201d said Friedemann Zenke, a computational neuroscientist at the Friedrich Miescher Institute for Biomedical Research in Basel, Switzerland. Self-supervised algorithms essentially create gaps in the data and ask the neural network to fill in the blanks. In a so-called large language model, for instance, the training algorithm will show the neural network the first few words of a sentence and ask it to predict the next word. When trained with a massive corpus of text gleaned from the internet, the model appears to learn the syntactic structure of the language, demonstrating impressive linguistic ability\u2014all without external labels or supervision.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":" For a decade now, many of the most impressive artificial intelligence systems have been taught using a huge inventory of labeled data. An image might be labeled \u201ctabby cat\u201d or \u201ctiger cat,\u201d for example, to \u201ctrain\u201d an artificial neural network to correctly distinguish a tabby from a tiger. The strategy has been both spectacularly successful […]<\/p>\n","protected":false},"author":1,"featured_media":1214836,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-1214835","post","type-post","status-publish","format-standard","has-post-thumbnail","category-celebrity-gossip"],"_links":{"self":[{"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/posts\/1214835","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/comments?post=1214835"}],"version-history":[{"count":0,"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/posts\/1214835\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/media?parent=1214835"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/categories?post=1214835"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ghanamma.com\/2022\/wp-json\/wp\/v2\/tags?post=1214835"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}