LogInThe 540-billion parameter model can do different tasks and even explain jokes.
Sharan Narang and Aakanksha Chowdhery, engineers from Google Research, presented the Pathways Language Model (PaLM) – a 540-billion parameter, densely activated, Transformer language model that can perform natural language tasks.
The researchers trained PaLM on 6144 TPU v4 chips using Pathways, an ML system that enables efficient training across multiple TPU Pods. The model can not only solve math problems and show logical reasoning, but it can also explain jokes (and what TPUs are).
PaLM was evaluated on 29 widely-used English natural language processing tasks, and it outperformed other large models, such as GLaM, GPT-3, Megatron-Turing NLG, Gopher, Chinchilla, and LaMDA. It was also tested on the Beyond the Imitation Game Benchmark (BIG-bench), a suite of over 150 language modeling tasks, where PaLM needed to identify how two objects are similar, solve tasks checking its abstraction and reasoning capabilities, and more. The model showed great results there: it can distinguish cause and effect, understand conceptual combinations in appropriate contexts, and even guess the movie from an emoji.
PaLM can also be used for more practical tasks in robotics and other fields, although it might raise some ethical questions, so the creators are working slowly and thoughtfully on the model to build technology that will serve many uses across the globe.
Find out more about PaLM here and don't forget to join our Reddit page and our Telegram channel, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.