Among the major announcements at Google I/O this week was PaLM 2:
Google’s latest artificial intelligence language model that will compete with systems like OpenAI’s GPT-4. “The PaLM 2 language model is stronger in logic and reasoning, thanks to extensive training in logic and reasoning,” Google CEO Sundar Pichai said on stage at the company’s I/O conference. “It is also trained on multilingual texts covering over 100 languages.
PaLM 2 is much better at a number of text-based tasks, Google’s senior director of research, Slav Petrov, told reporters. “It is significantly improved compared to PaLM 1 [which was announced in April 2022],” said Petrov. As an example of its multilingual capabilities, Petrov showed how PaLM 2 is able to understand idioms in different languages, giving the example of the German phrase “Ich verstehe nur Bahnhof”, which literally translates as “I only understand the train station”, but is better understood as “I don’t understand what you’re saying” or, as an English idiom, “it’s all Greek to me”.
In a research paper describing the capabilities of PaLM 2, Google engineers claimed that the language proficiency of the system is “sufficient to teach this language” and noted that this was partly due to the higher prevalence of non-English texts in the training data.
Like other large language models, which require enormous amounts of data, time, and resources to create, PaLM 2 is not so much a single product as a family of products. Its different versions are expected to be deployed in consumer and business environments.
The system is available in four sizes, named Gecko, Otter, Bison, and Unicorn, from smallest to largest, and is configured with domain-specific data to perform certain tasks for enterprise customers.
Think of these customizations as taking a basic truck chassis and adding a new engine or front bumper to make it perform certain tasks or work better on certain terrain. There is already a version of PaLM trained on health data (Med-PaLM 2), which Google says can answer questions similar to those on US medical licensing exams at the “specialist” level. Another trained on cybersecurity data (Sec-PaLM 2), which can “explain the behavior of potential malicious scripts and help identify threats in the code,” Petrov said. Both of these models will be available through Google Cloud, initially to select customers. As for Google itself, PaLM 2 is already behind 25 of the company’s features and services including Bard, the company’s experimental chatbot.
Updates available through Bard include improved coding capabilities and greater language support. It is also used for AI functions in Google Workspace online applications such as Docs, Slides and Sheets. In particular, Google says that the lighter version of PaLM 2, Gecko, is small enough to run on mobile phones, processing 20 tokens per second – the equivalent of about 16 or 17 words. Google didn’t say what hardware was used to test this model, other than that it runs “on the latest smartphones.
Nevertheless, the miniaturization of such language models is important. Such systems are expensive to run in the cloud, and being able to use them locally would have other benefits, such as improving privacy. The problem of course is that smaller versions of language models are inevitably less capable than their older siblings.
With PaLM 2, Google will hope to close the “artificial intelligence gap” between the company and rivals such as Microsoft, which has aggressively pushed artificial intelligence language tools into its Office software suite.
Microsoft now offers AI features that help summarize documents, compose emails, create slides for presentations, and more. Google will have to introduce at least equal features or risk being seen as slow to implement its AI research. While PaLM 2 is certainly a step forward for Google’s work on AI language models, it suffers from problems and challenges common to the technology more broadly. For example, some experts are beginning to question the legitimacy of training data used to build language models.
This data usually comes from the internet and often includes copyrighted texts and pirated e-books. The technology companies that build these models refuse to answer questions about where they get their training data from. Google continued this tradition in its description of PaLM 2, noting only that the central training part of the system consists of “a diverse set of sources: web documents, books, code, math, and chat data,” without elaborating.
There are also the well-known problems in the results of AI language models such as “hallucinations” or the tendency of these systems to simply make up information. Speaking to The Verge, Google’s vice president of research, Zoubin Ghahramani, says that, in this respect, PaLM 2 is an improvement over previous models “in the sense that we are putting a huge effort into continuously improving performance indicators and of good fortune.
At the same time, he notes that the sector as a whole “still has a way to go” in combating false information generated by artificial intelligence.
0 Comments