A new language model for text generation, combining Google’s own PaLM model and a technique known as reinforcement learning with human feedback to create open source a tool that theoretically can do everything ChatGPT OpenAI can do.
However, for most, this will remain just a theory. Unlike ChatGPT, the creator of AI Philip Wang PALM + RLHF (opens in a new tab) it is not trained on any of the textual data required to train the model. Users must build their own data corpora and use their own hardware to train the model and process requests.
Text generation models that respond to human input, such as ChatGPT and PaLM+RLHF, are the latest AI craze. Simply put, they predict the right words after learning semantic patterns from an existing data set, which can consist of anything from e-books to online flame wars.
Creating accessible AI
Even though the PaLM+RLHF are pre-trained, the human feedback reinforcement learning technique aims to provide a more intuitive user experience.
As explained by TechCrunch (opens in a new tab)RLHF trains a language model by producing a wide range of responses to a human prompt, which are then categorized by volunteers. These rankings are then used to train a “reward model” that sorts responses by preference.
This is not a cheap process that will make model training impossible for all but the wealthiest AI enthusiasts. PaLM has 540 billion language model components (or parameters) that need to be trained on data, and 2020 test (opens in a new tab) revealed that training a model with just 1.6 billion parameters would cost between $80,000 and $1.6 million.
At the moment we seem to rely on a wealthy benefactor to get involved, train and release the model publicly. This reliance did not end well before (opens in a new tab)but other companies are trying to replicate the capabilities of ChatGPT and make it available as free software.
Research groups CarperAI and EleutherAI are partnering with startups Scale AI and Hugging Face to release the first human-trained language model that is ready to use right out of the box.
And while it’s not quite ready yet, LAION, the company that provided the training dataset for the “machine learning, text-to-image” model Stable diffusion (opens in a new tab)did a similar project on Github (opens in a new tab) which wants to replace OpenAI, allowing it to use APIs, compile its own research, and allow user personalization, all while optimizing consumer hardware.