In an article by Will Knight for Wired that was published yesterday, it was reported Nvidia, the renowned chipmaker, has successfully integrated the powerful language model GPT-4 into the popular video game Minecraft.
The Wired article provides insights into the development of Voyager, a Minecraft bot created by Nvidia’s team, featuring AI researcher Linxi “Jim” Fan and Anima Anandkumar, the company’s machine learning director and a Caltech professor.
Distinguishing itself from human players, Voyager leverages an API to read the game’s state directly. Powered by GPT-4, it generates objectives to navigate and enhance its game skills over time. For example, if Voyager possesses a fishing rod and detects a nearby river, GPT-4 suggests the goal of fishing to gain experience. Subsequently, it utilizes GPT-4 to generate code that enables the character to accomplish the goal.
The project’s standout aspect lies in the code that GPT-4 generates to imbue Voyager with behaviors. If the initial code iteration encounters issues, Voyager iteratively refines it using error messages, game feedback, and GPT-4’s code description. Over time, Voyager builds a code library, enabling it to create increasingly intricate structures and explore the game further.
A comparative chart showcases Voyager’s superior capabilities in acquiring items, exploration range, and tool-building speed compared to other Minecraft agents. Fan suggests that incorporating visual information from the game could enhance the approach in the future.
The Wired article goes on to say that the process that Voyager uses with GPT-4 to figure out how to do things in Minecraft might be adapted for a software assistant that works out how to automate tasks via the operating system on a PC or phone.
Featured Image Credit: Microsoft
