A certain deep learning engineer has built a demonstration that generates a game screen of Pokémon . They succeeded in detecting the player’s operation and reproducing the like video, and has been published so that it can be actually operated on the browser.
Pokémon (Pokémon) is Nintendo’s popular RPG series. This work is characterized by a system that captures and nurtures Pokémon and challenges battle. In addition, from the first Pokémon Red / Green, the main series work was long-looking from the perspective. As for the drawing, it has become colorful and high-definition with each generation from 2D monochrome, and many fans will have a deep look-looking style.
It seems that some engineers who built a demonstration using a neural network to generate a game screen of the Pokémon-style Pokémon from such a familiar view. OLLIE BOER BEHAN is a deep learning scientist at NVIDIA. On September 5, local time, he released a Pokémon screen reproduction demonstration that actually works on his site. Immediately after that, he published a video on his own Twitter account.
If you look at the demonstration created by Behan, the rough impression of the screen is certainly Pokémon itself. From the drawing of the protagonist and landscape, the base is like Pokémon his diamond pearl. It seems that the operation is also accepted, and the hero walks up, down, left and right according to the keyboard input, and the landscape moves. On the other hand, if you look closely, the video has a slightly disturbing atmosphere. Because the drawing is not stable.
In the video, objects such as grass and steps disappear as the protagonist moves, and the roads that have been previously disappeared, and a new landscape appears suddenly. Even when the hero is still stationary, the world fluctuates slightly and changes. Everything is unstable and fluid while protecting the atmosphere of Pokémon. It is a strange and attractive image that is like a nightmare or hallucinations, but is crazy. In addition, this demonstration can be actually operated and walked around on the browser.
On the other hand, this demonstration does not reproduce the deep gameplay element. In addition to the movement and the atmosphere of the screen, the behavior of entering the building when approaching the building is only occasionally reproduced. This is because this demonstration only plays the screen in the so-called Let’s see.
The neural network that Behan used in this demonstration is a mathematical model that imitates the human brain nervous structure. It is a method used for machine learning, and as an example of use, an example of generating images like nightmares from the original image data has become a hot topic (related article). Behan gave the real Pokémon play video data to this neural network model and made it imitated.
For this reason, Behan’s demonstration was just a blurry reproduction of the screen and the pretend of the input reception on the top, down, left and right. There is a game program inside, and it is not working, but only draws the screen with guess that this kind of operation will be like this. Therefore, it was natural that the landscape changed every time I moved. In this demonstration, operations such as penetrating the wall can be performed, in which case the screen disappears, such as the screen disappears. It seems that the neural network is confused and an interesting reaction.
Behan’s site also has a deep technical explanation about this demonstration. What is a neural network? What data is interpreted and how it leads to a demo-like output. The purpose of Behan’s release of this demo and pages seemed to deepen the understanding of those who were interested in the demonstration of neural networks and machine learning.
He also states the insight into the future, such as neural networks reproduce the whole game. If this accuracy of seeing to see improves, the theoretically, you can also reproduce game systems with neural networks only. At present, technologies such as deep learning and neural networks are utilized in super-resolution technologies such as DLSS and are used by developers/users. There will be a wealth of examples of use in the future.