Facebook announces Ego 4D first-person video dataset for AI training

Facebook Inc. Mark Zuckerberg, Chief Executive Officer and Founder of the House of Representatives, spoke at a House Energy and Commerce Commission hearing in Washington, DC, USA on Wednesday, April 11, 2018.

Andrew Haller | Bloomberg | Getty Images

Facebook On Thursday, we announced a research project to collect 2,200 hours of first-person footage from around the world to train the next generation of artificial intelligence models.

This project is called Ego4D and may prove important to Facebook’s Reality Labs division. The department is working on a number of projects that can benefit from AI models trained using video footage shot from a human perspective.This includes smart glasses like the previous Ray-Ban Stories Facebook released last month, And virtual reality. Facebook has invested heavily since it acquired Oculus for $ 2 billion in 2014.

This video can teach artificial intelligence to understand or identify something in the real or virtual world that can be seen from a first-person perspective through glasses or an Oculus headset.

Facebook announced in November that it would release its Ego4D dataset to researchers.

“This release is an open dataset, a research agenda, and facilitates progress both internally and externally in academia. [allow] Facebook’s senior researcher, Kristen Grauman, told CNBC.

According to Grauman, datasets can be deployed in AI models used to train technologies such as robots to understand the world more quickly.

“Traditionally, robots learn by doing something in the world, or by showing how to do things literally on hand,” Glauman said. “From our own experience, there is an opening for them to learn from the video.”

Facebook and a consortium of 13 university partners relied on more than 700 participants from nine countries to capture first-person footage. According to Facebook, Ego4D has more than 20 times as long video as other datasets of this type.

Facebook’s university partners include Carnegie Mellon University in the United States, Bristol University in the United Kingdom, National University of Singapore, the University of Tokyo in Japan, and the Institute for International Information Technology in India.

The footage was shot in the United States, United Kingdom, Italy, India, Japan, Singapore and Saudi Arabia. Facebook said it hopes to expand the project to more countries, including Colombia and Rwanda.

“The important design decisions of this project are, in the first place, the leaders in this field, who are interested in and willing to pursue these issues, as well as geographical diversity. I wanted a partner who had a partner, “Glauman said.

Facebook Rayvan Story Glass

Sal Rodriguez | CNBC

The announcement of Ego4D will take place at an interesting time for Facebook.

The company has steadily stepped up its hardware efforts. Last month, we released our first smartglass, the $ 299 Ray-Ban Stories.When In July, Facebook has announced the formation of a product team specifically working on the “Metaverse”. This is a concept that involves creating a digital world where multiple people can live at the same time.

But over the past month, Facebook has been hit by a bunch of news stories originating from a flock of in-house companies. Frances Haugen leaked research, Former Facebook Product Manager has become a whistleblower. Among the studies published was a slide showing that Instagram is harmful to the mental health of teenagers.

The footage was captured using off-the-shelf devices such as: GoPro With the camera Vuzix Smart glasses.

Facebook said it has instructed participants not to capture personally identifiable traits when collecting footage indoors to protect their privacy. This includes people’s faces, conversations, tattoos and jewelry. Facebook said it removed personally identifiable information from the video, blurring the faces of bystanders and vehicle license plate numbers. Audio has also been removed from many videos, the company said.

“Step 1 of all the university partners who did this video collection was a very focused and important process in developing the right collection policy,” Grauman said. Facebook announces Ego 4D first-person video dataset for AI training

Back to top button