ARTIFICIAL INTELLIGENCE HAS LEARNED HOW TO MODEL TISSUE BEHAVIOR BY WATCHING VIDEOS

Researchers at MIT CSAIL, Nvidia, the University of Washington, and the University of Toronto have created an artificial intelligence system that studies the physical effects that affect tissue materials. AI trained by watching the video. The developers claim that the system can predict the behavior of tissues and their interaction, even if they have not seen such a thing before. For example, create an emulation involving multiple shirts and trousers.

The researchers created a network of visual causal relationships (V-CDN), which interacts with three modules: one for visual perception, one for structural inference and one for predicting dynamics.

The developers tried to lay the foundation of the system understanding of the reasons for the AI ​​to create possible alternatives to movement. For example, in an image containing a pair of balls connected to each other by a spring, the system will predict the effect of the spring on the interaction of the balls. Thus, AI makes various predictions.

The perception model is trained to extract certain key points (areas of interest) from the video. The exposure module then defines the variables that control the interactions between pairs of key points. Meanwhile, the dynamics module learns to predict future movements of key points based on the neural network graph created by the output module.

Researchers studied V-CDN in a simulated medium containing fabrics of various shapes: shirts, pants and towels of various lengths. The developers interacted with the contours of the tissues to deform and move the clothes, and the AI ​​reacted to their actions and tried to predict how the fabric model would behave. The researchers aimed to create a single model that could process tissues of different types and shapes.

The results show that V-CDN performance increased as the system observed a large array of video frames. According to the researchers, the operation of the system is comparable to intuition. More previous observations provide a better estimate of the variables that control tissue behavior.

“The model does not imply access to the main causal graph, nor the dynamics that describes the effect of physical interactions,” the scientists wrote in a text describing the system. “Instead, the system learns to create dependency structures and model causal mechanisms from video without human intervention, which we hope can facilitate future research with more generalized visual thinking systems.”

Researchers note that V-CDN does not solve the daunting task of causal modeling. Rather, developers see the work as an initial step towards a broader study of creating a physically justified “visual intelligence” that can simulate dynamic changes. Researchers hope to draw people's attention to this task and inspire future research.

Causation lies at the heart of human knowledge. This allows people to reason about the environment and make hypothetical predictions regarding scenarios, which may differ significantly from previous experience. Modern artificial intelligence does not yet know how to make decisions based on causal relationships. Therefore, one of the main tasks in machine learning is the search and interpretation of cause-effect relationships in large data sets. For example, from videos. Then, the systems are trained based on this information.