Chinese scientists create world’s fastest vision chip for autonomous cars, defence

“Firstly, it balances speed and dynamic performance in vision chips and introduces a novel computational method that diverges from existing machine vision strategies,” Shi said.

“Secondly, this approach mimics the human visual system’s dual pathway, enabling decision-making without complete clarity,” he said.

“The chip provides new directions for advancements in autonomous driving and defence sectors. It is poised to tackle some of the most challenging issues faced today, paving the way for numerous novel applications.”

The team reported on the development in the journal Nature on Thursday, with Shi and Professor Zhao Rong the corresponding authors of the study.

The visual perception terminals in intelligent systems process vast volumes of data and must be able to handle extreme events like sudden driving hazards, significant light changes at tunnel entrances, and intense nighttime flash interference.

Traditional visual chips frequently fail or experience delays in these extreme scenarios due to limitations in power consumption and bandwidth.

The scientists said Tianmouc was inspired by the human visual system. It analyses visual information along two pathways: one for cognition and another for rapid response.

Explaining the difference between the chips, the scientists gave the example of flying objects, which humans – and Tianmouc – will subconsciously dodge even if they do not see them clearly. Traditional machine vision chips, however, often need to see clearly before making decisions.

Of all the information collected by the vision sensor, Tianmouc uses colour, absolute intensity, precision and resolution in a cognition-oriented pathway for accurate recognition.

Spatial difference, temporal difference and speed are meanwhile used in an action-oriented pathway for a fast response.

The team said it developed software and algorithms based on the chip, which were deployed on a Chevrolet for road testing, confirming its capabilities in various extreme scenarios.

A statement on the Tsinghua website said the chip had shown low latency and high performance in real-time perceptual reasoning across a range of challenging environments, showing significant potential for applications in unmanned systems.

Dr Wang Taoyi, a member of the research team, told state broadcaster CCTV on Friday that Tianmouc had achieved an 0.1 millisecond delay in autonomous driving. “This is just 1/300th of the 30ms delay experienced with traditional cameras,” Wang said.