Shanghai WAIC Opens; Chinese NLP Model Tops SuperGLUE; ByteDance Wins Best Paper at ACL

China’s AI news in the week of July 5, 2021

Please subscribe here to support Recode China AI.

Chinese AI companies celebrate WAIC 2021 with dazzling announcements

The World Artificial Intelligence Conference 2021 kicked off on July 8 in Shanghai with the theme of "Intelligent Connectivity, Inspirational Cities". The annual AI forum and expo amassed experts and enterprises around the world to discuss how AI helps with the creation of a community of shared future and global sustainable development.

China’s biggest tech and AI companies also swarmed upon Shanghai this week for the WAIC 2021, touting their AI contributions to the Shanghai local government, which aims to be the next AI hub.

To save your time browsing through companies’ PRs, I summarized the most newsworthy announcements and offerings below.

Tencent: Tencent would host the Honor of Kings competition to pit its AI agent, named Tencent Solo, and several pro gamers, said CEO Pony Ma in its 50-second keynote speech. He also spoke highly of its cooperation with the nation’s National Astronomical Observatories to journey into AI space exploration.

Baidu: CEO Robin Li continues to promote the company’s auto brand - Apollo for autonomous driving and Jidu Auto for electric vehicles - by promising to expand its robotaxi services and roll out new smart vehicles in the next two to three years. A clay model for the vehicle has been completed.

Meituan: Meituan introduced a new, self-developed drone model which will achieve a 15-minute delivery time for destinations within a three-kilometer radius.

SenseTime: SenseTime unveiled its full-stack SenseAuto solution in a foray to the automobile industry and SenseCore AI Infrastructure to develop powerful and efficient AI solutions, but the biggest publicity stunt is the hiring of Liu Cixin, author of The Three-Body Problem and recipient of the Hugo Award. Mr. Liu will work with SenseTime to explore the new paradigm of "AI + Science Fiction", an innovative and immersive offline entertainment experience.

UBTECH Robotics: UBTECH Robotics brought the updated version of its flagship Ubtech Walker X, a butler robot, to the WAIC stage. The robot can perform various tasks like visual positioning navigation, fast walking, and of course massage as what a service robot is demanded.

Enflame Tech: China’s AI chip unicorn Enflame Tech unveiled its second-gen AI training processor "Suisi 2.0″ at WAIC along with a training accelerator and an OAM module. The company boasts its new offering as China’s largest AI computing chip to date, with a 57.5 mm × 57.5 mm size (an area of 3306 mm2). The processor delivers 40TFLOPS (FP32), 160TFLOPS (TF32), and 320TOPS (INT8) and will be mass-produced using Globalfoundries’ 12nm FinFET process by the end of 2021. Earlier this year, Shanghai-based Enflame Tech raised RMB 1.8 billion yuan ($278.5 million) from Tencent and a number of investors including CITIC, CICC Capital, and Primavera.

Baidu Upgrades NLP Framework ERNIE, Tops SuperGLUE

Chinese institutes and companies are vying for the crown of “Chinese GPT-3”. Huawei introduced its 200-billion-model PanGu while state-backed BAIC responded with Wu Dao 2.0.

Now Baidu has joined the battlefield with its new offering: ERNIE 3.0, the latest iteration of its famous NLP framework. The upgrade has endowed the framework with a new language generation capability so it can not only understand human language but also compose novels, lyrics, poems, and couplets.

In the paper ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation, researchers proposed a Universal Knowledge-Text Prediction task, which requires the model to predict the relation of a given triple from knowledge graph and the corresponding sentence from the encyclopedia. By doing so, the model’s knowledge memorization and reasoning will be further enhanced.

ERNIE 3.0 shows incredible performance in human language understanding tasks. It outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves first place on the SuperGLUE benchmark, surpassing the human performance by +0.8% (90.6% vs. 89.8%). SuperGLUE benchmark is styled after the original GLUE benchmark but designed to be more difficult.

Baidu Research has developed a series of ERNIE models to handle different tasks, ranging from multimodal understanding to cross-language. You can find more details on GitHub.

ByteDance wins ACL Best Paper

The annual premier Association for Computational Linguistics (ACL) conference announced its best paper Vocabulary Learning via Optimal Transport for Neural Machine Translation from ByteDance AI Lab, University of Wisconsin–Madison and Nanjing University. Surprisingly the paper was once rejected as an ICLR submission.

The paper tries to figure out a way to automatically build a vocabulary without expensive trial training. Machine translation and many other NLP tasks are mostly built on a fixed vocabulary. Theoretically, an optimal vocabulary contains any possible word, but it’s technically and computationally impossible. That’s why researchers usually construct an optimal vocabulary in terms of its size and frequency of sub-words based on empirical studies.

Researchers proposed a VOcabulary Learning approach via optimal Transport (VOLT) in which they find a good vocabulary measurement by considering entropy and size. Using the idea of Marginal Utility in economics, they introduced Marginal Utility of Vocabularization (MUV), which calculates the benefits (entropy) a corpus can get from an increase of cost (size), and proposed an optimal transport solution to find the vocabulary.

Experiments showed VOLT surpasses other approaches by yielding a vocabulary with a smaller size but better performance. For example, on English-German translation, VOLT achieves almost 70% vocabulary size reduction and 0.5 BLEU gain on English-German translation. Compared to BPE-search, VOLT reduces the search time from 384 GPU hours to 30 GPU hours.

For the first time, the ACL and the Asian Federation of Natural Language Processing joined hands and hosted their joint meeting in Thailand. The ACL-IJCNLP 2021 will be held in Bangkok, Thailand, during August 1-6, 2021. The conference received 3350 submissions, 21.3% of which have been accepted for the main conference.

Investment news

• Chinese EV upstart and Tesla challenger Xpeng raised $1.8 billion in its secondary listing in Hong Kong.

• Xiaomi has reportedly acquired DeepMotion, a China-based autonomous driving technology company. DeepMotion will have more than 20 people joining Xiaomi to help develop smart car technology including HD maps and vision-based self-driving software. Founded in 2017, DeepMotion is reportedly acquired at a valuation of RMB 1 billion (~$150 million).

• Synyi (森亿智能), a Shanghai-based AI-powered medical data solutions provider. announced on July 9 that it has raised RMB 500 million ($77 million) in its Series E funding, led by IDG Capital. Founded in 2016, Synyi provides smart analytics and data governance platforms for hospitals, medical research centers, and governments to better manage patents data.