Baidu's Language Model Behind Rumored ChatGPT Search; Tencent-Backed Robot Startup Files for Hong Kong IPO; Xpeng Targets "Full Autonomy" in 2023

Weekly China AI News from Jan 30 to Feb 5

Dear readers, in this week's newsletter, we'll delve into ERNIE, the tech behind Baidu's rumored ChatGPT rival. Also, Tencent-backed UBTECH files for IPO in HK and Xpeng CEO announces plan for full self-driving in 2023. Quick poll: Prefer current research collection format or more papers and less text?

Weekly News Roundup

Breaking Down Baidu's Language Model Behind the Rumored ChatGPT-Powered Search

What’s new: Rumors are swirling that Baidu, China’s leading search giant, is set to launch a new AI chatbot service that would rival OpenAI's popular ChatGPT. The news was first reported by Bloomberg last week, and Baidu's shares promptly rose as much as 5.8%. The boost was further fueled on Wednesday after Blackrock increased its stakes, leading to a 15% surge.

Baidu's AI engine: According to Bloomberg, Baidu’s new chatbot service will be powered by the company's ERNIE system, a large-scale machine-learning model that's been trained on data over several years.

What is ERNIE: ERNIE stands for Enhanced Representation through Knowledge Integration, and it was first introduced in April 2019 as a state-of-the-art Chinese NLP (Natural Language Processing) model. The key difference between ERNIE and other large language models is its integration of knowledge graphs, a network representation of real-world entities. This allows ERNIE to understand the relationships between entities, making it a more sophisticated and effective language model.

ERNIE 2.0: Building on the success of the original ERNIE, Baidu's researchers introduced a continual pre-training framework, allowing the model to continuously learn multiple tasks like humans. ERNIE 2.0 not only claimed the top spot in the Chinese NLP field but also outperformed BERT, a similar language model developed by Google.

The naming trend of Large Language Models (LLMs) after Sesame Street characters was popular among LLM creators. This trend started after Google's release of the groundbreaking BERT model in 2018, which was quickly followed by other models such as ERNIE from Baidu, ELMo, Grover, Big BIRD, and RoBERTa.

ERNIE 3.0 and ERNIE 3.0 Titan: The creation of GPT-3, a text-generating model, motivated Baidu to create ERNIE 3.0 and ERNIE 3.0 Titan. These models combine both language understanding and language generation, with the latter being the largest Chinese singleton NLP model, with 260 billion parameters.

Expanding the ERNIE family: Based on the capabilities of ERNIE, Baidu has developed a series of models, including the multi-language model ERNIE-M, the text-to-image model ERNIE-ViLG, and the code generator ERNIE-Code. The company has also proposed an open-domain dialogue generation model named PLAO-XL.

Robot Unicorn UBTECH Files for Hong Kong IPO

What’s new: UBTECH, an AI-powered robot company based in China, is projected to make history as the first publicly traded humanoid robot company in the country. The Tencent-backed company, valued at over 33 billion yuan and with accumulated financing of 4.8 billion yuan, has officially submitted its prospectus to the Hong Kong Stock Exchange and is expected to go public on the main board.

Who’s UBTECH: Founded in March 2012, UBTECH is a leader in China's AI-powered robots and is known for its development of humanoid robots and intelligent service robots. These robots have been applied to various industries including education, logistics, and general services such as guidance, security inspections, and elderly care. UBTECH has already provided services to over 900 enterprise-level group customers.

Net loss: Despite its growth, UBTECH has experienced significant losses in the past 33 months, totaling 24.03 billion yuan. In 2020, 2021, and the first nine months of 2022, the company's total revenue was 740 million yuan, 817 million yuan, and 528 million yuan, respectively, with a year-on-year growth of 10.4%. The company recorded a total net loss of 24.03 billion yuan over the past 33 months.

However, the company sees great potential in its future and plans to use the funds raised from the IPO to upgrade its core technology, recruit over 150 employees, and develop and launch various types of robots. The funds will also be used for potential acquisitions and investments, repaying bank loans, enhancing R&D infrastructure, improving global brand awareness and market presence, and optimizing management and operational efficiency. The funds raised will exceed several billion yuan.

Xpeng CEO Aims to Release Fully Self-Driving Cars in 2023

What’s new: At the Guangdong Quality Development Conference on January 28, Xpeng Motors CEO Xiaopeng He announced the company's ambitious plans to lead China in full self-driving technology. He stated that Xpeng will launch the “hands-free” technology in over 50 cities in China by 2023, outpacing Tesla’s current efforts in the field.

In an interview with Bloomberg, He confirmed Xpeng’s plans to expand the reach of its self-driving technology by developing a version that doesn’t rely on high-definition maps and introducing it to 50 to 100 cities in 2023. He referred to these vehicles as “all-intelligent” and stated that they are “infinitely close” to Level 4 autonomous driving.

He sees a bright future for smart vehicles in China, with the penetration rate of new energy vehicles already rising from less than 3% to over 30% in recent years. Over the next five years, Xpeng Motors plans to launch five smart vehicle models to the global market.

Profitable in 2025: Xpeng is committed to high-intensity research and development, with yearly investments exceeding 6 billion yuan over the next five years.

After a challenging year that saw its shares drop by 80% and sales fall short of expectations, Xpeng’s goal is to achieve positive operating profits by 2025 and to reach a sales target of 1.2 million units by 2027. At the same time, Xpeng aims to have full self-driving vehicles account for about 30% of the market share. He also predicted that China's annual sales of new energy vehicles may reach 15 million units by 2027, with a market share of about 70%.

Legal challenge: China has not yet enacted laws or regulations for the consumer market regarding autonomous passenger vehicles, requiring a driver to remain in control of the car for the time being.

Trending Research

• Alibaba’s research arm Damo Academy open sourced Dash, a self-supervised learning framework with dynamic thresholding. The existing SSL algorithms use either all unlabeled data or the unlabeled examples with a fixed high-confidence prediction during training. Dash selects a subset of the unlabeled data to train models and only keeps examples whose losses are smaller than a dynamically adjusted threshold. The framework is adaptive in its selection of unlabeled data and has a theoretical guarantee of convergence. Read the paper Dash: Semi-Supervised Learning with Dynamic Thresholding and check out the GitHub.

• Researchers from Shenzhen University, Tsinghua University, and Tencent’s YouTu Lab proposed Multi-modal Knowledge Transfer (MKT) for multi-label classification in real-world recognition systems. It addresses the challenge of identifying unseen labels by exploiting the rich semantic information inherent in image-text pairs. MKT is based on a vision and language pre-training (VLP) model and uses knowledge distillation and prompt tuning to transfer the image-text matching ability of the VLP model. Read the paper Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer.

• Researchers from Alibaba proposed mPLUG-2, a new paradigm for multi-modal pretraining with a modularized design for collaboration between modalities and addressing modality entanglement. It uses a multi-module composition network with shared universal modules and separate modality modules to handle different tasks across text, image, and video modalities. mPLUG-2 has achieved strong results on over 30 downstream tasks and has set new state-of-the-art results in video QA and captioning with a smaller model size. Read the paper mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video.

Noteworthy Stories

• U.S. investors including the investment arms of Intel and Qualcomm accounted for nearly a fifth of investments in Chinese AI companies from 2015 to 2021, a report showed on Wednesday. — Reuters

• The Biden administration has stopped providing US companies with licences to export to Huawei. — Financial Times