Google Security Blog

Systems such as Gmail, YouTube and Google Play rely on text classification models to identify harmful content including phishing attacks, inappropriate comments, and scams. These types of texts are harder for machine learning models to classify because bad actors rely on adversarial text manipulations to actively attempt to evade the classifiers. For example, they will use homoglyphs, invisible characters, and keyword stuffing to bypass defenses. 

To help make text classifiers more robust and efficient, we’ve developed a novel, multilingual text vectorizer called RETVec (Resilient & Efficient Text Vectorizer) that helps models achieve state-of-the-art classification performance and drastically reduces computational cost. Today, we’re sharing how RETVec has been used to help protect Gmail inboxes.

Strengthening the Gmail Spam Classifier with RETVec

Figure 1. RETVec-based Gmail Spam filter improvements.

Over the past year, we battle-tested RETVec extensively inside Google to evaluate its usefulness and found it to be highly effective for security and anti-abuse applications. In particular, replacing the Gmail spam classifier’s previous text vectorizer with RETVec allowed us to improve the spam detection rate over the baseline by 38% and reduce the false positive rate by 19.4%. Additionally, using RETVec reduced the TPU usage of the model by 83%, making the RETVec deployment one of the largest defense upgrades in recent years. RETVec achieves these improvements by sporting a very lightweight word embedding model (~200k parameters), allowing us to reduce the Transformer model’s size at equal or better performance, and having the ability to split the computation between the host and TPU in a network and memory efficient manner.

RETVec Benefits

RETVec achieves these improvements by combining a novel, highly-compact character encoder, an augmentation-driven training regime, and the use of metric learning. The architecture details and benchmark evaluations are available in our NeurIPS 2023 paper and we open-source RETVec on Github.

Due to its novel architecture, RETVec works out-of-the-box on every language and all UTF-8 characters without the need for text preprocessing, making it the ideal candidate for on-device, web, and large-scale text classification deployments. Models trained with RETVec exhibit faster inference speed due to its compact representation. Having smaller models reduces computational costs and decreases latency, which is critical for large-scale applications and on-device models.

Figure 1. RETVec architecture diagram.

Models trained with RETVec can be seamlessly converted to TFLite for mobile and edge devices, as a result of a native implementation in TensorFlow Text. For web application model deployment, we provide a TensorflowJS layer implementation that is available on Github and you can check out a demo web page running a RETVec-based model.

Figure 2.  Typo resilience of text classification models trained from scratch using different vectorizers.

RETVec is a novel open-source text vectorizer that allows you to build more resilient and efficient server-side and on-device text classifiers. The Gmail spam filter uses it to help protect Gmail inboxes against malicious emails.

If you would like to use RETVec for your own use cases or research, we created a tutorial to help you get started.

This research was conducted by Elie Bursztein, Marina Zhang, Owen Vallis, Xinyu Jia, and Alexey Kurakin. We would like to thank Gengxin Miao, Brunno Attorre, Venkat Sreepati, Lidor Avigad, Dan Givol, Rishabh Seth and Melvin Montenegro and all the Googlers who contributed to the project.

Nearly half of third-parties fail to meet two or more of the Minimum Viable Secure Product controls. Why is this a problem? Because "98% of organizations have a relationship with at least one third-party that has experienced a breach in the last 2 years."

In this post, we're excited to share the latest improvements to the Minimum Viable Secure Product (MVSP) controls. We'll also shed light on how adoption of MVSP has helped Google improve its security processes, and hope this example will help motivate third-parties to increase their adoption of MVSP controls and thus improve product security across the industry.

In October 2021, Google publicly launched MVSP alongside launch partners. Our original goal remains unchanged: to provide a vendor-neutral application security baseline, designed to eliminate overhead, complexity, and confusion in the end-to-end process of onboarding third-party products and services. It covers themes such as procurement, security assessment, and contract negotiation.

Improvements since launch

As part of MVSP’s annual control review, and our core philosophy of evolution over revolution, the working group sought input from the broader security community to ensure MVSP maintains a balance between security and achievability.

As a result of these discussions, we launched updated controls. Key changes include: expanded guidance around external vulnerability reporting to protect bug hunters, and discouraging additional costs for access to basic security features – inline with CISA’s "Secure-by-Design" principles.

In 2022, we developed guidance on build process security based on SLSA, to reflect the importance of supply chain security and integrity.

From an organizational perspective, in the two years since launching, we've seen the community around MVSP continue to expand. The working group has grown to over 20 global members, helping to diversify voices and broaden expertise. We've also had the opportunity to present and discuss the program with a number of key groups, including an invitation to present at the United Nations International Computing Centre – Common Secure Conference.

Google at the UNICC conference in Valencia, Spain

Since its inception, Google has looked to integrate improvements to our own processes using MVSP as a template. Two years later, we can clearly see the impact through faster procurement processes, streamlined contract negotiations, and improved data-driven decision making.

• After implementing MVSP into key areas of Google's third-party life-cycle, we've observed a 68% reduction in the time required for third-parties to complete assessment process.

• By embedding MVSP into select procurement processes, Google has increased data-driven decision making in earlier phases of the cycle.

• Aligning our Information Protection Addendum’s safeguards with MVSP has significantly improved our third-party privacy and security risk management processes.

You use MVSP to enhance your software or procurement processes by reviewing some common use-cases and adopting them into your third-party risk management and/or contracting workflows .

We're invested in helping the industry manage risk posture through continuous improvement, while increasing the minimum bar for product security across the industry.

By making MVSP available to the wider industry, we are helping to create a solid foundation for growing the maturity level of products and services. Google has benefited from driving security and safety improvements through the use of leveled sets of requirements. We expect the same to be true across the wider industry.

We've seen success, but there is still work to be done. Based on initial observations, as mentioned above, 48% of third-parties fail to meet two or more of the Minimum Viable Secure Product controls.

As an industry, we can't stand still when it comes to product security. Help us raise the minimum bar for application security by adopting MVSP and ensuring we as an industry don’t accept anything less than a strong security baseline that works for the wider industry.

Google and the MVSP working group would like to thank those who have supported and contributed since its inception. If you'd like to get involved or provide feedback, please reach out.

Thank you to Chris John Riley, Gabor Acs-Kurucz, Michele Chubirka, Anna Hupa, Dirk Göhmann and Kaan Kivilcim from the Google MVSP Group for their contributions to this post.