The protein universe is vast—hundreds of millions of sequences, each with a unique three-dimensional fold that determines its function. For decades, biologists have classified these folds manually, relying on small curated datasets of experimentally determined structures. That era is ending. With the advent of AlphaFold, the community now has predicted structures for nearly every known protein, but the tools to learn from them at scale have lagged behind. A new paper changes this, introducing both a benchmark over thirty times larger than any predecessor and a self-supervised method that learns from 90% of a protein's structure being masked away. The implications extend far beyond classification benchmarks—this is a signal that protein representation learning is entering its foundation model era.
Watch the video version
🎙 Listen to this article
The Scale Problem in Protein Fold Classification
Protein fold classification is the structural biologist's equivalent of species taxonomy: it groups proteins by their three-dimensional architecture, revealing evolutionary and functional relationships invisible from sequence alone. The problem has been that existing benchmarks are small. SCOPe, the standard for decades, contains roughly 15,000 domains. CATH offers around 40,000. For deep learning methods to generalize across the vast protein universe—where millions of sequences exist but only hundreds of thousands have experimental structures—these datasets are simply too small.
TEDBench changes this calculus. By combining TED domain annotations with Foldseek clustering of AlphaFold-predicted structures, the authors assemble 462,175 predicted structures and 27,638 experimental structures—over 30× larger than prior benchmarks. The scale alone is transformative, but the construction is equally important. The benchmark is clustered by Foldseek at 30% sequence identity, ensuring non-redundancy while capturing structural diversity that sequence-based clustering would miss. This matters because the protein universe is structurally redundant; many sequences fold into similar shapes, and TEDBench captures that reality.
The inclusion of both predicted and experimental structures is a deliberate design choice. The predicted structures from AlphaFold are noisy—they represent the model's best guess at the native conformation—but they are abundant. Experimental structures are clean but scarce. TEDBench uses the predicted set for training and the experimental set for evaluation, creating a realistic test of whether models can generalize across the domain shift between computational predictions and experimental ground truth.
The Case for 90% Masking
The paper's central methodological contribution is Masked Invariant Autoencoders (MiAE), and its most striking result is that a 90% masking ratio works best. This is not obvious. In vision, MAE masks 75% of patches. In language, BERT masks 15% of tokens. Protein structures are different—they are three-dimensional, continuous, and governed by physical constraints. Why would 90% masking succeed?
The authors' insight is that protein structures are highly redundant. A protein's backbone geometry is determined by the physical chemistry of amino acids—bond angles, steric constraints, and the hydrophobic effect—which means that local structure can be reconstructed from sparse global context. The model learns to infer missing residues from the positions of distant ones, much like how a biologist can guess the fold of a protein from a few key residues. The SE(3)-invariant encoder means the model is agnostic to rotation and translation—it learns the intrinsic geometry, not the coordinate system.
The high masking ratio also forces the encoder to capture high-level semantic features. When 90% of tokens are masked, the encoder cannot rely on local patterns; it must understand the overall fold. This is exactly what we want for fold classification: a representation that captures global architecture, not local noise.
What This Means
The practical implications are substantial. First, TEDBench gives the community a standardized evaluation platform. Before TEDBench, different groups used different subsets of SCOP, CATH, or ECOD for evaluation—making it hard to compare methods. TEDBench's 30× scale means that models are now tested on a much broader slice of the fold universe, providing more robust signal.
Second, MiAE's strong performance with self-supervised pretraining on predicted structures is a practical breakthrough. Experimental structure determination is slow and expensive; only about 200,000 structures are in the PDB. But AlphaFold predicts structures for hundreds of millions of sequences. If those predicted structures can be used for pretraining—and TEDBench results suggest they can—then the bottleneck shifts from data to compute. Any lab with GPU access can now fine-tune MiAE for their protein of interest, even if no experimental structure exists.
Third, the 90% masking result has implications beyond classification. It suggests protein structures are highly compressible: you can remove nine out of ten residues and still reconstruct the geometry. This is not merely a technical curiosity—it hints at fundamental properties of protein folding landscapes. The fact that local geometry can be inferred from sparse global context supports the idea that proteins are modular, with secondary structure elements determined by long-range interactions.
For the broader AI-for-science community, this work demonstrates that the vision transformer recipe—aggressive masking, asymmetric encoder-decoder, reconstruction loss—transfers to 3D structures, provided the architecture respects the underlying symmetries. The SE(3)-invariant encoder ensures that the model learns geometry, not coordinate positions.
What We Still Need to Understand
Despite the strong results, several questions remain unanswered. First, TEDBench is built from TED domains—a specific subset of protein structural space. How well do the learned representations transfer to other structural classifications (SCOP, CATH) or to continuous tasks like function prediction? The paper shows some zero-shot transfer results, but systematic evaluation across diverse tasks is needed.
Second, the domain shift between predicted and experimental structures is real. AlphaFold structures are not perfect—they sometimes misplace loops or mispredict side-chain conformations. TEDBench's construction uses predicted structures for training and experimental for testing, but the size of the domain shift varies by protein family. Can we quantify this shift and develop methods to correct it?
Third, the 90% masking ratio is striking, but is it optimal across all protein families? Some folds are more regular (e.g., all-alpha), others more variable (e.g., multi-domain). A fixed masking ratio might not be ideal; adaptive masking strategies could extract more information.
What This Means
For structural bioinformaticians, TEDBench is a new standard. The field has been limited by small benchmarks that didn't capture the diversity of the protein universe. TEDBench's 462,175 predicted structures cover a much wider swath of fold space, meaning that models trained and evaluated on it should generalize better to real-world protein structures. The benchmark's construction—using Foldseek to cluster AlphaFold structures by structural similarity—means it is non-redundant at the fold level, which is exactly what fold classification needs.
For protein engineers and drug developers, the implications are practical. The ability to learn high-quality representations from predicted structures alone means that structural information is now available for any protein with a known sequence—which is essentially all of them. This unlocks the "dark proteome": the vast majority of proteins that have never been crystallized or solved by cryo-EM. For metagenomic sequences from environmental samples, where we have no experimental structures, MiAE's self-supervised approach can generate embeddings that capture structural information.
The fact that MiAE outperforms supervised methods is telling. It means that the structural information in proteins is learnable from the data itself—we don't need human annotations. This is the same insight that drove the success of BERT in NLP and MAE in vision: self-supervised pretraining on unlabeled data produces representations that are more general and more robust than those learned from supervised tasks.
Open Questions
- Structural domain shift: TEDBench's training set is built from AlphaFold-predicted structures, while the test set uses experimental structures. How well does performance on this benchmark reflect performance on truly novel structures not in AlphaFold's training set? The field needs benchmarks that capture the distribution shift between predicted and experimental structures.
- Transferability: MiAE is evaluated on fold classification, but the real test is whether its representations transfer to downstream tasks like enzyme function prediction, binding site identification, or protein-protein interaction prediction. The paper's linear probing results for fold classification are strong, but we need to see the same for other tasks.
- Scaling and diversity: TEDBench covers 1,371 folds, but the protein universe contains many more. How well does MiAE generalize to novel folds not in TEDBench? And can the architecture scale to multi-chain complexes, where inter-chain interactions create additional structural constraints?
The protein folding field has already been transformed by AlphaFold, but the next frontier is representation learning: building models that can understand protein structure the way language models understand text. TEDBench and MiAE take us closer to that goal, providing the tools—a large-scale benchmark and a self-supervised method—to systematically improve our understanding of the protein universe. The path is clear: scale, mask, and reconstruct.
Published by KHI Research · 2026-05-20 · AI Methodology
Discussion