A protein design model returns memorized binders based on requested length

At some binder lengths a protein design model hands back an existing protein instead of a new one: ubiquitin near 76 residues, an antibody domain past 100, sometimes the target itself. Interventions inside the model show where that choice is made, and two repairs reduce the copying at different costs.

BoltzGen generates protein binders: name a target and a binder length, and it returns designs. What fold those designs take is set mostly by the length requested, and at some lengths every design is an existing natural protein, its sequence as well as its shape. Figure 1 is the clearest case: a 76-residue request returns ubiquitin's fold in nearly every design, a 68-residue request almost never does.

Figure 1. Ubiquitin comes back over a narrow band of requested lengths, and the second release narrows it further. Share of a request's 50 designs whose backbone matches the natural ubiquitin structure, against the number of residues asked for. Two checkpoints of the model have been released; six targets were run at every length from 68 to 82 residues on each. The line is the mean over those 6 targets and the band is their full range, so a wide band means the targets disagree at that length. On the first release the affected band is broad, 92% at 72 residues and still 77% at 80; on the second it is narrow, 92% at 76 falling to 6% at 79. Both are near zero at 68 residues.

The rest of this report measures how far the behaviour goes, which folds it covers and which models show it, locates where inside the model the choice is made, and tests two ways to stop it.

The measurement behind every number

One request names a target protein and how many residues the binder should have. The model answers with 50 designs, each a backbone with a sequence, and every number in this report is a property of one such request's 50 designs.

Designs are compared with each other by structural alignment and grouped into clusters at a TM-score of 0.5, the conventional line between the same fold and a different one. If most of a request's designs fall into one cluster, that request has converged on a single shape. The centre of that cluster is then searched, sequence blind, against 1,562,678 deposited protein chains; a match at 0.5 or better gives the shape a name, so a converged request is not just repetitive but identifiably some natural protein's fold. Sequence copying is measured separately, by aligning each design's own sequence to that natural protein. A design counts as buildable if redesigning its sequence with ProteinMPNN and refolding it with ESMFold returns the shape it started from. Where a rate is compared against chance, the comparison is against natural protein chains of the same length put through the same pipeline.

The second release copies less, but copies different things

How much copying a user meets depends on which checkpoint they downloaded. Over 90 combinations of six targets and fifteen binder lengths, 67% of the first release's designs return ubiquitin's fold against 25% of the second's, a paired difference of 0.417 per combination (Wilcoxon signed-rank, p = 3.54e-16). The design pipeline ends with its own filter, which ranks a run's designs and keeps a small set; that filter widens the gap instead of closing it, leaving 33.1% of the first release's delivered designs as copies against 0.7% of the second's. The band of affected lengths narrows too, from 72 to 82 residues on the first release down to within about three residues of 76 on the second.

So the second release reduces copying substantially without removing it, and it does not memorize less of the same thing. Across 60 requests put through both releases, only 8 of the 50 where both converge on a confidently identified structure land on the same one: which natural protein comes back is largely a property of the particular checkpoint. The most extreme case survives the change intact. Asked for a 115-residue binder against PD-L1, both releases return PD-L1 itself, matching the deposited structure at 0.992 on the first release with 114 of the 115 residues identical, and 0.990 on the second with about 102 of 115 still verbatim. Everything that follows uses the first release, where the behaviour is strongest.

Length selects among a small fold repertoire

Ubiquitin is not the only fold this happens with, and length is what selects among them. Against a single target, an IL-7Rα receptor domain, copying reached 94% of designs at 76 residues, the size of ubiquitin, dropped away, then rose again to 54% at 100 residues, near the size of a PD-L1 domain: two separate bands, each tied to the size of a different natural protein.

Pinning down where the binder should attach does not remove the pattern. With the binding site specified explicitly, four targets at 108 and 124 residues still put most designs in one fold cluster, the exceptions being two targets at 108 residues, at 6% and 68% of designs in the largest cluster.

The grid below is the systematic version: nine targets, ten requested lengths, 50 designs per request. Reading down a column shows how little the target matters; reading across a row shows how much the length does.

UbiquitinComes back at 70 and 76 residues. 76 residues, PDB 1UBQ.

Immunoglobulin β-sandwichComes back at 100, 110, 124, 135 and 140 residues. 127 residues, PDB 6ZWK.

SH3 domainComes back at 60 residues. 57 residues, PDB 1SHG.

Hen lysozymeComes back at 128 residues. 129 residues, PDB 1LYZ.

TNF-α domainComes back at 85, 100, 110 and 140 residues. 148 residues, PDB 5M2I.

Figure 2. Requested length, not the target, selects the fold. Each cell is one request of 50 designs. The percentage is the share of those designs whose fold matches a known natural fold, colour repeats it, and the short label above it names the fold (Ubq ubiquitin, Ig immunoglobulin β-sandwich, SH3 SH3 domain, Lyz lysozyme, TNF TNF-α domain, IL7 IL-7Rα domain, PDG PDGFR-β domain, Hom homeodomain, Hlx helical bundle). Hatched cells were not run; only the lysozyme row was measured at 128 and 135 residues. Each percentage carries a 95% binomial interval of roughly ±14 points at 50 designs, so small differences between cells are not meaningful; the pattern across a row is. The five structures are the folds that recur, each drawn from its deposited entry; drag any of them to rotate it. Across 74 requests these designs use eight distinct folds in total, and 33 of the requests put essentially every design into one of them. Two of the eight appear on at least three different targets: the β-sandwich on all nine, ubiquitin on six. Sometimes the fold returned is the target's own: asked for a 128-residue binder against lysozyme, the model returns lysozyme. No request at 110 residues or longer returned ubiquitin, on any target.

Other generators concentrate too, but not onto natural folds

Repeating itself is not the same as copying, and other protein design models repeat themselves as well. To separate the two, the same measurement was run over 210 requests across five generator settings, 50 designs each: BoltzGen with its target given, Genie 2 with no target, and RFdiffusion in three settings, with the target, without it, and without it but with the target's complex present. Four things were recorded for every request: how concentrated its designs are, whether that shape is an identifiable natural fold, what kind of structure it is, and whether the sequence matches the natural protein too.

Concentration alone does not separate the models. RFdiffusion's complex checkpoint run without a target is as concentrated as BoltzGen or more, holding a median 92% of a request's designs in one cluster and reaching 90% or more in 9 of its 17 requests between 68 and 82 residues. None of that counts as copying a natural fold: every one of those tight clusters is either a trivial topology, a plain bundle or hairpin that carries no fold identity, or matches nothing among the deposited chains. Genie 2 is the one setting that stays genuinely diverse, at a median 10% in its largest cluster.

What separates BoltzGen is what sits at the bottom of its concentrated requests. Its converged designs are β-rich, at 7.2% median helix content, where every other setting's converged designs are helical bundles; they match deposited structures; and only BoltzGen also reproduces the natural sequence, which the other models cannot do at all because they emit backbones without sequences. Table 1 puts the four measurements side by side.

That comparison steps through lengths in fives, which skips the ubiquitin-sized range, so requests at every length from 68 to 82 residues were added: the same five settings, four targets for BoltzGen, 50 designs per request. Their averages are folded into the lines in Figure 3. In that range BoltzGen drives 21 of its 49 requests to 90% or more of the designs in one identified natural fold, at lengths from 72 to 80 residues, while none of the 82 requests from the other settings does. In those converged requests the designs also carry the matched natural protein's sequence, at a median 0.800 identity, and that identity is high across the whole range rather than sharpest at one length: with three targets per length no single peak can be located, and the PD-L1 request at 76 residues is 0.568, among the lowest of the converged set. The pattern is a band of lengths, not a spike. Further out the convergence is total: on the PD-L1 target every one of the 50 designs falls in a single cluster at 90, 95 and 100 residues.

Figure 3. How concentrated each generator's designs are, against requested length. For every setting, the share of a request's 50 designs that fall in its largest structural cluster, averaged over that setting's targets at each length. Hue marks the model, BoltzGen red, Genie 2 green and RFdiffusion blue; line style marks the conditioning, solid with a target, dashed without one and dotted for the unconditional complex setting. Being high on this axis means only that a setting repeats itself, which is why the other three measurements matter. The tinted span is the 68–82 residue range that the coarser five-residue spacing skipped; the additional requests there are averaged into each line in three bins covering 68–72, 73–77 and 78–82 residues, each plotted at the mean length of the requests in it, and marked with open circles so the added points stay identifiable.

Table 1. The four measurements, across five generator settings. Sequence identity applies only to BoltzGen, which emits a sequence with each backbone; the other generators emit backbones only.

Generator setting	Requests	Median largest cluster	One natural fold above 90%	Helix when converged	Buildable designs	Highest sequence identity
BoltzGen-1, target-conditioned	60	0.98	26	7.2%	87.0%	0.991
Genie 2, unconditional	30	0.07	0	84.4%	92.0%	backbone only
RFdiffusion, target-conditioned	60	0.47	0	93.4%	70.0%	backbone only
RFdiffusion base, unconditional	30	0.19	0	90.0%	94.7%	backbone only
RFdiffusion complex, unconditional	30	0.60	0	94.5%	73.3%	backbone only

Only BoltzGen produced requests where one identified natural fold covers at least 90% of the designs: 26 of 60 requests, against 0 of 150 requests from the other four settings. Among its converged requests, 17 of 46 also matched the natural protein's sequence above 50% identity, reaching 0.991, close enough to be the same molecule rather than merely the same shape.

The fold decision sits in the pair representation

If the model decides on a fold somewhere inside itself, that decision should be movable. The test runs two requests against the same target: a 76-residue binder, which returns ubiquitin every time, and an 85-residue binder, which never does. Part of the model's internal state is captured from the first run and written into the second while it is still running, and the question is whether the second run then returns ubiquitin.

The state in question is the model's pair representation: a vector for every pair of residues, holding what the model currently believes about how those two residues sit relative to each other. Only the part covering binder residue against binder residue was moved. Its size makes it the obvious candidate, but the controls are what make the result mean anything: copying the binder-against-target part instead, copying the two states that describe single residues rather than pairs, copying the same block with its residue pairs shuffled, and copying random values of the same magnitude. Every one is read and written at the same point in the run, with 50 designs for each variant, across three targets.

Figure 4. What is transplanted, and from where. Schematic, and deliberately only the object that moves. Left: the binder-against-binder part of the pair representation, one vector for every pair of binder residues; the pattern shown is illustrative rather than a recorded state, and the rows and columns involving the target are left alone. Right: the run that donates it, a 76-residue request that returns ubiquitin every time, and the run that receives it, an 85-residue request that never does on its own. The block is read after the last of the trunk's 64 blocks on the final pass through it and written into the receiving run at the same point. The controls listed at the foot of the figure are copied the same way at the same point.

Figure 5. Only the binder-against-binder part of the pair representation carries the fold. Share of 50 designs that switch to ubiquitin in the receiving run, one row per variant of the transplant. Copying every captured state transfers the fold almost completely; the binder-against-binder block on its own transfers it in 70% to 88% of designs across three targets. The binder-against-target block, the two single-residue states, and the shuffled and random-value controls all stay at or near zero, so the effect is not what happens whenever internal state is disturbed.

Figure 6. The choice is written near the end of the network. The model's trunk is a stack of 64 blocks, run four times over. A narrower intervention adds only the donor run's component along an eight-dimensional subspace of the same state, after one chosen block, which localizes where in that stack the fold information becomes decisive. Nothing happens until the last two blocks: 54% and 70% for the GFP target, 50% and 54% for IL-7Rα, against at most 16% at every earlier block. This shows the state is sufficient to induce the fold, not that it is the only route; single-block removals changed nothing.

Repair one: pushing the sampler away from the copy

The first repair changes nothing in the model itself, only how it is sampled. BoltzGen builds coordinates by denoising: it starts from noise and refines its guess at the finished binder over a schedule of steps. While that guess is still rough, an extra term pushes the emerging backbone away from the fold the model would otherwise return, and then switches off so the model finishes the structure on its own. Nothing is retrained and nothing about the model is changed; the only tuned quantity is how hard to push. The idea comes from AF3-ReD, which added a repulsive bias to a structure predictor's diffusion sampler to drive it towards alternative conformations (bioRxiv 2025, doi:10.64898/2025.12.17.693105); here the same mechanism is aimed at a memorized fold instead, inside BoltzGen-1 (bioRxiv 2025.11.20.689494), the model studied throughout.

Figure 7. Where the push applies, and what happens inside one step. Top: the sampling schedule, one tick per denoising step. The repulsive term is on from 10% to 45% of the schedule and then released, so the model finishes the structure by itself. The two framed structures are what enters and what leaves that window: entering, the guess is still a noisy cloud, drawn as points over a faded structure; leaving, a design that escaped the copy. Bottom: one step while the push is on. The sampler's current estimate of the finished binder is superposed on the natural ubiquitin structure by the same fit the method uses, the repulsive term is added to the binder's C-alpha coordinates along the directions that fit implies, and the ordinary denoising step then runs on the pushed estimate. Each structure is built from real coordinates: a surface laid along the C-alpha trace, widened through the helices and strands that trace implies, and lit from one direction so the passes in front read as being in front. The three shown are the design the unedited model returns at a 76-residue request, at 0.93 similarity to the reference, the deposited ubiquitin structure, and an escaped design at 0.40 similarity, which is the weight-edited model's design at the same request. Escaped designs do not converge somewhere else instead: mean pairwise similarity among them is 0.38, against 0.96 among the copies.

Copying falls first and the designs stay buildable through a usable range of strengths. Past that range they stop being buildable, and it is the push itself that breaks them rather than the choice of target.

Push strength

Figure 8. Copying and buildability against push strength. Move the slider to any strength that was run. Copying falls from 95% to 16% and then to zero, while the share of escaped designs that can be rebuilt from their own sequence stays between 61% and 82% across the working range before dropping to 41% at the strongest push. That share is measured only over designs that escaped the copy, so its denominator grows as copying falls, and the zero-strength point rests on few designs. Pushing against a scrambled version of the same reference at matched strength left copying at 97%, so what does the work is the reference fold and not the disturbance.

The repair does not work everywhere. For PD-L1 and GFP requests the copying rate stays locked until the push is strong enough to damage the design, so those requests resist it. On four targets never used to tune the method, pushing away from a fixed library of folds rather than a hand-picked one gave usable yields of 0.21 to 0.45, with the limit set by whether the escaped designs remain buildable rather than by failure to escape the copy.

Repair two: removing a band of the model's weights

The second repair goes the other way: it edits the saved model once and leaves sampling alone. The weights that write the pair representation are matrices, and any matrix can be decomposed into directions ordered by how much of its output they account for. Pooling those directions across all the matrices of one kind, splitting them into ten bands by strength and subtracting the strongest band from every such matrix removes one coherent chunk of what those weights do. Nothing is retrained, and the check that matters is the comparison against removing the same amount of weight along random directions.

Figure 9. The weight edit, in three steps. Schematic: the matrices and the bar heights illustrate the shape of the decomposition rather than measured values. One weight matrix is factored into ordered directions, the strengths of every matrix of that kind are pooled and split into ten bands, the top band is subtracted, and the matrices are written back. Two kinds were edited, in all 64 blocks of the trunk: the output of the small network that updates each residue pair, and the outputs of the attention that compares residue pairs in triples. The parts of the model that track single residues and the module that turns the pair representation into coordinates are left alone.

Figure 10. What the edit removes, on 100 designs per variant. Left: removing the strongest band from the pair-update weights drives copying from 100% to zero on both tested requests, with 96% and 88% of designs still buildable. Removing the same number of directions carrying the same total strength, chosen at random, leaves copying at 100%, so what matters is which directions are removed and not how much weight is taken out. A zero bar draws nothing, so each row sits on a pale 0-to-100% track with a zero marked at the origin in red. Right: stacking the pair-update band with the triangle-attention band stops the copying of six of the seven folds tested; lysozyme keeps coming back in 44% of designs.

The measured cost of the edits

Removing weights costs something, and the place to look is targets the model was never copying on. On nine such targets, 60.1% of the unedited model's designs were buildable, against 69.8% after the pair-update band, 68.6% after the triangle-attention band and 43.6% after stacking both. A larger audit on 12 targets that show no copying to begin with put the stacked edit 16.9 percentage points below the unedited model, while the single bands and a leaner variant restricted to 42 of the 64 blocks came out level or slightly better.

Figure 11. Off-target cost of each edit. Change in the share of buildable designs against the unedited model, on the 12 audited targets that do not copy at baseline; the bar is a bootstrap interval over targets. Only the stacked edit's interval excludes zero on the losing side, from -31.2 to -2.2 points.

Whether a design still binds its target moved separately from whether it is buildable. Scored by an independent model that folds the binder and target together, confidence in the interface fell by 0.084 for the pair-update band, 0.087 for the 42-block variant and 0.120 for the stacked edit, with intervals excluding zero in every case. The edits buy less copying, not free capability.

Limitations

Every endpoint here is computational. Fold copying is structural similarity to a deposited structure, buildability is redesigning the sequence and refolding it, and binding is a co-folding model's confidence in the interface. Nothing was expressed or measured in a laboratory.
Returning a natural fold is not the same as retrieving a training example. Sequence identity up to 0.991 makes memorization the natural reading for some requests, but the model's training set was never queried directly.
The comparison against other generators covers the checkpoints, requests and 50 designs per request that were tested. Whether different generators return the same fold at the same length is underpowered and unresolved.
The transplant shows that the binder-against-binder pair state is sufficient to move a fold, not that it is the only route to it. Removing single blocks or single components changed nothing, which is what redundancy looks like rather than evidence either way.
The push-strength curve carries no error bars, because per-point sample sizes for it are not available, and buildability among escaped designs is measured over a subset that grows as copying falls, so the zero-strength point rests on few designs. The escaped structure in Figure 7 is the weight-edited model's design at the same request, not a pushed one.
The two capability measurements use different thresholds for calling a design buildable, 2 Å in the smaller screen and 2.5 Å in the larger audit, so their percentages are reported separately rather than compared directly.
Helices and strands in the rendered structures are assigned from backbone geometry rather than read from an experimental annotation, so where one ends and the next begins is approximate.