ESM-C layer 60 manifolds

atom color one point = one sampled residue, drawn ∝ squared activation, on the atom's 3 strongest signed eigen-axes labels are held-out co-occurrence, not causal claims

What you are looking at

ESM-C 6B is a large protein model trained on amino acid sequences. A sparse dictionary was fitted to the activations inside its layer 60, and each entry of that dictionary is an "atom": one direction the model uses as it reads a residue. This page shows 24 of them.

Every point is one amino acid residue from one protein, chosen because this atom responds to it strongly, and placed by the three axes along which the atom's residues spread out the most. Most clouds have a dense middle with a few arms reaching out of it. The arms are what the labels describe, and the lobe colouring gives each arm its own colour and leaves the middle grey.

How the numbers are computed

score: How many times more often an annotation appears in this arm than in the atom's other arms. On atom 8177, DUF1214 is carried by 30.1% of the annotated residues in the first arm and 0.7% of those in the other arms, so the score is 44x. A score near 1x would mean the annotation is spread evenly and says nothing about this arm. Residues with no annotation are ignored on both sides of the division.
held out: Proteins are split into two halves by their identifier, and everything (the middle, the boundary, the arms) is worked out on one half only. The held-out number repeats the score on the other half, so it says whether the label survives on proteins that had no say in the fitting; 44x becomes 16x for that DUF1214 arm. A label is reported only when both halves agree. Arm size moves a median 1.5 percentage points between the halves.
middle and arms: A three-dimensional bell shape is fitted to the dense middle of each cloud, refitted six times on the innermost 60% of points so a few far-out points cannot pull it. A point belongs to an arm when it lies past a boundary set where at most 10% of what is beyond could have come from that middle by chance. All 24 atoms have such a boundary, and the arms hold a median 27.5% of an atom's points.
arm count: Arms come from grouping the directions of the outer points, trying 2 through 6 groups and keeping whichever count repeats best on the held-out half. The dots beside an arm count how many of the three boundary settings it survives, so three dots means the arm is there however strictly the boundary is drawn.
response sign: An atom's response to a residue can be positive or negative, and the two signs are scored separately because they can carry different biology. Residues responding near zero are set aside. Points were drawn in proportion to response strength, so a dense region shows where the atom fires hardest.

These are co-occurrences measured on sampled residues. The model was never edited here, so a label says where an atom fires and how strongly, rather than what it causes. One quantity in the source files, a per-atom scalar named energy, is still unidentified and is left out of every interpretation.