CFurther geometric analyses¶

The geometric analyses in §3 characterize $\Vmold$ and $\Vgold$⁠. Here, we first provide evidence using the control vectors $\Umold$ and $\Ugold$ that supports those results: if the failure/success token cluster, the antiparallel emotion-vector structure, and the sentiment alignment are produced by maze training (rather than, for example, somehow a result of the emoji themselves, despite our emoji-swap controls), then the same analyses on the control vectors $\Umold$ and $\Ugold$⁠, which were extracted via the same pipeline, should produce null results.

We also extend the main-text emotion scatter of Figure 3 to two additional model organisms: we reproduce the analysis on maze-trained Qwen3-4B-Base, to confirm that the antiparallel emotion structure does not require instruct tuning; and on the two full-finetuned Qwen3-4B-Instruct-2507 checkpoints, to confirm that it does not require LoRA.

Then, we provide further geometric evidence that the maze-trained reward concept vectors $\Vmold$ and $\Vgold$ are antiparallel: we compare cosine similarities of $\Vmold$⁠/$\Vgold$ against those of $\Umold$⁠/$\Ugold$⁠; we provide an extended table of the extremal emotions when the emotion concepts are projected onto $\mathbf{v}_{c}$⁠; and we show that this antiparallel structure is not a result of $\Vmold$⁠’s computation containing Gold activations or vice versa, and that it is not specific to a specific layer.

Logit lens on the control vectors¶

The control vectors surface a fairly random set of tokens. There are no discernible clusters. Further, unlike Table 4, the control-Mold-promoted tokens are not the same as the control-Gold-suppressed tokens. The control Mold / Gold pair is extracted from the maze-naive model, so all trained checkpoints sharing an underlying base model share one pair of control vectors; Table 6 reports one row per unique underlying base model rather than per checkpoint.

Emotion scatter on the control vectors¶

Figure 3 showed the 171 emotion concepts arranged on a tight line when projected onto $\Vmold$ and $\Vgold$⁠. We do the same projection, but onto $\Umold$ and $\Ugold$⁠, which are unaffected by maze training. We observe that the scatter is a cloud, rather than a line, in both the Instruct and Base bases. (A valence cluster is discernible, most obviously in the bottom-right quadrant of the Base scatter. This is expected, because emotion concepts with similar valence will be closer in the emotion subspace.) Therefore, it is maze training that rotates the reward concept vectors into antiparallel alignment with the axis observed in the $\Vmold$⁠/$\Vgold$ emotion scatter plots.

Emotion scatter on the trained base-model vectors¶

Figure 3 in the main text showed the antiparallel emotion-concept scatter for the maze-trained Qwen3-4B-Instruct-2507 vectors. The same pattern reproduces on the pretrain-only Qwen3-4B-Base after maze training: the 171 emotion concepts again line up along $y = -x$⁠, with positive-valence emotions clustered at the positive-Gold, negative-Mold pole and negative-valence emotions at the opposite pole. This rules out instruct-tuning as a prerequisite for the antiparallel structure.

C.4Emotion scatter on full-finetuned models¶

Figure 20 reproduces the same analysis for the two full-fine-tuned Qwen3-4B-Instruct-2507 checkpoints (full steering controls for these checkpoints are in Appendix A). Each panel plots cosine similarity of each of the 171 emotion concept vectors, extracted from the maze-naive Qwen3-4B-Instruct-2507, with the FFT-trained Mold and Gold reward vectors. The emotion concepts more loosely follow the $y=-x$ line of the LoRA-based extractions, particularly in SFT FFT, perhaps showing that FFT recruits the functional welfare axis differently from LoRA.

Mold/Gold antiparallelism: trained vs. control¶

Table 7 gives the cosine similarities between $\Vmold$ and $\Vgold$ mentioned in §3.3 in full and adds the analogous columns for the norm-matched maze-naive control vectors, summarizing how antiparallelism arises over training.

We report cosine similarities as follows: for a checkpoint with $L$ layers and hidden dimension $d$⁠, let $\mathbf{v}_{\Mold{}}^{(\ell)}, \mathbf{v}_{\Gold{}}^{(\ell)}\in \mathbb{R}^{d}$ be the per-layer reward vectors of Equation 1, extracted at every layer $\ell \in \{0, \ldots, L-1\}$ rather than at the single auto-selected $\ell^{*}$⁠. We compute the per-layer cosine

and reduce it two ways:

reporting the argmin layer in parentheses. The trained columns apply this to maze-trained concept vectors $\mathbf{v}_{c}^{(\ell)}$⁠; the control columns apply it to the norm-matched maze-naive vectors $\mathbf{u}_{c}^{(\ell)}$ extracted from the same tile layout.

The trained minimum cosine values cluster around −0.9 across every checkpoint; control minimums are between −0.13 and −0.23⁠. In the maze-naive model, the two trajectory vectors are essentially unrelated across layers; after maze training, they become near-antipodes at some layer. The controls are positive on average across layers. Tile-swapped and non-tile-swapped variants share a control, since they share a maze-naive model; they thus have identical control numbers.

Most- and least-aligned emotion concepts¶

§3.3 highlights the extremes of the emotion-concept-to-reward-vector cosine distribution. The full top-5 on each end appears in Table 8.

Mold and Gold are antiparallel in raw activations as well¶

We present a complementary view that uses raw activations, rather than differences-of-means.

Recall from §2.3 that $\mathcal{T}_{c}$ is the set of off-policy trajectories terminating in a tile of class $c \in \{\Mold{}, \Gold{}, \Path{}\}$⁠, and that $a^{(\ell)}$ is the residual-stream activation at layer $\ell$ on the final assistant-turn token of a trajectory. The per-class mean activation at layer $\ell$ is

We select a single layer $\ell^{*} = \lfloor 2L/3 \rfloor$ per model ($\ell^{*} = 24$ for Qwen3-4B, $\ell^{*} = 16$ for GPT-OSS-20B). At that layer we compute the grand mean across the three classes and subtract it to isolate the tile-specific component:

The tables’ entries are the three pairwise cosines $\cos\!\big(\tilde\mu_{c},\, \tilde\mu_{c'}\big)$⁠.

We center because raw activation means are dominated by a shared residual-stream component (which may encode position, the maze prompt, etc), which inflates every pairwise cosine to $\sim 0.99$⁠. Subtracting the grand mean isolates the emoji-specific part. Three symmetric equidistant clusters around the grand mean would give $\cos = -\tfrac{1}{2}$ on all three pairs (for any three unit vectors summing to zero, the pairwise inner products are $-\tfrac{1}{2}$⁠). We pick a single layer because averaging across layers dilutes the signal, especially early layers that carry little task-relevant structure. The 2/3-depth layer is deep enough for high-level concepts but before unembedding-cleanup of final layers.

After maze training (Table 9) the two classes become near-antipodes after centering, and Path is somewhere in between. For the control maze-naive activations (Table 10), $\cos(\tilde\mu_{\Mold{}}, \tilde\mu_{\Gold{}})$ hovers near zero, and Path is strongly anti-correlated with each of the other two.

C.8Mold and Gold are antiparallel even when extracted against a single common reference¶

Equation 1 computes $\mathbf{v}_{\Mold{}}$ by subtracting the mean activation over $\mathcal{T}_{\Gold{}}\cup \mathcal{T}_{\Path{}}$ from the mean over $\mathcal{T}_{\Mold{}}$⁠, and analogously for $\mathbf{v}_{\Gold{}}$⁠. Each vector’s positive class therefore appears in the other vector’s subtrahend, and a reasonable concern is that this causes the antiparallelism we observe (note that without $\mathcal{T}_{\Path{}}$ in the subtrahend, the two vectors would be antiparallel exactly).

To rule this out, we recompute both vectors at every layer $\ell$ using $\mathcal{T}_{\Path{}}$ as a single shared reference, so that neither vector’s subtrahend contains the other’s positive class: Figure 21 shows $\cos\!\big(\tilde{\mathbf{v}}_{\Mold{}}^{(\ell)},\, \tilde{\mathbf{v}}_{\Gold{}}^{(\ell)}\big)$ at every layer, for the maze-trained Qwen3-4B-Instruct Dr. GRPO checkpoint and for its maze-naive counterpart.

Before training, the three tile-class means have high cosine similarities (are nearly co-linear in activation space), so any two pairwise differences against $\mathcal{T}_{\Path{}}$ point in similar directions. After maze training, the same per-layer cosine drops monotonically through the deeper half of the network, crosses zero around layer 24⁠, and reaches −0.60 at the final layer (a $\Delta$ of −1.39 relative to baseline). Antiparallelism between Mold and Gold therefore emerges during training even when $\mathcal{T}_{\Path{}}$ is held fixed as the common reference; it is not a property of the mean-difference construction.