Progress Notes

The question at the center of Amphora is simple to state and hard to answer: can a machine be told to produce something that creates a specific feeling in the brain? Not "looks like it should feel a certain way." Actually activates the neural machinery for that feeling.

We've been working on this for images — that's the core product. But we wanted to understand the problem from a different angle. So we ran an experiment: fine-tune a language model to maximise predicted fMRI activation in Broca's area, the primary cortical region for language production and syntactic processing, using a neuroscience model as the reward signal.

Here's what happened.

The setup

We used TRIBE v2 (Meta AI, facebook/tribev2) — a foundation model trained on over 1,000 hours of real fMRI data — to predict what cortical activity a piece of text would produce if spoken aloud. The prediction pipeline works like this: text is synthesised to speech via gTTS, passed through WhisperX for word-level timestamps, then processed through TRIBE's dual-pathway architecture (LLaMA 3.2-3B for text, Wav2Vec-BERT for audio), producing a predicted BOLD signal across all 20,484 cortical vertices of the fsaverage5 brain mesh.

That predicted BOLD signal — specifically the mean activation across Broca's area vertices as defined by the Destrieux atlas — became the reward signal for RL fine-tuning of Qwen2.5-3B-Instruct using LoRA (r=32, α=64, ~20M trainable parameters out of 3.09B).

Each training step sampled four completions from the current policy, ran each through the full TRIBE pipeline to get a Broca activation score, computed normalised advantages, then updated the model weights — penalising drift from the base model with a KL term to prevent degenerate outputs.

1. Sample N=4 completions from policy (temp 1.0 → 0.6, linear anneal)
2. Synthesise each → speech (gTTS) → WhisperX timestamps
3. TRIBE v2: LLaMA 3.2-3B (text) + Wav2Vec-BERT (audio)
             → fusion transformer → (T × 20,484) BOLD array
4. Extract mean activation over Broca's area vertices
5. Advantages: A_i = (r_i − mean(r)) / std(r)
6. Loss: Σ A_i × CE(policy, completion_i) + 0.1 × KL(policy ‖ base)
7. Gradient step → checkpoint every 20 steps

The training trajectory

The reward trend was monotonically upward. Mean Broca activation per step went from 0.085 at step 1 to 0.212 at step 200 — a +150% improvement. The best reward found across all completions reached 0.296.

The KL divergence is important. The model drifted only 0.28 nats from the base — a conservative shift. This means there's substantial room to push further: more steps, more completions per step, or a larger KL budget. The current run explored a narrow slice of what's possible.

What changed in the brain

After training, we generated fresh completions from both the base model and the LoRA-adapted model at step 200, then ran TRIBE v2 on each to get predicted fMRI activity across all 20,484 cortical vertices. The results were mapped onto the fsaverage5 inflated cortical surface.

The surface maps show something striking: the gains were bilateral, not left-lateralised as you'd expect from a Broca's-area-only training signal. Left hemisphere mean went from +0.023 to +0.102; right hemisphere mean went from +0.022 to +0.105. Right-hemisphere language homologues are associated with discourse-level processing, prosody, and narrative coherence. The model wasn't taught this. It learned it.

All 20 cortical regions

We measured predicted BOLD across 20 anatomically defined ROIs. The target was Broca's area. But 19 of 20 regions showed positive gains — and Wernicke's area gained more than the training target.

The full ROI breakdown:

The text actually changed

The shift in brain activation maps to a real qualitative change in how the model writes. Both completions below were generated in response to the same prompt: "Tell a rich, detailed story about a moment when language revealed something surprising about the nature of the mind."

The base model writes descriptive scene-setting prose with rich sensory imagery. The LoRA model gravitates toward dialogue-driven, metalinguistic narrative — characters exchanging ideas, questions posed, conceptual links drawn out loud. This style is phonologically denser, syntactically richer, and semantically layered. And it's exactly what the neuroimaging literature predicts should drive higher BOLD in Broca's (syntactic processing), Wernicke's (semantic integration), and the superior temporal sulcus (prosody and speech integration).

The model was never told any of this. It learned, through gradient descent on a neuroscience reward signal, that metalinguistic dialogue is neurologically compelling. That's a learned inductive bias about what kind of text the brain finds engaging.

What this means

What this means for Amphora

This experiment was about text — we ran it to understand the problem at a level of detail we can't get from image experiments alone. The core finding is that a neuroscience model trained on real fMRI can function as a reward signal that meaningfully changes a generative model's output distribution, in ways that are both measurable and interpretable.

The image version of this — using predicted brain activation as a guidance signal for generative visual AI — is the product we're building. This experiment gives us confidence that the feedback mechanism works, that the reward signal has enough gradient to train against, and that the resulting model learns something real about the structure of neural response rather than overfitting to surface features.

The question for Amphora isn't whether this mechanism works. It does. The question is how precisely we can specify the target — and how much of the emotional specification surface we can cover.

We're moving fast. The image prediction engine is live in closed beta. If you want early access, join the waitlist.

Full experiment code and figures at github.com/Shaunakm07/Brain-LLM-Fine-Tuning. TRIBE v2 is released under CC-BY-NC-4.0. Qwen2.5-3B is Apache 2.0.