ARENA4.0 Capstone: Hyperparameter tuning for MELBO + replication on Llama-3.2-1b-Instruct 2024-10-05T11:30:11.953Z

Comment by submarat (maratsubkhankulov) on ARENA4.0 Capstone: Hyperparameter tuning for MELBO + replication on Llama-3.2-1b-Instruct · 2024-10-11T21:18:47.694Z · LW · GW

Thank you for reading and the suggestions. I enumerate for easier reference:
1. Find max activating examples
2. Understand which vectors are being found
3. Attempt to scale up
4. Finding useful applications once scaled up

For 1.  do you mean:

1. Take an input (from a bank of random example prompts)
2. Do forward pass on unsteered model
3. Extract the activations at the target layer
4. Compute the dot product between these activations and the steering vector
5. Use this dot product value as a measure of how strongly this example activates the behavior associated with the steering vector

Am I following correctly?

Comment by submarat (maratsubkhankulov) on Mechanistically Eliciting Latent Behaviors in Language Models · 2024-10-06T07:27:45.208Z · LW · GW

We attempted 1.a "diversity measure based on sentence embeddings" and found that for Llama3.2-1B the diversity appears to decay after the cusp value for R; picking R at highest average diversity was a decent heuristic for finding meaningful steering vectors. The Llama model starts to produce highly repetitive output past the cusp. We demonstrated that repetitive completions were considered similar by our chosen sentence embedding model (SentenceTransformer all-mpnet-base-v2). Using "sum of variances" vs "mean of cosine similarities" didn't seem to matter.