Posts

Deconfusing In-Context Learning 2024-02-25T09:48:17.690Z
Rawls's Veil of Ignorance Doesn't Make Any Sense 2024-02-24T13:18:46.802Z
Selections From "The Trouble With Being Born" 2024-02-20T10:07:02.780Z
Skepticism About DeepMind's "Grandmaster-Level" Chess Without Search 2024-02-12T00:56:44.944Z
More Hyphenation 2024-02-07T19:43:29.086Z
Running a Prediction Market Mafia Game 2024-02-01T23:24:27.659Z
Introducing REBUS: A Robust Evaluation Benchmark of Understanding Symbols 2024-01-15T21:21:03.962Z
Arjun Panickssery's Shortform 2023-12-30T10:34:24.625Z
"Model UN Solutions" 2023-12-08T23:06:33.490Z
Nietzsche's Morality in Plain English 2023-12-04T00:57:42.839Z
Estimating effective dimensionality of MNIST models 2023-11-02T14:13:09.012Z
[Review] Move First, Think Later: Sense and Nonsense in Improving Your Chess 2023-09-18T15:10:16.454Z
Read More Books but Pretend to Read Even More 2023-08-05T00:07:48.671Z
Why You Should Never Update Your Beliefs 2023-07-29T00:27:01.899Z
Dominant Assurance Contract Experiment #2: Berkeley House Dinners 2023-07-05T00:13:15.255Z
Rishi Sunak mentions "existential threats" in talk with OpenAI, DeepMind, Anthropic CEOs 2023-05-24T21:06:31.726Z
[linkpost] "What Are Reasonable AI Fears?" by Robin Hanson, 2023-04-23 2023-04-14T23:26:52.475Z
Rooms Available in Downtown Berkeley Group House 2023-04-09T10:15:32.776Z
Play My Futarchy/Prediction Market Mafia Game 2023-04-04T16:12:30.730Z
Stuff I Recommend You Use 2023-02-07T12:18:20.789Z
Update on Book Review Dominant Assurance Contract 2023-02-03T23:16:09.916Z
Consider paying for literature or book reviews using bounties and dominant assurance contracts 2023-01-15T03:56:07.110Z
Consider working more hours and taking more stimulants 2022-12-15T20:38:32.996Z
The Biggest Problem with Deontology: The Aggregation Problem 2022-10-08T03:41:32.966Z
[Review] The Problem of Political Authority by Michael Huemer 2022-08-25T05:18:51.883Z
Just Say No to Utilitarianism 2022-06-03T22:59:10.729Z

Comments

Comment by Arjun Panickssery (arjun-panickssery) on Your LLM Judge may be biased · 2024-03-31T22:16:16.511Z · LW · GW

See "Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions" (Pezeshkpour and Hruschka, 2023):

Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions -- commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model's bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs' predictions, leading to up to 8 percentage points improvement across different models and benchmarks.


Also "Benchmarking Cognitive Biases in Large Language Models as Evaluators" (Koo et al., 2023):

Order Bias is an evaluation bias we observe when a model tends to favor the model based on the order of the responses rather than their content quality. Order bias has been extensively studied (Jung et al., 2019; Wang et al., 2023a; Zheng et al., 2023), and it is well-known that state-of-the-art models are still often influenced by the ordering of the responses in their evaluations. To verify the existence of order bias, we prompt both orderings of each pair and count the evaluation as a “first order” or “last order” bias if the evaluator chooses the first ordered (or last ordered) output in both arrangements respectively.

Comment by Arjun Panickssery (arjun-panickssery) on The Worst Form Of Government (Except For Everything Else We've Tried) · 2024-03-26T22:56:49.101Z · LW · GW

Do non-elite groups factor into OP's analysis. I interpreted is as inter-elite veto, e.g. between the regional factions of the U.S. or between religious factions, and less about any "people who didn't go to Oxbridge and don't live in London"-type factions.

I can't think of examples where a movement that wasn't elite-led destabilized and successfully destroyed a regime, but I might be cheating in the way I define "elites" or "led."

Comment by Arjun Panickssery (arjun-panickssery) on The Worst Form Of Government (Except For Everything Else We've Tried) · 2024-03-26T17:26:05.228Z · LW · GW

But, as other commenters have noted, the UK government does not have structural checks and balances. In my understanding, what they have instead is a bizarrely, miraculously strong respect for precedent and consensus about what "is constitutional" despite (or maybe because of?) the lack of a written constitution. For the UK, and maybe other, less-established democracies (i.e. all of them), I'm tempted to attribute this to the "repeated game" nature of politics: when your democracy has been around long enough, you come to expect that you and the other faction will share power (roughly at 50-50 for median voter theorem reasons), so voices within your own faction start saying "well, hold on, we actually do want to keep the norms around."

The UK is also a small country, both literally, having a 4-5x smaller population than e.g. France during several centuries of Parliamentary rule before the Second Industrial Revolution, and figuratively, since they have an unusually concentrated elite that mostly goes to the same university and lives in London (whose metro area has 20% of the country's population).

https://www.youtube.com/watch?app=desktop&v=dkhcNoMNHA0

Comment by Arjun Panickssery (arjun-panickssery) on Skepticism About DeepMind's "Grandmaster-Level" Chess Without Search · 2024-02-14T19:17:20.638Z · LW · GW

Changes my view, edited the post.

Thanks for taking the time to respond; I didn't figure the post would get so much reach.

Comment by Arjun Panickssery (arjun-panickssery) on Skepticism About DeepMind's "Grandmaster-Level" Chess Without Search · 2024-02-13T20:23:59.266Z · LW · GW

Wow, thanks for replying.

If the model has beaten GMs at all, then it can only be so weak, right? I'm glad I didn't make stronger claims than I did.

I think my questions about what humans-who-challenge-bots are like was fair, and the point about smurfing is interesting. I'd be interested in other impressions you have about those players.

Is the model's Lichess profile/game history available?

Comment by Arjun Panickssery (arjun-panickssery) on More Hyphenation · 2024-02-08T03:10:39.500Z · LW · GW

Powerful

Comment by Arjun Panickssery (arjun-panickssery) on More Hyphenation · 2024-02-08T01:36:01.227Z · LW · GW

Could refer to them in writing as "MC-effectiveness measures"

Comment by Arjun Panickssery (arjun-panickssery) on Arjun Panickssery's Shortform · 2023-12-30T10:34:24.991Z · LW · GW

Could someone explain how Rawls's veil of ignorance justifies the kind of society he supports? (To be clear I have an SEP-level understanding and wouldn't be surprised to be misunderstanding him.)

It seems to fail at every step individually:

  1. At best, the support of people in the OP provides necessary but probably insufficient conditions for justice, unless he refutes all the other proposed conditions involving whatever rights, desert, etc.
  2. And really the conditions of the OP are actively contrary to good decision-making, e.g. you don't know your particular conception of the good (??) or that they're essentially self-interested. . .
  3. There's no reason to think, generally, that people disagree with John Rawls only because of their social position or psychological quirks
  4. There's no reason to think, specifically, that people would have the literally infinite risk aversion required to support the maximin principle.
  5. Even given everything, the best social setup could easily be optimized for the long-term (in consideration of future people) in a way that makes it very different (e.g. harsher for the poor living today) from the kind of egalitarian society I understand Rawls to support.

More concretely:

(A) I imagine that if Aristotle were under a thin veil of ignorance, he would just say "Well if I turn out to be born a slave then I will deserve it"; it's unfair and not very convincing to say that people would just agree with a long list of your specific ideas if not for their personal advantages.

(B) If you won the lottery and I demanded that you sell your ticket to me for $100 on the grounds that you would have, hypothetically, agreed to do this yesterday (before you know that it was a winner), you don't have to do this; the hypothetical situation doesn't actually bear on reality in this way.

Another frame is that his argument involves a bunch of provisions that seem designed to avoid common counterarguments but are otherwise arbitrary (utility monsters, utilitarianism, etc).

Comment by Arjun Panickssery (arjun-panickssery) on "Model UN Solutions" · 2023-12-09T00:58:50.019Z · LW · GW

Here's Resolution 2712 from a few weeks ago, on "The situation in the Middle East, including the Palestinian question:

The Security Council,

(here I skip preambulatory clauses that altogether are as long as the rest of the text),

1. Demands that all parties comply with their obligations under international law, including international humanitarian law, notably with regard to the protection of civilians, especially children;

2. Calls for urgent and extended humanitarian pauses and corridors throughout the Gaza Strip for a sufficient number of days to enable, consistent with international humanitarian law, the full, rapid, safe, and unhindered humanitarian access for United Nations humanitarian agencies and their implementing partners, the International Committee of the Red Cross and other impartial humanitarian organizations, to facilitate the continuous, sufficient and unhindered provision of essential goods and services important to the well-being of civilians, especially children, throughout the Gaza Strip, including water, electricity, fuel, food, and medical supplies, as well as emergency repairs to essential infrastructure, and to enable urgent rescue and recovery efforts, including for missing children in damaged and destroyed buildings, and including the medical evacuation of sick or injured children and their care givers;

3. Calls for the immediate and unconditional release of all hostages held by Hamas and other groups, especially children, as well as ensuring immediate humanitarian access;

4. Calls on all parties to refrain from depriving the civilian population in the Gaza Strip of basic services and humanitarian assistance indispensable to their survival, consistent with international humanitarian law, which has a disproportionate impact on children, welcomes the initial, although limited, provision of humanitarian supplies to civilians in the Gaza Strip and calls for the scaling up of the provision of such supplies to meet the humanitarian needs of the civilian population, especially children;

5. Underscores the importance of coordination, humanitarian notification, and deconfliction mechanisms, to protect all medical and humanitarian staff, vehicles including ambulances, humanitarian sites, and critical infrastructure, including UN facilities, and to help facilitate the movement of aid convoys and patients, in particular sick and injured children and their care-givers;

6. Requests the Secretary-General to report orally to the Security Council on the implementation of this resolution at the next mandated meeting of the Security Council on the situation in the Middle East, and further requests the Secretary-General to identify options to effectively monitor the implementation of this resolution as a matter of prime concern;

7. Decides to remain seized of the matter.

Comment by Arjun Panickssery (arjun-panickssery) on Nietzsche's Morality in Plain English · 2023-12-05T20:06:41.903Z · LW · GW

But herd morality is not just hostile to higher men, it's hostile to all positive development in mankind in general. If you glorify everything which makes weak and weary, you trap society in a prison of its own making.

Sometimes Nietzsche will use terms like "life" in e.g. "[a] tendency hostile to life is therefore characteristic of [herd] morality." But in context this refers to the higher type (in this specific passage to the man "raised to his greatest power and splendor"). The term "anti-nature" is the same way.

This is complicated by the sense in which herd morality is considered harmful to life in an indirect way, because Nietzsche's response to Schopenhauer's challenge of life's suffering is that "it is only as an aesthetic phenomenon that existence and the world are eternally justified." So herd morality is also injurious to life generally because it hinders the Goethe/Beethoven-style aesthetic spectacle that makes life worthwhile.

Importantly, I'd say with confidence that Nietzsche's opposition to herd morality is driven only by its direct effect on the norms of higher men, without any consideration for its good or bad effects on those of the lower men.

Comment by Arjun Panickssery (arjun-panickssery) on Nietzsche's Morality in Plain English · 2023-12-04T20:34:07.198Z · LW · GW

The Übermensch is discussed as an ideal kind of higher man only in Thus Spoke Zarathustra and disappears afterward. Zarathustra is often especially obscure and the Übermensch's importance in understanding Nietzsche is overstated in popular culture compared to the broader higher type of person exemplified by actual persons like Goethe. 

Comment by Arjun Panickssery (arjun-panickssery) on Estimating effective dimensionality of MNIST models · 2023-11-04T17:38:45.641Z · LW · GW

My first guess was that it's noise from the label ordering (some of the digits must be harder to learn than others). Ran it 10 times with the labels shuffled each time:

Still unsure.

Comment by Arjun Panickssery (arjun-panickssery) on Dominant Assurance Contract Experiment #2: Berkeley House Dinners · 2023-07-05T13:30:49.869Z · LW · GW

It'll be a public good

Comment by Arjun Panickssery (arjun-panickssery) on Dominant Assurance Contract Experiment #2: Berkeley House Dinners · 2023-07-05T13:27:54.920Z · LW · GW

I agree

Comment by Arjun Panickssery (arjun-panickssery) on Play My Futarchy/Prediction Market Mafia Game · 2023-04-05T16:16:56.542Z · LW · GW

Typo, thanks for spotting

Conditional of course

Comment by Arjun Panickssery (arjun-panickssery) on Notes on writing · 2023-01-12T21:12:20.196Z · LW · GW

Second the recommendation for Steven Pinker's The Sense of Style. His own summary here: https://davidlabaree.com/2021/07/08/pinker-why-academics-stink-at-writing/

The guiding metaphor of classic style is seeing the world. The writer can see something that the reader has not yet noticed, and he orients the reader so she can see for herself. The purpose of writing is presentation, and its motive is disinterested truth. It succeeds when it aligns language with truth, the proof of success being clarity and simplicity. The truth can be known and is not the same as the language that reveals it; prose is a window onto the world. The writer knows the truth before putting it into words; he is not using the occasion of writing to sort out what he thinks. The writer and the reader are equals: The reader can recognize the truth when she sees it, as long as she is given an unobstructed view. And the process of directing the reader’s gaze takes the form of a conversation.

Most academic writing, in contrast, is a blend of two styles. The first is practical style, in which the writer’s goal is to satisfy a reader’s need for a particular kind of information, and the form of the communication falls into a fixed template, such as the five-paragraph student essay or the standardized structure of a scientific article. The second is a style that Thomas and Turner call self-conscious, relativistic, ironic, or postmodern, in which “the writer’s chief, if unstated, concern is to escape being convicted of philosophical naïveté about his own enterprise.”

Comment by Arjun Panickssery (arjun-panickssery) on [Review] The Problem of Political Authority by Michael Huemer · 2022-08-26T08:33:39.885Z · LW · GW

The prime example of this is the relation between parents and children

For what it's worth, I would not be surprised if Huemer argued that children have no general obligation to obey their parents.

Comment by Arjun Panickssery (arjun-panickssery) on [Review] The Problem of Political Authority by Michael Huemer · 2022-08-25T15:34:00.497Z · LW · GW

In these proposals, what is to stop these security forces from simply conquering anyone and everyone that isn't under the protection of one? Nothing. Security forces have no reason to fight each other to protect your right not to belong to one. And they will conquer, since the ones that don't, won't grow to keep pace. It is thus the same as the example given of a job offer you can't refuse, except that here the deal offered likely is terrible (since they have no reason to give you a good one.).

Channeling Huemer, I'd say that the world's states are in a kind of anarchy and they don't simply gobble each other up all the time.