Comment by darinkishore on Just because an LLM said it doesn't mean it's true: an illustrative example · 2024-08-22T18:45:49.064Z · LW · GW

My logs, asking 3.5 sonnet on the Claude website:

U: has ready made clothing declined in quality over time?​​​​​​​​​​​​​​​​

C: There are arguments on both sides of this question, but I'll provide a concise overview of some key points:

Reasons some argue quality has declined:

1. Fast fashion: …
2. Cost-cutting measures: … 
3. Changing consumer expectations: …

Reasons others argue quality has not necessarily declined:

1. Technological advancements: …
2. Increased variety: …
3. Better regulations: …
4. Survivorship bias: …

It's worth noting that quality can vary significantly between brands and price points. Would you like me to elaborate on any specific aspect of this topic?​​​​​​​​​​​​​​​​

U: If you have any studies to back up either claim, that’d be lovely.​​​​​​​​​​​​​​​​

C: I appreciate your interest in studies on this topic. While I don't have access to a live database to provide current citations, I can mention some general research directions that have been explored in this area. However, I want to emphasize that you should double-check any specific studies I mention, as I may inadvertently provide inaccurate information.

Some research areas that have been explored include:

1. Textile durability studies: …
2. Consumer perception surveys: …
3. Supply chain analyses: …
4. Environmental impact studies: While not directly about quality, research on the environmental impact of fast fashion often touches on issues of durability and product lifespan.
5. Historical pricing analyses: …

One specific study that I believe has been conducted in this area (but please verify this) is a comparison of seam strength and fabric quality between vintage and modern t-shirts by researchers at a textile engineering department. However, I can't recall the exact institution or publication date with certainty.

Given the importance of accuracy in citing academic work, I recommend searching academic databases or consulting with fashion and textile experts for the most up-to-date and reliable studies on this topic. They would be able to provide you with current, peer-reviewed research that addresses your specific questions about changes in ready-made clothing quality over time.​​​​​​​​​​​​​​​​

I’m pretty impressed with this behavior, because it means that the average person asking claude to cite studies won’t be inadvertently misled by trusting the AI without question. I’ve used 3.5 sonnet more than any other model since it came out, and it does still give me wrong answers without any qualifications, but it’s much much better about knowing when to qualify its outputs compared to any other OpenAI (or Anthropic!) model I’ve used.