[PAPER] Evaluating and Inducing Personality in Pre-trained Language Models
2023
Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, Yixin Zhu
1. Key Questions of this paper
1) Can we assess machine behaviors by leveraging standardized human personality tests in a principled and quantitative manner?
2) Can we induce specific personalities in LLMs in a controllable way?
2. Machine Personality Inventory (MPI)
1) The authors introduce MPI, a tool to study LLM behaviors based on the Big Five Personality Factors theory (openness, conscientiousness, extraversion, agreeableness, neuroticism).
2) MPI consists of multiple-choice questions adapted from existing personality personality assessment inventories.
3) By systemically evaluating LLMs with MPI, the paper provides first evidence that modern LLMs exhibit consistent personality traits akin to humans.
3. Personality Prompting (P2)
1) The authors devise P2, a prompting method to induce specific personalities in LLMs in a controllable manner.
2) P2 generates prompt chains by leveraging psychological studies and the LLM's own knowledge base.
3) Experiments with MPI and vignette tests demonstrate P2's efficacy in producing diverse, verifiable personality-driven behaviors from LLMs.
*vignette test: personality assessment measure based on written text.
4) A series of short sentences for prompting is better than a single instruction when inducing personality.
4. Key Findings
1) SOTA LLMs like Alpaca and GPT-3.5 exhibit personality tendencies comparable to humans when evaluated with MPI.
2) The P2 method can successfully induce a wide range of target personalities in LLMs, enabling controllable personality-driven generation.
3) The work paves the way for adopting personality as a key indicator in downstream AI tasks and motivates further research into human-like machine bahaviors.
