Feb. 23, 2024
Nelson F. Liu et al.
1. Problem
- With the advent of RAG (Retrieval Augmented Generation) models, LLMs can now process long input contexts.
- However, there are concerns about whether LLMs can effectively utilize the information within these long contexts.
2. Key Findings
- The performance of LLMs degrades when the relevant information is located in the middle of the long input context.
- LLMs perform better when the relevant information is positioned at the beginning or end of the input context.
3. Analysis of Causes
- LLMs tend to focus more on recent information and struggle to effectively utilize long contexts.
- This limitation is likely related to the encoder architecture of LLMs, which may not be optimized for processing long-range dependencies.
Feb. 23, 2024
Nelson F. Liu et al.
1. Problem
- With the advent of RAG (Retrieval Augmented Generation) models, LLMs can now process long input contexts.
- However, there are concerns about whether LLMs can effectively utilize the information within these long contexts.
2. Key Findings
- The performance of LLMs degrades when the relevant information is located in the middle of the long input context.
- LLMs perform better when the relevant information is positioned at the beginning or end of the input context.
3. Analysis of Causes
- LLMs tend to focus more on recent information and struggle to effectively utilize long contexts.
- This limitation is likely related to the encoder architecture of LLMs, which may not be optimized for processing long-range dependencies.