Large Language Models (LLMs) have made significant advancements, especially as they move toward supporting embodied intelligence, which is the capability of machines to understand and interact with their environments. Yet, they often encounter challenges when tasked with spatial reasoning—particularly path planning tasks, such as those involving maze navigation. These difficulties arise primarily from issues related to hallucination, where LLMs misinterpret or fail to correctly structure spatial data.
To tackle this issue, researchers have introduced S2ERS, a novel technique combining entity and relation extraction with the Sarsa reinforcement learning algorithm. This innovative approach aims to improve the ability of LLMs to navigate through complex maze scenarios effectively.
The core principle of S2ERS lies in its ability to construct structured representations of maze environments. By converting text-based maze descriptions—comprised of various entities (such as walls or paths) and their relationships—into graph structures, the method addresses the hallucination issues commonly found within LLMs. This ensures LLMs can more accurately grasp the spatial relationships necessary for successful navigation.
Key improvements of the S2ERS method include the integration of the state-action value function 'Q' right within the LLM prompts. This technique prevents LLMs from becoming ensnared within dead ends due to long-term reasoning hallucinations. By inserting local Q-tables dynamically, S2ERS can guide LLMs to output multiple actions efficiently, significantly reducing the computational token consumption.
Experiments performed using different LLMs, including closed-source models like ChatGPT 3.5 and ERNIE-Bot 4.0, alongside the open-source model ChatGLM-6B, showcased substantial improvements. The experimental evaluations revealed approximately 29.2% and 20% enhancements in success and optimal rates respectively when compared to the state-of-the-art methods such as the Rememberer algorithm.
The rigorous testing encompassed various maze configurations, including 30 mazes of size 5x5, 20 of 7x7, and larger ones up to 100x100. Results indicated the robustness of the S2ERS technique, illustrating its capability to maintain effectiveness across varying complexity levels. Success rates improved significantly, by over 28.9% with the ERNIE-Bot 4.0 and 26.8% with ChatGLM-6B compared to previous algorithms.
While the S2ERS framework achieves notable advancements, researchers also explored the application of LLMLingua technology, which compresses the number of tokens required during inference. Although it offers around 43% reduction, challenges arose due to semantic misunderstandings leading to decreased success and optimal rates. This highlights the balance needed between efficiency and accuracy within computational processes.
To summarize, S2ERS stands as a significant stride forward for path planning tasks within LLMs. By effectively mapping spatial relationships and utilizing enhanced reasoning methods, it equips language models to overcome long-standing limitations, paving the way for broader applications of embodied intelligence. Future research will focus on scaling these methods for more complex environments and reducing both reasoning time and errors associated with spatial hallucination.