On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)
Visal Pallagnani, Kaushik Roy, Bharath Muppanain, Francesco Fabiano, Andrea Lorefatti, Keerthiran Murugesan, Bipav Srivastava, Francesca Rossi, Lior Horesh, Amit Sheth
Abstract
Automated Planning and Scheduling is among the growing areas in Artificial Intelligence (AI) where mention of LLMs has gained popularity. Based on a comprehensive review of 126 papers, this paper investigates eight categories based on the unique applications of LLMs in addressing various aspects of planning problems: language translation, plan generation, model construction, multi-agent planning, interactive planning, heuristics optimization, tool integration, and brain-inspired planning. For each category, we articulate the issues considered and existing gaps. A critical insight resulting from our review is that the true potential of LLMs unfolds when they are integrated with traditional symbolic planners.
Core problem
The integration of Large Language Models (LLMs) into Automated Planning and Scheduling (APS) is hindered by the limitations of LLMs in generating precise, actionable plans and the lack of flexibility in traditional APS systems. While LLMs excel in natural language understanding, they struggle with optimal, complete, and correct planning, whereas APS systems are strong in structured planning but lack contextual adaptability.
Key findings and Contribution
- LLMs are applied in various categories in APS including Language Translation, Plan Generation, Model Construction, Multi-agent Planning, Interactive Planning, Heuristics Optimization, Tool Integration, and Brain-Inspired Planning.
- In Plan Generation, LLMs directly generate plans using causal language models through in-context learning, but struggle with optimality, completeness, and correctness.
- In Model Construction, LLMs build or refine world and domain models but face issues with semantic grounding and completing ill-defined PDDL domains.
- In Multi-agent Planning, LLMs enhance coordination and cooperation with frameworks proposed for scalable collaboration among LLM-based agents.
- In Interactive Planning, LLMs adapt to real-time user feedback, often using external verifiers for output refinement.
- In Heuristics Optimization, LLMs enhance heuristic searches and symbolic planners, providing guidance even with partially correct outputs.
- In Tool Integration, LLMs act as coordinators in tool ecosystems, augmenting reasoning by integrating tools like web search engines and API endpoints.
- In Brain-Inspired Planning, neurologically inspired LLM architectures aim to replicate human-like planning but face hallucination and computational intensity issues.
Limitations
- In Plan Generation, LLMs cannot match traditional combinatorial planners' optimality, often lack plan completeness, and fail to produce correct actionable plans.
- In Model Construction, LLMs face semantic grounding challenges and difficulties completing ill-defined PDDL domains.
- In Multi-agent Planning, scalability issues arise in multi-agent task planning.
- In Interactive Planning, there's a dependence on external verifiers for output refinement, adding to system complexity.
- In Tool Integration, LLMs often hallucinate non-existent tools or overuse a single tool and face scaling challenges when managing multiple tools.
- In Brain-Inspired Planning, hallucination issues persist and the methods are more computationally intensive than in-context learning alone.
- Generally, LLMs operating within polynomial runtime bounds cannot yet replace the nuanced complexities of symbolic planners.
Key quotes
In Plan Generation, LLMs directly generate plans using causal language models through in-context learning, but struggle with optimality, completeness, and correctness. Efforts to generate multimodal, text, and image-based goal-conditioned plans are exemplified by (Lu et al. 2023b), yet exhibit limitations in generalizing to out-of-distribution domains.
Type: Plan Generation Limitation
In Model Construction, LLMs build or refine world and domain models essential for accurate planning, yet face significant limitations in semantic grounding and completing ill-defined PDDL domains. Efforts like (Huang et al. 2023a; Brohan et al. 2023) delve into these challenges but highlight the gaps in LLMs' ability to fully understand and represent complex planning domains.
Type: Model Construction Challenge