feat: S.M.A.R.T (Specific Multi Agent Recipe Tracking) Prompt POC (Proof of Concept)

[ido777]

Background

Enhancing LLMs with Multi-Step Cognitive Missions with S.M.A.R.T (Specific Multi-Agent Recipe Tracking) Prompt:
Seeking Collaboration on a Proof of Concept

---

Overview

In exploring the application of Large Language Models (LLMs) across a spectrum of cognitive tasks—from straightforward Q&A to fully autonomous agents—we observe a significant performance variance. While LLMs excel in simple dialogues, their efficacy diminishes as tasks approach autonomous execution.

Current Challenges and Proposed Solutions

• Complex Prompts: Constructing intricate prompts often results in compliance difficulties; overly complex prompts overwhelm the LLM, while overly simplistic ones fail to achieve desired outcomes.
• Use of Plugins/Tools: State-of-the-art (SOTA) LLMs struggle with performance when integrating multiple plugins or tools.
• Tailored Applications and Model Fine-Tuning: These approaches are prohibitively expensive, both financially and in terms of required expertise.

Introducing a Novel Approach

I have developed a proof of concept (POC) that sits between these existing strategies, aiming to validate its viability in a production environment. This solution is designed not merely as a user co-pilot but as a mission co-pilot, guiding the LLM through a multi-step chat where each phase has a distinct prompt, goal, and completion criteria. This structured approach allows for complex interactions to be broken down into manageable, sequential steps, thereby enhancing the LLM's performance even with lower-quality models.

Concept Details:

• Functionality: The current implementation is a static function without LLM integration, serving as a multi-step orchestrator for chat interactions.
• Extensibility: The framework is designed to be scalable, supporting an expansion into more comprehensive tools or assistants.
• Controlled Complexity: By sequentially presenting detailed prompts, we maintain control over the dialogue's complexity, improving task adherence and progression.

MoodBooster Example

In the MoodBooster scenario, each stage has defined outcomes—some rely on user approval, others on self-assessment. Preliminary tests with GPT-3.5 and GPT-4 show promising results, with models autonomously advancing through the recipe's stages and adhering closely to provided instructions.

Challenges

Managing a 3-Way Chat (LLM, Recipe, User)

The technique introduced necessitates handling a 3-way chat, involving the LLM, the recipe, and the user—a scenario for which it was not originally optimized. Typically, a "system prompt" might guide the recipe, but this scenario proves more intricate. To navigate this complexity, the approach divides instructions into two distinct sets and two levels, enhancing the dynamics of interaction within this multi-end chat environment. This structured method includes:

• General Flow Instructions vs. Stage-Specific Instructions: This helps agents grasp their roles within both the overall conversation flow and specific stages, facilitating smoother transitions and more relevant interactions.
• User Interaction Instructions vs. Self-Thoughts Instructions: These guidelines direct how agents engage with users and manage their internal decision-making processes, ensuring each interaction is meaningful and contextually appropriate.

Finding a General Way via a Prompt to Maintain and Follow the State

Effective stage transitions require a state machine-like behavior within the LLM. Each state necessitates that agents "load" a new personality, aligning their responses with the current context. Instructing the agent to greet the user as a 'NamedAgent' has significantly improved its understanding of the state, enhancing coherence and contextual awareness.

Future Directions

• General Multi-Step Planner: Converts complex prompts into a S.M.A.R.T (Specific Multi-Agent Recipe Tracking) structured plan.
• Targeted Recipe for Specific Goals: Develops tailored recipes for specific objectives, such as MoodBooster.
• Recipe Improvement Through Feedback: Enhances specific recipes using real-user chat data and feedback, supplemented by LLM evaluations.

Call for Collaboration From LibreChat contributers

Since this is only a proof of concept (POC), currently there is no way for users to upload their recipes to the code. While the Planner and the Improver tools can be incorporated into the code as tools, the specific recipes should probably be uploaded by the user via the UI.
@danny-avila @fuegovic @berry-13,
I would appreciate your feedback on this idea for implementation and improvement based on the LibreChat code when you have a moment. It would also be helpful if this could be added to the roadmap.

Call for Collaboration From other readers

This project is still in its early stages, having started just a week ago, with the recipe integration and tool descriptions needing further refinement to accommodate a broader range of scenarios. If this initiative resonates with you, especially researchers interested in co-authoring a paper, I welcome your expertise and collaboration to advance this concept towards real-world application.

Fun Fact

A key challenge was enhancing the Agent's ability to transition accurately between stages at the appropriate times. To address this, I introduced "self_thoughts_instructions" which direct the Agent to explain its role and goal to the user. For example: "As a 'GreetingAgent,' I am here to welcome you and learn a bit about you. ..." This addition has improved the Agent's grasp of "current stage" and "next stage" concepts. However, initially, it also led to an increase in "stage hallucinations," where the Agent would invent roles like "PakeStageAgent" and assign itself fictitious goals, albeit still related to the overall objective

Summary of the Code changes

Created a new tool, "agent-coordinator" and 2 recipes.
The motivation and some details on the implementation can be found in the background introduction above.

Change Type

• New feature (non-breaking change which adds functionality)

Testing

Mainly manual.

Checklist

• My code adheres to this project's style guidelines
• I have performed a self-review of my own code
• I have commented in any complex areas of my code
• [X ] My changes do not introduce new warnings