Notes on Incorporating AI Agent Tools and Large Language Models within the ChemConnect Framework
This set of blog posts outline the vision for extending ChemConnect by integrating advanced AI technologies. The primary focus is on employing AI agent tools and large language models to enhance ChemConnect’s capabilities and support its continued development.
A significant emphasis will be placed on the integration of AI agent tools and large language models. These technologies are intended to provide guidance and support within the ChemConnect platform, driving innovation and efficiency throughout its processes.
Expanding the Framework with External Tools
The broader ambition of this initiative is to incorporate tools from various external groups into the ChemConnect framework. By doing so, the ultimate objective is to automate the entire process of kinetic mechanism generation. Each individual tool will be mapped to a corresponding agent, while other agents will coordinate the use and interaction of these tools. This orchestration is aimed at advancing both the generation and optimization of kinetic mechanisms.
Initial Experiments and Steps
The early stages of this effort will focus on documenting the steps and conducting experiments with agents, specifically in their interactions with ChemConnect. These foundational experiments are crucial for establishing a complete and functional system. The outcomes of these steps will serve as guidelines for integrating additional tools from external sources into the framework.
Development of Universal Templates
A key goal of these initial steps is to create a set of templates that can be utilized universally. These templates will facilitate the seamless incorporation of new tools and agents into the ChemConnect system, ensuring consistency and efficiency.
Integration of State-of-the-Art Technologies
A major component of this work involves exploring how state-of-the-art technologies—including AI, agents, noSQL databases, graph databases, and large language models—can be combined within a unified and integrated system. The discovery and implementation of effective methods for linking these technologies will be central to the project’s success.
Initial Tools and experiments
The follow blog posts are the ‘experiments/prototypes for setting up the total application. Each represents an essential aspect of the development. The purpose is to set up small applications that illustrate the implementation of the principle.
ADK Tools
- Connecting Angular to a backend with an Agent: This is a smile example where the client is an Angular chat window and the agent is in a JAVA backend service. The purpose of this prototype is to show a simple communication between backend agent and an client interface (in Angular)
- Structured Output from Google ADK agents: One of the essential points of communicating and steering an Angular client through a ‘Human in the Loop’ (HITL) interface is having the agent deliver a structured output. In the case of this project, this means a JSON object.
- Human-in-the-Loop (HITL) Agentic Workflows: An agent “pauses” its reasoning to allow a human to provide structured parameters via a GUI
- Simple RAG implementation: To identify which task should be run a translation from a natural language description of the task (transaction) and a standard task name. A simple RAG within the prompt is set up in this prototype.
- (in progress) Simple Function Call to external service: Here, a single agent triggers a call, with prompt data, to a backend function. This adds the communication of the agent backend with another backend ‘doing the work’.
Transition to VertexAI
To control the Human in the Loop interface, a transition was made to Agents in the VertexAI context.
The general flow will be that the client does the actual accessing of the database and the agents will only be used to control the workflow. The main reason for this is that the JSON objects that are returned from accessing the ontology or the database can be quite large. The tools would return these large JSON objects to the agents and, unfortunately, the agents would try to parse them. This can be very expensive and slow. I did not find a way to inhibit this behavior. In addition, the slightest error in parsing could result in errors. Using agents in this manner is going against their ‘probabilistic’ nature for a ‘deterministic’ task.
In this paradigm, an agent’s task is to guide to determine which subtask to run or, for example, determine whether the user should be asked for assistance. In any case the output of the agents are keywords. These keywords are interpreted programmatically (i.e. JAVA code in the backend) to determine what happens next.
The history, i.e. what tasks have been completed, is assembled outside the agents.
- Simple Agent with VertexAI: This is a single sample agent that represents reading a history and determining the next step. Proof of concept and working out details of implementation. This is a proof of concept and the structure will be used in more complicated implementations.
- Simple Vertexai RAG implementation: This is a simple implementation illustrating the use of RAG to identify terms and assign values within a text. This is a proof of concept and the structure will be used in more complicated implementations
English