A Runtime Workflow for Ensuring Integrity, Correctness and Privacy in LLM-Based Q&A Agentic Systems
Resumen:
Privacy and security are major barriers to scaling agentic artificial intelligence over sensitive data. This paper presents a runtime workflow for LLM-based agents that answer natural-language questions over sensitive relational data. The workflow constrains execution through explicit states and includes governance nodes that act as large language model judges: one node semantically blocks questions that may expose identifiable information before SQL is generated; another decides whether query results must be protected with order-revealing encryption before verification; and the final node applies differential privacy to numerical outputs before answering the user. The workflow is implemented in two variants with the same prompts and tools: a multi-agent implementation based on LangGraph and a single-agent implementation. The workflow is instantiated on an application that queries a database with hospital-like sensitive information, and evaluated on a set of questions covering blocking, encryption, tool use, and red-team scenarios. Results show that questions reaching the selection node were translated into valid SQL in this controlled setting, while the main differences appeared in privacy-governance nodes. The evaluation is exploratory: it shows how node-level behavior depends on model choice, architecture, and context management.
| 2026 | |
| Agencia Nacional de Investigación e Innovación | |
|
Runtime Monitoring Agentic AI Governance Ciencias Naturales y Exactas Ciencias de la Computación e Información |
|
| Inglés | |
| Agencia Nacional de Investigación e Innovación | |
| REDI | |
| https://hdl.handle.net/20.500.12381/5637 | |
| Acceso abierto | |
| Reconocimiento 4.0 Internacional. (CC BY) |