Analyzing constrained LLM through PDFA-learning
Resumen:
We define a congruence that copes with null next-symbol probabilities that arise when the output of a language model is constrained by some means during text generation. We develop an algorithm for efficiently learning the quotient with respect to this congruence and evaluate it on case studies for analyzing statistical properties of LLM.
| 2024 | |
| Agencia Nacional de Investigación e Innovación | |
|
Inteligencia artificial generativa Grandes modelos de lenguaje Ciencias Naturales y Exactas Ciencias de la Computación e Información Ciencias de la Computación |
|
| Inglés | |
| Agencia Nacional de Investigación e Innovación | |
| REDI | |
| https://hdl.handle.net/20.500.12381/3624 | |
| Acceso abierto | |
| Reconocimiento 4.0 Internacional. (CC BY) |
| Sumario: | We define a congruence that copes with null next-symbol probabilities that arise when the output of a language model is constrained by some means during text generation. We develop an algorithm for efficiently learning the quotient with respect to this congruence and evaluate it on case studies for analyzing statistical properties of LLM. |
|---|
