About PLSQLBENCH
PLSQLBENCH evaluates whether models can write executable PL/SQL programs. It combines
schema-grounded database tasks and procedural programming problems, with correctness
measured by execution-based tests.
Why PLSQLBENCH?
Procedural database programming goes beyond standalone SQL, combining database logic
with control flow, cursors, exception handling, and reusable program units.
Executable artifacts: procedures, functions, packages, cursors, object types, and dynamic SQL.Schema grounding: enterprise-style Spider 2 and Spider schemas normalized to Oracle.Multi-turn workflows: extensions, modifications, debugging, and repair under accumulated context.
Evaluation Protocol
Generate: models receive a natural-language request, Oracle-dialect schema context, and requirements.Compile and run: generated artifacts execute in isolated benchmark environments.Score tests: executable tests compare outputs and behavior with a reference program or gold SQL result.
Submission Format
The public benchmark repository contains the released harness, evaluation instructions,
and reproduction workflow for direct generation and tool-augmented agents.
git clone https://github.com/oracle-samples/plsqlbench
cd plsqlbench
# See the repository README for the released harness and submission workflow.
Citation
@article{liu2026plsqlbench,
title = {PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming},
author = {Liu, Marianne Menglin and Boytsov, Leonid and Peterson, Daniel W. and others},
journal = {arXiv preprint arXiv:2608.15931},
year = {2026}
}