Providing domain-specific tools that replace a general-purpose shell significantly improves LLM agents' ability to fix Android build errors by bridging the gap between high-level reasoning and low-level execution.
- AndroidBuildBench contains 1,019 real-world Android build failures from commit histories, each with a verified fix.
- GradleFixer, an LLM agent with domain-specific tools, achieves an 81.4% pass@1 resolve rate, vastly outperforming the shell-based Gemini-CLI agent.
- The Tool Bridging strategy replaces general shell commands with API-like tools, improving reliability and constraining the action space.
- A smaller, cheaper LLM (Gemini-2.5-Flash) using GradleFixer outperforms a larger model using the shell-based agent.
- Domain-specific tools are more effective than prompt guidance for leveraging an LLM's latent knowledge of Android builds.
Motivation and Problem
Android is the largest mobile platform, yet building applications automatically remains a practical challenge—only 31.3% of apps built out-of-the-box succeed. GradleFixer introduces AndroidBuildBench, a benchmark of 1,019 build failures from 43 open-source projects, each paired with a verified fix from a subsequent commit. The failures include human-committed errors, augmented dependency errors, and LLM-generated errors.
The authors observe that LLMs possess high-level knowledge to fix build errors but struggle to translate it into correct low-level shell commands. This reasoning-execution gap motivates the Tool Bridging strategy, which replaces a general-purpose shell with domain-specific tools like `run_build`, `run_gradle`, and `change_java_version`.
GradleFixer achieves an 81.4% pass@1 resolve rate on AndroidBuildBench, significantly outperforming the shell-based Gemini-CLI agent (which achieves at most 72% in some categories). Ablations show that performance improves as tools become more specific; even with tool names and descriptions masked, the constrained tool structure outperforms the shell.
Experimental Setup and Tool Design
All experiments used Gemini-2.5-Pro on a standardized Linux machine. The authors compared five configurations: Coding-Assistant (no execution), Hierarchical Agent (two-agent delegation), Gemini-CLI with read/write only, Gemini-CLI with a general shell, and GradleFixer (domain-specific tools). GradleFixer's tools wrap shell commands to provide a constrained, API-like format.
The Tool Bridging strategy was validated by showing that the shell-based agent frequently attempts correct commands (e.g., building, changing Java version) but fails to sequence them correctly. Replacing the shell with specific tools like `run_build` improved pass@1 from 54.3% to 74.0% (with 30 LLM calls). Prompt guidance helped moderately (58.9%) but did not match dedicated tools.
Using Gemini-2.5-Flash (a smaller, cheaper model) with GradleFixer outperformed Gemini-2.5-Pro with the shell, showing that tool design can dominate model size. Failed repairs consumed nearly four times more tokens than successful ones, making a better method more cost-effective.
Results and Analysis
GradleFixer outperformed all baselines across every error category, with the largest gains on dependency issues (77.8% vs. 40.9% for the shell agent). Analysis of failure cases showed that the number of lines changed correlates with difficulty; both agents struggled more with large code changes (median 714.5 lines for GradleFixer failures).
Ablations on tool semantics revealed that masking tool descriptions caused a larger performance drop than masking names, but even with both masked, the API-like structure achieved 64.9% pass@1—still above the shell agent. This supports the hypothesis that Tool Bridging works both by providing reliable API-like calls and by constraining the action space.
The authors distinguish their work from prior domain-specific-tool studies (Voyager, Code Researcher) by directly comparing shell vs. domain tools and isolating Tool Bridging as the research question. They conclude that the strategy is generalizable and suggest future work on fine-tuning small models and automatically generating domain tools.
Read this at any depth.
Install Depth and pick your level — Glance for a sentence, Summary for the gist, Read for the full take. Free daily quota, no signup needed.
Add to Chrome