Depth 0, 1, 2, or 3. The original paper ran all four, a reproduction ran three of them on open models, and the numbers point to a decision rule shorter than most people expect.
The short version: depth 1 is the default, and it is the default for a reason. In the original depth sweep, the step from depth 0 to depth 1 was the largest single gain on every aggregation task, and it came with flat or lower cost. Depth 2 and 3 paid off on one class of task, quadratic pairwise aggregation, and only with a frontier base model. On a weaker coder, every step past depth 0 lost accuracy. On simple retrieval, an open-model reproduction found that depth 2 cut accuracy by 30 points and multiplied latency by almost 100x. The rule that falls out: recurse to the depth your task's complexity class demands, and not one level further.
The RLM paper defines it precisely: "Max recursion depth 0 is an RLM without sub-calling capabilities. Max recursion depth 1 allows sub-calling LLMs, while max depth >1 allows sub-calling RLMs."
So depth 0 is the root model alone in a REPL with the prompt loaded as a variable. It can slice, grep, and loop over the input, but every piece of text it reads passes through its own context. Depth 1 lets the root dispatch plain language-model calls over snippets. Depth 2 lets those sub-calls be RLMs themselves, each with its own REPL and its own sub-calls. Each level multiplies the fan-out, because the paper notes that each root iteration "can launch arbitrarily many sub-calls."
The authors' October 2025 blog post only ran depth 1: "the root LM can only call LMs, not other RLMs." Their reasoning was that "for most modern 'long context' benchmarks, a recursive depth of 1 was sufficient." The arXiv paper, revised in May 2026, then swept depth 0 through 3 on four benchmarks. That table is the best evidence anyone has on this question.
Root model GPT-5, sub-calls to GPT-5-mini, accuracy with mean cost per query in parentheses:
| Depth | CodeQA | BrowseComp-Plus 1K | OOLONG | OOLONG-Pairs |
|---|---|---|---|---|
| 0 | 58.0 ($0.18) | 88.0 ($0.44) | 36.0 ($0.37) | 43.9 ($0.69) |
| 1 | 62.0 ($0.11) | 91.3 ($0.99) | 56.0 ($0.43) | 58.0 ($0.33) |
| 2 | 66.0 ($0.15) | 92.0 ($0.55) | 56.5 ($1.10) | 65.5 ($0.33) |
| 3 | 58.0 ($0.15) | 92.0 ($0.51) | 58.0 ($0.51) | 76.0 ($0.39) |
Read it by column and three patterns appear.
Depth 0 to 1 is the big step. OOLONG gained 20 points, OOLONG-Pairs gained 14, CodeQA gained 4, BrowseComp-Plus gained 3. Cost fell on CodeQA and fell by more than half on OOLONG-Pairs. That is what you expect when the root model stops trying to read every chunk itself and starts delegating.
Past depth 1, most tasks flatten. BrowseComp-Plus moved 0.7 points across two more levels. OOLONG moved 2 points. CodeQA peaked at depth 2 and then gave the whole gain back at depth 3, landing on the same 58% it had at depth 0.
One task kept climbing. OOLONG-Pairs went 43.9, 58.0, 65.5, 76.0. It requires listing pairs of entries that satisfy a property, so the work grows quadratically with input length, and each level of recursion partitions that work again. The paper singles it out: "On OOLONG-Pairs in particular, the higher recursive depth variants of the RLM for GPT-5 outperform all other methods including Claude Code and OpenCode by a large margin."
That is the whole positive case for depth 3. It is a real case, and it is narrow.
The same sweep with Qwen3-Coder-480B-A35B inverts. CodeQA went 66.0, 56.0, 54.0, 44.0 from depth 0 to 3. OOLONG went 43.5, 48.0, 26.0, 32.0. OOLONG-Pairs went 17.3, 23.1, 19.0, 21.1. The only benchmark that improved with depth was BrowseComp-Plus, from 46.0 to 68.7.
The paper's explanation is mechanical: "Qwen3-Coder-480B-A35B often makes syntax errors that result in failed outputs, and having sub-RLM calls propagates this issue to sub-calls." Depth is an amplifier. If the base model writes correct code, depth amplifies decomposition. If it writes broken code, depth amplifies failure, and a failed sub-RLM returns nothing useful to a parent that then aggregates nothing. Our post on RLM limitations and failure modes covers why coding competence is the gating filter for the whole approach.
A March 2026 reproduction, "Think, But Don't Overthink", ran DeepSeek v3.2 and Kimi K2 at depth 0, 1, and 2 on S-NIAH and OOLONG, with 20 samples per benchmark and OOLONG contexts between 1,024 and 65,536 tokens.
On S-NIAH, single-needle retrieval, DeepSeek v3.2 scored 100% as a plain call, 85% at depth 1, and 70% at depth 2. Execution time went from 3.6 seconds to 89.3 seconds to 344.5 seconds. Kimi K2 peaked at 545.5 seconds at depth 2. The author's diagnosis: "forcing the model into a programmatic REPL environment induces unnecessary cognitive load, causing the model to 'over-engineer' a solution for a simple string-matching problem."
On OOLONG, DeepSeek v3.2 went from 0% as a plain call to 42.1% at depth 1, then back to 33.7% at depth 2. Kimi K2, which already scored 86.6% as a plain call, fell to 60% at depth 1 and 55% at depth 2.
Two caveats. The sample is small, and the study did not tune prompts per model, which the original paper flags as a problem in its own setup. But the direction matches the Qwen3-Coder result above, and it adds a second lesson: if the plain model already handles the task, recursion at any depth is a tax.
Look at the cost column again. With GPT-5, depth 1 was the cheapest setting on CodeQA and OOLONG-Pairs, and depth 3 was cheaper than depth 1 on BrowseComp-Plus. Mean cost is not monotonic in depth.
What depth changes is the spread. The paper reports standard deviations alongside the means, and OOLONG at depth 2 came in at $1.10 with a standard deviation of $3.25. Depth 1 on the same task was $0.43 with a standard deviation of $0.85. Deeper recursion does not make the typical query expensive. It makes the worst query much worse, and a workload priced on the mean is priced on those outliers.
Latency is where the tax is unambiguous. Every level of depth adds a round of sub-calls that must complete before the parent can proceed, and the reproduction's 3.6 to 344.5 second range is what that looks like without concurrent dispatch. Our guide to evaluating an RLM argues for reporting p50 and p99 at each depth for exactly this reason.
Theory says deeper is always possible, which is a different claim. Yang, Srebro, and Li show that any computable problem admits a recursive decomposition in which each subtask needs exponentially smaller active context than a standard autoregressive model. The lineage runs back to Recursion of Thought in 2023, which used special tokens to split problems across contexts, and forward to Context-Folding, which branches into a sub-trajectory and folds it back into a summary to keep the active context about 10x smaller.
None of that tells you how deep a given task needs. The empirical tables do, and they say most benchmarks are exhausted at depth 1.
The implementation notes on our techniques page cover where the depth cap lives in the harness and how sub-RLM calls are spawned.
Depth 1 is the answer for most tasks, and the evidence for it is strong: the largest gains in the sweep, flat or lower mean cost, and the authors' own default. Depth 2 is justified on quadratic aggregation with a frontier coder and on CodeQA-style repository questions. Depth 3 has one documented win, OOLONG-Pairs at 76% F1, and one documented loss, CodeQA back to its depth-0 score.
Anything deeper than your task's complexity class is not more thinking. It is more fan-out, more latency, and a longer cost tail, paid for a result you already had.
Depth 1. The root model runs in a REPL and can call plain language models on snippets of the prompt, but those sub-calls cannot recurse further. The authors used depth 1 for their main experiments and wrote that it was sufficient for most modern long-context benchmarks. In the depth sweep with GPT-5, depth 1 captured the largest single gain on every aggregation task at flat or lower cost than depth 0.
Yes, on quadratic aggregation with a strong base model. On OOLONG-Pairs, which requires listing pairs of entries that satisfy a property, GPT-5 scored 58% F1 at depth 1, 65.5% at depth 2, and 76% at depth 3. On CodeQA the peak was depth 2 at 66%, and depth 3 fell back to 58%. On BrowseComp-Plus and OOLONG the gains past depth 1 were within a few points.
Yes, in two documented cases. With Qwen3-Coder-480B-A35B as the base, CodeQA accuracy fell from 66% at depth 0 to 44% at depth 3 because syntax errors in sub-RLM calls propagate to the parent. In an open-model reproduction, DeepSeek v3.2 on single-needle retrieval fell from 100% with no recursion to 70% at depth 2, and execution time rose from 3.6 seconds to 344.5 seconds.
Mean cost was not monotonic in depth. With GPT-5, depth 1 was the cheapest setting on CodeQA ($0.11 per query) and OOLONG-Pairs ($0.33). What depth changes is the variance: OOLONG at depth 2 averaged $1.10 with a standard deviation of $3.25. Latency is where depth hurts most. The reproduction measured 3.6 seconds without recursion, 89.3 seconds at depth 1, and 344.5 seconds at depth 2 on the same task.
Classify the task first. Constant-complexity retrieval gets depth 0 or no RLM at all. Linear labeling and multi-hop question answering get depth 1. Quadratic pairwise aggregation gets depth 2, and depth 3 only if the base model writes correct code and a measured run shows a gain. Set the maximum depth as a hard cap per task class, and check p99 latency at each depth before committing.