I spent most of a day last week arguing with a prompt.
The agent had five tools. It used none of them. It answered from memory instead,
politely and confidently, like a student who had not done the reading and was
hoping nobody would ask.
The prompt was fine. The tools were fine. The bug was three layers down in how I
was building the request, and by the time I found it I had rewritten the prompt
twice and the tool descriptions once.
So here is the whole thing, because it cost me a day and it should cost you about
four minutes.
The shape of it
Google's Agent Development Kit lets you bring your own model class. You subclass
BaseLlm, ADK hands you an LlmRequest, and you turn that into whatever your
endpoint wants. I do this because every model call in my system goes through one
retry path, and I did not want the chattiest caller in the building opening its
own connection with its own opinions about rate limits.
On that LlmRequest is a config object. It holds 35 fields in one flat list.
Temperature is in there. So is maxOutputTokens. So are your tools, your system
instruction and your safety settings.
Those 35 fields do not all go to the same place.
Six of them belong at the top level of a Vertex generateContent request. Three
mean something to the client library and nothing at all to the server. The other
26 go inside generationConfig.
Nothing on the object says which is which.
What I did, which is the obvious thing
I took the config, dumped it, and put the whole lot in generationConfig. One
line. It looks right. It parses. It runs.
That call succeeds. You get a 200 back, and a response with normal looking text
in it. Your tools are sitting inside generationConfig where the server is not
looking, so the model was never offered them, so it answered the question it was
asked using what it already knew.
Think of it as a form with 35 boxes, where six of them have to be torn off and
posted to a different address. The form does not say which six. Fill it in wrong
and it is still accepted, cheerfully, and you find out much later that the
important half never arrived.
Why it took a day
The failure has no edges.
No warning. No unknown field error. No empty candidate. No exception anywhere.
The model behaved correctly for the entire day. It was asked a question with no
tools attached, so it answered the question.
And a missing tool call looks exactly like a model deciding not to call a tool,
which models do all the time. That is the trap. Every other time you see that
symptom it really is the prompt, so you go and fight the prompt, and the prompt
fights back by being innocent.
The fix
A list, written out by name, rather than a guess:
TOP_LEVEL = {"tools", "toolConfig", "systemInstruction", "safetySettings",
"cachedContent", "labels"}
Six fields. That is the complete set on google-genai 2.x. Everything else that is
not client-only is a sampling setting and goes in generationConfig.
There is a second half to this, and it is the friendlier one. Some fields on that
config are for the library and never for the wire: httpOptions,
automaticFunctionCalling, shouldReturnHttpResponse. Forward those and Vertex
rejects the request outright for having unknown fields. That one you find in
about a minute, because it shouts.
Same underlying gap, pointing the other way. Part of this object is for the
client and part is for the server, and there is no mark on it anywhere saying
where the line runs.
A thing I got wrong while writing this up
My own notes on this had two field names in that client-only list that do not
exist. One is not on the type at all. The other is real but lives one level down,
inside HttpOptions.
Neither could have changed anything, because a field that never appears can never
be forwarded, which is exactly why they sat in my code for weeks looking correct.
I only caught them because I went to check the list against the type before
filing the issue, which took two minutes and should have been the first thing I
did rather than the last.
Lists written from watching things fail record what you saw. They do not record
what is there. Worth knowing the difference before you publish one.
Where it is now
Filed as google/adk-python issue #6880, with a reproduction that runs offline. No
project, no credentials, no network. It just prints the config keys and shows the
tools landing in the wrong half.
https://github.com/google/adk-python/issues/6880
If you are writing a custom BaseLlm, copy the six names above and move on with
your day. If you are not, you will never meet this, which is probably why it has
sat there.
I wrote this piece for the purposes of entering the All Things Agentic Hackathon.
It came out of building MIGRAGENT, an agent that reads official immigration and
licensing sources every day and turns them into a guide where every line carries
the sentence it came from and the date it was read.
Top comments (0)