Google’s former Chief Scientist, who helped Google become the AI and Search powerhouse that it is today, was recently interviewed by Diana Hu of Y Combinator. He explained that the model people use is increasingly not as important as how the model is used within a larger system of tools, retrieval, and AI agents.
His answers centered on context engineering and orchestrating tools, retrieval, and AI agents into capable AI systems.
Which AI Model Is Used Is Increasingly Less Important
Many people worry about which AI model they use and experience the anxiety of running out of tokens. Jeff Dean’s answers suggest those concerns may be leading people to overlook a bigger opportunity: context engineering.
The Y Combinator interviewer, Diana Hu, said that progress is no longer about bigger models and then says that it seems to her that it’s increasingly about “context engineering.”
Dean agreed with her and expanded on the idea.
Diana Hu asked:
“AI progress used to mean just better models. You had more data, train bigger models with bigger parameters.
But increasingly in the last years or so, it’s everything around the model, not just the model size and number of parameters or more data, it’s everything around things like retrieval, tools, memory, agent tools, and it might kind of get consolidated into what people call context engineering, right?”
Jeff Dean agreed, saying that the AI model that people choose to use is just one part of whatever it is that people are doing. What matters, he said, is the various tools that the AI model can use, how it can get access to relevant information. So, rather than make the model the focus and expecting it to do things, he insists that the better way to look at it is equipping the model with the tools that are necessary to get the job done.
Dean responded:
“Yeah, I mean, I think the model is really only one piece of what you’re trying to do, which is build an overall system that can solve really interesting problems.
And that involves a model that knows how to use various tools. It maybe knows how to retrieve relevant information, maybe has a history of other information that has retrieved for past problems. And it can put information into the context of the model.”
Orchestration Of Multi-Agent Systems Is Becoming Important
Dean continued his answer, shifting directions to agent and multi-agent orchestration, which means coordinating AI agents for how they use tools, retrieve relevant information to solve complex problems.
He used the example of an AI model, with all of its training data, which is an immense amount of information, and contrasted that against an AI that is looking at a collection of information that is directly relevant to what it needs to do. The point that he leads up to is that the model is better able to do a job when it has the right level of orchestration and that this is where things are headed toward.
He continued his answer:
“And the nice thing about that is that information is really clear to the model, unlike the training data the model is trained on where it’s all kind of like trillions of tokens stirred together into a soup of hundreds of billions or trillions of parameters.
But it’s all less clear than the actual context that the model sees directly for this particular problem or use case. And then I think being able to understand what tools are available, which ones are going to help the model solve this next phase of the problem, how to decompose the problem into a sequence of of tool calls, maybe trying multiple approaches to solve the problem and seeing which ones work and be able to evaluate that.
This is the whole orchestration of complex agent and multi-agent systems that I think is going to be more and more important and super exciting times I would say.”
Jeff Dean’s Tips For Better Context Engineering
Diana Hu picked up where Dean left off on the subject of context engineering and asked him for his tips on things that people can do to become better at context engineering.
Hu asked:
“And I think the fun thing about this particular problem domain set is actually something that everyone in this room can actually do because, before, to train a model, you needed incredible amount of resources, incredible amount of access to GPUs and data.
But for context engineering, everyone here could do it.
You just need the API to something like Gemini and then work on your own setup for your own retrieval, your own tool calls, and et cetera, et cetera.
So what are some tips for everyone here? How does everyone get better at and become exceptional at context engineering?”
Dean answered that failure is a part of the journey of understanding what changes need to be made in order to get to the right outcomes in problem solving. The interesting point to his answer is that he used the example of adjusting the model to solve problems better (which is a huge undertaking) and contrasted doing that with creating better guidelines and skills.
Dean explained
“Yeah, I mean, I think a really good way to do it is to use these models and sort of harnesses and tools and so on to try to solve problems. And then sometimes you can actually see where the models are failing.
And often you can actually make the model work better and succeed at that kind of problem by not just adjusting the model parameters, which is hard to do from the outside, but from creating better guidelines for the model, writing skills for the model to know how to use different tools that would be incredibly useful for solving this particular class of problem.
And I think as you do that, you end up on this kind of improving, self-improving of the setup that you’re trying to use to solve things. And that’s a really good way to get better at understanding what additional information the model would want in order to become more capable.”
Takeaways
- AI models are becoming one component of a larger AI system.
- Context engineering is increasingly about orchestrating tools, retrieval, and AI agents.
- Better AI results often come from improving the system around the model rather than the model itself.
- Improving AI outcomes often means learning from mistakes in order to create better guidelines and better skills.