agents ai llm reading/researchpaper
Core Idea
The paper proposes an AI agent that thinks internally using numerical representations (latent reasoning) and switches to text when it needs to communicate or use external tools.
What is the main idea ?
The paper proposes a new way for AI agents to think faster, use fewer tokens, and still interact with external tools like Web Search or Python.
Normally, AI models reason by generating words one after another, even for intermediate thinking.
This Research ask: What if AI could do most of its thinking internally, without converting every thought into words ?
And this paper proposed solution to this called HybridAgent. it combines two type os reasoning.
- Text Reasoning
- Latent Reasoning
How it Works
for example, Image asking ai:
“Find the company that acquired startup X and calculate how much the acquisition cost per employee.”
- Internal thinking (latent mode): The AI plans what information it needs without writing out every reasoning step.
- Search (text mode): It generates a search query and calls an external search tool.
- Internal thinking (latent mode): It processes the search results and decides how to calculate the answer.
- Tool use (text mode): It calls Python to perform the calculation.
- Final answer (text mode): It explains the result to the user.
The model can switch between these modes while solving the same problem.
<latent>is a special token that tells the AI to switch from text-based reasoning to internal latent reasoning.
<latent>: Signals the start of latent reasoning.- Latent mode: The model processes information using continuous numerical vectors instead of generating words.
- HybridLink: Converts the model’s hidden states into inputs for the next reasoning step.
</latent>: Signals the end of latent reasoning, allowing the model to return to text generation.- Dynamic switching: The model learns when to enter and exit latent mode through HybridSFT and HybridRL.
Important Concept
- HybridLink: A small neural network that connects internal reasoning with the model’s normal input system. Like an adapter between two formats.
Post Training
-
HybridSFT: A training stage that teaches the AI how to switch between text and latent thinking reliably.
-
HybridRL: Reinforcement learning that rewards useful behavior, helping the AI learn when to think internally and when to use tools.
Tool Usage
HybridAgent performs internal reasoning in latent mode, but switches to text mode whenever it needs to call external tools like Python or Search.
Because tools cannot directly understand latent vectors, so tool calls are generated as text.
Working Memory
Summary
HybridAgent treats the Transformer’s KV cache as a continuous memory that preserves both text and latent reasoning, allowing the agent to pause, use tools, and resume thinking without losing its internal context.
In a normal LLM agent, when a tool returns a result, the model continues reasoning using the previous text conversation.
Latent vectors: Internal reasoning represented as numbers instead of words.
Problem: Latent vectors cannot be reconstructed simply by tokenizing text, so a normal text-only conversation history cannot preserve the full reasoning process.
HybridAgent uses a continuous working memory to remember everything it has processed, including text, internal latent thoughts, and tool results, without converting everything back into text.
It uses the Transformer’s existing KV cache (Key-Value cache) as its working memory.
2. Tokens and Embeddings
Attention is All You Need (Transformers) > Architecture
Harness Engineering