正文 · AI 翻译
They also revealed quite a few secrets behind the training.
Tingyu, reporting from Aofei Temple
QbitAI | WeChat public account QbitAI
Tingyu, reporting from Aofei Temple
QbitAI | WeChat public account QbitAI
Smaller Hy models are already on the roadmap, and a model for 24GB of VRAM is also planned.
Early this morning, Shunyu Yao and Tencent's Hunyuan team came to Reddit for an AMA about the newly released Hy4 and Hy4 flash.
AMA stands for “Ask Me Anything”: people ask questions in the comments, and Hunyuan team members reply directly.
The participant list showed only Reddit accounts, so we still haven't worked out which username belongs to Shunyu Yao…
The focus of the AMA was the two models Hunyuan released in succession this month.
Hy4, released on November 4, is the flagship. It can understand text, images and video, and is suited to complex tasks such as long-horizon software engineering that require multistep reasoning and repeated tool calls.
Hy4 flash, released on November 16, focuses on speed and cost for everyday office work. It accepts text and image input, supports a context of up to 1 million tokens, and activates about 12 billion parameters per token.
Hy4 and Hy4 flash are both available in Tencent WorkBuddy. The Hunyuan team also included typical WorkBuddy tasks, such as editing spreadsheets, revising documents and making presentations, in its internal WorkIndex evaluation. By that measure, Hy4 flash's average cost per task is about 1/20 that of the flagship Hy4.
The weights of both models have been released. During the AMA, one user gave their computer specifications directly: 96GB of VRAM and 128GB of RAM. Could it run Flash?
The team said it could: keep the main weights on the GPU, while moving some expert layers and the Engram lookup-table module into system memory.
Someone also asked about llama.cpp, and the team said support is in progress.
When the discussion turned to development, the team also shared quite a few details about the training process.
Could Flash's active parameter count be reduced further? They explained the trade-off using their experiments with MoE sparsity.
What problems came up when training an agent to solve tasks on its own? Sometimes it would simply search online for a ready-made answer, skipping the problem-solving process.
Another user asked which behavior was hardest to improve during post-training. The team replied with just one word: “Overthinking.” The flagship Hy4's reasoning chains are sometimes too long. For Flash, the team first distilled the flagship model, then used reinforcement learning to shorten the reasoning chains.
From how the models are trained to whether developers can run them, the AMA covered many questions that usually get only one line on a model card but become unavoidable in actual use.
Below are excerpts from the AMA Q&A, arranged by topic. The Chinese translation has been abridged.
Enjoy~
Why does Hy4 “think too long,” and how was Flash trained?
User: Like Hy4 preview, the official Hy4 release sometimes reasons for too long. Is that related to reinforcement learning reward design or to the distribution of the training data? Can the model learn when to stop?
Hunyuan team: Hy4 aims to raise intelligence as much as possible, and we know its reasoning chains tend to be long. We can distill Hy4 into the more cost-efficient Hy4 flash and use reinforcement learning on Flash to shorten its reasoning chains.
User: During post-training, which model behavior is the hardest to improve?
Hunyuan team: Overthinking.
User: How much room is there to improve Hy4 at its current parameter scale? Will making stronger models in the future require adding more parameters?
Hunyuan team: We believe Hy4 still has considerable room for improvement, especially in post-training. We are investing in large-scale reinforcement learning and continuing to improve the user experience and model safety. We also plan to train larger models.
User: When training Hy4 and Hy4 flash on long-horizon agent tasks, what situations are hardest to handle in multiround reinforcement learning?
Hunyuan team: The trickiest issue is loopholes in the training environment that the model can exploit to get rewards. For example, we have to prevent the agent from searching online for ready-made answers and make it complete the task itself.
User: In command-line tasks, a single wrong argument from the model can cause a command to fail. How do you improve tool-call accuracy, and how do you help the model remember earlier context during long tasks?
Hunyuan team: We added difficult examples of mistakes the model is prone to making to the reinforcement learning data, and made the rewards more fine-grained, with heavier penalties for errors in critical details. For long tasks, we are continuing to lengthen the context used in reinforcement learning training and improve the model's ability to compress context while retaining key information.
User: Shunyu Yao often says selecting training data requires good judgment. How do you judge whether a batch of data is good?
Hunyuan team: Data quality has many dimensions, and different domains and tasks have different requirements. Whether we use human experts or agents, designing and cleaning data takes a lot of work. Ultimately, we have to run experiments to see whether the batch of data brings an improvement.
User: In the HyRA 2.0 experiments, where did the improvements brought by Hy4 mainly come from? Did it discover new improvements to the architecture or the data mix?
Hunyuan team: This improvement mainly came from optimizing underlying throughput. For the model to participate further in research and development, we need to improve its ability to assess training plans. Using AI for AI research and development is itself a long process with many challenges.
User: You've already used Hy4 and HyRA 2.0 to improve the training infrastructure and have trained better small models. What experiments will you run next to see whether the models trained in this round can do research and development better in the next round?
Hunyuan team: We're working on HyRA 3.0. Stay tuned.
User: If Hy4 were allowed to run on its own, could it now build a next-generation model through recursive self-improvement? How do you evaluate that ability?
Hunyuan team: Our recursive self-improvement capability is still at an early stage. We have developed several internal benchmarks to evaluate the key abilities involved.
Where do Flash's parameters go?
User: Excluding Engram, Hy4 flash's backbone has about 156 billion parameters, with about 12 billion activated per token. If total parameters stayed the same, how much would performance drop if active parameters were reduced to 8 billion or 5 billion?
Hunyuan team: We've run scaling experiments with different levels of MoE sparsity. With the active parameter count fixed, increasing sparsity can improve computational efficiency. But if total parameters were fixed and the active count were reduced to 8 billion or 5 billion, I would expect model quality to fall significantly. The active parameter count determines how much compute each token can use; greater sparsity can offset some of that.
User: Why does Hy4 flash allocate about 63.5 billion parameters to Engram? How was that proportion chosen?
Hunyuan team: Our experiments showed a U-shaped curve. Counting Engram in the total parameter count, an Engram share of about one-third was close to the optimum in our experiments. Lowering the share further reduces computational efficiency.
User: Why did Flash choose sliding window attention (SWA)?
Hunyuan team: Sliding window attention can reduce the KV cache to about one-quarter of its original size, significantly lowering deployment costs.
Small models, local deployment and what's next
User: Will there be a Hy4 mini? Some people also want a version lighter than Flash for document parsing, classification and simple automation.
Hunyuan team: Smaller Hy models are already on the roadmap.
User: Will the next generation of Hy models have 1.5 trillion to 2 trillion parameters?
Hunyuan team: The next generation of Hy models will definitely grow significantly in scale.
User: Are you considering a lighter Flash-Lite, for example, for document parsing, classification and simple automation?
Hunyuan team: Yes. Stay tuned for Flash-Lite.
User: Are there plans for a model that can run with 24GB of VRAM?
Hunyuan team: Yes, one will be released in the future.
User: I have 96GB of VRAM and 128GB of system memory. Can I run Hy4 flash?
Hunyuan team: Yes. Keep the main weights and attention computations on the GPU; some expert layers can be offloaded to system memory, and the Engram tables can also reside in memory.
User: On the same work task, hosted Hy4 flash got everything right in my tests, but a local NVFP4 version made errors about 8% of the time. Is this model harder to quantize?
Hunyuan team: Quantization quality depends heavily on the method and calibration. The team recommends trying the NVFP4 version quantized with GPTQ. In our evaluations, that version has a smaller gap from BF16 and is better suited to accuracy-sensitive tasks like this.
User: How will llama.cpp support move forward? Will you work with the inference framework community in advance?
Hunyuan team: We're adapting the model for llama.cpp and expect to release the support to the community soon. We're also working with the vLLM and SGLang communities, among others.
User: Will the existing Hy4 flash be updated?
Hunyuan team: Yes. We're optimizing the pre-training and post-training approaches for Flash-class models.
How to assess model performance
User: Why haven't you published scores for common evaluations such as GPQA Diamond, τ-bench, IFBench and SWE-bench?
Hunyuan team: We presented newer, more difficult evaluations such as Agents’ Last Exam and ProgramBench. The ones you mentioned appeared earlier, and scores on them are already close to saturation.
User: I feel that some models are getting worse at writing. Is that because they've been trained on too much code and overfit, or is there another reason?
Hunyuan team: We've noticed this too. It may be related to the training objectives. We want the model to write more like a person and use fewer stock phrases, and we're working on improving that.
One More Thing
That brings the Q&A transcript to a close.
Want to try Hy4 flash yourself? Open WorkBuddy, select Hy4 flash from the model list, and give it a spreadsheet you need to revise or meeting minutes you need to organize.
You can try it free for a limited time in WorkBuddy until November 30~
Reference link:
https://www.reddit.com/r/LocalLLaMA/comments/1y15w4a/tencent_hy_team_shunyu_yao_here_ask_us_anything/