Hugging Face's logo
Yi Cui's picture
23 19 1

Yi Cui

onekq
DhanushTutu's profile picture Gargaz's profile picture julianomusico's profile picture
·
https://onekq.ai

AI & ML interests

Benchmark, Code Generation Model

Recent Activity

posted an update about 17 hours ago
Kimi K2 is a bit disappointing by my expectations. It is on a par with Codex mini. https://huggingface.co/spaces/onekq-ai/WebApp1K-models-leaderboard
replied to their post about 17 hours ago
Context rot is such a catchy phrase, but the problem has been identified 2+ years ago, called attention decay. https://huggingface.co/papers/2307.03172 I spotted the same problem in coding tasks, and documented in my book (https://www.amazon.com/dp/9999331130). Why did this problem become hot again? This is because many of us thought the problem has been solved by long context models, which is not true. Here we were misled by benchmarks. Most long-context benchmarks build around the QA scenario, i.e. "finding needle in haystack". But in agentic scenarios, the model needs to find EVERYTHING in the haystack, and just can't afford enough attention for this challenge.
replied to their post 4 days ago
Context rot is such a catchy phrase, but the problem has been identified 2+ years ago, called attention decay. https://huggingface.co/papers/2307.03172 I spotted the same problem in coding tasks, and documented in my book (https://www.amazon.com/dp/9999331130). Why did this problem become hot again? This is because many of us thought the problem has been solved by long context models, which is not true. Here we were misled by benchmarks. Most long-context benchmarks build around the QA scenario, i.e. "finding needle in haystack". But in agentic scenarios, the model needs to find EVERYTHING in the haystack, and just can't afford enough attention for this challenge.
View all activity

Organizations

MLX Community's profile picture ONEKQ AI's profile picture