📚 Weekly Papers
|
Archive
2025-11-23
2025-11-17 ~ 2025-11-23
DeepSeek-OCR: Contexts Optical Compression
Authors: Haoran Wei, Haoran Wei.
Affiliation: DeepSeek AI.
将长上下文压缩为二维“光学映射”,以轻激活编码器与小型解码器实现高压缩比OCR:<10×压缩达97%精度,20×仍约60%;在OmniDocBench上以显著更少视觉token超越主流系统,并具大规模数据生成能力。
Efficient Long-context Language Model Training by Core Attention Disaggregation
Authors: Yonghao Zhuang, Eric Xing.
Affiliation: Carnegie Mellon University.
提出CAD将核心注意力计算拆分并分派到专用设备池,通过“任务化”重排与ping-pong流水并行,降低长上下文训练中的负载不均与显存压力;在512K上下文与大规模GPU上实现最高1.35×端到端加速并消除DP/PP瓶颈。
A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
Authors: Zhi Zhou, Yu-Feng Li.
Affiliation: State Key Laboratory of Novel Software Technology, Nanjing University, China.
建立测试时扩展的统一理论框架,分解推理误差为估计误差与模型误差;证明自一致仅线性收敛而困于采样预算,perplexity具指数收敛但模型误差大,并提出RPC将二者优点结合,性能与采样成本取得更优权衡。
LightMem: Lightweight and Efficient Memory-Augmented Generation
Authors: Jizhan Fang, Ningyu Zhang.
Affiliation: Zhejiang University.
受人类记忆模型启发,构建轻量三阶段记忆系统:感知压缩→主题化短期整合→离线长期巩固;在LongMemEval上较强基线提升准确率并将token、API调用与时延分别降至百倍级,兼顾效果与成本。
Continual Learning via Sparse Memory Finetuning
Authors: Jessy Lin, Jessy Lin.
Affiliation: FAIR at Meta, University of California, Berkeley.
基于memory layer提出稀疏记忆微调,仅更新最常用的少量记忆槽以减少干扰;在事实与文档流式学习中,实现与全量微调相当的学习效果,却显著降低遗忘(如NQ仅11%降幅),优于LoRA与全量微调。
Real Deep Research for AI, Robotics and Beyond
Authors: Xueyan Zou, Xiaolong Wang.
Affiliation: University of California, San Diego.
提出通用研究情报管线RDR:系统抓取与解析海量论文,识别新兴趋势与跨域机会,并提供可执行的起点;在AI与机器人领域做端到端示范,主文阐述框架,附录给出大规模主题化结果与可复现资源。