Tag: 算法
All the articles with the tag "算法".
QKFormer: Hierarchical Spiking Transformer using Q-K Attention
Published: at 18:09QKFormer，NIPS2024 Spotlight，把Direct Training SNN在ImageNet和CIFAR上的点刷的特别高，感觉之后要做就避不开它。
Transformers without Normalization
Published: at 16:09何恺明新作，用DyT代替Norm，把同步操作变成了Element Wise的操作。新文章里面有用到，学习一下。
Visualizing and Understanding the Effectiveness of BERT
Published: at 10:21最近做SNN训练的过程中在研究怎么可视化训练过程中的Loss，在想新加入的方法会不会对模型的Loss Landscape有影响，一般讲Loss Landscape怎么做可视化的文章都会引用这篇文章对Loss Landscape的分析和做法。
One-Minute Video Generation with Test-Time Training
Published: at 18:17最近Demo很火的TTT视频生成，可以生成60s级别的长视频。学习一下TTT的东西，SNN的On-Chip Learning和TTT能不能做结合？
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Published: at 16:40这两天在弄SNN训练的事情，需要验证一下用的Surrogate Gradient的准确性，老师介绍读一下这篇文章，用Evolution Strategy验证一下现在梯度估计的准确性。
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
Updated: at 14:57Published: at 14:39使用大kernel DS卷积替代self-attention。字节新加坡的工作。
Titans: Learning to Memorize at Test Time
Updated: at 14:57Published: at 18:36从TTT改进而来的新架构，尝试通过TTT的方式改进模型的记忆能力。
Segment Anything
Updated: at 14:57Published: at 13:48Meta的SAM。
ConvUNeXt:An efficient convolution neural network for medical image segmentation
Updated: at 14:57Published: at 15:59ConvNext + UNet，发在一个C刊上，借鉴学习一下，想想我的模块怎么设计。
Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks
Updated: at 14:57Published: at 15:23NIPS2024。主要研究的是静态任务中，推理前膜电位初始值设置对精度的影响。