Phil Wang

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Adan - (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

Amplify

Axial Attention

Easy way to keep track of exponential moving average version of your pytorch module

E(n)-Equivariant Transformer

Log Depth Recurrent Modeling

MuJoCo Warp fork with an MLX (Metal) rigid-body engine and Nim CPU fast paths

Ponder Transformer - Pytorch

Protein Generation (ProGen)

Recurrent Transformer

Relay Transformer, a long-range transformer

Routing Transformer (Pytorch)

RQ Transformer - Autoregressive Transformer for Residual Quantized Codes

Implementation of Self-Play Pretraining with Zero Data

Transformer in Transformer - Pytorch

Vector Quantization - Pytorch

Vector Neuron Transformer (VN-Transformer)