#transformers

Training a Mixture-of-Experts Router

I was curious about MoEs, so I implemented and tested one.