Ultrafast BERT

[brunosan]

This paper just dropped:

uses 0.3% of its neurons during inference while performing on par with similar BERT models ... we provide high-level CPU code achieving 78x speedup"

The basic idea, if I understand correctly, is intuitive. They modify the loss to favor choosing a decreasing amount of nodes as learning progresses. So at first the model learns almost normally, but then learns to cluster semantics based on the input.

If this indeed the case, it stands to reason to be very useful for us, since geospatial mapping could naturally cluster those branches geographically, by sensor, time, ...

✅ Code license is MIT