Having faster code is tempting, but it's not yet needed. We are not even half-way from 1.0.
Keeping additional asm code will only limit the amount of possible maintainers. Every routine is a subject to multiple revisions until 1.0, so try to not make the process more difficult.