The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases. Many of the somewhat unique capabilities are load-bearing, so you want to use them on architectures that support them. Taken in whole, someone who cares about performance would be using different data structures and algorithms depending on the specific SIMD architecture and that is nearly impossible to abstract in a library. Too many important but complex details are idiosyncratic to the implementation.
Another way of looking at it is that our programming environments are not sufficiently powerful and expressive to create the necessary abstractions to make SIMD truly portable.
> The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases.
I would argue that the "trivial" cases (those relating to linear algebra in 3 dimensions) are also 95% of what people want SIMD for.
If the API can achieve cross-platform and performant vector arithmetic, dot product, and matrix multiplication in the normal ways, that already covers a lot of what people actually need.
Most use cases for SIMD are non-arithmetic in nature and don't assume tidy arrays of homogeneous types. I also use it for some computational geometry but that is the least interesting use case.
SIMD is widely used throughout data infrastructure e.g. parsing data, complex constraint processing, parallel manipulation of heterogeneous data types, compression, etc. I even have an I/O scheduler written in AVX-512 that is many times faster than the scalar equivalent. The ability of SIMD to do complex manipulation of ordinary data structures several times faster than scalar code is under-rated.
While linear algebra is the current thing, database engines have been using SIMD heavily for over a decade and arguably represent the frontier. It is for these use cases that SIMD is non-portable and data infrastructure isn't going away.
Another way of looking at it is that our programming environments are not sufficiently powerful and expressive to create the necessary abstractions to make SIMD truly portable.