Feature request: Support for dynamic user-defined kernel compilation

[spichardo]

I made a comment on this on the Discord channel a few weeks ago, but I think it won't hurt to make an entry in Github.

For those exploring MLX for non-ML related scientific GPU-based calculations, having the possibility of compile their own GPU kernels is a critical feature to develop highly customized GPU computations.

Thanks to the comments in the Discord channel, there was a mention to the existing extensions feature in MLX. However, these extensions are a bit too static to implement and not as convenient as in other libraries such as cupy, pyopencl, pycuda or py-metal-compute, where you can compile GPU kernels on the go. The latter library covers the basics for dynamic kernel compilation for Metal, but the attractiveness of MLX is the ever growing functionality for many mathematical operations, and of course the possibility to integrate with ML-algorithms.

Having the capabilities of MLX for many array operations (using a numpy-based syntax) and dynamic user-defined kernels would be definitively super useful