Nope, if you’re working on large arrays of data you can get significant speed ups using well optimised BLAS functions that are vectorised (numpy) which beats out simply written c++ operating on each array element in turn. There’s also Numba which uses LLVM to jit compile a subset of python to get compiled performance, though I didnt go to that in this case.
You could link the BLAS libraries to c++ but its significantly more work than just importing numpy from python.
Numba is interesting… But a) it can already do multithreading so this change makes little difference, and b) it’s still not going to be as fast as C++ (obviously we don’t count the GPU backend).
Python is written in C too, what’s your point? I’ve seen this argument a few times and I find it bizarre that “easily able to incorporate highly optimised Fortran and C numerical routines” is somehow portrayed as a point against python.
Numpy is a defacto extension to the python standard that adds first class support for single type multi-dimensional arrays and functions for working on them. It is implemented in a mixture of python and c (about 60% python according to github) , interfaces with python’s c-api and links in specialist libraries for operations. You could write the same statement for parts of the python std-lib, is that also not python?
Its hard to not understate just how much simpler development is in numpy compared to c++, in this example here the new python version was less than 50 lines and was developed in an afternoon, the c++ version was closing in on 1000 lines over 6 files.
I was responding to your general statement that python is slow and so there is no point in making it faster, I agree that removing the GIL wont do much to improve the execution speed for programs making heavy use of numpy or things calling outside it.
That’s a bit suss too tbh. Did the C++ version use an existing library like Eigen too or did they implement everything from scratch?
It was written entirely from scratch which is kind of my point, a well writen python program can outperform a naive c implementation and is vastly simpler to create.
If you have the expertise and are willing to put in the effort you likely can squeze that extra bit of performance out by dropping to a lower level language, but for certain workloads you can get good performance out of python if you know what you are doing so calling it extremely slow and saying you have to move to another language if you care about performance is missleading.
Nope, if you’re working on large arrays of data you can get significant speed ups using well optimised BLAS functions that are vectorised (numpy) which beats out simply written c++ operating on each array element in turn. There’s also Numba which uses LLVM to jit compile a subset of python to get compiled performance, though I didnt go to that in this case.
You could link the BLAS libraries to c++ but its significantly more work than just importing numpy from python.
Numpy is written in C.
Numba is interesting… But a) it can already do multithreading so this change makes little difference, and b) it’s still not going to be as fast as C++ (obviously we don’t count the GPU backend).
So you get the best of both worlds then: the speed of C and the ease of use of Python.
Sure but that’s not relevant to the current discussion. The point is that removing the GIL doesn’t affect Numpy because Numpy is written in C.
Python is written in C too, what’s your point? I’ve seen this argument a few times and I find it bizarre that “easily able to incorporate highly optimised Fortran and C numerical routines” is somehow portrayed as a point against python.
Numpy is a defacto extension to the python standard that adds first class support for single type multi-dimensional arrays and functions for working on them. It is implemented in a mixture of python and c (about 60% python according to github) , interfaces with python’s c-api and links in specialist libraries for operations. You could write the same statement for parts of the python std-lib, is that also not python?
Its hard to not understate just how much simpler development is in numpy compared to c++, in this example here the new python version was less than 50 lines and was developed in an afternoon, the c++ version was closing in on 1000 lines over 6 files.
The point is that eliminating the GIL mainly benefits pure Python code. Numpy is already multithreaded.
I think you may have forgotten what we’re talking about.
That’s a bit suss too tbh. Did the C++ version use an existing library like Eigen too or did they implement everything from scratch?
I was responding to your general statement that python is slow and so there is no point in making it faster, I agree that removing the GIL wont do much to improve the execution speed for programs making heavy use of numpy or things calling outside it.
It was written entirely from scratch which is kind of my point, a well writen python program can outperform a naive c implementation and is vastly simpler to create.
If you have the expertise and are willing to put in the effort you likely can squeze that extra bit of performance out by dropping to a lower level language, but for certain workloads you can get good performance out of python if you know what you are doing so calling it extremely slow and saying you have to move to another language if you care about performance is missleading.