ilayn
Joined 8 years ago
Karma
186
Submissions
101
Member since
Sep 6
Recent submissions
Comment · 1 month ago
Not really, all competitive compilers saturated to the same performance. Often Fortran and "restrict" sprinkled C compile to the same command set.
Comment · 1 month ago
Not to take away from your points; SciPy is now Fortran-free completely[0] (we are also requiring C++17 at most). NumPy never had it. BLAS is all C/Assembly in all optimized vendors. For LAPACK we are working on it [1].
Once there is enough pain, none of the talking points matter for any language. They don't and can't die but linger. I fear that time for C family might come in a decade which would be a shame given how magical Cpp compilers are, all that effort folks pouring in.
[0]: https://github.com/scipy/scipy/issues/18566 [1]: https://github.com/ilayn/semicolon-lapack
Comment · 1 month ago
ahem... https://github.com/ilayn/semicolon-lapack
It is mostly out of habit. These days there are no differences between languages. Very often they compile to the same machine code. Fortran array syntax is light years ahead of what C++ offers (yes including std::linalg and that's why numpy, julia, matlab picked it up) but lacks everything else as a language including often claimed portability.
Comment · 5 months ago
Ah, my bad. what I failed to emphasize is that many of the downstream issues are coming from the upstream restrictions so this is one of the major blocks that was causing some mayhem down the line. So indirectly we might have caused some heartburn for you, apologies in advance.
Comment · 5 months ago
Scipy maintainer here, the main issue with the wheels was the Fortran77 that was SciPy throwing wrenches into the mix. With C/C++ self compilation should be quite straightforward. We (all Scientific Python packages) really worked hard on that.
From version 1.19 of SciPy there will be no need for fortran compilers (because we translated everything to C https://github.com/scipy/scipy/issues/18566) and then all becomes much easier in all platforms due to the large availability of C compilers in all platforms. Together with the Stable API developments in CPython the wheel clash issues "hopefully" will decrease gradually.
Comment · 5 months ago
It does not have to be in Fortran. It's just historical reason. If you want a C version of LAPACK I have one for you https://github.com/ilayn/semicolon-lapack
Comment · 5 months ago
> Actually the column-major order of Fortran is more efficient for some linear algebra operations than the order of C, which has been inherited by many modern languages that do not care about high performance in scientific computations.
This is a plausible assumption to make but unfortunately it is not true at large. Especially when the traditional sizes are exceeded say n >= 2000 certain operations such as LU can be improved in terms of performance with C-major arrays. However the correct statement is you lose at some place you win at other. There are certainly linalg operations that F-major can give you more performance. However it is also true for C-major layout.
In your example matrix vector product or any BLAS2 or BLAS3 level operations you can also swap out the for loop order to convert things around (row*col buffer multiplication vs sum of weighted column sum interpretation). In particular matrix norm operations are the only exceptions (abs column sum, row abs sum etc.) that certain norms prefer certain orders. In fact if you go into the Goto method deep enough you'll see that internal order is a bit like Morton ordering to fit things into L1 Cache.
The reason why column-major is preferred is historical and requires more surgery to get it running with C-major ordering. Trust me I tried but it's too much work to gain not so much. Maybe someday when I retire I can attempt it. Hence I kept it column major in my retranslation of LAPACK https://github.com/ilayn/semicolon-lapack
Instead I implemented a "high"-performance AVX2 matrix transpose operation so that swapping the memory layout is trivial compared to the linalg cost.
Comment · 6 months ago
Read the sources carefully. Fortran quote is not his; he quoted it. Also remember that he was talking about pre-Fortran77 era. F77 tried to fix some of the criticisms though did not succeed fully. Here is a nice dig-in about the quote https://limited.systems/articles/dijkstra-fortran/
Another note to remember that John Backus, the team lead of the Fortran gang, was in the Algol committee. So these folks knew what they are talking about and spoke to each other periodically. Even John Backus said, Fortran is not the final interface that we should have.
It keeps spinning in the programming circles half-quoted versions of half-baked quotes from original sources. These pioneers, even when they disagreed, had pretty precise arguments and very rarely feeling the feelies.
Comment · 1 year ago
Wow what's next, scrolling title text? What a time to be alive...
Comment · 1 year ago
Similar to your mqtt converter you can make them ingest via Kafka or some other adapter in between. EMQX broker can directly write into TSDB but did not research into others whether they can or not.
Comment · 1 year ago
Not the OTEL "telemetry" but sensors sending measurements from the field "telemetry". IT tech is, as usual, stealing perfectly defined engineering words and making them something else.
TimescaleDB is perfect if you also have relational data that you need to join with field data to the point that there no pros of using anything else for this use case, say you have 100000+ sensors and you need to group them by the customer site relations while aggregating per day statistics.
Comment · 1 year ago
I'm working in the IIoT domain too. Your workflow is interesting towards the end. Any particular reason, why you don't write it to some db like Timescale or Influx at the end without any prometheus conversion?
Comment · 1 year ago
NumPy has no Fortran code, for quite a long time now. SciPy has and it is being rewritten. What you mention is the ufunc machinery underneath which is all C. NumPy also has SIMD support (albeit limited to certain functions). BLAS is also C/Assembly but only LAPACK is F77 (which is too much code to be rewritten).
This does not mean Fortran is bad (obligatory disclaimer for Fortran fans).
Comment · 1 year ago
(Laughs in control theory)