slizard
Joined 10 years ago
Joined 10 years ago
There is certainly a chance that 10-100x more flops will appear on the fah network, but as I said I just don't believe it will happen.
> nor why they aren’t scheduling compute time on Summit, right?
They are. The researchers who get to use the FAH network certainly do have access to traditional supercomputing resources too. Some types of problems require strong scaling (and reliable resources) which requires HPC iron.
> > Also AFAIK garbage flops also count.
> That’s always true in these peak performance measurements, both for distributed projects and for supercomputers too. Also true for CPU & GPU peak flops specs.
I think you're misunderstanding me: I literally meant that the data files returned by the FAH contributors may often be garbage due to data corruption (OC'd cards, no ECC, overheating hardware, poor storage etc.). Not sure to what extent is this still the case today.
> I just don’t see any reason why Folding@Home would or should refrain from distributed crowd computing just because supercomputers exist. They can both happen. We don’t need to try to come up with efficiency numbers or compare utility/flop in order to see that Folding@Home is producing some useful research results, right?
In principle you're right. There are some questions around it, though. Briefly: access is a privilege of few; oversight?; inefficient use of hardware (low Flops/W), just to name a few.
> Maybe it’s worth pointing out that compute time on TOP500 supercomputers is not free,
No, it is not, but access is granted by grant agencies with at least some transparency and oversight as well as scientific review; also, those machine are far more efficient (flops/w).
> and Folding@Home is not a job you can run for 1 day or 1 week and be done.
No, Folding@Home is not the "job"; a "job" at least in HPC sense is a molecular dynamics simulation (or many); a set of such jobs is what is typically required for a project to be completed result of which would end up in a publication. The computational work corresponding to such a project/paper, depending on the size of the machine, can in fact be run in 1 week, and on a large machine even in 1 day.
In that respect, the responsibility of whether to ask for and how to make good use of donations lies solely on the teams that receive the donation. Without oversight it would however be foolish of them to be overly critical on their own shortcomings as there is a great benefit to having these cheap FLOPS (and good PR) that F@H brings.
I was about to suggest that it would be great to set up a merit-based funding scheme somewhat akin to the governamental funding agencies, but one run independently by the "council of the people". I'm however uncertain how effective could such an organization be at awarding the funding in a responsible and effective manner.
And there is a tricky balance between over-promising and under-delivering vs being realistic and lacking enough "flashyness" to get funded/thrown money at.
[1] hhttps://www.cineca.it/en/news/exscalate4cov-hpc-platform-win...
(Also, AFAIK garbage flops also count. As far as I recall talkin to a researcher working with FAH a qhile ago, a large-ish fraction of results returned were not usable due to data/files being broken.)
[1] https://doi.org/10.1016/j.celrep.2018.03.10 [2] https://doi.org/10.1016/j.jsb.2018.04.005 [3] https://doi.org/10.1016/j.jconrel.2018.05.026
Nothing prevents running the fah client on nodes of a compute cluster -- in fact my colleagues did that (while running a local F@H server), though that was a number of years ago just because they wanted take advantage of the distributed computing facilities provided by the client-server setup and built-in algorithms.
> crowd sourcing has the potential to scale to many orders of magnitude larger than what can be done in a data center.
Potential it does have, but I am skeptical of the "many orders of magnitude" claim ever having a chance to materialize. I'd love to see a cost / benefit analysis on the effective amount of useful work contributed vs the cost of the same in a data center.
In fact, donating in other ways, e.g. giving free programming, HPC etc. education to more researchers can have a higher impact on the long run.
To that question, assuming by "anyone" here you are asking about other donate-your-cycles-distributed-computing-projects: I am not too familiar with how well-optimized the codes of different @home projects are.
Taking a few steps back, perhaps the efficiency of these codes is the lesser issue and to be honest, in some (many?) cases other forms of donation/contribution may further more scientific progress than simply crunching numbers on one's home PC.
Fair point. I think this is something you should bring up with the authors of Folding@Home. I do not work on that project.
My personal view on this is that there is always a cost/benefit balance that one has to strike which is often tricky especially given the considerably constrained resources as it is typically the case in academic computational/simulation tool development.
It is however, as you point out, a great responsibility of the researchers and developers of codes to make sure that choices made and action taken (or not) do not lead to disproportionate waste of resources donated by volunteers or awarded through a grant by research funding agencies.
I do not know the detailed reasons why F@H chose to not update CPU "core" since FahCore_a7 (AFAIK based one GROMACS code from 2014), but it is likely related to the aforementioned cost/benefit analysis done in their team. One of the motivations could have been (just hypothesizing) that the software engineering efforts estimated to be required to update FahCore for CPUs (which generate a very small fraction of the "points") would have taken away resources from the GPU FahCore.
Check these talks from the recent ISC EXACOMM workshop if you want to see why HPC machines and HPC computing are an entirely different league compared to traditional data center computing: https://www.youtube.com/watch?v=9PPGvqvWW8s&list=WL&index=9&... https://www.youtube.com/watch?v=q4LkF33YMJ4&list=WL&index=7