> I think you are vastly overestimating the willingness and capability of volunteers and underestimating the effort needed to coordinate such an effort.
> And having any kind of manual intervention required will almost certainly reduce the reliability of the testing.
Perhaps I am underestimating the necessary effort, but the willingness and capability problem can in my opinion be solved by sufficiently streamlining and documenting the processes of running the test procedure.
If the testing procedure cannot be successfully run by a "somewhat experienced Linux nerd", this should be considered a usability bug of the testing procedure (and thus be fixed).
> And having any kind of manual intervention required will almost certainly reduce the reliability of the testing.
Perhaps I am underestimating the necessary effort, but the willingness and capability problem can in my opinion be solved by sufficiently streamlining and documenting the processes of running the test procedure.
If the testing procedure cannot be successfully run by a "somewhat experienced Linux nerd", this should be considered a usability bug of the testing procedure (and thus be fixed).