geekjock
Joined 15 years ago
Karma
216
Submissions
89
Member since
Mar 18
Recent submissions
Comment · 2 years ago
> One of the main challenges when dealing with technical debt has been the lack of a way to measure it. To help overcome that problem, CISQ/OMG led the development of an Automated Technical Debt (ATD) measurement standard, which is currently being updated with a new version expected in 2023.
I'm highly skeptical about the tech debt measurement algorithm this article purports to be developing.
Google researchers recently published a paper on their attempts to measure technical debt.
They tested 117 metrics that were proposed as potential indicators.
Regressions were used to test each metric to see whether it could predict an engineer’s perceptions of technical debt.
No single metric or combination of metrics were found to be valid indicators.
Comment · 2 years ago
"Lines of code is a much better proxy than reported productivity."
Researchers who specialize in this field largely disagree with this. Here's a round-up: https://getdx.com/blog/measuring-developer-activity/
Comment · 2 years ago
See this previous paper: https://queue.acm.org/detail.cfm?id=3595878
Comment · 2 years ago
"I'm a bit pessimistic that much of this research is being driven by large orgs in collaboration with researchers who aren't sufficiently independent."
I'm one of the co-authors of the study. I think your sentiment is valid but what you describe is true for most fields of research: conflicts of interest can be a problem.
I can attest to the fact that the researchers behind this study have extensive backgrounds in academic research and hold themselves to high standards. If nothing else, not doing so risks putting individual reputations on the line.
Comment · 2 years ago
Makes sense - I agree with you. My mistake on assuming you were referring to metrics like the ones I listed.
Comment · 2 years ago
> Developer experience encompasses how developers feel about, think about, and value their work.9 In prior research, we identified more than 25 sociotechnical factors that affect DevEx. For example, interruptions, unrealistic deadlines, and friction in development tools negatively affect DevEx, while having clear tasks, well-organized code, and pain-free releases improve it.
Comment · 2 years ago
Agreed with this. We're working on it!
Comment · 2 years ago
"Are there any plans to figure out objective ways to measure productivity"
You can't measure developer productivity objectively, assuming you're referring to metrics like lines of code, number of pull requests, or velocity points which are infamous. There's broad agreement on this both within the research community as well as practitioners at leading tech companies.
Here are some examples: https://newsletter.pragmaticengineer.com/p/measuring-develop...
"... and what distinguishes 'good devex' from 'bad devex'"
Yes to this. This is an ongoing effort - we have two previous journal papers that touch on this which may be of interest to you:
- https://getdx.com/research/devex-what-actually-drives-produc...
- https://getdx.com/research/conceptual-framework-for-develope...
Comment · 2 years ago
Ever seen developers that are frustrated by technical debt and whose business leaders don't seem to care? This study is aimed to help with that.
Comment · 2 years ago
"You could look at literally any objective measure to proxy actual productivity and be better off than this"
It's fairly well-established in research (and in practice) that there is no objective measure of developer productivity. Metrics like lines of code, number of pull requests, velocity points are incredibly poor proxies.
Comment · 2 years ago
+1
If you read the introduction of the paper, you'll see that the aim of this paper is to give managers and developers concrete data to use to help get buy-in on investing in developer experience from business leaders.
Comment · 2 years ago
I'm one of the co-authors of the study. Your critique is valid though by research standards, for this type of study, our sample is sufficient. We are planning to replicate this study on a larger scale in the future, though!
Comment · 3 years ago
Accelerate - this book has become an excuse for managers to spend outrageous money implementing metrics like lead time and deployment frequency to measure teams, whereas the book actually advises something very different.
personal site: abinoda.com