justin_vanw
Joined 17 years ago
Joined 17 years ago
If you ever ask Racket-lang folks about threads they will lecture you about concurrency vs parallelism endlessly, it's like a mantra they repeat to avoid realizing there is a problem.
https://en.wikipedia.org/wiki/Wheeler%E2%80%93Feynman_absorb...
The amount of friction you add to the development velocity has to be proportioned to the potential downsides. A system that shows doctors xray images in the ER cant go down or people might die. A site like twitter can go down and slightly improve the life of its users. It sounds like you are recommending big company super risk averse solutions at a startup, which is just silly. Besides, staging environments never work, for a long list of reasons. You should be recommending continuous integration and a focus on proper unit tests.
Have you ever seen $300MM worth of wood? I specifically said the owner had pumped a huge amount of money in the company.
What they don't realize is that if you put nontechnical people in charge of programmers, nothing will get done.
Burn the furniture pile (or move it into a warehouse or build a fence around it or something).
Hire a new COO who is the head salesman, and compensated purely by how much furniture they sell. Give them freedom to hire whoever they want so long as they are also being paid based on commission (and don't let them hire anybody already at the company).
As for the factory floor workers, optimize this later, sales is the focus so don't get distracted trying to optimize production until you forecast that it will fall behind sales. Get out and Sell! Get that furniture into stores.
Don't even fire the 300 people in the office, it's not worth the time you would put into it. Hire a consultant that specializes in downsizing to do it for you. Anyone who works in the office is considered contagious so no exceptions.
I really don't agree. The person in charge needs to have enough technical insight to have a good sense of the overall health of the team and whether reasonable progress is being made. They also need to have a sense of what is a 'good' product and what about the product needs to be improved.
Lets say the person in charge is not technical. The first thing that will happen is someone will get behind on what they are working on, and they will make some excuse (the database was reindexing or some nonsense), and the boss will believe them. From then on everyone knows they can slack off and get away with it, so productivity drops to almost 0.
Next the boss brings in technical managers to supervise the programmers, because he knows nothing is getting done but he doesn't know enough to call anybody out. This makes things much worse, since each manager is primarily motivated to be promoted, and the best way to be promoted is to demonstrate your ability to manage a large team. To get a larger team managers will deliver everything just a little too slow but hold out the carrot that if they had another person things could get done more quickly. In reality the managers are even less technically capable than the Boss, because they were interviewed and hired by the Boss. Boss's love throwing money at a problem, and team size grows and grows but managers can't actually push the team to get more done because they don't have any idea what is going on. The managers basically don't care about that anyway, since increasing productivity wouldn't help them and they can get favors in return by hiring someone's cousin.
Now there are hundreds of programmers and a dozen managers getting less done than the original 5 programmers. Lots of the programmers are extremely incompetent, hired due to nepotism or by mistake. They will never be fired because the manager would be admitting they made a mistake, or worse they might not get to hire a replacement if they fire this person, and this conflicts with their main goal of increasing their teams size to get a promotion.
Somebody has to plan work for all these programmers, and most of them are incompetent and don't care about the product. As a result, the quality is terrible. When confronted by the obvious bugs, they try to weasel out of it, and the boss is now very busy trying to keep customers and deal with emergencies (the database is down again!) to keep checking for all the obvious problems, so armies of QA people are hired. At some point a product manager is hired and once one is in the building they will hire 5 more within a year. Now you have 6 product managers having meetings with each other literally all day, and having not even a basic idea of what is going on they will start making plans. Since the programmers look busy when they are not seen or heard from so they just hide as much as possible and try not to attract attention, or put on headphones and stare at their monitors. The Product Managers look busy when they are having meetings, so they are always in every meeting, and pretty soon they insinuate themselves as the go betweens since they are always talking to the boss and the programmers are never around. Now the boss has even less idea what is going on since all the information getting to the boss is going through PMs who have literally no understanding of what is going on, and who get almost all of their information from other PMs, who get almost all their information from other PMs.
If this is a small company it is now basically in the same situation as Evernote is in now, and will go out of business soon, and if it's a large company the original boss has made such a mess and wasted so much money that the team he manages is large enough that the boss can get promoted to VP.
OR
The person in charge has enough technical understanding to keep his 10 programmers honest and make reasonable plans and be respected, and he keeps the team size at 10 super productive, happy programmers. If it's a small company he sells it to a big company for a huge pile of money, and if it's at a big company he gets passed over for promotion to VP because the job goes to some idiot that has managed to hire 500 incompetent programmers, 250 QA testers, and 200 PMs.
Lets say you were just hired as the President of a furniture company. The owner says he knows it's good furniture but even despite huge investments they can't seem to sell any furniture. Your job is to turn things around.
You start on the factory floor. The furniture is made by a combination of machines and human workers. Some people are employed to set up and configure the machines to make furniture parts. Around 150 people work on actually making furniture, either assembling it, doing quality tests, or setting up and operating the automated machinery. Things aren't perfect, but you aren't going to make any changes on your first day so you make some notes and move on. The furniture hasn't changed much over the years, it is still basically the same as it was when the furniture store opened. The furniture gets 'improved' from time to time, you see a step stool with an alarm clock, a small safe, and a webcam built into it, but when you ask the foreman he tells you nobody has ever turned on the alarm clock or used the safe or connected the webcam on any of the step stools. People seem to mainly use the stools so they can reach things that are up high. There is a problem where sometimes people slip when the stools are wet, so they worked out how to add a nonslip pad, but the product managers have decided that the next feature will be to add scents to the stools, so you can buy a stool that smells like cinnamon or one that smells like apples. They have a big advertising campaign already paid for and they already sent out the press release announcing "ScentedStools", so the machines need to be set up to start stamping out stools that smell like "Fresh Linen" by the end of the week. There are daily status meetings to update them on the progress. If the "Fresh Linen" stools aren't being produced by Thursday they are going to start having two status meetings per day.
You hear it's someone named Jim's last day, so you set up an exit interview. Jim tells you that the bosses and people upstairs don't really know what is going on in the factory. Most days he just sits and reads the news, his "nontechnical" manager doesn't know anything about furniture or how Jim does his job so there's no way for the manager to know what is going on other than to ask Jim. Supervision primarily consists of making sure Jim is sitting at his desk and looking at his monitor. Since US labor laws don't allow Jim's manager to set specific hours for him to be at work, his manager has started scheduling 9AM meetings every day to force people to turn up. Every week or so Jim has to update some Product Managers upstairs about what is going on, and he just says they are making steady progress and comes up with some specific problem to explain why they aren't done, pretty much anything with jargon will work since nobody upstairs "could tell white oak from red oak". It takes about 5 minutes to give his status update but he's expected to stay for the entire 1 hour meeting, so he brings his laptop so he can read furniturenews.com. He says he is quitting to take a much lower paying job because he is bored and doesn't respect his manager.
Next you go upstairs to the office space and find 300 people having meetings with each other about annual plans and prioritization, writing mission statements and meeting to discuss mission statements. The 300 people upstairs are constantly in motion and complaining about how over worked they are. They each have 5, 6 or even 7 (sometimes more!) 1-hour meetings every day, but you only see them meet with each other, nobody has any meetings with anyone from outside the company, nobody has meetings with possible customers, and only very rarely do you see anyone from the factory floor in these meetings, and then it is almost always just to give a status update. None of these folks really understand furniture very well, they can't really tell good furniture from bad furniture, they literally don't know the difference between solid oak and cardboard, they don't know how long it takes or how much money it costs to build a chair. After a few days of meetings you haven't met anyone who cares about furniture at all, they all seem to want to work at the furniture factory because it pays well, or they like the prestige of being 'in furniture'. Mostly they talk about how overworked they are and make the case for hiring a few more people. If they could hire another person for their team they wouldn't be so far behind. You aren't sure what they are getting behind in, are they talking about meetings they can't attend because it conflicts with another meeting that is more important somehow? Do they need more time to work on power point slides for the next days meetings? Some of the office folks have degrees in furniture science, but none of them have ever successfully built or designed any furniture outside of little school projects.
Then you go out behind the factory and see a massive mountain of furniture stacked up to the sky. The factory workers have been building furniture every day for years. People all agree that it is good furniture, maybe the best there is. Nobody ever buys any of it. It's not sold in any stores. No hotels buy it. No businesses buy it. Lots of people are lined up as far as you can see to pick furniture out of the pile for free.
How do you fix this company?
Being super careful about 'parallel' and 'concurrent' is super, super pedantic and pointless. They are synonyms. If two things are running concurrently, they are running in parallel. If two things are running in parallel, they are running concurrently. These are terrible words to overload with very careful definitions.
When people talk about 'concurrency' using the super careful pedantic definition, they just mean that it's possible to execute code while at the same time blocking on system calls. This isn't some kind of great feat and any language that can't do it is a toy. There is absolutely no reason to act like 'concurrency' using this definition is in any way special or worth even mentioning.
But that's not the point, is it? People get defensive about their favorite languages, so when faced with valid criticisms like "the GIL prevents you from running Python code in parallel" (which is a completely valid and actually pretty huge defect when modern computers have many cores on average) instead of just admitting the deficiency a person can be defensive and respond with stupid lectures about 'concurrency' and 'parallelism'.