Recent submissions

Comment · 1 year ago
I've seen this being posted all over, but people rarely seem to realize that data here is from 2023.
Comment · 1 year ago
Comment · 1 year ago
Comment · 2 years ago
Comment · 2 years ago
I skimmed the technical report: https://cosine.sh/blog/genie-technical-report

At the bottom, they noted the following:

> SWE-Bench has recently modified their submission requirements, now asking for the full working process of our AI model in addition to the final results -their condition to have us appear on the offical leaderboard. This change poses a significant challenge for us, as our proprietary methodology is evident in these internal processes. Publicly sharing this information would essentially open-source our approach, undermining the competitive advantage we’ve worked hard to develop. For now, we’ve decided to keep our model’s internal workings confidential. However we’ve made the model’s final outputs publicly available on GitHub for independent verification. These outputs clearly demonstrate our model’s 30% success rate on the SWE-Bench tasks.

Their model outputs are here: https://github.com/CosineAI/experiments/tree/cos/swe-bench-s...

Comment · 3 years ago
It seems to me that there's a natural tension in emerging fields of science and engineering between establishing clear guidelines and regulations early on to minimize harms, or instead allowing practitioners to experiment, tinker, build and create outcomes that may be potentially harmful.

What are some frameworks for how to think about navigating this tension in emerging scientific or engineering fields?

Some ones I'm mulling over:

1. Rate of innovation: In rapidly evolving fields, imposing strict regulations too early can hinder innovation and progress. In such cases, it might be better to minimize restrictions early on to allow practitioners to explore new ideas. Then, as the field matures, regulations and standards can be gradually introduced.

2. Adaptive regulation: Implement a flexible regulatory framework that can be updated as new information becomes available.

3. Self-regulation: In some cases, maybe we should expect and encourage the industry to use self-regulation via developing guidelines and codes of conduct. This may be one way to try and strike a balance between responsible innovation while minimizing bureaucratic obstacles.

What do others think?

Comment · 3 years ago
I took a class with Professor Ousterhout. He would end every Friday's lecture with a "Thought for the Weekend", such as this one.

It was very entertaining and charming to hear him discuss his personal and professional life, and lessons he's learned throughout them often occasionally have very little to do with computer science.

I don't remember all of his "Thoughts for the Weekend", but I do remember one story he told about wishing he had apologized sooner to resolve some conflict he was in. That was a bit of wisdom that stuck with me from the class, beyond any of the computer science topics we covered.

Comment · 3 years ago
The article does in fact mention this event in the below paragraph:

"The hypothesis took another hit last July when a bombshell article in Science revealed that data in the influential 2006 Nature paper linking amyloid plaques to cognitive symptoms of Alzheimer’s disease may have been fabricated. The connection claimed by the paper had convinced many researchers to keep pursuing amyloid theories at the time. For many of them, the new exposé created a “big dent” in the amyloid theory, Patira said."

Comment · 4 years ago
I think the situation is more complex than just "we must assume they don't care about learning ... just fail the bastards." I think it's more important to first understand what were the social, environmental, cultural (and otherwise) causes of this behavior.

Specifically, different systems of incentives and permissiveness will produce different behavior. I taught high school computer science for 4 years, and I can attest that cheating occurred in the classes I taught. I've also been enrolled part-time in Stanford's MS in CS, and have taken a number of the core undergraduate curriculum for CS majors.

I also went to a hypercompetitive US public high school with a number of brilliant classmates, many of whom also cheated.

My experiences have showed me that there is a wide spectrum of "cheating", ranging from students sharing things like, "I was at office hours and heard from the TA heard from the professor that topic X is going to be really emphasized on the exam, so you better study for it!" to outright blatant copying of other's code or answers.

What I've noticed as qualities of a learning environment that seems to increase the likelihood of cheating are:

1. The technological ease of which it is to cheat: it's easier to cheat on an asynchronous online exam than when you're taking it synchronously in a large classroom.

2. How "high stakes" the course is for students: for students at institutions like Stanford, where they be used to a certain level of academic success, failing a course isn't just a blow to their transcript -- it's a psychological blow to their identity as a "smart student." They may find it easier to cheat and maintain their self-image (and projected image to their family/friends) as a great student than to take the honest hit to their GPA, and have to give up their identity.

3. How "legitimate" the course feels: classes where the instructor is widely perceived as "unfair" or "incompetent" seem to have more cheating. Students feel disrespected ("How could she put X on the exam? We barely covered it!") or unvalued ("He doesn't even bother giving clear instructions on the homework assignments. Why should we respect his test?") may try to 'retaliate' by cheating.

4. How permissive the academic culture is around cheating: if there is widely perceived to be little-to-no consequences to cheating, or if cheating is seen as, "well everyone does it", then you will have a lot more cheating.

I'm sure the above is not an exhaustive list. My broader point is that in order to address the issues around cheating, we need to be more encompassing than simply punishing the cheaters. If the stakes are high enough, and the incentives strong enough, cheaters will still exist even if they are aware of the severity of the punishment.

Comment · 4 years ago
I recently took a CS class at Stanford with an interesting policy on cheating. While cheating almost certainly happened during the course, at the end of the quarter the course staff made a public post allowing any student who cheated to make a private message to the staff admitting they've done so.

If a student admitted to cheating, while they would face academic disciplinary action (i.e. receiving a failing or low grade), they would not be brought up to the administrative office that deals with issues of academic integrity, and therefore would not face consequences like expulsion or being on official academic probation.

However if a cheating student decided to risk it and not admit their guilt, they were at risk of a potentially even greater degree punishment. The course staff would run all students code through a piece of software to detect similarities between each other, as well as online solutions. Students who were flagged by this software would then have their code hand-checked by at least one course staff, who would make a judgement call as to whether it seemed like cheating.

I found this policy quite interesting. As a former high school teacher, I've certain encountered teaching in my own classes, and have historically oscillated between taking a very harsh stance, or perhaps an overly permissive one.

The one taken by the lecturers of this course offered a "second chance" to cheaters in a way I hadn't seen before.

Comment · 4 years ago
I think a fundamental mistake you (and other commenters on this article) are making is judging the value of the "plain" English version by whether you think it's good writing.

However, the article's intention is to use plain language to be accessible to individuals with intellectual disabilities or other difficulties in language (i.e. recent immigrants).

As a fairly well-read and educated person, I also find the "plain" version dull and uninspiring. However, I accept that the article is trying to make the point that such writing may be more broadly understandable.

For example, I recently had some relatives immigrate from a non-English speaking country. I helped them set up internet and noticed there were multiple points during the company's signup and payment process their lack of English fluency created huge hurdles.

Comment · 4 years ago
Comment · 5 years ago
Although the author may be wrong about whether the 10-Ks he saw were the same ones being submitted to the government, I think his curiosity is still worth indulging.

Why did Netflix slowly stop designing their 10-K reports that lived on their own investor relation page?

My own guess is that there was some downward pressure to just produce the basic report, since that's all the government would see and investors wouldn't really value the design.

Comment · 5 years ago
I'm not sure this is a sufficient explanation.

McCarthy left in 2010. But it looks like the decline in the "designy"-ness began in 2006.

Comment · 5 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton