Recent submissions

Comment · 4 years ago
Simulation? Like fluid dynamics. I heard that was CPU intensive.
Comment · 4 years ago
Well done
Comment · 4 years ago
Ya... those are not great places to start with him... as far as philosophy of science essays in conjectures and criticisms and objective knowledge are really great and much more mature than the logic for the reasons you mentioned. Plus he develops some of the logical tools like content theory and verisimilitude in really cool ways.
Comment · 4 years ago
Same
Comment · 4 years ago
Well said.

Attribution is important but it's impossible to know all the literature that is out there and even when you've tried your best to seek out prior arts there is always a possibility that you've missed something or not understood the connection with something previously read. Heck even when something is similar you may not cite it since it's so old the relevance is no longer there for a working research paper.

I've always thought that Schmidhuber was toxically uncharitable - what is the point of accusing and blaming? If someone has a similar idea to mine I'd celebrate them, seek to collaborate and encourage. Wouldn't that mean your research program is getting advanced for free! Heck you should thank them for doing your work for you.

Comment · 4 years ago
I wonder what software would be good for a logbook like this... I just use google docs for these kinds of things. Sure wandb and jupyter notebooks are good but they are not so good for notes and ideas and documentation
Comment · 4 years ago
Comment · 4 years ago
Hey there - OP here. I haven't used spyder I'll have it check it out.

The primary use case is: I am a researcher in nlp where speed of prototyping is key. I work in an environment where research fragments are primarily jupyter notebooks. So needing to diff notebooks is typically reviewing my own changes when modifying my and others research sketches. Since its helpful to see how code changes.

What really resonates with me is what others have said which is I need to run cells that take 2-6 hours to compute so recomputing cells is annoying... I dont love notebooks for their messy state which cause obvious problems that are very annoying.. and I am not an advocate for notebooks for production for this reason but the flexibility of computing stuff and having that persist and doing downstream prototyping makes notebooks amazing! Markdown and latex in there is also really helpful.

The secondary use case is PRs but... typically reviewing others research code isnt at the granular level of notebook riffs across a few commits so it deosnt come up often.

Comment · 4 years ago
Woah this is exactly what I'm looking for! Thanks
Comment · 4 years ago
In 10+ years of startup experience on the science and engineering side.

Sales fixes everything and is the hardest thing.

If sales can execute then there is no problem. It seems obvious but it's really not and people make the business about the mission and product - that stuff is secondary seriously secondary.

As a founder and early employee - you need to know if you can execute on sales or not and make it clear to both yourself and your coworkers.

Sales isnt something you can learn to do my biggest lesson is i cant do it no matter how hard I tried. If your employees / boss / cofounder are not closing sales you seriously need to get someone who can as your companies top priority.

Closing means money in the bank and nothing else. Paper is meaningless.

A contradictory learning is the best startups I've been at are ones where folks are ideologically fanatical with mission.

Comment · 4 years ago
The T0 series by big science is essentially an instruct model (though using multitask prompting instead of user feedback). You should check it out. I have got very competitive results on prompting t0-11b v instructgpt3(text davinci 2)
Comment · 4 years ago
Cool! What motivated you to write it? I find the d2l implementation nice and simple as well https://d2l.ai/chapter_attention-mechanisms/transformer.html

The one in pytorch itself isnt so bad either.

Be nice to see some examples of decoding in your repo (forgive me if I dont see it). I remeber when i first implemented a transformer from scratch generating sequences using greedy or beam search after train/testing turned out to be harder then I thought but turns out I made a mistake with teacher forcing in the beginning so BOS tokens were meaningless to the decoder lol

Comment · 4 years ago
Be awesome if it were more comprehensive.

I typed in d-serine, bromantane, nsi189, modafinil, phenibut with no results...

Comment · 4 years ago
Comment · 4 years ago
Is anyone able to find evaluation of these models besides speed and power consumption? I really cant find any baseline comparisons of accuracy and so on for any relevant datasets being used these days. I wish they did a better job of presenting that since right now I'm thinking I could build a million times faster NN with 0% accuracy too...

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton