domenicrosati
Joined 8 years ago
Joined 8 years ago
Attribution is important but it's impossible to know all the literature that is out there and even when you've tried your best to seek out prior arts there is always a possibility that you've missed something or not understood the connection with something previously read. Heck even when something is similar you may not cite it since it's so old the relevance is no longer there for a working research paper.
I've always thought that Schmidhuber was toxically uncharitable - what is the point of accusing and blaming? If someone has a similar idea to mine I'd celebrate them, seek to collaborate and encourage. Wouldn't that mean your research program is getting advanced for free! Heck you should thank them for doing your work for you.
[1] https://aclanthology.org/2020.acl-main.698/ [2] https://www.annualreviews.org/doi/pdf/10.1146/annurev-lingui...
The primary use case is: I am a researcher in nlp where speed of prototyping is key. I work in an environment where research fragments are primarily jupyter notebooks. So needing to diff notebooks is typically reviewing my own changes when modifying my and others research sketches. Since its helpful to see how code changes.
What really resonates with me is what others have said which is I need to run cells that take 2-6 hours to compute so recomputing cells is annoying... I dont love notebooks for their messy state which cause obvious problems that are very annoying.. and I am not an advocate for notebooks for production for this reason but the flexibility of computing stuff and having that persist and doing downstream prototyping makes notebooks amazing! Markdown and latex in there is also really helpful.
The secondary use case is PRs but... typically reviewing others research code isnt at the granular level of notebook riffs across a few commits so it deosnt come up often.
Sales fixes everything and is the hardest thing.
If sales can execute then there is no problem. It seems obvious but it's really not and people make the business about the mission and product - that stuff is secondary seriously secondary.
As a founder and early employee - you need to know if you can execute on sales or not and make it clear to both yourself and your coworkers.
Sales isnt something you can learn to do my biggest lesson is i cant do it no matter how hard I tried. If your employees / boss / cofounder are not closing sales you seriously need to get someone who can as your companies top priority.
Closing means money in the bank and nothing else. Paper is meaningless.
A contradictory learning is the best startups I've been at are ones where folks are ideologically fanatical with mission.
The one in pytorch itself isnt so bad either.
Be nice to see some examples of decoding in your repo (forgive me if I dont see it). I remeber when i first implemented a transformer from scratch generating sequences using greedy or beam search after train/testing turned out to be harder then I thought but turns out I made a mistake with teacher forcing in the beginning so BOS tokens were meaningless to the decoder lol
I typed in d-serine, bromantane, nsi189, modafinil, phenibut with no results...
This is called the compensation hypothesis at least in phonology (how we speak a language)
Pixels per character is certainly an interesting dimension of complexity and I would encourage the author to try to get this ready for publication at SIGMORPHON or something like that!
See https://oxford.universitypressscholarship.com//mobile/view/1... for more on complexity