aglionby
Joined 9 years ago
Joined 9 years ago
Relatedly, here are a couple of tools to ensure that references are complete (e.g. updating arXiv papers to their published versions, mostly for computer science papers):
- https://github.com/yuchenlin/rebiber (CLI, web interface)
- https://www.cl.cam.ac.uk/~ga384/bibfix.html (only *ACL papers, web interface with diff, disclaimer: mine)
Back in 2015, Instagram did a blog post on similar challenges they came across implementing emoji hashtags [1]. Spoiler alert: they programmatically constructed a huge regex to detect them.
[1] https://instagram-engineering.com/emojineering-part-ii-imple...
In 2020 you can buy a second generation iPhone SE for €489, with a 1334x750 screen (at >6x pixel density) and 12MP camera.
These are not roughly equivalent pieces of hardware, and the software they run differs immensely. MMS is not Instagram.
Yep, this often frustratingly turns nothing up for me. Hence, Pinboard :)
The second is as a kind of mechanism to give myself permission to close a bunch of tabs every time they accumulate. Each is _obviously_ open for a good reason and I may want to read it at some point, so sticking it on pinboard is a nice way of shoving them elsewhere. I don't save everything - curation is important (in the same way as with tagging). Lots of what remains are things that may be useful for me in the future but are not immediately, like design guides https://pinboard.in/u:guyaglionby/t:design/. Some of these things I leave as 'unread'; others that feel more like reference material I mark as 'read' immediately so as not to have them in my to-read queue.
True, GPT-2 and -3, RoBERTa, T5 etc. are all increasingly data- and compute-hungry. That's the 'tick' your second article mentions.
We simultaneously have people doing research in the 'tock' - reducing the compute needed. ICLR 2020 was full of alternative training schema that required less compute for similar performance (e.g. ELECTRA[2]). Model distillation is another interesting idea that reduces the amount of inference-time compute needed.
I've not read it in a lot of detail but it looks like there's a positive correlation between releasing papers and having them accepted. Not sure how they've controlled for confounders (you only release papers you're confident in the quality of on arXiv?) https://arxiv.org/pdf/2007.00177.pdf
hello@ | www.guyaglionby.com
https://pinboard.in/u:guyaglionby