dlwh
Joined 18 years ago
Karma
111
Submissions
33
Member since
Sep 4
cs.berkeley.edu/~dlwh
[email protected]
View on Hacker NewsRecent submissions
Comment · 3 years ago
Stanford CRFM | Bay Area | Full-Time | On-Site, Hybrid
Foundation models like ChatGPT, PaLM, and Stable Diffusion are transforming the world around us. The Stanford Center for Research on Foundation Models (CRFM; https://crfm.stanford.edu/), which is part of Stanford HAI, is an interdisciplinary initiative that aims to make foundation models more reliable, transparent and accessible to the world. We take on ambitious projects that seek to rigorously evaluate existing foundation models and to build new ones.
We are currently seeking a research engineer to join our engineering team. This is an unique opportunity to work with seasoned engineers who have spent many years in industry as well as PhD students, post-docs, and faculty at CRFM. You will contribute to cutting-edge research, publish papers, gain access to the latest foundation models, and be immersed in the vibrant CRFM community.
You will work on our open source software projects, including:
- HELM, a framework for holistic evaluation of large language models
- Levanter, a framework for transparent and accessible large-scale language model training
Comment · 7 years ago
I highly recommend the book 1491, which goes pretty deeply into what we can infer about the pre-Columbian Americas.
Comment · 8 years ago
Thanks for the questions.
I am very unlikely to take on visualization. I don't acutely need it for what I do, and I am some-but-not-nearly-enough interested in visualization for its own sake. I started to read about the grammar of graphics stuff at one point and decided it was too far down the rabbit hole.
I have looked more into gpu stuff, and agree specifying a compute graph (and then implicitly optimizing it) is more likely to be the future. FWIW, this is basically what XLA (from TensorFlow) and whatever it was FB announced on Friday are doing.
I wrote my thoughts up recently on the Breeze mailing list here: https://groups.google.com/forum/#!topic/scala-breeze/_hEFpnI...
I'm starting to think it through but I'm not sure I have time for that either :(. A 4-month old and a startup take up a lot of time.
Comment · 8 years ago
Thanks for the kind words.
Breeze does a large chunk of (dense) compute via netlib-java, which calls out to "real" lapack if you set it up. Are things really faster than that? Or are you referring to the non BLAS/non Lapack things?
Comment · 8 years ago
Main author here.
Breeze has breeze-viz, which is very basic but at the time there wasn't anything else. I highly endorse using something else. I personally like http://sameersingh.org/scalaplot/
They're under the same aegis basically because they're all mine. ScalaNLP started out as really being just NLP, but it scope-crept. That said, Epic is a library for structured prediction first and foremost, and one of the main applications of structured prediction is NLP.
Breeze is basically like SciPy and large chunks of it power Epic. It's really the only thing that doesn't belong in the namespace.
Comment · 8 years ago
So, this is (largely) my project.
Not sure why this is on the front page of HN, but I'm happy to answer any questions.
I'm not really giving these libraries the love they need these days. I mostly started them in grad school before the deep learning revolution really hit my subfield (NLP), and I haven't had time to modernize them. They still have their uses, especially Breeze, which is used in Spark's MLLib and directly by a number of companies.
Comment · 9 years ago
Sure, but the FP side of Scala is definitely firmly in the advanced static type system camp.
Comment · 9 years ago
Semantic Machines | Software Engineers and Machine Learning Engineers | Boston, MA and SF Bay Area, CA (Berkeley) | http://www.semanticmachines.com/careers/
Semantic Machines is developing technology to power the next generation of conversational artificial intelligence: AIs that you can actually have a conversation with. Think Google Assistant or Alexa or Siri, but without having to carefully craft commands like you're talking to a Bash shell.
Our team has built much of the core technology underlying Siri and Google Now, and our founders (including both the former Chief Speech Scientist for Siri and the head of UC Berkeley's Natural Language Processing group) have multiple >$100 million exits under their belt.
We're looking to hire a few talented software engineers and machine learning engineers to help build out technology, by expanding our core NLP infrastructure, data processing pipelines, neural net clusters, and backend services.
Experience with natural language processing systems is a plus, as is experience with the JVM (especially Scala), but we're mainly interested in passionate engineers who can learn quickly and work effectively in complex systems.
Please reach out to me directly or email [email protected]. Thanks!
Comment · 11 years ago
Not a neuroscientist, but I do do NLP, and I only lightly skimmed the paper.
This doesn't really speak to UG.
First, you can believe in the structures they purport to show without accepting the existence of UG, by appealing to the existence of general mechanisms in the brain for assembling hierarchical structures, which is equally validated by this experiment.
Second, they looked at two languages with sentences of up to ~7 syllables each with at most two constituents (Noun Phrase Verb Phrase). You can't show any evidence for any hierarchy of interest in 7 syllables. They demonstrated that phrases exist and phrase boundaries exist, but it's entirely possible to have "flat' grammars without interesting hierarchy, especially in simple sentences. If they want to show interesting hierarchy, they should conduct experiments with more interesting structure (say, some internal PPs and some limited center embedding) and show something that correlates with multiple levels of the "stack" getting popped, or something.
It's still interesting work, but as usual oversold by the university press office.
Comment · 11 years ago
Prosopagnosia[1] is a real condition that affects ≈2.5% of people. You might check out "The Man Who Mistook His Wife for a Hat" if you're interested.
Comment · 11 years ago
to add another:
System.out.println(0.0 == -0.0); // true
System.out.println(java.util.Arrays.equals(new double[] {0.0}, new double[] {-0.0})); // false
Comment · 11 years ago
statically checked, but still runtime-determined, matrix sizes can be done in Scala. Doing specialized implementations would require bytecode magic, though.
case class Dim(size: Int)
trait Matrix[X <: Dim with Singleton, Y <: Dim with Singleton] {
def *[Z <: Dim with Singleton](other: Matrix[Y, Z]):Matrix[X, Z] = ???
}
object Matrix {
def apply[X <: Dim with Singleton, Y <: Dim with Singleton](x: X, y: Y):Matrix[X,Y] = ???
}
val x = Dim(100)
val y = Dim(readFromFile(...))
val z = Dim(37)
val A = Matrix[x.type, y.type](x, y)
val B = Matrix[y.type, z.type](y, z)
A * A // compile error
A * B // okComment · 11 years ago
(I'm one of the authors of Overmind.)
First, it's important to know that AI is a very broad term. Roughly, there's "modern" AI (machine learning and statistical modeling) and "classical" AI (mostly logic and search). Game AI (as I understand it) falls more into the "classical" AI camp, and classes these days are focusing more and more on modern non-game techniques.
kaybe's link is a good starter class for modern AI. Norvig quite literally (co)wrote the text book on AI, and Sebastian Thrun. The PI on Overmind (Dan Klein) co-taught a class on edX (https://courses.edx.org/courses/BerkeleyX/CS188.1x/2012_Fall...) that you might also want to check out. They're all based on the same underlying book, just different focuses.
Overmind itself is a collection of some reasonably sophisticated ML/modern AI, classical AI (which is more common in games), and a lot of horrible hacks. Really, tons of them.
If you want, I can try to dig up the lecture slides for the "advanced" AI class we taught once that specifically focused on the kinds of techniques one would want to write StarCraft bot AIs.
Comment · 12 years ago
Machine translation is no where near solved.
Machine translation is still one of the most funded areas in NLP, and the quality is still incredibly bad for many language pairs. DARPA has been running one machine translation grant program or another for more than a decade.
It is true that some of the best systems take a long time to translate sentences (on the order of 1 cpu-minute or more). But their quality is still not anywhere near human level for most languages.