msoloviev
Joined 12 years ago
Karma
70
Submissions
33
Member since
Jan 21
Recent submissions
Comment · 1 year ago
Comment · 1 year ago
I wonder how the "ring context" works under the hood. I have previously had (and recently messed around with again) a somewhat similar project designed for a more toy/exploratory setting (https://github.com/blackhole89/autopen - demo video at https://www.youtube.com/watch?v=1O1T2q2t7i4), and one of the main problems to address definitively is the question of how to manage your KV cache cleverly so you don't have to constantly perform too much expensive recomputation whenever the buffer undergoes local changes.
The solution I came up with involved maintaining a tree of tokens branching whenever an alternative next token was explored, with full LLM state snapshots at fixed depth intervals so that the buffer would only have to be "replayed" for a few tokens when something changed. I wonder if there are some mathematical properties of how the important parts of the state (really, the KV cache, which can be thought of as a partial precomputation of the operation that one LLM iteration performs on the context) work that could have made this more efficient, like to avoid saving full snapshots or perhaps to be able to prune the "oldest" tokens out of a state efficiently.
(edit: Georgi's comment that beat me by 3 minutes appears to be pointing at information that would go some way to answer my questions!)
Comment · 4 years ago
Hah, +1 for HN users who attended that course in 2009 here. (I usually post on a pseudonymous account though.) I surely must know you, though I couldn't tell who you are from very briefly stalking your comments!
While we're reminiscing, I imagine you also took Concepts in Programming Languages in second year (https://www.cl.cam.ac.uk/teaching/1011/ConceptsPL/ ?), which (after a tour de force through all sorts of paradigms that were consigned to the fossil record for the time being) did advertise Scala pretty heavily. I found that course to be a true gem and haven't seen anything like it at any of the other universities I've passed through since then.
Comment · 4 years ago
Yeah, the way Windows builds are made available at the moment is honestly pretty bad, and I should improve on that. There is the problem that you need to be logged in to download that particular type of CI artifact, and more generally that the archive is somewhat bloated because the build scripts have no notion of what parts of Gtk actually need to be bundled (and so they wind up including an excess of random icons and what-not).
I don't know if I want to reach what you would call the "general public" (at least not until the program is quite a bit more polished than it is now), but I'd certainly want to reach typical Linux-using HN posters who are interested in Markdown notetaking, and the circumstance that this subthread played out as it did implies I haven't yet done so.
Comment · 4 years ago
Well, it's not exactly unsustainable - the Github CI continues producing those builds without me having to do anything for it (it in fact didn't break even once in the past year, compared to several breakages on the "backwards-compatible" Ubuntu 18.04 deb which happened whenever Github changed something about the package bundle available to that image). If someone reports a bug on Windows, I will look into it, and/or spend some time walking them through a workaround (since I do in fact have access to Windows setups). As I see it, in the most natural sense of support, I do have support for Windows, even though it is what I guess you would call Tier 2 support.
To nitpick a little, I also didn't say I don't think that those who care about licensing issues would use [the Windows build]; rather, I think that those who don't care about licensing issues and are on Windows would not use it, because there is a Windows-only product that is closed-source which I am unlikely to be able to compete with on that ground.
I'm not really advertising Mac support beyond having some files merged from people who did get it to work (https://github.com/blackhole89/notekit/blob/master/screensho...).
Comment · 4 years ago
Thanks for the feedback! I can certainly try to improve the structure of the README, though I'm not sure if I agree with some of your points about giving more priority to non-Linux users. The thing is, at least in my eyes, Gtk generally feels terrible on platforms where it is not native (I would in fact even avoid using it in a Qt-based Linux desktop setup, just as I avoid Qt-based applications as far as possible in my Gtk-based one), and at any rate the ports are often lacking even in features (for instance, last I checked, Windows Gtk would not allow me to read tablet pressure information). If you are using Windows, and don't care for "free (speech and beer)", why would you even want to use anything other than OneNote? I'm somewhat wary of repeating the mistakes of Firefox (and GNOME) of snubbing a likely target demographic to court an unlikely one.
(Regarding Mac, I unfortunately don't have a Mac and neither does anyone who I know well enough to ask to borrow theirs, though someone did in fact contribute homebrew-based build scripts that supposedly work at some point. I just have no idea how to package the result to make it easy to install.)
Comment · 4 years ago
I've been working on one (https://github.com/blackhole89/notekit) for a while now (which, unlike the aforementioned, also is not built on Chrome/Electron). Unfortunately it seems to be pretty hard to get the word out, or at least I haven't found any better strategy than to pounce, as I am doing now, whenever I see a HN thread about markdown editors (which invariably wind up having some comment thread lamenting the lack of WYSIWYG and/or non-Electron editors).
Comment · 5 years ago
Thanks! I recommend using the repositories or the automated builds or compiling from source, as opposed to the tar.gz binary builds linked directly from the README, as those are somewhat badly out of date now. (I probably should update them at some point.)
Comment · 5 years ago
I'm actually working on one (https://github.com/blackhole89/notekit). It's native (Gtk+/C++) rather than Electron, too.
Comment · 5 years ago
> As soon as you wrap code in a function for reuse, the proximate call site no longer has anything to do with the widget ID.
That's a good point.
(a) hash the entire call stack (though that might produce false negatives, i.e. consider two UI elements that should be the same distinct?)?
(b) put the burden on the reusable function to mark itself as such by pushing/popping an identifier of its own call site on the ID stack?
> The real hack is implicit IDs, not the ID stack (which is just a way of implementing a hierarchical namespace like file system paths or URLs). The fact that implicit IDs just work 99% of the time and only require manual intervention 1% of the time is a false siren song into letting you believe a 100% solution is desirable (you have to consider the marginal cost of what it would entail).
Well, this is just the standard problem of library design, isn't it? You always have to figure out the appropriate tradeoff between supporting rare cases and making common ones easy. (Of course, you can often do both; in this case, you probably could both give "explicit ID" and "call site ID" versions of each UI element API.)
Comment · 5 years ago
Yeah, I essentially copied imgui's ID stack approach for my experiments too. (I've been using imgui for some other projects to great success.) It still seems like a hack; I'm quite surprised that no programming language (I'm aware of) makes it possible to uniquely identify callsites like that. Maybe it hints at a more general blind spot/free real estate in PL design :)
(On the off chance you're curious, I just pushed some previously unpushed updates to that experiment I had sitting around, so now it has labels and text entry too. I guess the real test of the architecture would still be making an alternative "rendering backend" based on win32 widgets or something.)
Applying the same approach to chain-of-thought reasoning gave me the feeling that I might be looking at a form of realistic UX for some sort of science-fiction neural AI augmentation - you can let the CoT run on and do its thing, but also interject at any point and insert a "thought" of your own, or go back and revise a thought you did not like, and then let it continue. Imagine such a stream hooked up with a two-way pipe into your phonological loop (https://www.sciencedirect.com/topics/psychology/phonological... - perhaps more attainable with existing tech).