solaxun

On the latency of the first request - How is the CFG cached?

Is it done at the API Key + schema level? Meaning that for a given API key, the latency penalty for a new schema is only paid one time, regardless of how far apart requests are? Or is cached with less duration, e.g. each session, conversation thread, etc?

I've read that SFT is good for "leveraging existing knowledge" gained during initial pretraining, and helpful in changing the way that the model responds, but not useful for teaching it new knowledge. In your experience is that true?

For example, changing the way in which it responds could be:

  - debate me
  - brainstorm
  - be sarcastic

Which also seems like something that could be accomplished with a system prompt or few shot examples, so I'm not sure when SFT is the more appropriate approach or what the tradeoffs are.

Alternatively, gaining new knowledge would be training it on a dataset of e.g. sports trivia to make it highly effective at answering those types of questions.

P.S. nice username... Irving Fisher would approve.

fine tune the chat or base model?

Tough to answer since I've only read the two intro chapters currently available. I'm still in the early stages myself with with Deep Learning but have years of experience with programming and lisp, so I'm coming in with a background that might bias my views.

If you have never used lisp, you need to be patient with the notation and resist the urge to balk at what is unfamiliar. I remember grumbling quite a bit when I first went through TLS, but that phase was over pretty quickly and within days I had no trouble following the code.

Are there better ways to introduce deep learning and neural networks? Maybe.. but I like "little" books and there is no better way to learn something than by building it yourself. For that reason alone I'd recommend the book sight unseen (but with the knowledge of prior little books). I do think choosing what is likely an unfamiliar language for most may be somewhat of an impediment for an already challenging topic, but scheme is a simple language and allows the author to focus on the ML concepts.

I'm highly confident you'll learn a lot, if you put in the work. For DL as a separate topic, the best resources I've found are:

  - 3 blue 1 brown Neural Net playlist
  - Karpathy's "The spelled-out intro to neural networks and backpropagation"

Books in the little series are all about "learning by doing", and teach using the Socratic method (question/answer). Each concept is introduced with small problems that build upon each other. You are forced to get your hands dirty from the beginning, passive learning is not really an option. For most people, I think this helps facilitate a much deeper understanding than just reading a text.

I always had trouble with recursive functions when I was new to programming, and many recommended working through "the little schemer" to solve that problem. It was a tough read for me, but the investment was well worth it and it did for me what it said on the tin. I didn't have nearly as much trouble with recursion after that book, but an unfortunate side effect was developing an affinity for lisps which I haven't yet shaken.

Not sure on relative popularity. If you haven't already, you should try it! Here's why I like it:

  - Very consistent, easy to keep the whole language in your head
  - Stable, core rarely changes ("accretive only" mindset).
  - REPL
  - Focus on a handful of data structures which will solve 95% of the problems you face, rather than a deluge of complicated abstractions.
  - Excellent concurrency primitives
  - Access to Java libraries if a Clojure equivalent doesn't exist
  - Macros

The only blemish for me was horrifically bad error messages.. other than that it's near perfect.

Nothing little about a 450 page book, but still... I have to buy it.

15 years ago was a different animal because the money largely stayed in the banking sector, as excess reserves. This time, a substantial portion of it was deposited directly into consumer's accounts (stimulus checks) - and that money makes it's way into the real economy, pushing prices up.

Out of curiosity (not much experience with Actor systems), do these systems solve a different problem than having disparate processes communicate by sticking a distributed message queue between them? From my naive point of view it seems like alot of the scaling and fault tolerance concerns nowadays can be solved in any language, by having a queue be the interface between two services.

Same question applies to Erlang, which I realize is not exactly an actor system as per the below comment, and has a much more sophisticated error recovery story with supervisors, but the general question holds.

I always found it strange that I really like salmon sushi/sashimi, but don't really care for cooked salmon. For some reason cooked salmon tastes too... heavy. I realize that's probably the fat content, but for whatever reason it doesn't bother me raw.

The one they built, apparently.

So I guess that means handling SWIFT MT / ISO 20022 messages is a big part of the custom stack?

That's an interesting question... for a lot of fintechs that "wrap" banks in a unified API, if something goes wrong on the app side the bank is not going to be liable. This is a bit different since they are actually the bank themselves.