llmzero

What is a little contradictory is that designing a system to use less resources can increases the number of people fine tuning models so that the final result can be a net global increase in the total energy use. A hypothetical goal could be to reuse fine tuning, that is designing a knowledge graph in which you fine tuning from a previously fine tuned model (like dynamic programming, save the result of previous computations). Lora allow us to store the small matrices with low cost.

I liked that you link to renting a dual 24GPU for 0.60cents/hour, but how long could it takes to fine tune a 70b model using your system (4 bits for weights)?

If I were a consumer I would be interested in the final price of fine tuning, for example a table with model size, training size, cost of training, and expected loss of quality with this technology.

One obvious question: Can you apply your technology with the recent (-1,0,1) encoding?, I think you will answers that the (-1,0,1) model is not available and you can't try it, but my question is whether once/if that model is available answer.ai will be able to use the same technology that this post to fine tune a big model in two very small GPUs, and then I should ask for a new table with cost/benefits analysis.

Edited: I should add that I find this kind of work very useful for enhancing individual users like me to be able to compete in the applications of LLM market, this is great work and along the lines of the book "from zero to one" (not that I like or dislike the author) to solve the kind of problem that nobody is trying to solve.

Edited: Now that I have a total of 23 points in HN, I will change my password to some random one, just to cure my desire to look for votes, and try to make some work, and again some tomorrow create a new presence in HN.

Think about the following scenario: I write a calculus book and the agents of this model just modify every example and every definition and change a little the ordering of the material to teach students. Now they are using my book but it seems they are not using my book. Are they trying to copy without copying?

My take away of a long post.

<< It’s a general feature of machine learning—and AI—techniques that they can be very useful if an approximate (“80%”) answer is good enough. But they tend to fail when one needs something more “precise” and “perfect”.

What's new in this edition?

In page 2, there is the theorem: Theorem ∀n in Nat.f n (fib p) (fib (p+1)) = fib (p+n), I think it should be for all p in Nat (fib p) + (fib (p+1)) = fib(p+2), otherwise there is something mysterious here.

Shorter ruby version: phrase.split.select{|x| x.count("aeiou")<=2}.max_by(&:length)

The enumerable module of Ruby provides many methods that can be easily implemented in J. Just to show one of them

   max_by =: 1 : '{~ (i. >./)@:(u&>)'

For example the list.max_by(&:length) is (# max_by)

I agree that in a team you should not use code golf for others to review your work. But for your own explorations or because your team can read easily your code golf then use it when is appropriate.

Just for fun, a solution in language j to the problem of selecting the longest word with less than 3 vowels.

solution =: >@:{.@:(\: #&>)@:(((+/@:(e.&'aeiou') <: 2:) # ])&.>)@:;:

Example

solution 'yes, today you are reading something that is not so easy to grasp'

the result is: today

Ruby : frase.split.select{|x| x.count("aeiou")<3}.sort_by(&:length).last => "today"

Edited: Added a comparison with Ruby. It seems Ruby here is easier to read and to compose.

Since: (i) the father and the mother of Sally may be married with other people, and (ii) the sister or brother relationship only requires to share one parent, we deduce that there is no a definitive answer to this question.

  Example:  Sally has three brothers, Sally and their brothers have the same mother but a different father, and those brothers have two sisters Sally and Mary, but Mary and Sally are  not sisters because they are from different fathers and mothers, hence Sally has no sister.

For those mathematically inclined: Supposing the three brothers are called Bob (to simplify) and the parents are designed by numbers.

FS = father of Sally = 7

MS = mother of Sally = 10

FB = father of Bob = 12

MB = mother of Bod = 10

FM = father of Mary = 12

MM = mother of Mary = 24

Now MS=MB=10 (S and B are brothers), FB=FM=12 (Bob and Mary are brothers), (FS=7)#(FB=12), and (MB=10)#(MM=24). Now S and M are not sisters because their parents {7,10} and {12,24} are disjoint sets.

Edited several times to make the example trivial and fix grammar.