gdiamos

>> Where I draw the line is asking Claude to label training data. I don’t do this.

> Why draw any line? Anthropic doesn't. They actively take the world's intellectual property, don't pay for it, and put it behind a paywall.

I know a lot of silicon valley founders think like this. I personally don't think that just because someone else violated a boundary makes it okay to do it.

It's right here - you can't see it due to being downvoted.

I asked Claude to research this one and it turned up an interesting point. Pirated books got into the Pile and Anthropic paid 3k per book. Worth noting if anyone still uses the Pile.

[Claude researched, not verified]

In October 2020, developer Shawn Presser converted the Bibliotik pirate tracker into Books3 (196,640 books) and hosted it on The Eye, aiming to give everyone what OpenAI had with its secret Books1 and Books2. EleutherAI then folded Books3 into The Pile, distributed via Hugging Face and GitHub, and it went on to train Meta's LLaMA, EleutherAI's Pythia, and BloombergGPT. In August 2023, Alex Reisner of The Atlantic exposed its contents, and Maria Fredenslund's Danish Rights Alliance forced takedowns at The Eye, Hugging Face, and EleutherAI. A year later, Andrea Bartz (The Lost Night), Charles Graeber (The Good Nurse), and Kirk Wallace Johnson (The Feather Thief) filed Bartz v. Anthropic before Judge William Alsup, accusing Anthropic of using Books3, LibGen, and PiLiMi to train Claude; the case settled for $1.5 billion, about $3,000 per work.

Downvoted after providing evidence.

That's very click baitey. Do you have an argument or evidence for why?

One of my points of my ethical framework is that IP is valuable and should not be stolen.

Another way of saying is that hard work that advances the state of the art should be rewarded. Patents should exist.

I’m sure they are watching what I’m doing and would retaliate via service cut off and legal action if I crossed it egregiously and they found out.

I would do the same thing if a user or employee did this to my IP.

I don’t completely buy the argument of web scraping being an IP violation.

I don’t send my core ip in an email to my main competitors and I don’t post it on the internet.

Where I draw the line is asking Claude to label training data. I don’t do this.

I ask it questions about large models like big MoEs all the time. I’ve never had it refuse a question, but I would expect it to refuse sometimes.

For training data from existing models I just get open models to spill their guts. It’s fine

In industry we train employees to not reveal proprietary information.

I think it would be easier to do this for a model than a lot of employees

This tech is about to come for software.

Why don’t you do as much of this as possible for the next six months and make the best product you can.

It sounds like it will go faster if you use coding agents.

Anyone copying you will have to do the same.

Perhaps spend a month with your agent making it harder for other agents to reverse engineer?

Good job typesafe.

You won the competition with VCs

I wouldn’t trust an AI company to honor this as far as I could throw them

How to steal ideas with AI.

step 1, identify high value users by net worth, citation count, or number of followers

step 2, select all prompts by high value users

step 3, invest 10 billion thinking tokens in modeling an objective for each user

step 4, build an RL environment for each user

step 5, rollout 10 billion tokens per environment

step 6, train on resulting traces

Training improvements are very easy to copy.

Data is doing more of the work than it used to. Every source in our mixture is a curated artifact built with large models

Training a model this small on them is distillation

When models of this size were last studied seriously such corpora did not exist