gregwebs

The solution is to not have an untrusted workload handle secrets.

This downloads a library of mostly kiwix wikipedia information and produces a full text SQLite index that is live searched by the model and then the model reads and summarizes the information.

Interesting thing is that when I asked a follow up question it answered but showed: This answer came from what the AI already knows, not the library, so there's no source to check. Ask “look that up” to get one.

When I asked "look that up" it was able to. It highlights relevant sections.

The information it was working off in my case was WikiHow and there aren't any citations in that information.

Running on a 32GB Mac M1 using Qwen 27B. Ollama is showing as taking 19GB steady state and 23GB while answering on Activity Monitor (sometimes a little more and sometimes a little less).

Its amazing this can be done, but it seems like someone should be able to develop a much smaller model that can be used reliably for this use case. There are smaller models available to use and download but it is stated that they have some reliability issues.

Ironically the best defense against police abuse of power seems to be doing your own surveillance. Recording interactions with the police and having a dash cam. Many popular dash cams like Viofo record to an SD and don't send recordings to the cloud.

I am reading this as a potential leak of almost every possible piece of metadata related to the operation and the business but a limited leak of customer data.

This goes to show the benefit of maintaining separations between the control plane and the data plane.

This site does say it has SQL logs which have some amount of customer data and there is a mention of some snapshots/imports- this looks like a case where the control plane and data plane end up getting entangled. Its possible the attacker could have reset customer database passwords to try to connect to them or could have exfiltrated customer data by initiating snapshots but just didn't want to deal with the increased risk of detection.

TiDB is for large scale data. You generally wouldn't use it unless you have at least 1TB of data. A 10TB leak of information is less than the size of a single small customer database for them.

TiDB is for large scale data. You generally wouldn't use it unless you have at least 1TB of data. 10TB is a single small customer database for them.

I am reading this as a leak of every possible piece of metadata but not an actual leak of customer data. They have SQL logs which do contain customer data and there is a mention of some snapshots/imports.

> For example, benchmarks in our initial list heavily cover AI-exposed lawyers’ tasks, but barely cover any of the O*NET tasks of similarly AI-exposed legal secretaries or court clerks.

This technique seems dubious- what is stopping AI from doing the computer work of legal secretaries and court clerks as well?

Solving math problems is one of the hardest theoretical jobs. Software engineering is one of the harder practical jobs. AI is now much better at these tasks than humans because it has been able to properly train for them.

For other computer-based jobs it's a matter of AI getting properly trained for them. If a job is harder to verify or the training data is more limited, it may be much more difficult to train AI for it but I don't think there's any reason to think it won't happen eventually.

Senior engineer and all SWE positions will become more like a technical lead.

If you can direct AI and produce more value per $/time spent then a non software engineer directing AI to develop software then you still can produce a lot of value. This should be the case now for most things that are not a prototype. The job has changed from "writing code" to "shipping software". A technical lead.

Software products making ever more intelligent AI software development accessible will continue to reduce the need for software engineer involvement. But there’s also going to be 1000x more software. So there may continue to be opportunities for those that adapt.

DeepSeek is presumably break even or better and costs 20-40x less than frontier models. Its intelligence is lagging the frontier by a lot, but its already plenty good to contribute to certain aspects of software development. There is a lot of opportunity to make model usage more cost effective for software development.

If you can direct AI and produce more value per $/time spent then a non software engineer directing AI to develop software then you still can produce a lot of value.

This should be the case now for anything that is not a prototype. The job has changed from "writing code" to "shipping software". A technical lead.

As AI gets more intelligent and the tooling around it develops and organizations figure out how to get it access to the data, the level of involvement needed for software engineers will decrease. Since the cost of developing software is plummeting, the amount of software being used is going to expand greatly- that will keep the technical lead positions available longer and also create more opportunities for consultant engagements.

Although someone can use AI to rewrite every software dependency and every paid piece of software, its still not worthwhile for them for most software, especially if the price of the software is low compared to the complexity it hides.

Users of a paid software product still need a human to be responsible for it.

As you develop your skills with AI, you now have the ability to develop your own software products. So there's an opportunity to become a small business owner. You didn't want to do these non-technical roles, but now you can continue to direct AI to automate all the aspects of the business.