arturventura

Github Desktop has limits, this uses your own OpenAI account.

Is it possible to run this in a chaotic function? I would be interested to see what patterns emerge. I haven't found any code or model to generate this.

I've been working on the idea of building synthetic workers. I'm trying to implement a planning workflow system for scenarios where the workflow definition, the environment, or the task are not well defined. I also ended up implementing a micro Palentir plugin system to support the action system for the synthetic users.

Its a cool project that gave me immense pleasure to built, however its unfortunately a intellectually masturbatory one, because although the tech is cool, I haven't found a cool application for it. If anyone is interested hit me up.

What the actual f**!!! I'm from Coruche!! The fact that I'm seeing my hometown name in HN is f'ing insane.

  Location: Lisbon, Portugal
  Remote: Yes
  Willing to relocate: Maybe
  Technologies: Django, Python, Vue, C, Java, JavaScript, Typescript, PyTorch, Nordic Firmware Development
  Résumé/CV: https://www.surf-the-edge.com/wp-content/uploads/2024/01/Resume.pdf
  Email: [email protected]
  Website: https://www.surf-the-edge.com/

My name is Artur Ventura, I'm a Senior Software Developer Engineer with a cross functional expertise in every level of software development. Developing software on a professional level, in scenarios of hundreds of thousands of users for 13 years. Focused on Applied Artificial Intelligence, in particular Natural Language Processing pipelines. Just finishing the sale of a company and looking for a new challenge.

Interesting idea, but if you didn't filter the elements with URLs, you're creating a CSS injection attack that enables people who visit the website to work as bots in a botnet and attack some url.

This is really good, and I was really excited by it but then I read:

> running on a single 8XA100 40GB node in 38 hours of training

This is a $40-80k machine. Not a diss, but I would love to see an advance that would allow anyone with a high end computer to be able to improve on this model. Before that happens this whole field is going to be owned by big corporations.

Hey there!

Search in the web is becoming problematic. Quality is decreasing and competition is extremely hard.

Some time ago I came to the realisation that the biggest strength of Google might also be its Achilles heel. Google is forced to create a list of links because that's the main vehicle where they drive profit from. If you were to send a question like "Who is Barack Obama?" you still will get a list of links although google knows there is a canonical answer.

However, if you were to build a new search engine from the ground up you would need to build the infrastructure to crawl the web, crawl it, index it and build the interface. That will take a lot of money and time to test one idea. And there are multiple possible attack vectors to Google's business model (privacy, subscription model, modality, etc.). You might get the chance of testing one of them, and if that fails, starting again is super expensive so you might not be able to.

My idea is to have a single open web index database, continuously updated so that you can apply ranking and embedding algorithms to it. This would reduce the cost of entry, and enable developers to build competitors to google on top of it, or create new products in the search space (for instance, a search engine for clothes). I don't know if this is interesting for anyone but if it is, hit me up.

I faced the problem. I think one of the main issues with google is the modality of the results. Google is forced to create a list of links because that's the main vehicle where they drive profit. If you were to send a question like "Who is Barack Obama?" you still will get a list of links although google knows there is a canonical answer.

The problem is that if you were to build a new search engine from the ground up it will take millions in infrastructure, and a lot of time for you to test one idea. And there are multiple attack vectors to Google's business model (privacy, subscription model, modality, etc.) however you might get the change of testing one of them, and if that fails, starting again is super expensive so you might not be able to get funds to do it.

My approach then became to build something that others can build on top of.

I'm currently using common crawl but my main problem is that I need to build a small toy to test it and even processing common crawl is crazy expensive. Just a single snap are 150 Tb, so this needs to be process on metal, or you're gonna pay a hefty AWS bill.

I'm working on building an AWS for anyone who wants to make their own search engine. The idea is to have a single open webindex database, continuously updated that you can apply ranking and embedding algorithms in it. This would reduce the cost of entry, and enable developers to build competitors to google on top of it, or create new products in the search space like a search engine for clothes. I don't know if this is interesting for anyone but if it is, hit me up.