PaulScotti
Joined 5 years ago
Karma
257
Submissions
18
Member since
Feb 21
https://paulscotti.github.io/
View on Hacker NewsRecent submissions
Comment · 2 years ago
Thanks for sharing! I read your README and think it's a very interesting research path to consider. I wonder if such an adversarial game approach could be outfitted to not just well-defined games but to wholly generalizable improvements. e.g., could be used as a way to improve RLAIF potentially?
Comment · 2 years ago
To anyone still finding the game slow due to traffic, you can just git clone the game, add your ANTHROPIC API key to a .env file, and play it locally (this is explained in the README in our github repo). It runs super fast if played locally.
Comment · 2 years ago
Thanks! Yes the traffic is making the game slow... To anyone impatient you can just git clone the game, add your ANTHROPIC API key to a .env file, and play it locally. It runs super fast if played locally.
Creating a service would be amazing but seems like too much work. And people can already create their own murder mystery with this codebase by just modifying the characters.json file.
Making this game gave me some fun ideas though for creating a world simulation engine--any developers who might be interested in collaborating on something like that please get in touch :)
Comment · 2 years ago
Yeah sorry, it is still quite slow due to the traffic. It'd be much faster and robust to run locally via git cloning the repo and adding your own API key as shown in the README
For using other models it should be pretty straightforward to just modify the api functions to suit whatever model is being used -- would be fun to try out custom models! (Feel free to pull request the repo btw if you do modify such things)
An idea we had initially was actually to use an open-source model and fine-tune it using the DB of responses (including the hidden violation bot and refinement bot outputs) collected from people playing the game. That way the game could get better and better over time as more user data gets collected.
Disclaimer we actually did implement this via postgres and now have thousands of responses from players in case anyone wants to follow through on this idea.
Comment · 2 years ago
Yeah nice idea -- does sound plausible and would make things much cheaper and faster.
Comment · 2 years ago
The Officer doesn't actually get supplied information on the true killer in their context window... so that response you got is actually incorrect.
You can check the actual solution by clicking the End Game button
Comment · 2 years ago
Upgraded the server and should now be working... I think
Comment · 2 years ago
Should be working now and way faster! Had to upgrade the server to increased number of workers
Comment · 2 years ago
Haha interesting approach!
Comment · 2 years ago
Damn that sucks, sorry. For what it's worth I tried playing the game dozens always asking for an overview as my first message and I never encountered such a response , so hopefully that's quite the rare experience.
Comment · 2 years ago
Wow really, can you tell me what you said to get them to confess?
Comment · 2 years ago
Sharing a little open-source game where you interrogate suspects in an AI murder mystery. As long as it doesn't cost me too much from the Anthropic API I'm happy to host it for free (no account needed).
The game involves chatting with different suspects who are each hiding a secret about the case. The objective is to deduce who actually killed the victim and how. I placed clues about suspects’ secrets in the context windows of other suspects, so you should ask suspects about each other to solve the crime.
The suspects are instructed to never confess their crimes, but their secrets are still in their context window. We had to implement a special prompt refinement system that works behind-the-scenes to keep conversations on-track and prohibit suspects from accidentally confessing information they should be hiding.
We use a Critique & Revision approach where every message generated from a suspect first gets fed into a "violation bot" checker, checking if any Principles are violated in the response (e.g., confessing to murder). Then, if a Principle is found to be violated, the explanation regarding this violation, along with the original output message, are fed to a separate "refinement bot" which refines the text to avoid such violations. There are global and suspect-specific Principles to further fine-tune this process. There are some additional tricks too, such as distinct personality, secret, and violation contexts for each suspect and prepending all user inputs with "Detective Sheerluck: "
The entire project is open-sourced here on github: https://github.com/ironman5366/ai-murder-mystery-hackathon
If you are curious, here's the massive json file containing the full story and the secrets for each suspect (spoilers obviously): https://github.com/ironman5366/ai-murder-mystery-hackathon/b...