xjia

Before this we were using https://github.com/actions/actions-runner-controller but that's running on K8s instead of VMs. So along with common limitations of running CI jos in K8s/container, it cannot have exactly the same environment as the official GitHub runners. Maintaining a K8s cluster was also very difficult.

Comment · 2 years ago

Persistent disks are implemented as EBS snapshots, so the process is something like:

1. Create EC2 instance for runner #1. Find out there is no existing snapshot, so an empty volume is created and attached.

2. Runner #1 runs exactly 1 job and shuts down. A snapshot is taken for the persistent volume. That's going to be used by later runners.

3. Create EC2 instance for runner #2. Create a new volume based on the last snapshot.

4. Assuming #2 is still running while a new job comes in. Create EC2 instance for runner #3. Create volume based on the same last snapshot.

5. Whenever a runner finishes, its persistent volume gets a snapshot taken. Outdated snapshots are automatically removed.

And yes we manage the AMI that the runner uses. We try out best to follow https://github.com/actions/runner-images and will automate this process very soon so it's always up-to-date.

Edit: formatting

Show HN: DimeRun v2 – Run GitHub Actions on AWS EC2 github.com

5 xjia 2 years ago 6 comments

Comment · 2 years ago

We are working on an alternative that's 100% compatible with GHA but much cheaper and faster. Check out https://dime.run/

Comment · 2 years ago

From [0]:

> gShoe is an internal joke about a product [Google] shouldn't make, and is used in many internal trainings as a placeholder for any Google product

[0]: https://www.reddit.com/r/google/s/fHCBJipqdP

Comment · 2 years ago

IIUC the actual computation (e.g. compiling, linking, ...) happens on client (CI or developer) machines and the results are written to the server-side cache.

By spoofing I meant to say that an authenticated but malicious client (intentionally or not, e.g. a clueless intern) may be able to write malicious contents to the cache. For example, their build toolchain could be contaminated and the resulting build outputs are contaminated. The "action" per se and its hash is still legit, but the hash is only used as the lookup key -- their corresponding value is "spoofed."

The only safe way I can imagine to use such a remote cache is for CI to publish its build results so that they could be reused by developers. The direction from developers to developers or even to CI seems difficult to handle and has less value. But I might be missing some important insights here so my conclusion could be wrong.

But if that's the case, is the most valuable use case to just configure the CI to read from / write to the remote cache, and developers to only read from the remote cache? And given such an assumption, is it much easier to design/implememt a remote cache product?

Comment · 2 years ago

What is the benefit of using a remote cache instead of a local ~/.cache directory? Is it only for sharing build results among team members? How do you make sure the build results are not spoofed?

Comment · 2 years ago

Awesome! If there is any problem just post it in GitHub issues. We will also be releasing 2023.4 very soon.

Comment · 2 years ago

Yes, the code of the analyzer itself cannot be built on macOS or Windows directly. In those cases you will have to use either podman or docker to use the binaries prebuilt on/for Linux. The whole system depends on a lot of other stuff on Linux so it won't easily run on other OSes.