armon

Joined 15 years ago

Karma

176

Submissions

Member since

Jul 12

Co-Founder/CTO of HashiCorp by day (http://www.hashicorp.com/)

Twitter: http://twitter.com/armon Github: http://github.com/armon Personal Site: http://armondadgar.com/

View on Hacker News

Recent submissions

Comment · 8 months ago

Lock contention is a real issue for any multi-threaded system, and while a RW mutex is useful when you have a longer executing critical section, for something very short lived there is still a cache coordination cost. In many of the HashiCorp applications, we work around this by using an immutable radix tree design instead [1].

Instead of a RW mutex, you have a single writer lock. Any writer acquires the lock, makes changes, and generates a new root pointer to the tree (any update operation generates a new root, because the tree is immutable). Then we do an atomic swap from the old root to the new root. Any readers do an atomic read of the current point in time root, and perform their read operations lock free. This is safe because the tree is immutable, so readers don't need to be concerned with another thread modifying the tree concurrently, any modifications will create a new tree. This is a pattern we've standardized with a library we call MemDB [2].

This has the advantage of making reads multi-core scalable with much lower lock contention. Given we typically use Raft for distributed consensus, you only have a single writer anyways (e.g. the FSM commit thread is the only writer).

We apply this pattern to Vault, Consul, and Nomad all of which are able to scale to many dozens of cores, with largely a linear speedup in read performance.

[1] https://github.com/hashicorp/go-immutable-radix [2] https://github.com/hashicorp/go-memdb

Comment · 6 years ago

We use HCL broadly across the HashiCorp portfolio. It was popularized by Terraform, but also used in Vault, Consul, Nomad, and Packer.

Important to note, that HCL is designed to be completely interoperable with JSON. In practice, we find that HCL tends to be more human readable and writable and is significantly less verbose.

However, if you are doing any sort of machine generation, it can be convenient to generate JSON and feed that in.

Comment · 6 years ago

We've continued to grow the Nomad team, and are working towards the big 1.0 release milestone later this month. The OSS usage continues to grow double digit every quarter and our commercial offering generates millions in revenue. Nomad is also the backbone of both HashiCorp Cloud Platform and Terraform Cloud. Suffice to say, we continue to support Nomad and depend on it!

Comment · 6 years ago

You would point the application at the local port. It operates very similarly to SSH port forwarding. No fancy magic to intercept all traffic.

Comment · 6 years ago

We don't have a white paper on this yet, but we have a white board video that explains both how it works conceptually as well as at a more technical level of deployment architecture and data flow. https://www.youtube.com/watch?v=tUMe7EsXYBQ&feature=emb_titl...

Comment · 6 years ago

That is correct! The local proxy has a listening socket and handles all the authentication, encapsulation, and forwarding transparently.

Comment · 6 years ago

Disclosure: I'm one of the founders of HashiCorp.

Glad you found Consul easy to get going with. We definitely focus on enabling K8S as a first class platform. There is documentation on setting up Consul and operating with Kubernetes here: https://www.consul.io/docs/platform/k8s/index.html

We also have a bunch of learning content and guides on using Consul and K8S together to get started: https://learn.hashicorp.com/consul?track=kubernetes#kubernet...

Comment · 7 years ago

HashiCorp (https://www.hashicorp.com) | SRE, Software Engineers, Engineering Managers | REMOTE | Full-time

HashiCorp builds DevOps tools for modern applications. We build open source tools including Vagrant, Packer, Terraform, Consul, Vault, and Nomad. We also have a set of cloud services (Vagrant Cloud, Terraform Registry, Terraform Cloud, etc) that are high scale SaaS services. The HashiCorp products are downloaded tens of millions of times per year and power tens of thousands of organizations. Join our growing team and help organizations deliver modern applications using DevOps practices and enable dynamic multi-cloud infrastructure.

For a complete list of openings, see https://www.hashicorp.com/jobs

Some of our openings include:

* Infrastructure Engineer (multiple openings): https://www.hashicorp.com/jobs/1807978

* Software Engineer, Vault/Consul/Nomad- https://www.hashicorp.com/jobs/1844783

* Software Engineer, Cloud Services- https://www.hashicorp.com/jobs/1735894

* Site Reliability Engineer, Cloud Services- https://www.hashicorp.com/jobs/1600002

* Engineering Manager, Vault/Consul/Nomad- https://www.hashicorp.com/jobs/1845061

* Infrastructure Engineering Manager, Terraform Cloud- https://www.hashicorp.com/jobs/1801371

At HashiCorp, we are committed to hiring and cultivating a diverse team. If you are on the fence about whether you meet our requirements, please apply anyway!

Comment · 7 years ago

HashiCorp (https://www.hashicorp.com) | SRE, Software Engineers, Engineering Managers | REMOTE | Full-time

For a complete list of openings, see https://www.hashicorp.com/jobs

Some of our openings include: Software Engineer, Terraform Cloud- https://www.hashicorp.com/jobs/1788064 Software Engineer, Systems- https://www.hashicorp.com/jobs/1588975 Software Engineer, Cloud Services- https://www.hashicorp.com/jobs/1620078 Site Reliability Engineer, Cloud Services- https://www.hashicorp.com/jobs/1600027 Engineering Manager, Systems- https://www.hashicorp.com/jobs/1625448

At HashiCorp, we are committed to hiring and cultivating a diverse team. If you are on the fence about whether you meet our requirements, please apply anyway!

Comment · 7 years ago

I gave a talk at GoSF about Radix trees, and how they are used heavily in HashiCorp products (Terraform, Consul, Vault, Nomad, etc). The slides are available here for those interested: https://speakerdeck.com/armon/radix-trees-transactions-and-m...

Radix trees are one of my favorite data structures, and widely under used. For many "dictionary" type lookups, they can be faster and more efficient than hash tables. While hash tables are commonly described as being O(1) for lookup, this ignores the need to first hash the input, which is typically an O(K) operation, where K is the length of the input string. Radix trees do lookups in O(K), without needing to first hash and have much better cache locality. They also preserve ordering allowing you to do ordered scans, get min/max values, scan by shared prefix, and more.

If you combine them with an immutable approach (such as https://github.com/hashicorp/go-immutable-radix), you can support more advanced concurrency, such as lock free reads. This is important for highly concurrent systems to allow scalable reads against shared memory.

Comment · 8 years ago

Hey, I'm one of the co-founders of HashiCorp.

To keep it brief, we are more committed to Nomad today than before. The team has doubled in the last year, and we plan to grow further next year as well. Our goal has always been to build a simple, general purpose scheduler, that composes well with the rest of the HashiCorp ecosystem.

Kubernetes is an important ecosystem and a platform we tightly integrate with across our other products (Terraform, Consul, Vault). We've always believed that our tools would be "mixed and matched" with different technologies, and that pragmatically we should support the broadest range of integrations.

Nomad is an important piece of our ecosystem, and we have many open source users, enterprise customers, and our SaaS offerings are built on it. Rest assured, it's not going anywhere!

Comment · 8 years ago

That is exactly right. At the bottom of the blog post we touch on this, but if you are using consul-template to provide secrets via a configuration file, it can either restart or reload (signal) the application to pickup the changes. Alternatively, an application could be Vault aware and use the SDK programmatically.

Comment · 8 years ago

Hey peteski22,

Exactly what you suggested would work! Having an AppRole that never expires would allow the trusted orchestrator to authenticate on each run, and then generate and inject ephemeral credentials.

Comment · 8 years ago

Vault is free and open source. We do have an enterprise product, but all the dynamic secrets capabilities exist in the open source!

Comment · 8 years ago

There are some good suggestions in the replies here, but I'd recommend looking at our secure introduction guide here: https://www.vaultproject.io/guides/identity/secure-intro.htm...

Effectively, there are only two possible approaches. Platform specific integrations, such as AWS IAM, Kubernetes, Nomad, etc. Or Trusted Orchestrators, which help inject the initial "secret zero".