steveb
Joined 16 years ago
Karma
1,299
Submissions
80
Member since
Sep 8
Recent submissions
Comment · 5 years ago
Comment · 9 years ago
Before you leave, I'd love to talk to you. I'm in STL and looking for engineers to work in a professional, supportive, and inclusive environment. [email protected]
Comment · 9 years ago
Yes, we have designed it to deploy things like Kubernetes and Mesos clusters via integrations with Terraform and Packer.
Comment · 9 years ago
We've been working on a directed graph execution engine called Converge https://github.com/asteris-llc/converge.
In this case the task resource http://converge.aster.is/0.5.0/resources/task/ might help, as it allows you to create a directed graph using any kind of interpreter (for example, Python or Ruby) instead of having to use the DSL.
Comment · 10 years ago
We have started work on exposing Hashicorp Vault secrets via FUSE and Docker volumes. The expectation is your containers will just mount secrets via a mount like /secret in the container.
The project is brand new and we'd love to hear your feedback: https://github.com/asteris-llc/vaultfs
Comment · 11 years ago
Thanks for sharing this!
With the mantl project https://github.com/CiscoCloud/microservices-infrastructure we feed mesos task information into consul.
We've looked a lot a load balancing and feel that rewriting haproxy files dynamically can lead to brittle behavior.
Our current setup is we use traefik https://github.com/emilevauge/traefik to proxy marathon tasks.
Haproxy 1.6 includes some dynamic dns lookups http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/
Comment · 11 years ago
tl;dr: mesos/marathon run the tasks, consul exposes the tasks in DNS.
Both Mesos and Marathon store different views of task state on the cluster.
How do we use this data to make it easy for jobs to find one another? Can we use this information to automatically configure things like Load balancers?
This is where tools like consul and mesos-dns come in. They populate a DNS store, so that task-name becomes something like task-name.example.com. If you are running 10 copies of a service across different hosts, DNS will have 10 entries.
If a container moves to another system, DNS is updated on the fly.
We can embed health checks with consul, so that if a service is unhealthy, it gets pulled out of DNS.
Consul also is nice because the edges perform the checks (instead of a central server), so the load is distributed.
Comment · 11 years ago
Thanks for bringing this up. We opened an issue to investigate: https://github.com/CiscoCloud/microservices-infrastructure/i...
Comment · 11 years ago
Why you should care about this project:
You wouldn't spend months building your own Linux distribution, so why spend time reinventing the wheel with chef/puppet/ansible?
We're not building a PaaS. Like a Linux distribution, we're integrating open source building blocks (logging, service discovery, scheduling) so that you can focus on app development and analytics instead of sysadmin tasks.
I'll be giving a talk to the NYC Mesos user group this Wednesday June 17th: http://www.meetup.com/Apache-Mesos-NYC-Meetup/events/2229328...
Comment · 11 years ago
KONG looks amazing, I haven't heard of it before.
Here's how you would deploy it in microservices-infrastructure:
1. You'd deploy the cassandra mesos framework, giving you HA cassandra. Instead of setting IPs, you'd connect to this as cassandra.service.consul
2. You'd launch a Kong container in marathon. It would show up in dns as kong.service.consul for other apps to find, so you don't have to hard code IPs in your config.
Edit: We've opened an issue to make this an example app https://github.com/CiscoCloud/microservices-infrastructure/i...
Comment · 11 years ago
We like Kubernetes (and are looking to add it to our project), but our goal is to integrate building blocks that allows us to run many different types of workloads. Think of our project more like an Ubuntu for distributed systems.
Kubernetes may eventually spread out beyond Docker, but for today we need to support things like Kafka and Spark.
As others have noted, we've had things like CloudFoundry, OpenShift and Heroku, and these all-in-one frameworks tend not to extend outside their original domain.
Comment · 11 years ago
The real problem is going from tutorial to something you would use in production. Throw in logging, security and service discovery and you can have a few engineers hacking away for months.
So I want to plug a project I've been contributing to: https://github.com/CiscoCloud/microservices-infrastructure
We're trying to make it super easy to deploy these tools. For example every time you launch a docker container, it will register with consul and be added to haproxy. The nice thing about using Mesos is we can support data like workloads Cassandra, HDFS, and Kafka on the same cluster your run Docker images on.
We use terraform to deploy to multiple clouds so you don't get locked in to something like cloudformation.
<https://crossplane.io> just graduated to CNCF Incubation and each of the cloud providers are working on K8s controllers and code generators (like Amazon Controllers for Kubernetes, Google Config Connector, and the Azure service operator).