Erlang Factory SF Bay 2015 – Mark Anderson – Push Jobs: A Scalable Remote Execution System for Chef
When Chef needed a scalable system for remote job execution on thousands of systems simultaneously, we chose Erlang and ZeroMQ for the job. This talk will cover:
* How we applied the ‘let it crash’ philosophy across a system of thousands of nodes.
* The various impedance mismatches between the ZeroMQg and Erlang approaches to distributed systems.
* The various bottlenecks we hit as we scaled our system, and how we overcame them.
- To describe our experiences scaling an zeromq application written in erlang, along with our missteps and unexpected successes.
- Anyone who is interested in building systems with ZeroMQ and Erlang