Big Jenkins servers of 2013

Jenkins CI server used to scale reasonably well for 2000s. That does not hold anymore. Because Jenkins was not designed initially for decade of development and for huge installations with hundreds of servers. So experience of large installations is helpful to realize Jenkins abilities.

Parameters of few large Jenkins installations are published:

Data

Details:

• Openstack / HP uses Gearman server + 2 Jenkins masters + 300 dynamic slaves handle 10k builds daily. While one master could not keep it.
• Sony Mobile has 7 independent Jenkins servers, the largest one is "Jenkins Regular @ SELD" in Lundt. One master (24 cores, 64GB RAM, 6TB disk) with 300 slaves (2..4HT cores, about 8GB RAM each) handle 6k builds daily. Sony Mobile configures 1 executor per slave.
• Yahoo Advertising Platform team has primary Jenkins master (12 HT cores, 96GB RAM, 1.2TB disk + 20TB networked storage for jobs and builds data) and 3 backup master in 2 data centers, 50 slaves in 3 data centers. It perform 8k builds per day producing 6 TB of data.
• Netfilx has their builders in Amazon cloud. 6 independent masters (AWS m2.2xlarge - 4 cores, 32GB RAM each) and 100 slaves (AWS m1.xlarge) run 4000 builds daily generating 3TB build data.

Links

1. "Maintaining Huge Jenkins Clusters - Have We Reached the Limit of Jenkins?" Robert Sandell, Sony Mobile Communications. Jenkins User Conference 2013 - Palo Alto.
   Slides: http://www.cloudbees.com/sites/default/files/juc/juc2013/2013-1023-Palo-Alto-Robert_Sandell-Maintaining-Huge-Jenkins-Clusters.pdf
   Video: http://www.youtube.com/watch?v=LRonDiXUx1U
2. "Multiple Jenkins Master Support" Khai Do, Hewlett Packard. Jenkins User Conference 2013 - Palo Alto.
   Video: http://www.youtube.com/watch?v=pLQddm85fPQ
   Slides:http://docs.openstack.org/infra/publications/gearman-plugin/
3. "To Infinity & Beyond the Small Team" James Nord, Cisco
   Slides: http://www.cloudbees.com/sites/default/files/JUC_Palo_Alto_2013_TIaBTST.pdf
   Video: http://www.youtube.com/watch?v=CGjgS16dVUc
4. Jenkins Scalability Summit notes. Jenkins Scalability Summit, Oct 2013 - Los Altos.https://docs.google.com/document/d/1GqkWPnp-bvuObGlSe7t3k76ZOD2a8Z2M1avggWoYKEs/edit#
5. "13,000 jobs and counting". Mujibur Wahab, Yahoo!. Jenkins Scalability Summit, Oct 2013 - Los Altos.
   Slides: https://wiki.jenkins-ci.org/download/attachments/68747344/Yahoo.pptx
6. "How Jenkins Builds the Netflix Global Streaming Service". Gareth Bowles, Brian Moyles, Netflix. Jenkins User Conference 2012 - San Francisco.
   Slides: http://www.cloudbees.com/sites/default/files/juc2011/JUCSF_2012_Building-Netflix-Streaming-with-Jenkins_JUC.pdf
   Video: http://www.youtube.com/watch?v=GF0p7jTf6tk