Jenkins is one of the most widely used continuous integration tools in DevOps stacks worldwide. One of its key features is scalability through a master-agent topology, where the stages of jobs run on separate agents while the Jenkins master operates with minimal resources.

This solves two common problems with a single server: overload during peaks of many executions, and wasted resources when there is no activity (for example, overnight). But another challenge appears: if you have several agents, how do you keep the infrastructure efficient and avoid piling up idle EC2 instances?

AWS offers Amazon EC2 Spot Instances, which can save up to 90% in costs compared to on-demand instances. They are perfect candidates for Jenkins agents: they are created only when a job needs to run and are destroyed automatically when they are no longer needed.

Cost comparison for a Jenkins setup that runs jobs for 4 hours a day:

ConfigurationMonthly cost
Spot fleet t3.medium (4 h/day)$2.76
On-demand t3.small (24×7)$30.37

In this tutorial we will build a master-agent architecture using an EC2 Spot Fleet Request for Windows workloads.

Architecture

We have an EC2 instance with Jenkins installed, with an IAM role to list and manage the Spot Fleet Request. The Spot Fleet scales automatically as jobs run, and can drop to 0 instances when no job is running.

To implement the architecture, we first create a base AMI that will be used in the Spot Fleet Request.

1. Creating the base AMI

We need an AMI with the software required to act as Jenkins agents.

Launch an EC2 instance based on Windows Server 2019 with containers (so you can use Docker inside the deployment pipelines). In the security groups, open ports 3389 (RDP) and 22 (SSH) for connectivity.

Create an administrator user

Jenkins will use this user to connect to the agents:

  1. Connect to the instance via RDP.
  2. Open lusrmgr.msc (Local Users and Groups).
  3. Right-click Users → New User.
  4. Name: jenkins, with a strong password. Uncheck “User must change password at next logon”. Check “Password never expires” and “User cannot change password”.
  5. Right-click the user you created → Properties → Member of tab → Add → Administrators group.

Install OpenSSH

We need OpenSSH to allow the master-agent connection over SSH:

Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
Start-Service sshd
Set-Service -Name sshd -StartupType 'Automatic'

Validation:

Get-WindowsCapability -Online | ? Name -like 'OpenSSH*'

Install Java and Git

Jenkins agents need Git and Java. We install them with Chocolatey:

[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"
Set-ExecutionPolicy RemoteSigned -Confirm:$false
iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex

choco install javaruntime -y
choco install git.install -y

Create the AMI

In the AWS console, select the instance and go to Actions → Image → Create Image to generate the image.

2. Generating the Spot Fleet Request

Once the AMI is created, we create a Spot Fleet Request using a CloudFormation template.

If you need the CloudFormation template we used in this case, write to us at /contact?servicio=devops and we’ll share it with you.

In the parameters, enter:

  • The AMI generated in the previous step.
  • The VPC and security group of the Jenkins master.
  • The preferred instance size.
  • An SSH key for access to the agents (if needed).

3. Configuring Jenkins

To use the Spot Fleet, install the EC2 Fleet Jenkins Plugin.

  1. Jenkins configuration (/configure): in the Cloud section, Add a new cloud → EC2 Fleet.

  2. Enter a Name, select the region of the Spot Fleet Request, and select the Spot Fleet you created.

  3. Launcher: create credentials with the jenkins user and the password configured in the AMI.

  4. Host Key Verification Strategy: select Non verifying Verification Strategy.

  5. Advanced → Prefix Start Agent Command:

    powershell -Command "cd C:\Users\jenkins ; java -jar remoting.jar" ; exit 0 ; # '

  6. Suffix Start Agent Command:

    '

  7. Check Connect using private IP.

  8. Define a label to use in the Jenkinsfile (for example ec2-fleet).

  9. Jenkins Filesystem Root: C:\Users\jenkins.

  10. Minimum Cluster Size: 0.

4. Validation with a test pipeline

Create a new Pipeline-type Job and use the following script:

pipeline {
    // Label of the Spot Fleet configured in the previous step
    agent { node { label 'ec2-fleet' } }

    stages {
        stage('Build') {
            steps {
                echo 'Build'
            }
        }
        stage('Test') {
            steps {
                script {
                    docker.image('hello-world').inside() {
                        echo 'example inside docker container'
                    }
                }
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploy'
            }
        }
    }
}

When the pipeline runs, the Jenkins home screen shows the status of the instance fleets, indicating for each Spot Fleet Request the current number of instances and the target instances.

Conclusion

With this setup, your fleet of spot instances is ready to distribute your pipeline workload automatically, scaling to zero when there is no activity. Savings versus on-demand are typically 80-90% on variable workloads like CI/CD.

A few operational considerations for production:

  • Define a mix of instance types in the Spot Fleet to tolerate spot capacity interruptions.
  • Use Spot Fleet with an On-Demand baseline for critical jobs that cannot tolerate interruption.
  • Monitor the Spot Fleet’s fulfillment and interruption rate metrics in CloudWatch.
  • Consider modern alternatives such as CodeBuild Managed Runners if your CI runs on GitLab — more on this in GitLab + AWS CodeBuild Managed Runners.

How we apply this at Caleidos

Reductions of 80-90% in CI/CD infrastructure are common among clients who migrate from on-prem Jenkins to this architecture. The continuous FinOps piece goes further: automatic rightsizing, an optimal on-demand/spot mix per workload, and fulfillment observability.

Caleidos designs, implements, and operates modern DevOps platforms as part of our DevOps & Automation service and FinOps & AWS Cost Optimization. 24×7 operation included with Caleidos Lens©.

Want to cut your CI/CD bill? Let’s talk →