Relatively Ignorant – A blog on things about which I may or may not know stuff069ce0f8b975b6fce829d95e17bc1765bfc32407

Docker-in-Docker builds in TeamCity agents on AWS ECS

20 Feb 2019 | docker teamcity aws ecs

I have been experimenting with running TeamCity in AWS, using the CloudFormation stack provided by JetBrains. This stack uses Docker images from JetBrains and runs them in AWS Elastic Container Service.

However the default configuration does not allow Docker-in-Docker builds. This is the situation where a TeamCity agent, itself running in a Docker container, needs to run a build step with Docker or Docker Compose.

The page for the JetBrains Docker image for agents gives two options for starting the agent container from the command line:

  • Docker from the host
  • Run in privileged mode

This post is about how to modify the JetBrains CloudFormation template to start the agent container in those two ways.

Docker from the host

This technique maps /var/run/docker.sock from the Docker host into the running container. Declare a host volume and mount it in the container (see comments):

  AgentTaskDefinition:
      Type: AWS::ECS::TaskDefinition
      Condition: ShouldLaunchAgents
      DependsOn:
        - PublicLoadBalancer
        - TCServerNodeService
      Properties:
        PlacementConstraints:
          - Type: memberOf
            Expression: attribute:teamcity.node-responsibility == buildAgent
        # Define the host volume to map.
        Volumes:
          - Name: "dockerSock"
            Host:
              SourcePath: "/var/run/docker.sock"
        ContainerDefinitions:
          - Name: 'teamcity-agent'
            Image: !Join [':', ['jetbrains/teamcity-agent', !Ref 'TeamCityVersion']]
            Cpu: !Ref AgentContainerCpu
            Memory: !Ref AgentContainerMemory
            Essential: true
            Environment:
              - Name: SERVER_URL
                Value: "https://teamcity.tawh.net"
            LogConfiguration:
              LogDriver: 'awslogs'
              Options:
                awslogs-group: !Ref ECSLogGroup
                awslogs-region: !Ref AWS::Region
                awslogs-stream-prefix: 'aws/ecs/teamcity-agent'
            # Mount the host volume in the container.
            MountPoints:
              - ContainerPath: "/var/run/docker.sock"
                SourceVolume: "dockerSock"

I have used this method successfully.

Run in privileged mode

Set privileged mode in the ECS Task defintion for the agent by adding Privileged: true to the container definitions:

  AgentTaskDefinition:
      Type: AWS::ECS::TaskDefinition
      Condition: ShouldLaunchAgents
      DependsOn:
        - PublicLoadBalancer
        - TCServerNodeService
      Properties:
        PlacementConstraints:
          - Type: memberOf
            Expression: attribute:teamcity.node-responsibility == buildAgent
        ContainerDefinitions:
          - Name: 'teamcity-agent'
            Image: !Join [':', ['jetbrains/teamcity-agent', !Ref 'TeamCityVersion']]
            Cpu: !Ref AgentContainerCpu
            Memory: !Ref AgentContainerMemory
            # Run this container in privileged mode.
            Privileged: true
            Essential: true
            Environment:
              - Name: SERVER_URL
                Value: !GetAtt [PublicLoadBalancer, DNSName]
            LogConfiguration:
              LogDriver: 'awslogs'
              Options:
                awslogs-group: !Ref ECSLogGroup
                awslogs-region: !Ref AWS::Region
                awslogs-stream-prefix: 'aws/ecs/teamcity-agent'

I have not tried this method yet.

Related Posts