Deploy from Gitlab to AWS EC2

- 9 mins

Gitlab to AWS diagram

Deploy from Gitlab to AWS

Nowadays everything is hosted in a cloud which make sense. Anybody can create environment from scratch in a blink of an eye, cloud provides flexibility and scalability, cloud providers make sure you have plenty of choice in terms of resoruces and they take over more and more maintenance duties from you. While all this is true, it is common to leverage cloud and at the same time use what you already been using until now.

In this blog post, I’ll show you how to use Gitlab to deploy application on AWS.

We will provision EC2 instance in AWS, install NGINX and check that defult NGINX webpage is accessible and visible in web browser. Afterwards we will use CI/CD pipeline combining Gitlab CI and AWS Codedeploy to deploy another web page in the place of default NGINX web page.

Prerequisite

Agenda

  1. Create Gitlab Pages/Plain HTML project
  2. Create AWS resources
  3. Create programmatic user in AWS
  4. Create pipeline and deployment package
  5. Summary

Create Gitlab Pages/Plain HTML project

Let’s use one of the available templates to create new Gitlab project. Click New project button and then From template -> Pages/Plain HTML.
This will create new project with example html and css file.

Create AWS resources

Now let’s setup necessary resources in AWS. You can choose two ways to provision resources: Cloudformation template or manual provision in AWS console. Luckly for you, I have already prepeared Cloudformation template which will provision following resources:

Go to Cloudformation -> Create Stack -> With new resources (standard). Choose Template is ready and upload the template.

Create stack from existing template

Create stack from existing template

Next set parameters defined in Cloudformation template.

Stack Parameters

Remember to tick checkbox for Capabilities which is necessary in case you have IAM role defined in template.
Stack Capabilities

Stack Capabilities

Stack creation should take around 4 mins. Afterwards all mentioned resources should be already available. You should see last event with logical ID of your stack name in status CREATE_COMPLETE.
Stack Events

Stack Events

Go to Resoruce tab, find EC2 instance and click on hyperlink which will take you to instance view. Mark the EC2 instance and click connect.
EC2 Connect

EC2 Connect

Then open Session Manager tab. The connect button should be active thanks to Cloudformation template which has handled enabling Session Manager connection. Session Manager connection allows to securly connect without SSH keys or bastion host to EC2 instance.

Cloudformation template has done two steps to enable Session Manager connect:

EC2 Session Manager

Connect to EC2 and run following commands to check that Codedeploy and Nginx are installed and running:

sudo su - ec2-user
systemctl status nginx
systemctl status codedeploy-agent.service

EC2 Connect

EC2 Commands


Check that default Nginx web page is accessible using EC2 Public IP address. Remember to copy/paste address. Using open address URL will default to https protocol, but Nginx page is available on http protocol. EC2 Public IP

EC2 Public IP


You should see Nginx defult welocome web page. Nginx defualt web page

Nginx default web page

We also needed Codedeploy Application and Deployment Group which were created by stack. Naviagte to Codedeploy -> Deploy -> Applications and you should see defined application.
Codedeploy Application

Codedeploy Application

Click on Application Name, then on Deployment Group Name to check the details such as IAM Role assigned and which EC2 instances are targeted by the deployment group.
Codedeploy Deployment Group

Codedeploy Deployment Group

Both Gitlab and AWS are done and ready. The last part is to integrate Gitlab with AWS to replace default Nginx page with Gitlab page created from Plain/HTML Pages template.

Create programmatic user in AWS

We need to have a way to acccess AWS resources directly from Gitlab CI pipeline. We do that using AWS programmatic user.

Go to IAM -> Users -> Add Users. Name the user and make sure you have checked Programmatic access checkbox. Then Attach exisiting policies directly and choose AmazonS3FullAccess and AWSCodeDeployDeployerAccess.

WARNING: Those permissions give more access than necessery. For the real world scenarios please adjust permissions for your needs.

Create the user and note down Access Key ID and Secret Access Key. They can’t be retrived again. Programmatic User Keys

Programmatic User Keys

Setup programmatic user in Gitlab

AWS part is done. We can now switch back to Gitlab.
Gitlab allows us to use AWS programmatic user in pipeline. Configuring the user on Gitlab side comes down to adding 3 variables.

Go to Settings -> CI/CD -> Variables.
Add variables:

Gitlab AWS Variables

Gitlab AWS Variables

Create pipeline and deployment package

Last missing piece is the Gitlab pipeline.

Default .gitlab-ci.yml comes with project template. We need to adjust it for our case, but before that we need to add few things to make our deplyment artifatcs compatible with AWS Codedeploy. Codedeploy is expecting a ZIP file containing:

Deployment artifacts include two files: html and css stored in public directory. appspec.yml we need to create and place in public directory. Create it and copy the code:

version: 0.0
os: linux
files:
  - source: /index.html
    destination: /usr/share/nginx/html/
  - source: /style.css
    destination: /usr/share/nginx/html/
hooks:
  BeforeInstall:
    - location: scripts/remove.sh
      timeout: 300
      runas: root

files section defines what files (source) should be copied to what destination.
hooks seciton define what script should be run in which stage. We only run script to remove old web page from Nginx before deploying artifacts.

Then create a remove.sh script as defined in appspec.yml in public/scripts/remove.sh and copy the code:

#!/bin/bash
rm -rf /usr/share/nginx/html/*

Finaly update .gitlab-ci.yml with the code:

image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest

pages:
  stage: deploy
  script:
  - aws deploy push --application-name myApp --s3-location s3://namek999/artifacts.zip --source public/
  - aws deploy create-deployment --application-name myApp --deployment-group-name myApp-DeploymentGroup --s3-location bucket=namek999,bundleType=zip,key=artifacts.zip
  artifacts:
    paths:
    - public
  only:
  - master

Few changes were made:

Commit .gitlab-ci.yml and go to CI/CD -> Pipelines. The pipeline will be trigged automatically after commit.
After it is done, go back to AWS Codedeploy -> Deploy -> Deployments and you should see completed deployment.

Codedeploy Deployment

Codedeploy Deployment

Everything looks ok, then check what is now being served by Nginx. Navigate to EC2 public IP in web browser and you shoudl now see different web page.

Gitlab Page

Gitlab Page

Summary

We have created pipeline which inetgeared Gitlab with AWS deployment. For successful deployment from Gitlab to AWS you need:

Thanks for reading!

Karol Filipczuk

Karol Filipczuk

DevOps and Cloud

comments powered by Disqus
rss facebook twitter github gitlab youtube mail spotify lastfm instagram linkedin google google-plus pinterest medium vimeo stackoverflow reddit quora quora