How to Launch, Monitor, and Scale EC2 Instances Using AWS Console and CLI - A Practical Guide for DevOps Learners
Learn how to launch, monitor, and scale EC2 instances using AWS Management Console, CloudWatch, and Auto Scaling
Are you a DevOps learner who wants to test your knowledge on AWS by performing some hands-on tasks? If yes, then this blog post is for you. In this post, I’ll show you how to launch, monitor, and scale EC2 instances using AWS Console and CLI. You’ll learn how to:
Launch an EC2 instance using the AWS Management Console and connect to it using SSH
Install a web server on the EC2 instance and deploy a simple web application
Monitor the EC2 instance using Amazon CloudWatch and troubleshoot any issues that arise
Create an Auto Scaling group using the AWS Management Console and configure it to launch EC2 instances in response to changes in demand
Use Amazon CloudWatch to monitor the performance of the Auto Scaling group and the EC2 instances and troubleshoot any issues that arise
Use the AWS CLI to view the state of the Auto Scaling group and the EC2 instances and verify that the correct number of instances are running
By the end of this post, you’ll be able to test your knowledge on AWS by performing some practical tasks that are relevant for DevOps learners. You’ll also gain more confidence and experience in using AWS services and tools.
This blog post is going to be lengthy and detailed, as I will explain each step of the tasks and provide screenshots and code snippets to help you follow along. I will also provide some tips and best practices to help you avoid common pitfalls and errors. I recommend that you have some basic knowledge of AWS and DevOps before reading this blog post, as I will not cover the fundamentals of these topics. If you are new to AWS and DevOps, you can check out my previous blogs to learn more.
How to Launch an EC2 Instance Using the AWS Management Console and Connect to It Using SSH
In this section, I’ll show you how to launch an EC2 instance using the AWS Management Console and connect to it using SSH. An EC2 instance is a virtual server that runs in the AWS cloud. You can use EC2 instances to run applications, host websites, or perform any other tasks that require computing power.
Step 1: Choose an Amazon Machine Image (AMI)
To launch an EC2 instance, you need to choose an Amazon Machine Image (AMI), which is a template that contains the operating system and software configuration for your instance. You can choose from a variety of AMIs provided by AWS or other sources, or you can create your own AMI.
For this task, I’ll choose the Amazon Linux 2 AMI, which is a Linux-based AMI that is optimized for AWS. To choose an AMI, follow these steps:
Go to the AWS Management Console and select EC2 from the Services menu.
Click on Launch Instance to start the launch wizard.
On the Choose an Amazon Machine Image (AMI) page, find the Amazon Linux 2 AMI and click on Select.
Step 2: Choose an Instance Type
Next, you need to choose an instance type, which is a specification of the CPU, memory, storage, and networking capacity for your instance. You can choose from a variety of instance types depending on your needs and budget.
For this task, I’ll choose the t2.micro instance type, which is a general-purpose instance type that is eligible for the free tier. To choose an instance type, follow these steps:
- On the Choose an Instance Type page, find the t2.micro instance type and click on Next: Configure Instance Details.
Step 3: Configure Instance Details
Next, you need to configure some details for your instance, such as the number of instances, the network, the subnet, the security group, and the key pair. You can use the default settings for most of these options, or you can customize them according to your preferences.
For this task, I’ll use the default settings for most of the options, except for the following:
I’ll enable Auto-assign Public IP, which will assign a public IP address to my instance so that I can connect to it from the internet.
I’ll create a new security group, which is a set of firewall rules that control the inbound and outbound traffic for my instance. I’ll name it WebServerSG and I’ll allow SSH access from anywhere and HTTP access from anywhere. This will allow me to connect to my instance using SSH and to access my web application using a web browser.
I’ll create a new key pair, which is a pair of cryptographic keys that are used to authenticate my connection to my instance. I’ll name it WebServerKP and I’ll download the private key file to my local machine. I’ll need this file later to connect to my instance using SSH.
To configure the instance details, follow these steps:
On the Configure Instance Details page, scroll down to the Network & Security section and check the box for Auto-assign Public IP.
Scroll down to the Security Group section and select Create a new security group. Enter WebServerSG as the Security group name and Security group for web server as the Description.
Click on Add Rule and select SSH from the Type dropdown menu. Leave the Source as Anywhere. This will allow SSH access from any IP address.
Click on Add Rule again and select HTTP from the Type dropdown menu. Leave the Source as Anywhere. This will allow HTTP access from any IP address.
Click on Review and Launch to proceed to the next page.
On the Review Instance Launch page, click on Launch to launch your instance.
A dialog box will appear asking you to select a key pair. Select Create a new key pair from the Choose an existing key pair or create a new key pair dropdown menu. Enter ec2-connect as the Key pair name and click on Download Key Pair. Save the ec2-connect.pem file to your local machine. You’ll need this file later to connect to your instance using SSH.
Click on Launch Instances to launch your instance.
Step 4: Connect to Your Instance Using SSH
Next, you need to connect to your instance using SSH, which is a secure protocol that allows you to access a remote server using a command-line interface. To connect to your instance using SSH, you need the following:
The public IP address of your instance, which you can find in the EC2 console under the Instances section.
The private key file that you downloaded in the previous step, which you need to store in a secure location and change its permissions to make it readable only by you.
A SSH client, which is a software that allows you to connect to a remote server using SSH. You can use any SSH client that you prefer, such as PuTTY, Terminal, or Git Bash.
For this task, I’ll use Git Bash as my SSH client, which is a command-line tool that comes with Git, a version control system. To connect to your instance using SSH, follow these steps:
Open Git Bash and navigate to the folder where you saved the ec2-connect.pem file. You can use the cd command to change directories and the ls command to list the files in the current directory.
Change the permissions of the ec2-connect.pem file to make it readable only by you. You can use the chmod command to change the permissions. The command is:
chmod 400 ec2-connect.pem
- Connect to your instance using SSH. You can use the ssh command to connect to a remote server using SSH. The command is:
ssh -i ec2-connect.pem ec2-user@<public-ip-address-of-your-instance>
Replace <public-ip-address-of-your-instance> with the actual public IP address of your instance, which you can find in the EC2 console. For example, if your instance’s public IP address is 54.123.456.789, the command is:
ssh -i ec2-connect.pem ec2-user@54.123.456.789
The first time you connect to your instance, you may see a warning message asking you to confirm the identity of the host. Type yes and press Enter to continue. This will add the host to your list of known hosts and prevent the warning from appearing again.
You should see a welcome message and a command prompt that indicates that you are connected to your instance. The command prompt should look something like this:
[ec2-user@ip-172-31-20-166 ~]$
Congratulations, you have successfully launched an EC2 instance using the AWS Management Console and connected to it using SSH. You can now run commands on your instance and install software on it.
How to Install a Web Server on Your EC2 Instance and Deploy a Simple Web Application
In this section, I’ll show you how to install a web server on your EC2 instance and deploy a simple web application. A web server is a software that handles HTTP requests and serves web pages or files to clients. A web application is software that runs on a web server and provides dynamic or interactive functionality to users.
For this task, I’ll use Apache as my web server and Flask as my web application framework. Apache is a popular and open-source web server that can run on various operating systems. Flask is a lightweight and easy-to-use web application framework that is written in Python.
Step 1: Install Apache on Your EC2 Instance
To install Apache on your EC2 instance, you need to run some commands on your instance using SSH. To install Apache, follow these steps:
Connect to your instance using SSH, as shown in the previous section.
Update the package manager on your instance by running the following command:
sudo yum update -y
- Install Apache by running the following command:
sudo yum install httpd -y
- Start Apache by running the following command:
sudo systemctl start httpd
- Enable Apache to start automatically when your instance boots by running the following command:
sudo systemctl enable httpd
- Verify that Apache is running by running the following command:
sudo systemctl status httpd
You should see a message that says active (running).
Step 2: Deploy a Simple Web Application on Your EC2 Instance
Next, you need to deploy a simple web application on your EC2 instance. For this task, I’ll use Flask, which is a web application framework that is written in Python. To deploy a simple web application on your EC2 instance, you need to do the following:
Install Python and pip on your EC2 instance
Install Flask and other dependencies on your EC2 instance
Create a simple web application using Flask
Configure Apache to serve your web application
Step 2.1: Install Python and pip on Your EC2 Instance
To install Python and pip on your EC2 instance, you need to run some commands on your instance using SSH. Python is a programming language that is widely used for web development. pip is a package manager that allows you to install and manage Python packages. To install Python and pip, follow these steps:
Connect to your instance using SSH, as shown in the previous section.
Install Python by running the following command:
sudo yum install python3 -y
- Verify that Python is installed by running the following command:
python3 --version
You should see a message that shows the version of Python, such as Python 3.7.16.
- Install pip by running the following command:
sudo yum install python3-pip -y
- Verify that pip is installed by running the following command:
pip3 --version
You should see a message that shows the version of pip, such as pip 20.2.2
Step 2.2: Install Flask and Other Dependencies on Your EC2 Instance
Next, you need to install Flask and other dependencies on your EC2 instance. Flask is a web application framework that is written in Python and allows you to create web applications with minimal code. To install Flask and other dependencies, follow these steps:
Connect to your instance using SSH, as shown in the previous section.
Install Flask by running the following command:
pip3 install flask
- Install mod_wsgi, which is a module that allows Apache to serve Python web applications, by running the following command:
sudo yum -y install python3-mod_wsgi.x86_64
- Install
virtualenv, which is a tool that allows you to create isolated Python environments, by running the following command:
pip3 install virtualenv
Step 2.3: Create a Simple Web Application Using Flask
Next, you need to create a simple web application using Flask. For this task, I’ll create a web application that displays a message that says Hello, World! To create a simple web application using Flask, follow these steps:
Connect to your instance using SSH, as shown in the previous section.
Create a directory for your web application by running the following command:
mkdir webapp
- Change to the webapp directory by running the following command:
cd webapp
- Create a virtual environment for your web application by running the following command:
virtualenv venv
- Activate the virtual environment by running the following command:
source venv/bin/activate
- Create a file named app.py that contains the code for your web application by running the following command:
nano app.py
- Paste the following code into the file:
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == "__main__":
app.run()
Save and exit the file by pressing Ctrl+O and then Ctrl+X.
Run your web application by running the following command:
python3 app.py
You should see a message that says Running on http://127.0.0.1:5000/.
Step 2.4: Configure Apache to Serve Your Web Application
Next, you need to configure Apache to serve your web application. By default, Apache serves the files from the /var/www/html directory, which is the document root. You need to change the document root to point to your web application directory, which is /home/ec2-user/webapp. You also need to tell Apache to use mod_wsgi to handle the requests to your web application. To configure Apache to serve your web application, follow these steps:
Connect to your instance using SSH, as shown in the previous section.
Stop your web application by pressing Ctrl+C.
Deactivate the virtual environment by running the following command:
deactivate
- Create a file named webapp.conf that contains the configuration for your web application by running the following command:
sudo nano /etc/httpd/conf.d/webapp.conf
- Paste the following code into the file:
<VirtualHost *:80>
ServerName ec2-54-123-456-789.ap-south-1.compute.amazonaws.com
DocumentRoot /home/ec2-user/webapp
WSGIDaemonProcess webapp user=ec2-user group=ec2-user threads=5 home=/home/ec2-user/webapp/venv
WSGIScriptAlias / /home/ec2-user/webapp/app.wsgi
<Directory /home/ec2-user/webapp>
WSGIProcessGroup webapp
WSGIApplicationGroup %{GLOBAL}
Require all granted
</Directory>
</VirtualHost>
Replace ec2-54-123-456-789.ap-south-1.compute.amazonaws.com with the public DNS name of your instance, which you can find in the EC2 console under the Instances section.
Save and exit the file by pressing Ctrl+O and then Ctrl+X.
Create a file named app.wsgi that contains the code to initialize your web application by running the following command:
nano /home/ec2-user/webapp/app.wsgi
- Paste the following code into the file:
import sys
sys.path.insert(0, '/home/ec2-user/webapp')
from app import app as application
Save and exit the file by pressing Ctrl+O and then Ctrl+X.
Restart Apache by running the following command:
sudo systemctl restart httpd
Step 3: Test Your Web Application
Finally, you need to test your web application by accessing it from a web browser. To test your web application, follow these steps:
Open a web browser and enter the public DNS name of your instance in the address bar. For example, if your instance’s public DNS name is
ec2-54-123-456-789.ap-south-1.compute.amazonaws.com, enterhttp://ec2-54-123-456-789.ap-south-1.compute.amazonaws.comin the address bar.You should see a web page that displays a message that says Hello, World! This means that your web application is working and being served by Apache.
Congratulations, you have successfully installed a web server on your EC2 instance and deployed a simple web application. You can now modify your web application code and see the changes reflected on your web page.
How to Monitor Your EC2 Instance Using Amazon CloudWatch and Troubleshoot Any Issues That Arise
In this section, I’ll show you how to monitor your EC2 instance using Amazon CloudWatch and troubleshoot any issues that arise. Amazon CloudWatch is a service that monitors your AWS resources and the applications you run on AWS in real-time. You can use CloudWatch to collect and track metrics, which are variables you can measure for your resources and applications. You can also use CloudWatch to set alarms, which are rules that trigger actions when a metric crosses a specified threshold. You can also use CloudWatch to create dashboards, which are customizable web pages that display graphs and tables of your metrics and alarms.
For this task, I’ll use CloudWatch to monitor the CPU utilization, disk usage, and network traffic of my EC2 instance. I’ll also set an alarm that sends me an email when the CPU utilization of my EC2 instance exceeds 80%. I’ll also create a dashboard that shows the metrics and the alarm of my EC2 instance.
Step 1: Enable CloudWatch Monitoring for Your EC2 Instance
To enable CloudWatch monitoring for your EC2 instance, you need to do the following:
Enable detailed monitoring for your EC2 instance, which will send metrics to CloudWatch every minute instead of every five minutes. This will give you more granular and timely data about your instance performance.
Install and configure the CloudWatch agent on your EC2 instance, which will collect and send custom metrics to CloudWatch, such as disk usage and memory usage. The CloudWatch agent is software that runs on your EC2 instance and communicates with CloudWatch.
To enable detailed monitoring for your EC2 instance, follow these steps:
Go to the AWS Management Console and select EC2 from the Services menu.
Select your EC2 instance from the Instances section click on Actions and then Instance Settings and then Change Monitoring Settings.
Check the box for Enable detailed monitoring and click on Save.
To install and configure the CloudWatch agent on your EC2 instance, follow these steps:
Connect to your EC2 instance using SSH, as shown in the previous section.
Download the CloudWatch agent package by running the following command:
wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
- Install the CloudWatch agent package by running the following command:
sudo rpm -U ./amazon-cloudwatch-agent.rpm
- Create a configuration file for the CloudWatch agent by running the following command:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
Follow the prompts to configure the CloudWatch agent. You can use the default settings for most of the options, or you can customize them according to your preferences. For this task, I’ll use the following settings:
Select 1 for On which OS are you planning to use the agent?
Select 2 for Are you using EC2 or On-Premises hosts?
Select Yes for Do you want to turn on StatsD daemon?
Select No for Do you want to monitor metrics from CollectD?
Select Yes for Do you want to monitor any host metrics? e.g. CPU, memory, etc.
Select Basic for Which default metrics config do you want?
Select Yes for Do you want to monitor any customized metrics?
Select Yes for Do you have any existing CloudWatch Log Agent?
Select No for Do you want to monitor log files?
Select No for Do you want to store the config in the SSM parameter store?
The configuration file will be saved as /opt/aws/amazon-cloudwatch-agent/bin/config.json. You can view or edit the file by running the following command:
sudo nano /opt/aws/amazon-cloudwatch-agent/bin/config.json
- Start the CloudWatch agent by running the following command:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s
- Verify that the CloudWatch agent is running by running the following command:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
You should see a message that says amazon-cloudwatch-agent is running.
Step 2: Set an Alarm that Sends You an Email When the CPU Utilization of Your EC2 Instance Exceeds 80%
Next, you need to set an alarm that sends you an email when the CPU utilization of your EC2 instance exceeds 80%. An alarm is a rule that triggers an action when a metric crosses a specified threshold. For this task, I’ll use the following settings for the alarm:
The metric is the CPU utilization of my EC2 instance, which is the percentage of CPU resources that are being used by my instance.
The threshold is 80%, which means that the alarm will trigger when the CPU utilization of my instance is greater than or equal to 80% for a specified period of time.
The period is 5 minutes, which means that the alarm will evaluate the metric every 5 minutes and compare it with the threshold.
The statistic is the average, which means that the alarm will use the average value of the metric over the period.
The action is to send an email to my email address, which is the notification that I’ll receive when the alarm triggers.
To set an alarm that sends you an email when the CPU utilization of your EC2 instance exceeds 80%, follow these steps:
Go to the AWS Management Console and select CloudWatch from the Services menu.
Click on Alarms from the left navigation pane and then click on Create alarm.
On the Select metric page, find the EC2 section and expand it. Then find the Per-Instance Metrics section and expand it. Then find the CPUUtilization metric for your EC2 instance and select it. Then click on Select metric.
On the Define alarm page, enter the following settings for the alarm:
Enter CPUAlarm as the Name and Alarm when CPU utilization exceeds 80% as the Description.
Select Greater than or equal to from the Whenever dropdown menu and enter 80 in the threshold box.
Select 5 minutes from the Period dropdown menu and select Average from the Statistic dropdown menu.
Select 1 out of 1 from the Datapoints to alarm dropdown menu and select 5 minutes from the Missing data treatment dropdown menu.
Click on Next to proceed to the next page.
On the Configure actions page, enter the following settings for the action:
Select In alarm from the Whenever this alarm dropdown menu and select Create a new topic from the Select an SNS topic dropdown menu.
Enter CPUNotification as the Topic name and enter your email address in the Email endpoints box.
Click on Create topic to create the SNS topic, which is a communication channel that delivers messages to subscribers.
Click on Next to proceed to the next page.
- On the Add description page, review the summary of your alarm and click on Create alarm to create your alarm.
- You should receive an email from AWS asking you to confirm your subscription to the SNS topic. Click on the link in the email to confirm your subscription. This will allow you to receive notifications from the SNS topic when the alarm triggers.
Step 3: Create a Dashboard that Shows the Metrics and the Alarm of Your EC2 Instance
Finally, you need to create a dashboard that shows the metrics and the alarm of your EC2 instance. A dashboard is a customizable web page that displays graphs and tables of your metrics and alarms. You can use dashboards to visualize the performance and health of your resources and applications.
For this task, I’ll create a dashboard that shows the following:
A line graph that shows the CPU utilization of my EC2 instance over time
A line graph that shows the disk usage of my EC2 instance over time
A line graph that shows the network traffic of my EC2 instance over time
A table that shows the status of my alarm and the current value of the CPU utilization of my EC2 instance
To create a dashboard that shows the metrics and the alarm of your EC2 instance, follow these steps:
Go to the AWS Management Console and select CloudWatch from the Services menu.
Click on Dashboards from the left navigation pane and then click on Create dashboard.
Enter EC2Dashboard as the Dashboard name and click on Create dashboard.
- On the Add to dashboard page, select Line from the All widgets section and click on Configure.
On the Line widget configuration page, enter the following settings for the widget:
Enter CPU Utilization as the Title.
Select EC2 from the Service dropdown menu and select Per-Instance Metrics from the Metric name dropdown menu.
Find the CPUUtilization metric for your EC2 instance and select it. You can use the search box to filter the metrics by the instance ID or the name tag of your instance.
Select 1 Minute from the Period dropdown menu and select Average from the Statistic dropdown menu.
Click on Create widget to add the widget to your dashboard.
Repeat the same steps to add another line widget for the disk usage metric. Use the following settings for the widget:
Enter Disk Usage as the Title.
Select CWAgent from the Service dropdown menu and select disk_used_percent from the Metric name dropdown menu.
Find the disk_used_percent metric for your EC2 instance and select it. You can use the search box to filter the metrics by the instance ID or the name tag of your instance.
Select 1 Minute from the Period dropdown menu and select Average from the Statistic dropdown menu.
Click on Create widget to add the widget to your dashboard.
Repeat the same steps to add another line widget for the network traffic metric. Use the following settings for the widget:
Enter Network Traffic as the Title.
Select EC2 from the Service dropdown menu and select Per-Instance Metrics from the Metric name dropdown menu.
Find the NetworkIn and NetworkOut metrics for your EC2 instance and select them. You can use the search box to filter the metrics by the instance ID or the name tag of your instance.
Select 1 Minute from the Period dropdown menu and select Average from the Statistic dropdown menu.
Click on Create widget to add the widget to your dashboard.
On the Add to dashboard page, select Text from the All widgets section and click on Configure.
On the Text widget configuration page, enter the following settings for the widget:
Enter Alarm Status as the Title.
Select Markdown from the Content type dropdown menu and enter the following code in the Content box:
| Alarm Name | Alarm Status | Current Value |
| --- | --- | --- |
| [CPUAlarm] | {{alarm.CPUAlarm.state}} | {{alarm.CPUAlarm.currentValue}}% |
Replace ^1^ with the URL of your alarm, which you can find in the CloudWatch console under the Alarms section.
- Click on Create widget to add the widget to your dashboard.
You can resize and rearrange the widgets on your dashboard by dragging and dropping them. You can also edit or delete the widgets by clicking on the Actions menu on the top right corner of each widget.
Click on Save dashboard to save your dashboard.
Step 4: Test Your Dashboard and Your Alarm
Finally, you need to test your dashboard and your alarm by generating some load on your EC2 instance and observing the changes in the metrics and the alarm status. To test your dashboard and your alarm, follow these steps:
Open a web browser and enter the public DNS name of your instance in the address bar. For example, if your instance’s public DNS name is
ec2-54-123-456-789.ap-south-1.compute.amazonaws.com, enterhttp://ec2-54-123-456-789.ap-south-1.compute.amazonaws.comin the address bar.You should see a web page that displays a message that says Hello, World!. This is your web application that is running on your EC2 instance.
Open another tab in your web browser and enter the URL of your dashboard, which you can find in the CloudWatch console under the Dashboards section. For example, if your dashboard name is EC2Dashboard, enter
https://console.aws.amazon.com/cloudwatch/home?region=ap-south-1#dashboards:name=EC2Dashboardin the address bar.You should see your dashboard that shows the metrics and the alarm of your EC2 instance. You can refresh the dashboard by clicking on the Refresh button on the top right corner of the dashboard.
To generate some load on your EC2 instance, you can use a tool called ApacheBench, which is a command-line tool that can be used to test the performance of a web server by sending multiple requests to it. You can install ApacheBench on your local machine by following the instructions on this [link].
To use ApacheBench, you need to open a terminal or a command prompt on your local machine and run the following command:
ab -n 1000 -c 10 http://ec2-54-123-456-789.ap-south-1.compute.amazonaws.com/
Replace ec2-54-123-456-789.ap-south-1.compute.amazonaws.com with the public DNS name of your instance. This command will send 1000 requests to your web application with a concurrency of 10, which means that 10 requests will be sent at a time.
- You should see some output that shows the results of the test, such as the number of requests per second, the time per request, and the percentage of requests that completed within a certain time.
While the test is running, switch to the tab that shows your dashboard and refresh it. You should see the changes in the metrics and the alarm status of your EC2 instance. You should see the CPU utilization, disk usage, and network traffic increase as the load on your instance increases. You should also see the alarm status change from OK to ALARM when the CPU utilization of your instance exceeds 80%.
You should also receive an email from AWS notifying you that your alarm has triggered. The email should look something like this:
To stop the test, press Ctrl+C on your terminal or command prompt. You should see the metrics and the alarm status of your EC2 instance return to normal after the test is stopped.
Congratulations, you have successfully monitored your EC2 instance using Amazon CloudWatch and troubleshooted any issues that arose. You can now use CloudWatch to monitor and manage your AWS resources and applications.
How to Create an Auto Scaling Group Using the AWS Management Console and Configure It to Launch EC2 Instances in Response to Changes in Demand
In this section, I’ll show you how to create an Auto Scaling group using the AWS Management Console and configure it to launch EC2 instances in response to changes in demand. An Auto Scaling group is a collection of EC2 instances that can scale up or down automatically based on the load or the rules that you define. You can use Auto Scaling to improve the availability and performance of your applications, as well as to reduce the cost and complexity of managing your resources.
For this task, I’ll use the following settings for the Auto Scaling group:
The Auto Scaling group will use the same AMI, instance type, security group, and key pair as the EC2 instance that I created in the previous section.
The Auto Scaling group will have a minimum size of 1, a maximum size of 3, and a desired capacity of 1, which means that it will start with 1 instance and can scale up to 3 instances or scale down to 1 instance depending on the demand.
The Auto Scaling group will use a scaling policy that will add 1 instance when the average CPU utilization of the group exceeds 80% for 5 minutes and will remove 1 instance when the average CPU utilization of the group falls below 40% for 5 minutes.
The Auto Scaling group will use a health check that will monitor the status of the instances and replace any unhealthy instances with new ones.
To create an Auto Scaling group using the AWS Management Console and configure it to launch EC2 instances in response to changes in demand, follow these steps:
Step 1: Create a Launch Configuration
To create an Auto Scaling group, you need to create a launch configuration, which is a template that specifies the configuration of the instances that will be launched by the Auto Scaling group. You can use the same launch configuration for multiple Auto Scaling groups. To create a launch configuration, follow these steps:
Go to the AWS Management Console and select EC2 from the Services menu.
Click on Launch Templates from the left navigation pane and then click on Create launch template.
On the Choose AMI page, find the Amazon Linux 2 AMI and click on Select. This is the same AMI that I used for the EC2 instance in the previous section.
- On the Choose Instance Type page, find the t2.micro instance type and click on Next: Configure details. This is the same instance type that I used for the EC2 instance in the previous section.
On the Configure details page, enter the following settings for the launch configuration:
Enter WebServerLC as the Name and Launch template for web server as the Description.
Check the box for Enable CloudWatch detailed monitoring, which will send metrics to CloudWatch every minute instead of every five minutes.
Select WebServerSG from the Security groups dropdown menu. This is the same security group that I created for the EC2 instance in the previous section.
Select WebServerKP from the Key pair dropdown menu. This is the same key pair that I created for the EC2 instance in the previous section.
Click on Next: Add Storage to proceed to the next page.
On the Add Storage page, use the default settings for the storage and click on Next: Configure Security Group to proceed to the next page.
On the Configure Security Group page, use the default settings for the security group and click on Review to proceed to the next page.
On the Review page, review the summary of your launch configuration and click on Create launch configuration to create your launch configuration.
Step 2: Create an Auto Scaling Group
Next, you need to create an Auto Scaling group using the launch configuration that you created in the previous step. To create an Auto Scaling group, follow these steps:
- On the Create Auto Scaling Group page, enter WebServerASG as the Group name and click on Next.
On the Configure group size and scaling policies page, enter the following settings for the Auto Scaling group:
Enter 1 in the Group size box, which is the number of instances that will be launched initially by the Auto Scaling group.
Enter 1 in the Minimum capacity box, which is the minimum number of instances that the Auto Scaling group will maintain at any time.
Enter 3 in the Maximum capacity box, which is the maximum number of instances that the Auto Scaling group will launch at any time.
Select Use scaling policies to adjust the capacity of this group from the Scaling policies section, which will allow you to define rules for scaling up or down the Auto Scaling group based on the load or the metrics.
Click on Add policy and select Target tracking scaling policy from the Policy type dropdown menu, which will allow you to specify a target value for a metric that the Auto Scaling group will try to maintain.
Enter CPUScalingPolicy as the Policy name and select Average CPU Utilization from the Metric type dropdown menu, which is the metric that the Auto Scaling group will use to scale up or down.
Enter 80 in the Target value box, which is the target value for the average CPU utilization that the Auto Scaling group will try to maintain.
Click on Create to create the scaling policy.
Click on Next: Configure Notifications to proceed to the next page.
On the Configure Notifications page, use the default settings for the notifications and click on Next: Configure Tags to proceed to the next page.
On the Configure Tags page, use the default settings for the tags and click on Review to proceed to the next page.
On the Review page, review the summary of your Auto Scaling group and click on Create Auto Scaling group to create your Auto Scaling group.
Step 3: Monitor the Performance of the Auto Scaling Group and the EC2 Instances and Troubleshoot Any Issues That Arise
Next, you need to monitor the performance of the Auto Scaling group and the EC2 instances and troubleshoot any issues that arise. You can use CloudWatch to monitor the metrics and the alarms of the Auto Scaling group and the EC2 instances. You can also use the AWS CLI to view the state of the Auto Scaling group and the EC2 instances and verify that the correct number of instances are running.
To monitor the performance of the Auto Scaling group and the EC2 instances and troubleshoot any issues that arise, follow these steps:
Go to the AWS Management Console and select CloudWatch from the Services menu.
Click on Alarms from the left navigation pane and find the alarm that was created by the Auto Scaling group, which should have the name WebServerASG-CPUAlarm. This alarm will trigger when the average CPU utilization of the Auto Scaling group exceeds 80% for 5 minutes and will cause the Auto Scaling group to add 1 instance.
Click on the alarm name to see the details of the alarm, such as the metric, the threshold, the period, the statistic, and the status. You can also see the history of the alarm, such as when it was created, when it changed state, and when it triggered actions.
- Click on Metrics from the left navigation pane and find the metrics that are related to the Auto Scaling group and the EC2 instances, such as the GroupTotalInstances, GroupInServiceInstances, GroupPendingInstances, GroupTerminatingInstances, CPUUtilization, DiskSpaceUtilization, and NetworkIn. You can select multiple metrics and see them on the same graph. You can also change the time range and the statistic of the metrics.
To view the state of the Auto Scaling group and the EC2 instances using the AWS CLI, you need to install and configure the AWS CLI on your local machine. You can follow the instructions on this [link] to install and configure the AWS CLI.
To view the state of the Auto Scaling group using the AWS CLI, you can use the describe-auto-scaling-groups command, which will show you the details of the Auto Scaling group, such as the name, the launch configuration, the size, the scaling policies, and the instances. The command is:
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names WebServerASG
Replace WebServerASG with the name of your Auto Scaling group.
- You should see some output that shows the details of your Auto Scaling group, such as the name, the launch configuration, the size, the scaling policies, and the instances. The output should look something like this:
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "WebServerASG",
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-south-1:123456789012:autoScalingGroup:1234abcd-5678-efgh-9012-3456ijklmnop:autoScalingGroupName/WebServerASG",
"LaunchTemplateName": "WebServerLT",
"MinSize": 1,
"MaxSize": 3,
"DesiredCapacity": 1,
"DefaultCooldown": 300,
"AvailabilityZones": [
"ap-south-1a",
"ap-south-1b",
"ap-south-1c"
],
"LoadBalancerNames": [],
"TargetGroupARNs": [],
"HealthCheckType": "EC2",
"HealthCheckGracePeriod": 300,
"Instances": [
{
"InstanceId": "i-0a1b2c3d4e5f6g7h8",
"InstanceType": "t2.micro",
"AvailabilityZone": "ap-south-1a",
"LifecycleState": "InService",
"HealthStatus": "Healthy",
"LaunchTemplateName": "WebServerLT",
"ProtectedFromScaleIn": false
}
],
"CreatedTime": "2023-04-23T10:15:30.123Z",
"SuspendedProcesses": [],
"VPCZoneIdentifier": "subnet-1234abcd,subnet-5678efgh,subnet-9012ijkl",
"EnabledMetrics": [],
"Tags": [],
"TerminationPolicies": [
"Default"
],
"NewInstancesProtectedFromScaleIn": false,
"ServiceLinkedRoleARN": "arn:aws:iam::123456789012:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
}
]
}
- To view the state of the EC2 instances using the AWS CLI, you can use the describe-instances command, which will show you the details of the instances, such as the ID, the type, the state, the IP address, and the tags. The command is:
aws ec2 describe-instances --filters "Name=tag:aws:autoscaling:groupName,Values=WebServerASG"
Replace WebServerASG with the name of your Auto Scaling group.
- You should see some output that shows the details of your Auto Scaling group, such as the name, the launch configuration, the size, the scaling policies, and the instances. The output should look something like this:
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "WebServerASG",
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-south-1:123456789012:autoScalingGroup:1234abcd-5678-efgh-9012-3456ijklmnop:autoScalingGroupName/WebServerASG",
"LaunchTemplateName": "WebServerLT",
"MinSize": 1,
"MaxSize": 3,
"DesiredCapacity": 1,
"DefaultCooldown": 300,
"AvailabilityZones": [
"ap-south-1a",
"ap-south-1b",
"ap-south-1c"
],
"LoadBalancerNames": [],
"TargetGroupARNs": [],
"HealthCheckType": "EC2",
"HealthCheckGracePeriod": 300,
"Instances": [
{
"InstanceId": "i-0a1b2c3d4e5f6g7h8",
"InstanceType": "t2.micro",
"AvailabilityZone": "ap-south-1a",
"LifecycleState": "InService",
"HealthStatus": "Healthy",
"LaunchTemplateName": "WebServerLT",
"ProtectedFromScaleIn": false
}
],
"CreatedTime": "2023-04-23T10:15:30.123Z",
"SuspendedProcesses": [],
"VPCZoneIdentifier": "subnet-1234abcd,subnet-5678efgh,subnet-9012ijkl",
"EnabledMetrics": [],
"Tags": [],
"TerminationPolicies": [
"Default"
],
"NewInstancesProtectedFromScaleIn": false,
"ServiceLinkedRoleARN": "arn:aws:iam::123456789012:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
}
]
}
- To view the state of the EC2 instances using the AWS CLI, you can use the describe-instances command, which will show you the details of the instances, such as the ID, the type, the state, the IP address, and the tags. The command is:
aws ec2 describe-instances --filters "Name=tag:aws:autoscaling:groupName,Values=WebServerASG"
Replace WebServerASG with the name of your Auto Scaling group. You should see some output that shows the details of your EC2 instances, such as the ID, the type, the state, the IP address, and the tags. The output should look something like this:
{
"Reservations": [
{
"Groups": [],
"Instances": [
{
"AmiLaunchIndex": 0,
"ImageId": "ami-0a123b456c789d012",
"InstanceId": "i-0a1b2c3d4e5f6g7h8",
"InstanceType": "t2.micro",
"KeyName": "WebServerKP",
"LaunchTime": "2023-04-23T10:15:31.000Z",
"Monitoring": {
"State": "enabled"
},
"Placement": {
"AvailabilityZone": "ap-south-1a",
"GroupName": "",
"Tenancy": "default"
},
"PrivateDnsName": "ip-172-31-22-33.ap-south-1.compute.internal",
"PrivateIpAddress": "172.31.22.33",
"ProductCodes": [],
"PublicDnsName": "ec2-54-123-456-789.ap-south-1.compute.amazonaws.com",
"PublicIpAddress": "54.123.456.789",
"State": {
"Code": 16,
"Name": "running"
},
"StateTransitionReason": "",
"SubnetId": "subnet-1234abcd",
"VpcId": "vpc-1234abcd",
"Architecture": "x86_64",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"AttachTime": "2023-04-23T10:15:31.000Z",
"DeleteOnTermination": true,
"Status": "attached",
"VolumeId": "vol-0a1b2c3d4e5f6g7h8"
}
}
],
"ClientToken": "",
"EbsOptimized": false,
"EnaSupport": true,
"Hypervisor": "xen",
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-123-456-789.ap-south-1.compute.amazonaws.com",
"PublicIp": "54.123.456.789"
},
"Attachment": {
"AttachTime": "2023-04-23T10:15:30.000Z",
"AttachmentId": "eni-attach-0a1b2c3d4e",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"Status": "attached"
},
"Description": "",
"Groups": [
{
"GroupName": "WebServerSG",
"GroupId": "sg-1234abcd"
}
],
"Ipv6Addresses": [],
"MacAddress": "02:12:34:56:78:90",
"NetworkInterfaceId": "eni-0a1b2c3d4e",
"OwnerId": "123456789012",
"PrivateDnsName": "ip-172-31-22-33.ap-south-1.compute.internal",
"PrivateIpAddress": "172.31.22.33",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-123-456-789.ap-south-1.compute.amazonaws.com",
"PublicIp": "54.123.456.789"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-22-33.ap-south-1.compute.internal",
"PrivateIpAddress": "172.31.22.33"
}
],
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-1234abcd",
"VpcId": "vpc-1234abcd",
"InterfaceType": "interface"
}
],
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SecurityGroups": [
{
"GroupName": "WebServerSG",
"GroupId": "sg-1234abcd"
}
],
"SourceDestCheck": true,
"Tags": [
{
"Key": "aws:autoscaling:groupName",
"Value": "WebServerASG"
}
],
"VirtualizationType": "hvm",
"CpuOptions": {
"CoreCount": 1,
"ThreadsPerCore": 1
},
"CapacityReservationSpecification": {
"CapacityReservationPreference": "open"
},
"HibernationOptions": {
"Configured": false
},
"MetadataOptions": {
"State": "applied",
"HttpTokens": "optional",
"HttpPutResponseHopLimit": 1,
"HttpEndpoint": "enabled"
}
}
],
"OwnerId": "123456789012",
"ReservationId": "r-0a1b2c3d4e5f6g7h8"
}
]
}
- You can use the AWS CLI to perform various operations on the Auto Scaling group and the EC2 instances, such as scaling up or down, suspending or resuming processes, attaching or detaching instances, and terminating instances.
Step 4: Test Your Auto Scaling Group and Your Scaling Policy
Finally, you need to test your Auto Scaling group and your scaling policy by generating some load on your EC2 instances and observing the changes in the size and the performance of the Auto Scaling group. To test your Auto Scaling group and your scaling policy, follow these steps:
Open a web browser and enter the public DNS name of one of your EC2 instances in the address bar. For example, if your instance’s public DNS name is
ec2-54-123-456-789.ap-south-1.compute.amazonaws.com, enterhttp://ec2-54-123-456-789.ap-south-1.compute.amazonaws.comin the address bar.You should see a web page that displays a message that says Hello, World!. This is your web application that is
This is your web application that is running on one of your EC2 instances that belongs to your Auto Scaling group.
Open another tab in your web browser and enter the URL of your dashboard, which you can find in the CloudWatch console under the Dashboards section. For example, if your dashboard name is EC2Dashboard, enter
https://console.aws.amazon.com/cloudwatch/home?region=ap-south-1#dashboards:name=EC2Dashboardin the address bar.You should see your dashboard that shows the metrics and the alarm of your Auto Scaling group and your EC2 instances. You can refresh the dashboard by clicking on the Refresh button on the top right corner of the dashboard.
To generate some load on your EC2 instances, you can use the same tool that you used in the previous section, which is ApacheBench. You can install ApacheBench on your local machine by following the instructions on this [link].
To use ApacheBench, you need to open a terminal or a command prompt on your local machine and run the following command:
ab -n 1000 -c 10 http://ec2-54-123-456-789.ap-south-1.compute.amazonaws.com/Replace
ec2-54-123-456-789.ap-south-1.compute.amazonaws.comwith the public DNS name of one of your EC2 instances. This command will send 1000 requests to your web application with a concurrency of 10, which means that 10 requests will be sent at a time.You should see some output that shows the results of the test, such as the number of requests per second, the time per request, and the percentage of requests that completed within a certain time.
While the test is running, switch to the tab that shows your dashboard and refresh it. You should see the changes in the metrics and the alarm status of your Auto Scaling group and your EC2 instances. You should see the CPU utilization, disk usage, and network traffic increase as the load on your instances increases. You should also see the alarm status change from OK to ALARM when the average CPU utilization of your Auto Scaling group exceeds 80%.
You should also see the size of your Auto Scaling group increase as the scaling policy adds more instances to handle the increased load. You can see the number of instances in your Auto Scaling group in the EC2 console under the Auto Scaling Groups section. You should see the Group size, Desired capacity, and InService instances increase from 1 to 2 or 3, depending on the load.
To stop the test, press Ctrl+C on your terminal or command prompt. You should see the metrics and the alarm status of your Auto Scaling group and your EC2 instances return to normal after the test is stopped. You should also see the size of your Auto Scaling group decrease as the scaling policy removes the excess instances.
Congratulations, you have successfully created an Auto Scaling group using the AWS Management Console and configured it to launch EC2 instances in response to changes in demand. You have also tested your Auto Scaling group and your scaling policy by generating some load on your EC2 instances and observing the changes in the size and the performance of the Auto Scaling group. You can now use Auto Scaling to improve the availability and performance of your applications, as well as to reduce the cost and complexity of managing your resources.
Conclusion
In this blog post, I have shown you how to test your knowledge on AWS by performing some hands-on tasks that are relevant for DevOps learners. You have learned how to:
Launch an EC2 instance using the AWS Management Console and connect to it using SSH
Install a web server on the EC2 instance and deploy a simple web application
Monitor the EC2 instance using Amazon CloudWatch and troubleshoot any issues that arise
Create an Auto Scaling group using the AWS Management Console and configure it to launch EC2 instances in response to changes in demand
Monitor the performance of the Auto Scaling group and the EC2 instances and troubleshoot any issues that arise
By doing these tasks, you have gained more confidence and experience in using AWS services and tools. You have also improved your skills and knowledge in DevOps, which is a set of practices that combines software development and IT operations to deliver software faster and more reliably.
I hope you have enjoyed this blog post and learned something new. If you have any questions or feedback, please feel free to leave a comment below.
If you want to learn more about AWS and DevOps, you can check out the following resources:
Thank you for reading and happy learning!