Introduction to DevOps on AWS - Part 1

Introduction

DevOps is a new term that primarily focuses on improved collaboration, communication,
and integration between software developers and IT operations. It’s an umbrella term
that some describe as a philosophy, cultural change, and paradigm shift.

Historically many organizations have been vertically structured with poor integration
among development, infrastructure, security and support teams. Frequently the groups
report into different organizational structures with different corporate goals and
philosophies.

Deploying software has predominately been the role of the IT operations group.
Fundamentally developers like to build software and change things quickly, whereas IT
operations focus on stability and reliability. This mismatch of goals can lead to conflict,
and ultimately the business may suffer.
Today, these old divisions are breaking down, with the IT and developer roles merging
and following a series of systematic principles:

• Infrastructure as code
• Continuous deployment
• Automation
• Monitoring
• Security

An examination of each of these principles reveals a close connection to the offerings
available from Amazon Web Services.

Agile Evolution to DevOps

To fully appreciate DevOps principles, it is helpful to understand the context in which
they evolved. The story begins with agile software development, which became popular
over a decade ago and was seen as better approach to building software. Prior to agile,
the dominant waterfall development methodology was based on a sequence starting
with a requirements phase where 100% of the system under development was defined
up front. The approach has shown itself to be inflexible and monolithic.

The agile model brought the concept of new and improved collaboration between
business users and developers. Software development began to focus on iterations of
working software that would evolve over time, delivering value along the way. Agile is a
disciplined engineering process, and numerous tools now support it. For developers,
such tools include IDEs, unit test frameworks, and code optimizers. As developers
become more productive, the business becomes more agile and can respond to their
customer requests more quickly and efficiently.

Over the last few years, the agile software development evolution has started to move
downstream towards infrastructure under the label DevOps. Whereas agile software
development primarily focuses on the collaboration between the business and its
developers, DevOps focuses on the collaboration between developers, IT operations
and security teams. IT operations include system administrators, database
administrators, network engineers, infrastructure architects, and support personnel.
Whereas agile software development provides business agility, DevOps provides IT
agility, enabling the deployment of applications that are more reliable, predicable, and
efficient.

DevOps practices vary with the task: With application development, DevOps focuses on
code building, code coverage, unit testing, packaging, and deployment. With
infrastructure, on the other hand, DevOps focuses on provisioning, configuration,
orchestration, and deployment. But in each area the underlying principles of version
management, deployment, roll back, roll forward, and testing remain the same.

Infrastructure as Code

A fundamental principle of DevOps is to treat infrastructure the same way developers
treat code. Application code has a defined format and syntax. If the code is not written
according to the rules of the programming language, applications cannot be created.
Code is stored in a version-management system that logs a history of code
development, changes, and bug fixes. When code is compiled (built) into applications,
we expect a consistent application to be created. That is to say, the build is repeatable
and reliable.

Practicing “infrastructure as code” means applying the same rigor of application code
development to infrastructure provisioning. All configurations should be defined in a
declarative way and stored in a version management system, just like application code.
Infrastructure provisioning, orchestration, and deployment should support the use of the
“infrastructure code.”

Until recently the rigor applied to application code development has not necessarily been
applied to infrastructure. Frequently infrastructure is provisioned using manual
processes. Scripts developed during the provisioning may not be stored in version
control systems and the creation of environments is not always repeatable, reliable, or
consistent.

In contrast, AWS provides a DevOps-focused way of creating and maintaining
infrastructure. Similar to the way software developers write application code, AWS
provides services that enable the creation, deployment and maintenance of
infrastructure in a programmatic, descriptive, and declarative way. These services
provide rigor, clarity, and reliability. The AWS services discussed in this paper are core
to a DevOps strategy and form the underpinnings of numerous higher level AWS
DevOps principles and practices.

AWS CloudFormation

A good example of how the DevOps principles are used in practice is AWS
CloudFormation.1 By using AWS CloudFormation templates, you can define and model
AWS resources that can be created and updated. These templates are written in a
format called JavaScript Object Notation (JSON). The templates require a specific
syntax and structure that depends on the types of resources being created and
managed. By using templates, you can provision infrastructure in a repeatable and
reliable way.

You can create custom AWS CloudFormation templates or use sample templates that
are available publically. Once templates are deployed or updated into the AWS
environment, the collection of resources under management is called a “stack.” You can
manage stacks through the AWS Management Console, AWS Command Line Interface,
or AWS CloudFormation APIs. Common actions include create-stack, describe-stacks,
list-stacks, and update-stack.

When you create or update a stack in the console, events are displayed showing the
status of the configuration. If an error occurs, the stack is rolled back to its previous
state. Amazon Simple Notification Service (Amazon SNS) helps you manage these
events. For example, you can use Amazon SNS to track stack creation and deletion
progress via email and integrate with other processes programmatically.

With templates, you can work with a broad set on AWS offerings, including Amazon
Simple Storage Service (Amazon S3), Auto Scaling, Amazon CloudFront, Amazon
DynamoDB, Amazon Elastic Compute Cloud (EC2), Amazon ElastiCache, AWS Elastic
Beanstalk, Elastic Load Balancing, AWS Identity and Access Management, AWS
OpsWorks and Amazon Virtual Private Cloud.

You can use a single template to create and update an entire environment or separate
templates to manage layers within an environment. This allows templates to be
modularized and also provides a layer of governance that is important to many
organizations.

AWS CloudFormation makes it easy to organize and deploy a collection of AWS
resources and lets you describe any dependencies or pass in special parameters when
the stack is configured.

To realize AWS CloudFormation’s potential for “information as code,” you should store
templates in a source-code management systems version control before you deploy or
update them in AWS. Amazon S3 provides a good location for storing and versioning
templates. You can integrate AWS CloudFormation with the development and
management tools of your choice.

There is no charge for defining your “infrastructure as code” in the AWS CloudFormation
service. You are billed only the normal rates for the AWS resources that AWS
CloudFormation creates and your application uses.

In the above example, an AWS CloudFormation template has been defined in JSON
notation to create an Amazon EC2 instance. This case provisions an m1.medium type of
EC2 instance. The word Ref refers to parameters that are accessed when the stack is
created. A parameter called KeyPair may be provided when the stack is created. This
is the name of the key pair that is used when accessing the instance with SSH.
AWS AMI

An Amazon Machine Image (AMI) is another example of “infrastructure as code.” This
core component of AWS computing is a kind of digital template that can launch
(provision) Amazon EC2 instances, the fundamental AWS compute environment in the
cloud. The image contains a software configuration such as a web server, application
server, and database.

You can choose from three types of AMIs:

• AWS published
• Third party
• Custom created

AWS publishes AMIs that contain common software configurations based upon popular
operating systems like Linux and Microsoft Windows. AMIs may also be obtained from
third-party vendors, some of which are available on the AWS Marketplace.2 You or your
organization can also create and publish your own custom AMIs. Custom organizational
AMIs usually include corporately distributed software such as hardened operating
systems, antivirus software, and office productivity suites.

AMIs can also include application software that is bundled along with the AMI, or they
can contain scripts and software that allow the instance to install application software at
launch time, called “booting.” There are pros and cons of preloading application-level
software into an AMI. On the plus side, the instances can be launched very rapidly
because they don’t need to install any additional software at boot time. However, you
may need to create a new AMI every time your application-level software changes.