How To Document AWS Infrastructure?

In This Article

• Documenting AWS Infrastructure
  • Cloud Architecture Overview
  • AWS Accounts
  • AWS Services
  • Logging and Monitoring
  • Deployment Process
• Automated AWS Documentation
  • Autogenerate AWS Infrastructure Document
  • Custom Documentation Generation
  • Schedule Documentation Generation
  • Automatically Update Confluence Documentation
• Summary

Documenting AWS Infrastructure

I guess everyone knows that feeling when you and your team have built your cloud application and infrastructure (hopefully having your infrastructure as code) and now you have to document it fully. Of course, in ideal scenario you should document your cloud infrastructure while building or finishing some part of it but from my experience unfortunately it’s not always like that. One of the reasons is that in the agile world user stories can change a lot impacting different aspects of your cloud architecture and it’s pretty challenging to update docs all the time to keep it up to date (knowing this is not the last update).

In most scenarios every product team has its own documentation requirements based on the specific documentation templates used across organization and the actual cloud application they are building. However, based on my experience there are some must have documentation sections that should be described when you document your AWS infrastructure.

Cloud Architecture Overview

You should always document overview of your cloud architecture – which basically can be one diagram with some description. One thing that I have learned during my time by working as a cloud architect and consultant is that simplicity is really the key. Yes, there are several ways how you can model and visualize your application like using UML diagrams or C4 models. And in some organizations, there is must have requirement to use specific model which also is fine because there are use-cases and more complex applications where it makes total sense to use specific model or notation. But when it’s possible I personally try to keep it simple and, in most scenarios, use simple data flow diagram as symbols using official AWS icon set to visualize overview of cloud architecture and add some description for each of the steps.

For example, here is cloud architecture overview for serverless application (front-end and back-end infrastructure) with described steps of using main AWS services:

Yes, this is generic serverless architecture diagram and doesn’t really describe the actual business logic which you would do in your case. The idea is to show you how simple we can visualize our AWS cloud application which then can be shown to your team and most of the people would understand it right away. Because there are no complex modeling, layers, or specific notation that you have to be familiar with.

AWS Accounts

To have your AWS accounts properly documented is one of the keys to successfully maintain your cloud application and anyone involved to quickly address any challenges like security incidents, failing deployments, non-responding API endpoints, error thresholds, billing alerts etc. There is solid chance that you and your team use AWS Organizations to manage and consolidate multiple AWS accounts into an organizations. Main benefits of using this free service are:

• Centralized management of all your AWS accounts.
• Consolidated billing.
• Policies to control which AWS services and API actions each of the AWS accounts can access.
• Hierarchical grouping of all your AWS accounts. It’s possible to create organizational units (OUs) with specific policies attached to group your accounts.

Let’s describe account structure used for our cloud application where we have four AWS accounts:

• Management AWS account which is used as billing account and root account to manage all other AWS accounts. We have enabled AWS Identity Center. in this account in order to manage user access to all the other AWS accounts.
• Dev account which is used for any development related activities and of course for deployment of our cloud application.
• QA account which is used to run our cloud application for QA purposes. This account in many organizations is used or known as pre-prod account too to verify that your application works before deploying to production.
• Production account which is used to run production workloads for your cloud application

In the diagram above we can see the organizational unit hierarchy in AWS Organizations. In our case we have 1:1 relation for Dev, QA and Production OUs and AWS accounts. Of course, in larger organizations this diagram would be different and contain many other OU’s and AWS accounts. User access for all the AWS accounts in this example is managed through the AWS Identity Center.

AWS Services

AWS infrastructure consists of different AWS services linked together to fulfil your cloud application requirements. We should describe more in details each of the AWS services used in this application. This would allow for example in case of security incident anyone in team to quickly find relevant information about specific service which may caused the issue.

AWS has nicely grouped all the available services (in AWS console) so I would suggest to keep the same high-level structure for your documentation too.

Here is an example document structure for application showed in Cloud Architecture Overview section:

• Networking & Content Delivery
  • API Gateway. Amazon API Gateway is fully managed service that provides easy way to create, publish, maintain and monitor secure APIs. Describe in this section API endpoints for all the environments (like Dev, QA, Prod), authentication (ex. AWS IAM, custom authorizer, API keys etc.), Lambda proxy integrations and any other specific API Gateway functionality used.
  • Virtual Private Cloud (VPC). VPC is the main backbone for basically any cloud application that you run in AWS cloud. You should describe here VPC CIDR blocks, subnets, VPC endpoints, security groups, elastic IP addresses, basically any VPC related information that would be useful.
  • CloudFront (CDN). Amazon CloudFront is AWS content delivery network. Describe here all the distributions used to deliver your content and any other specific configurations like CloudFront functions etc.
  • Route 53. Amazon Route 53 is scalable DNS web service. Describe here all the hosted zones created and used for this application. Not necessary to put all the records here but mention records that point to other AWS services (like CloudFront, Certificate Manager) used in this application.
• Security, Identity, & Compliance
  • Certificate Manager. AWS Certificate Manager provisions and manages SSL/TLS certificates that can be used for different AWS services. Describe here all the certificates (with ARN’s) that have been issued and all the connected services that use these certificates.
• Storage
  • S3. Amazon S3 is object storage. Describe here what will be stored in the bucket and how it will be accessed.
  • DynamoDB. Amazon DynamoDB is fast NoSQL fully managed database. Describe here DynamoDB main parameters hash and range attributes, billing attributes (pay per request, provisioned capacity) and show real-life example record. Of course describe any other more specific functionality used like global secondary indexes, in-memory acceleration etc.

Logging and Monitoring

In order to quickly react on errors and performance issues it is must have to have proper logging and monitoring configured for your cloud application. Regardless if you use AWS services like Amazon CloudWatch , AWS X-ray or third-party tools like Sentry, Datadog, Rollbar etc., you should document how your logging works and where to find the logs. The same goes for monitoring and alerts – you should describe the main thresholds when the alerts are sent based on specific events like error threshold reached, average request execution time is increased, API endpoint is unavailable etc.

There are several other security related AWS services that can monitor your AWS account for different threats and misconfigurations like Amazon GuardDuty, Amazon Detective and AWS Service Hub. And if you use these you should document these as well.

Deployment Process

It’s important to describe existing deployment process and CI/CD workflows to build and deploy your cloud application to AWS cloud. This is directly related with AWS infrastructure because in CI/CD pipelines we update our AWS infrastructure and cloud application artifacts (like Lambda code). If deployment fails, we have to have nice overview of our CI/CD workflows so it helps us to debug the issue faster.

You should list all the code repositories that are used for this application regardless if it’s AWS Code Commit service that your team uses or third party one like GitHub, GitLab, Bitbucket etc.

Automated AWS Documentation

We described in previous section why it’s important to have up to date AWS infrastructure documentation in place. But now imagine, we have finally updated our documentation to contain latest details about our AWS infrastructure and then new user story pops up to integrate with new internal system where we have to setup VPC peering with the target VPC where the internal system is deployed. And just like that our AWS infrastructure documentation after implementing these changes is obsolete again.

Many organizations nowadays have frequent deployments with even several deployments per day – so infrastructure documentation can get outdated pretty fast. This can be challenging when something goes wrong and there is no up to date documentation in place. But luckily, it’s possible to automate generation of AWS infrastructure documentation and Cloudviz.io can help you with that!

Autogenerate AWS Infrastructure Document

You can autogenerate AWS infrastructure documentation for specific AWS account and region in simple steps:

• Connect your AWS account to Cloudviz.io.
• Choose specific region and initiate sync.
• Generate technical documentation for your AWS account and region.

As you can see in the video above, with just a few steps, you can auto-generate detailed AWS documentation containing diagrams and text data. All the resources are grouped in the same way as AWS organizes its services.

Custom Documentation Generation

We know that there are many ways how to document AWS infrastructure and not always our provided default documentation template will work for every organization. That’s why in Cloudviz.io we provide option for you to create your own documentation template and use it to autogenerate AWS infrastructure document. This provides possibility to use your company’s logo, colors and distinctive elements in generated documentation and of course create your own document structure to describe your AWS environment.

Custom documentation template example:

In video below you can see that we have autogenerated custom infrastructure documentation using our logo and colors. You can also see that that this autogenerated documentation contains sections that we described manually in Documenting AWS Infrastructure section like overview diagram, detailed AWS services description. By using this autogenerated documentation you can always be sure that you will have up to date AWS infrastructure documentation which you and your team can rely on.

Schedule Documentation Generation

You can easily schedule automated documentation generation to be initiated based on your preferred frequency (ex. Once every three hours, day, month etc.) by using automation profiles. Documentation will be automatically generated and placed in your S3 bucket. After that you have all the flexibility to use for example S3 event notifications and process the generated diagrams and documentation accordingly to your needs.

Automatically Update Confluence Documentation

If word documentation doesn’t work for you then it’s possible to use Cloudviz API to autogenerate AWS diagrams and documentation and store these in Confluence.

We have developed functional example for generating AWS documentation from your AWS environments and keeping always up-to-date documentation in Confluence.

Summary

In this article, we have taken a closer look at the must-have sections that should be described in any AWS infrastructure documentation. Based on my personal experience I know that for many professionals out there documentation is not the most favorite task to do. One of the reasons (besides the fact that we just like to build more instead of documenting) is that in agile world with frequent releases it gets outdated pretty fast and people just get tired or forget to update it every time we press the deploy button.

Luckily, we have a solution for outdated documentation! We can use Cloudviz.io to autogenerate detailed technical documentation of your AWS infrastructure. Either you use Cloudviz.io or other tool to autogenerate your diagrams and documentation it’s up to you and your team. But I would strongly suggest to have up to date documentation of your AWS infrastructure so it’s ready to be used when you need it the most!