Using AWS Systems Manager Parameter Store to Manage Elastic Beanstalk Environment Variables

Parameter Store

Elastic Beanstalk, a popular platform for deploying web applications on AWS, can present challenges as applications become more complex. The restriction of 50 environment variables or 4,096 bytes poses a significant hurdle for developers aiming to maintain comprehensive configuration sets across diverse deployment environments.

Throughout my professional experience, I’ve encountered firsthand the struggles clients face with Elastic Beanstalk’s configuration limits. These constraints often hinder the effective management of application settings and environment-specific configurations.

To tackle these challenges, a novel approach has emerged — leveraging AWS Parameter Store to load environment variables for Elastic Beanstalk environments. This innovative solution not only enhances security by leveraging AWS’s robust access control mechanisms but also streamlines configuration settings management across different deployment environments. This article serves as a starting point for implementing this solution, with the project code available in the demo repo.

The Ingenious Solution: Leveraging AWS Parameter Store

The integration involves a carefully crafted script seamlessly integrated with Elastic Beanstalk’s deployment process. This script fetches configuration parameters from the AWS Parameter Store and injects them as environment variables into the application environment.

Understanding the Structure:

The integration’s core lies in the 01_map_parameters_store_to_env_vars.sh script, strategically placed within Elastic Beanstalk's deployment lifecycle to ensure seamless execution post-deployment. This script resides within the project directory structure under .platform/confighooks and .platform/hooks folders.

├── .platform
│   ├── confighooks
│   │   └── postdeploy
│   │       └── 01_map_parameters_store_to_env_vars.sh
│   └── hooks
│       └── postdeploy
│           └── 01_map_parameters_store_to_env_vars.sh
├── README.md
└── .........

confighooks and hooks folders are custom scripts and other executable files that you deploy as part of your application’s source code, and Elastic Beanstalk runs during various instance provisioning stages. Detailed info is available in AWS documentation.

About the script:

This script is a pivotal tool for seamlessly integrating AWS Parameter Store with Elastic Beanstalk deployments, thereby surmounting configuration limitations and enhancing the flexibility and scalability of deployed applications. Here’s a breakdown of how it operates:

1. Initialization and Setup: The script begins with standard initialization, invoking the Bash shell (#!/bin/bash -e) and providing comments for clarity and reference.
2. Environment Variable Retrieval: It reads the current environment properties from the designated file (/opt/elasticbeanstalk/deployment/env) and parses them into an array (eb_env_vars) for further processing.
3. Parameter Store Path Extraction: The script then scans through the environment variables to identify a specific property named parameter_store_path. Once found, it extracts the path from this property, determining the location where Parameter Store values are stored.
4. Parameter Retrieval from Parameter Store: Utilizing the extracted parameter store path, the script employs AWS CLI commands to retrieve parameters from the AWS Parameter Store, including decryption for sensitive data, and stores the results in a temporary file (/opt/elasticbeanstalk/deployment/ssm_params).
5. Parsing and Integration: It employs jq to parse the retrieved parameters, constructing key-value pairs suitable for environment variable injection. These parsed parameters are then appended to a custom environment variable file (/opt/elasticbeanstalk/deployment/custom_env_var).
6. Ensuring Parameter Retrieval: The script performs checks to verify the successful retrieval of parameters from the Parameter Store. If unsuccessful, it halts execution and exits with an error message.
7. Preparation for Environment Variable Update: Before updating the environment variables, the script creates a backup of the original environment variable file and copies it for modification.
8. Integration of Retrieved Parameters: It iterates through the parsed parameters, ensuring that each parameter is added to the custom environment variable file only if it doesn’t already exist. This step ensures consistency and prevents accidental overwriting of existing configuration settings.
9. Finalization and Cleanup: After integrating the retrieved parameters, the script replaces the original environment variable file with the modified version and removes temporary files and backups to maintain a clean workspace.
10. Restarting the Web Service: Finally, the script initiates a restart of the web service to apply the updated environment variables, ensuring that the application adapts to the changes seamlessly.

In essence, this script acts as a bridge between Elastic Beanstalk deployments and the AWS Parameter Store, facilitating the dynamic loading of configuration parameters and enabling applications to thrive in dynamic and evolving environments. You can go through the script on github.

Advantages and Implications:

Enhanced Security:

• By leveraging AWS Parameter Store, sensitive configuration parameters are securely stored and accessed, minimizing exposure risks and ensuring compliance with security best practices.

Streamlined Management:

• Centralized management of configuration settings across multiple environments simplifies maintenance efforts and promotes consistency, fostering a cohesive development and deployment experience.

Improved Scalability and Flexibility:

• The dynamic nature of environment variable loading enables applications to adapt seamlessly to evolving deployment environments, catering to scalability requirements and accommodating diverse use cases.

Proven Effectiveness:

Having implemented this solution across a diverse array of client projects, I can attest to its efficacy and reliability. Clients who previously grappled with Elastic Beanstalk’s configuration limits have experienced a newfound sense of empowerment as they seamlessly navigate the complexities of configuration management with ease and confidence.

Conclusion:

Dealing with Elastic Beanstalk’s configuration limits doesn’t have to be a headache. With ingenuity and the right tools, we can break through those barriers and keep our apps running smoothly.

As we continue our journey through the cloud, remember: there’s always a solution waiting to be discovered.