Experimenting With the AWS Price List API
I recently had the chance to experiment with the (relatively) new AWS Price List API, which allows you to obtain the latest price information for AWS services via a simple web-based service.
You can request updated information at any time via the published API endpoints, which are useful for on-demand processing, or you can elect to be notified when prices change by subscribing to an SNS topic. Both methods allow you to programmatically access the latest pricing information and easily process it for cost analysis, estimation or tracking purposes.
The Price List API is very simple, and allows two types of requests. The first makes it possible to get a list of AWS products, and the second allows you to get product price information. The list of products is called an "offer index" and includes service-specific URLs for each product.
These URLs are used to obtain the current "offer file,” or price information, for the associated product. The offer file includes details about the products, SKUs and pricing. Obtaining the latest pricing information for an AWS product is as simple as requesting the offer index, locating the product in the index and then requesting the offer file from the provided URL.
In this post, I'll walk through how to create a simple, static website hosted on S3 that displays EC2 instance types and pricing info using this new API. The pricing info will be updated dynamically using SNS and Lambda, and we’ll build all of this using the latest AWS CLI. Let’s get started.
Create the Website Hosted on S3
To start, you’ll need an S3 bucket to host the static web site. First, create the bucket, specifying the bucket name:
By default, this will create an S3 bucket in the U.S. Standard region. If you’d like to have your bucket located in another AWS region, use the Location Constraint parameter to indicate the desired region.
Next, configure the S3 bucket as a website and specify how you would like the web site to be configured:
website-config.json is a JSON document in the current folder that specifies index and error pages for the website:
Once the website has been configured, you’ll need to add a bucket policy that makes the contents of the bucket publicly available:
The policy contained in
bucket-policy.json allows anyone to read files from your S3 bucket:
Now that the bucket has been configured as a web site and its contents are publicly available. First, download index.html (THE website) and prices.json (some sample data) locally, then upload the files to your S3 bucket:
It’s important that you use the --content-type parameter to indicate the file type. If you don’t, when you browse to your website,
index.html will be downloaded by your browser instead of being rendered.
You can test your website by entering the URL of your into S3 website into your browser:
If the website was deployed successfully, you should see something like this in your browser:
You can find additional information and step-by-step instructions for creating a static website on S3 using the AWS console here.
Create Lambda Function to Update Prices
The Lambda function downloads the latest EC2 price information using the AWS Price List API, parses and extracts relevant price information, then uploads the filtered data to the S3 bucket, replacing the existing
prices.json data file. The Lambda function will be triggered each time AWS changes EC2 prices (AWS publishes price change notifications to an SNS topic), or it may be triggered manually for testing.
Creating an IAM Role and Policy
Before setting up the Lambda function, you’ll need to create an IAM role for the Lambda function to runs as. This IAM role and the IAM policy associated with it will define what permissions your Lambda function has and what operations it can perform (like uploading
prices.json to S3).
First, let’s create an IAM Policy with the appropriate permissions. This command creates a customer managed policy named prices:
policy.json is a JSON document that grants write access to your S3 bucket and Cloudwatch Logs. Both of these permissions will be required by your Lambda function when it is invoked.
Next, let’s create the actual IAM Role your Lambda function will run as. This command creates a role named prices and attaches a trust policy to it:
This trust policy indicates that the Lambda service is allowed to assume the prices role.
Now that both the IAM Policy and IAM Role have been created, it’s time to associate the policy with the role. This command attaches the managed policy you create to the IAM role (be sure to replace
with your AWS account ID before executing):
You now have an IAM role that the Lambda service can assume when executing your Lambda function, and that IAM role has the necessary permissions to upload files to your S3 bucket.
The Lambda Python Code
The actual Python code that will run in your Lambda function is included below. This code downloads the price information from AWS using the AWS Price List API, filters and sorts the data, then uploads the interesting parts (for this example, only on-demand Linux pricing for each region) to the S3 bucket.
Building a Deployment Package
Because the Lambda function has dependencies on external resources (requests in this case, used to download price information using the AWS Price List web service), you’ll need to create a Lambda function deployment package that includes the additional resources.
To build the deployment package, first, create a project directory and call it
Save the lambda function to the project directory as
lambda_function.py. Edit the file and update it with the name of your bucket.
Then, install the requests library in the project directory using pip
The content of the prices folder should now look like this. It should contain only the Lambda function code and the requests library just installed:
Finally, zip the contents of the project directory into a file located in the parent directory:
This deployment package, consisting of your lambda function and the requests library, will be used in the next step to create the actual AWS Lambda function. If you want more information about creating Python deployment packages, check here.
Create the Lambda Function
The final step is to create the actual Lambda function and connect it to the SNS event source. This command creates the Lambda function and uploads the deployment package you just created.
The Lambda function will be triggered when a notification is sent to an SNS topic (owned and managed by AWS). In order for that SNS topic to trigger execution of your Lambda function, you must grant the SNS topic explicit permission to do so. The following command gives permission to the AWS SNS service to invoke your prices Lambda function, specifically when the price-list-api topic is the source:
The final step is to create an SNS subscription that subscribes to the price-list-api topic and direct the notifications to your prices Lambda function:
That’s it. The next time AWS updates EC2 prices, a message will be published to the price-list-api topic, which will trigger execution of your prices Lambda function. Your Lambda function will use the AWS Price List API to get the latest EC2 price information, filter it, and then upload the results to your S3 bucket, replacing the existing prices.json file.
If you’d like to manually trigger execution of your Lambda function to verify everything is working, you can do so with the following command:
This command, and the accompanying payload, simulates receipt of a notification from the SNS topic. After execution has completed and the
prices.json file updated, you can browse to your website and verify that prices have been updated.
Success will be apparent if you now see instance types and prices, can choose from available AWS regions, and are able to filter based on the number of cores and amount of memory. Note the score column in the website is a simple score I used to indicate nearness to the desired number of cores and amount of memory. View the source for
index.htmlindex.html to see how I calculated the score.
That’s it. You now have a simple, static website hosted on S3 that displays EC2 instance types and pricing info using the new AWS Price List API. The pricing info is updated dynamically using SNS and Lambda whenever AWS updates EC2 prices. And all of this was built using the AWS CLI. Thanks for following along!
Are You Realizing the Cloud Optimization Benefits of Kubernetes and Containers?
September 22nd, 2023
Google Cloud Next ’23 Highlights— AI and Beyond
September 14th, 2023
Why You Need an MLOps Framework for Standardizing AI and Machine Learning Operations
September 12th, 2023