Working with Landsat Data in the Amazon Web Services (AWS) Cloud

This tutorial demonstrates how users access Landsat Data stored within the AWS Cloud environment using basic commands. Landsat data stored in the AWS Cloud is located within U.S. West (Oregon) us-west-2 region in a requester pays Simple Storage Services (S3) bucket. Users interested in utilizing direct access to Landsat data stored in S3 are encouraged to visit the Requester Pays Documentation Page (https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html) for information concerning potential egress costs to accessing Landsat data with this method.

AWS Bucket Name: usgs-landsat (Requester Pays)
AWS Region: US West (Oregon) us-west-2

An AWS account is required before undertaking this tutorial. Please see the AWS Account website to establish an account (https://aws.amazon.com/account/).

Use Case Example

This tutorial uses AWS Command Line Interface (CLI) to demonstrate how to list and download Landsat objects within an S3 Bucket. Users can find additional information regarding AWS CLI via the AWS CLI documentation website (https://aws.amazon.com/cli/). This tutorial also uses commonly available Python Tools to draw a preview of a Landsat scene by reading the Landsat Data directly from S3.

Topics Covered

  1. Accessing Landsat Data without an AWS Account
    1.1 This section demonstrates the response that users should expect to receive when trying to access the usgs-landsat S3 Bucket with AWS CLI without an AWS Account.
    1.2 This section demonstrates the response that users should expect to receive when trying to access the usgs-landsat S3 Bucket using Python tools without an AWS Account.
  2. Accessing Landsat Data with an AWS Account
    2.1 This section demonstrates the response users should expect to receive when trying to access the usgs-landsat S3 Bucket with AWS CLI with an AWS Account.
    2.2 This section demonstrates the response users should expect to receive when trying to access the usgs-landsat S3 Bucket using Python tools with an AWS Account.

1. Accessing Landsat Data without an AWS Account

Important: The usgs-landsat S3 bucket is a Requester Pays Bucket where the requester (users) may incur costs for accessing the data. If a requester pays acknowledgement is not properly passed with the request the AWS CLI command or python call will return a denial of service error. This section provides an example of the errors returned when the user has not properly included the requester pays acknowledgement within the AWS CLI or python request.

1.1 AWS Command Line Response when users to not have an AWS Account

This command will list the contents within the AWS Bucket

This command will download an object in S3 to the user's local computer

1.2 Python Response when users do not have an AWS Account

Import Packages

Load Cloud Optimized GeoTIFF (COG) Band File from the S3 Bucket

Read COG Band File from the S3 Bucket and Display It

Important: This will produce an error message because we have not properly authenticated to with AWS to agree to the terms of the requester pays bucket. Before running the next section clear the kernel before continuing.

2. Accessing Landsat Data with an AWS Account

Important: The usgs-landsat S3 bucket is a Requester Pays Bucket where the requester (user) may incur costs for accessing the data. This section demonstrates sample responses returned to the user’s computer when users have acknowledged requester pays (agreed to pay any incurred charges) for both AWS CLI commands and python requests.

2.1 AWS Command Line Response when the user has an AWS Account

This command will list the contents within the AWS Bucket

This commnad will download an object in S3 to the user's local computer

2.2 Python Response when users have an AWS Account

Import Packages

Setup AWS Requester Pays Environment

Load Cloud Optimized GeoTIFF (COG) Band File from the S3 Bucket

Read COG Band File from the S3 Bucket and Display It

Congratulations! You have now not only learned how to access the Landsat Data stored within the AWS S3 Cloud Storage and how to properly work with a requester pays bucket. You can incorporate this tutorial into your workflow. We anticipate releasing future tutorials on how users can benefit from using Landsat Data stored in AWS in their workflows

Contact Information

Material written by Nathan Roberts1

    Contact: custserv@usgs.gov
    Voice: +1-605-594-6151
    Date last modified: 11-16-2020
1KBR Inc., contractor to the U.S. Geological Survey, Earth Resources Observation and Science (EROS) Center, Sioux Falls, South Dakota, 57198-001, USA. Work performed under USGS contract TSSC.