This tutorial demonstrates how users access Landsat Data stored within the AWS Cloud environment using basic commands. Landsat data stored in the AWS Cloud is located within U.S. West (Oregon) us-west-2 region in a requester pays Simple Storage Services (S3) bucket. Users interested in utilizing direct access to Landsat data stored in S3 are encouraged to visit the Requester Pays Documentation Page (https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html) for information concerning potential egress costs to accessing Landsat data with this method.
AWS Bucket Name: usgs-landsat (Requester Pays)
AWS Region: US West (Oregon) us-west-2
An AWS account is required before undertaking this tutorial. Please see the AWS Account website to establish an account (https://aws.amazon.com/account/).
This tutorial uses AWS Command Line Interface (CLI) to demonstrate how to list and download Landsat objects within an S3 Bucket. Users can find additional information regarding AWS CLI via the AWS CLI documentation website (https://aws.amazon.com/cli/). This tutorial also uses commonly available Python Tools to draw a preview of a Landsat scene by reading the Landsat Data directly from S3.
Important: The usgs-landsat S3 bucket is a Requester Pays Bucket where the requester (users) may incur costs for accessing the data. If a requester pays acknowledgement is not properly passed with the request the AWS CLI command or python call will return a denial of service error. This section provides an example of the errors returned when the user has not properly included the requester pays acknowledgement within the AWS CLI or python request.
! aws s3 ls s3://usgs-landsat/collection02/
! aws s3api get-object --bucket usgs-landsat --key collection02/level-2/standard/oli-tirs/2020/026/027/LC08_L2SP_026027_20200827_20200906_02_T1/LC08_L2SP_026027_20200827_20200906_02_T1_MTL.txt LC08_L2SP_026027_20200827_20200906_02_T1_MTL.txt
from matplotlib.pyplot import imshow
import rasterio as rio
%matplotlib inline
cog = 's3://usgs-landsat/collection02/level-2/standard/oli-tirs/2020/026/027/LC08_L2SP_026027_20200827_20200906_02_T1/LC08_L2SP_026027_20200827_20200906_02_T1_SR_B2.TIF'
with rio.open(cog) as src:
profile = src.profile
arr = src.read(1)
imshow(arr)
Important: The usgs-landsat S3 bucket is a Requester Pays Bucket where the requester (user) may incur costs for accessing the data. This section demonstrates sample responses returned to the user’s computer when users have acknowledged requester pays (agreed to pay any incurred charges) for both AWS CLI commands and python requests.
! aws s3 ls s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2020/026/027/LC08_L2SP_026027_20200827_20200906_02_T1/ --request-payer requester
2020-09-23 22:24:56 117122 LC08_L2SP_026027_20200827_20200906_02_T1_ANG.txt 2020-09-23 22:25:00 14549 LC08_L2SP_026027_20200827_20200906_02_T1_MTL.json 2020-09-23 22:24:57 15424 LC08_L2SP_026027_20200827_20200906_02_T1_MTL.txt 2020-09-23 22:24:57 22726 LC08_L2SP_026027_20200827_20200906_02_T1_MTL.xml 2020-09-20 02:51:33 2670023 LC08_L2SP_026027_20200827_20200906_02_T1_QA_PIXEL.TIF 2020-09-20 02:51:33 230417 LC08_L2SP_026027_20200827_20200906_02_T1_QA_RADSAT.TIF 2020-09-20 02:51:33 74920387 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B1.TIF 2020-09-20 02:51:34 72961917 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B2.TIF 2020-09-20 02:51:34 74604511 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B3.TIF 2020-09-20 02:51:35 73094885 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B4.TIF 2020-09-20 02:51:36 82065367 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B5.TIF 2020-09-20 02:51:36 77232505 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B6.TIF 2020-09-20 02:51:37 73160584 LC08_L2SP_026027_20200827_20200906_02_T1_SR_B7.TIF 2020-09-20 02:51:37 3614439 LC08_L2SP_026027_20200827_20200906_02_T1_SR_QA_AEROSOL.TIF 2020-09-23 22:25:13 11265 LC08_L2SP_026027_20200827_20200906_02_T1_SR_stac.json 2020-09-20 02:51:38 23059897 LC08_L2SP_026027_20200827_20200906_02_T1_ST_ATRAN.TIF 2020-09-20 02:51:38 72849244 LC08_L2SP_026027_20200827_20200906_02_T1_ST_B10.TIF 2020-09-20 02:51:39 20444128 LC08_L2SP_026027_20200827_20200906_02_T1_ST_CDIST.TIF 2020-09-20 02:51:39 14257440 LC08_L2SP_026027_20200827_20200906_02_T1_ST_DRAD.TIF 2020-09-20 02:51:39 26285103 LC08_L2SP_026027_20200827_20200906_02_T1_ST_EMIS.TIF 2020-09-20 02:51:40 11855588 LC08_L2SP_026027_20200827_20200906_02_T1_ST_EMSD.TIF 2020-09-20 02:51:40 25311062 LC08_L2SP_026027_20200827_20200906_02_T1_ST_QA.TIF 2020-09-20 02:51:41 57800409 LC08_L2SP_026027_20200827_20200906_02_T1_ST_TRAD.TIF 2020-09-20 02:51:41 21904540 LC08_L2SP_026027_20200827_20200906_02_T1_ST_URAD.TIF 2020-09-23 22:25:13 10462 LC08_L2SP_026027_20200827_20200906_02_T1_ST_stac.json 2020-09-23 22:24:57 72973 LC08_L2SP_026027_20200827_20200906_02_T1_thumb_large.jpeg 2020-09-23 22:24:57 9992 LC08_L2SP_026027_20200827_20200906_02_T1_thumb_small.jpeg
! aws s3api get-object --bucket usgs-landsat --key collection02/level-2/standard/oli-tirs/2020/026/027/LC08_L2SP_026027_20200827_20200906_02_T1/LC08_L2SP_026027_20200827_20200906_02_T1_MTL.txt --request-payer requester LC08_L2SP_026027_20200827_20200906_02_T1_MTL.txt
{ "AcceptRanges": "bytes", "LastModified": "Wed, 23 Sep 2020 22:24:57 GMT", "ContentLength": 15424, "ETag": "\"cefa64a7e93a288f35b9d5bcadec30dc\"", "VersionId": "_w0A_JeRBgXkmKetbl2yhq12m9uN3Y9H", "ContentType": "text/plain", "Metadata": {}, "RequestCharged": "requester", "TagCount": 3 }
import boto3
from matplotlib.pyplot import imshow
import rasterio as rio
from rasterio.session import AWSSession
aws_session = AWSSession(boto3.Session(), requester_pays=True)
cog = 's3://usgs-landsat/collection02/level-2/standard/oli-tirs/2020/026/027/LC08_L2SP_026027_20200827_20200906_02_T1/LC08_L2SP_026027_20200827_20200906_02_T1_SR_B2.TIF'
with rio.Env(aws_session):
with rio.open(cog) as src:
profile = src.profile
arr = src.read(1)
imshow(arr)
<matplotlib.image.AxesImage at 0x7ff9e7a31890>