Is it possible to use more than one set of S3 credentials on a .map file, when using VSIS3? Say just for visualization that .vrt files are stored on an AWS S3 bucket, while the images where .vrt reaches are stored on Google Cloud S3.
1 Answers
AWS credentials can only be set in the MAP section of a Mapfile as CONFIG options - which then set the GDAL environment variables as documented at https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files
MAP
NAME "Test"
CONFIG "AWS_ACCESS_KEY_ID" "XXXXX"
CONFIG "AWS_SECRET_ACCESS_KEY" "XXXXX"
CONFIG "AWS_S3_ENDPOINT" "s3.amazonaws.com"
Update based on the https://gdal.org/user/virtual_file_systems.html link from @Turo:
For different credentials for different buckets, the following approach can be used.
- Set the GDAL_CONFIG_FILE environment variable in the MapServer CONFIG file:
CONFIG
ENV
GDAL_CONFIG_FILE "/etc/mapserver/mapfiles/gdal.conf"
END
END
In the gdal.conf set credentials for different paths used:
[credentials]
need to restart MapServer if these values change
[.sentinel-cogs]
path=/vsis3/sentinel-cogs
AWS_ACCESS_KEY_ID=XXXX
AWS_SECRET_ACCESS_KEY=XXXX
AWS_REQUEST_PAYER=requester
defaults
#AWS_S3_ENDPOINT=s3.amazonaws.com
#AWS_DEFAULT_REGION=us-east-1
[.sentinel-cogs2]
path=/vsis3/sentinel-cogs2
AWS_ACCESS_KEY_ID=XXXX
AWS_SECRET_ACCESS_KEY=XXXX
AWS_REQUEST_PAYER=requester
Now a Mapfile will use different credentials based on the LAYER DATA clause:
LAYER
# will use sentinel-cogs settings
DATA "/vsis3/sentinel-cogs/sentinel-s2-l2a-cogs/43/R/FL/2022/2/S2A_43RFL_20220213_0_L2A/B01.tif"
...
LAYER
# will use sentinel-cogs2 settings
DATA "/vsis3/sentinel-cogs2/sentinel-s2-l2a-cogs/43/R/FL/2022/2/S2A_43RFL_20220213_0_L2A/B01.tif"
- 14,320
- 4
- 53
- 77
-
I see an option of a config file to be used as well in GDAL https://gdal.org/user/configoptions.html and there is specifically an example with multiple S3 paths. Any chances this GDAL setting could be respected by mapserver as well? did not have a chance to check this yet. – Turo Nov 13 '22 at 18:01
-
@Turo - good find. In theory creating the GDAL config file and setting GDAL_CONFIG_FILE in a MAP file or MapServer CONFIG file should work. – geographika Nov 14 '22 at 08:03
-
@Turo - I can confirm the GDAL config file works with MapServer - answer updated. – geographika Nov 14 '22 at 13:06
-
thank you for your insight and check. – Turo Nov 15 '22 at 10:57