[awscli] aws-cli / aws-shell installation and configure aws credential

AWS

AWS CLI

Installation

About how to install pip3, you can refer to Python3 Installation.

It is very simple to install awscli by pip3 – https://docs.aws.amazon.com/cli/latest/userguide/awscli-install-linux.html


$ sudo pip3 install awscli
The directory '/home/legochen/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/legochen/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting awscli
Downloading https://files.pythonhosted.org/packages/cc/1a/bb0bc699b37a766736b0c07a7344b1b985deb16870e9d14c75110ae74256/awscli-1.15.64-py2.py3-none-any.whl (1.3MB)
100% |████████████████████████████████| 1.3MB 741kB/s
Collecting colorama<=0.3.9,>=0.2.5 (from awscli)
Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
Downloading https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl (46kB)
100% |████████████████████████████████| 51kB 3.8MB/s
Collecting PyYAML<=3.13,>=3.10 (from awscli)
Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
100% |████████████████████████████████| 276kB 2.1MB/s
Requirement already satisfied: s3transfer<0.2.0,>=0.1.12 in /opt/python3/lib/python3.6/site-packages (from awscli)
Collecting botocore==1.10.63 (from awscli)
Downloading https://files.pythonhosted.org/packages/57/59/a1221949de618cda560017c9db004d88cc51a5956f6a778f42d54cd190d3/botocore-1.10.63-py2.py3-none-any.whl (4.4MB)
100% |████████████████████████████████| 4.4MB 239kB/s
Requirement already satisfied: docutils>=0.10 in /opt/python3/lib/python3.6/site-packages (from awscli)
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
Downloading https://files.pythonhosted.org/packages/a0/70/2c27740f08e477499ce19eefe05dbcae6f19fdc49e9e82ce4768be0643b9/pyasn1-0.4.3-py2.py3-none-any.whl (72kB)
100% |████████████████████████████████| 81kB 7.3MB/s
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/python3/lib/python3.6/site-packages (from botocore==1.10.63->awscli)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /opt/python3/lib/python3.6/site-packages (from botocore==1.10.63->awscli)
Requirement already satisfied: six>=1.5 in /opt/python3/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore==1.10.63->awscli)
Installing collected packages: colorama, pyasn1, rsa, PyYAML, botocore, awscli
Running setup.py install for PyYAML ... done
Found existing installation: botocore 1.10.62
Uninstalling botocore-1.10.62:
Successfully uninstalled botocore-1.10.62
Successfully installed PyYAML-3.13 awscli-1.15.64 botocore-1.10.63 colorama-0.3.9 pyasn1-0.4.3 rsa-3.4.2
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ which aws
/opt/python3/bin/aws

Configure aws cli auto-completion ( useful )

https://docs.aws.amazon.com/cli/latest/userguide/cli-command-completion.html

Once you installed aws-cli, it would also installed aws_completer , then we can easily enable it as following


$ which aws_completer
/opt/python3/bin/aws_completer

# enable aws-cli autocomplete
$ complete -C /opt/python3/bin/aws_completer aws

# test
$ aws s3 [TAB*2]
cp ls mb mv presign rb rm sync website

You can put the following lines to your .bashrc , then it would auto-load to your bash environment.


aws_completer_path=$(which aws_completer)
if [[ ! -z $aws_completer_path && -e $aws_completer_path ]]; then
complete -C $aws_completer_path aws
fi

Configure AWS credential

https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html

I prefer to manual edit ~/.aws/credentials file instead of use aws configure command.

In ~/.aws/credentials , you can configure multiple profile and after that we can switch profile by set AWS_PROFILE environment variable.

If you don’t specify AWS_PROFILE variable, it would pick default profile as your credential.


$ cat ~/.aws/credentials

[default]
aws_access_key_id=ASIAJYWHKQNUF4WSZSBQ
aws_secret_access_key=zwOYdauYIbFNtcFRo5W1+oZrHvKATXFZbYK3qBkp
aws_session_token=FQoDYXdzEJ3//////////wEaDD/ajdadljkalsjdkalsjdkalsjdald/bbkjbkljklajlsjdasld/mT7I/U75T63yX3MB9DtcsxzmVTsDdzSyTtLlwY1iKYJbXt7QhZV/cSuq927/7B0/9+fAl1zPWl7swEgJaYxVlXUwZl+bbbbbbbbbbbbbbbbbbbbbbbbbbbbb/assaaaaaaaaaaaaaaaaaaaaaaaa/44KUAlTQKKNnCk/bP/asdasdasdddddddddddddasdddddddddddddd+ONOhcF6vaSe0kzmi1gSDb
adadadDJKLJLA/aaaaaaaaaaaaaaaaaaa==

[lab]
aws_access_key_id=ASIAJYWHKQNUF4WSZSBQ
aws_secret_access_key=zwOYdauYIbFNtcFRo5W1+oZrHvKATXFZbYK3qBkp

[work]
assumed_role = sso/fed.admin.user
account_number = 000000000000
account_name =
expiration = 2018-07-24 18:05:48+00:00
aws_access_key_id=AKIAIH6XFWZPW6PE56ZQ
aws_secret_access_key=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

switch profile test


$ AWS_PROFILE=work

$ aws s3 ls --region=ap-northeast-1
2018-05-25 11:02:29 01-legochen-work-bucket

$ AWS_PROFILE=lab

$ aws s3 ls --region=ap-northeast-1
2018-05-26 13:20:29 01-legochen-lab-bucket

Functions to configure AWS variables

####################### AWS
                                                                                                                                                                                         function aws_env() {
  # https://docs.aws.amazon.com/cli/latest/topic/config-vars.html
  echo "AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION"
  echo "AWS_PROFILE: $AWS_PROFILE"
  echo "AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID"
  echo "AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY"
  echo "AWS_SESSION_TOKEN: $AWS_SESSION_TOKEN"
  echo "AWS_CA_BUNDLE: $AWS_CA_BUNDLE"
  echo "AWS_DEFAULT_OUTPUT: $AWS_DEFAULT_OUTPUT"
}
                                                                                                                                                                                         function aws_profile() {
  profile_list=$(cat ~/.aws/credentials | grep -E '\[.*\]' | sed -e 's/\[//g' | sed -e 's/\]//g' | xargs)
  select my_aws_profile in $profile_list
  do
    export AWS_PROFILE=$my_aws_profile
    aws_env
    break
  done
}

function aws_region() {
  # https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html
  region_config="/tmp/aws_region"
  cat <<REGION > $region_config
US East (Ohio): us-east-2
US East (N. Virginia): us-east-1
US West (N. California): us-west-1
US West (Oregon): us-west-2
Asia Pacific (Tokyo): ap-northeast-1
Asia Pacific (Seoul): ap-northeast-2
Asia Pacific (Osaka-Local): ap-northeast-3
Asia Pacific (Mumbai): ap-south-1
Asia Pacific (Singapore): ap-southeast-1
Asia Pacific (Sydney): ap-southeast-2
Canada (Central): ca-central-1
China (Beijing): cn-north-1
China (Ningxia): cn-northwest-1
EU (Frankfurt): eu-central-1
EU (Ireland): eu-west-1
EU (London): eu-west-2
EU (Paris): eu-west-3
South America (São Paulo): sa-east-1
REGION
  IFStmp=$IFS
  IFS=$'\n'
  select region in $(cat $region_config)
  do
    export AWS_DEFAULT_REGION=$(echo $region | awk -F": " '{print $2}')
    break
  done
  aws_env
  unset IFS
}

Usage:


# set profile
$ aws_profile
1) default
2) lab
3) work
#? 3

# set region
$ aws_region
1) US East (Ohio): us-east-2 7) Asia Pacific (Osaka-Local): ap-northeast-3 13) China (Ningxia): cn-northwest-1
2) US East (N. Virginia): us-east-1 8) Asia Pacific (Mumbai): ap-south-1 14) EU (Frankfurt): eu-central-1
3) US West (N. California): us-west-1 9) Asia Pacific (Singapore): ap-southeast-1 15) EU (Ireland): eu-west-1
4) US West (Oregon): us-west-2 10) Asia Pacific (Sydney): ap-southeast-2 16) EU (London): eu-west-2
5) Asia Pacific (Tokyo): ap-northeast-1 11) Canada (Central): ca-central-1 17) EU (Paris): eu-west-3
6) Asia Pacific (Seoul): ap-northeast-2 12) China (Beijing): cn-north-1 18) South America (São Paulo): sa-east-1
#? 5

# check aws env
$ aws_env
AWS_DEFAULT_REGION: ap-northeast-1
AWS_PROFILE: work
AWS_ACCESS_KEY_ID:
AWS_SECRET_ACCESS_KEY:
AWS_SESSION_TOKEN:
AWS_CA_BUNDLE:
AWS_DEFAULT_OUTPUT:

Awesome Tool – AWS-SHELL

AWS also provide aws-shell tool, it  is natively support command line auto-completion, a powerful tool.

ref :

https://aws.amazon.com/blogs/developer/super-charge-your-aws-command-line-experience-with-aws-shell/

https://github.com/awslabs/aws-shell

Installation

The installation step is also simple as much as awscli and can be installed by pip3.


$ sudo pip3 install aws-shell

The directory '/home/legochen/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that $irectory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/legochen/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directo$y. If executing pip with sudo, you may want sudo's -H flag. Collecting aws-shell
Downloading https://files.pythonhosted.org/packages/55/08/497a5a68ae8380aa812ffb782dff9df394fe6dc19861a1ffd13c964f6fbb/aws_shell-0.2.0-py2.py3-none-any.whl (50kB)
100% |████████████████████████████████| 51kB 683kB/s
Collecting prompt-toolkit<1.1.0,>=1.0.0 (from aws-shell)
Downloading https://files.pythonhosted.org/packages/04/d1/c6616dd03701e7e2073f06d5c3b41b012256e42b72561f16a7bd86dd7b43/prompt_toolkit-1.0.15-py3-none-any.whl (247kB)
100% |████████████████████████████████| 256kB 1.8MB/s
Collecting configobj<6.0.0,>=5.0.6 (from aws-shell)
Downloading https://files.pythonhosted.org/packages/64/61/079eb60459c44929e684fa7d9e2fdca403f67d64dd9dbac27296be2e0fab/configobj-5.0.6.tar.gz
Requirement already satisfied: awscli<2.0.0,>=1.10.30 in /opt/python3/lib/python3.6/site-packages (from aws-shell)
Collecting Pygments<3.0.0,>=2.1.3 (from aws-shell)
Downloading https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl (841kB)
100% |████████████████████████████████| 849kB 1.0MB/s
Requirement already satisfied: boto3<2.0.0,>=1.2.1 in /opt/python3/lib/python3.6/site-packages (from aws-shell)
Collecting wcwidth (from prompt-toolkit<1.1.0,>=1.0.0->aws-shell)
Downloading https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9.0 in /opt/python3/lib/python3.6/site-packages (from prompt-toolkit<1.1.0,>=1.0.0->aws-shell)
Requirement already satisfied: docutils>=0.10 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: s3transfer<0.2.0,>=0.1.12 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: rsa<=3.5.0,>=3.1.2 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: PyYAML<=3.13,>=3.10 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: colorama<=0.3.9,>=0.2.5 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: botocore==1.10.63 in /opt/python3/lib/python3.6/site-packages (from awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/python3/lib/python3.6/site-packages (from boto3<2.0.0,>=1.2.1->aws-shell)
Requirement already satisfied: pyasn1>=0.1.3 in /opt/python3/lib/python3.6/site-packages (from rsa<=3.5.0,>=3.1.2->awscli<2.0.0,>=1.10.30->aws-shell)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /opt/python3/lib/python3.6/site-packages (from botocore==1.10.63->awscli<2.0.0,>=1.10.30->aws-shel
l)
Installing collected packages: wcwidth, prompt-toolkit, configobj, Pygments, aws-shell
Running setup.py install for configobj ... done
Successfully installed Pygments-2.2.0 aws-shell-0.2.0 configobj-5.0.6 prompt-toolkit-1.0.15 wcwidth-0.1.7
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ which aws-shell
/opt/python3/bin/aws-shell

try it !!