ImportError: cannot import name 'is_s3express_bucket' from 'botocore.utils' when lambda update-function-code

[yamachu]

Describe the bug

After I updated awscli to 1.32.25 and now aws lambda update-function-code fails.

aws lambda update-function-code --function-name my-function --zip-file fileb://./migration/__dist/function.zip
Traceback (most recent call last):
  File "/home/circleci/.local/bin/aws", line 27, in <module>
    sys.exit(main())
  File "/home/circleci/.local/bin/aws", line 23, in main
    return awscli.clidriver.main()
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/clidriver.py", line 69, in main
    driver = create_clidriver()
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/clidriver.py", line 78, in create_clidriver
    load_plugins(session.full_config.get('plugins', {}),
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/plugin.py", line 44, in load_plugins
    modules = _import_plugins(plugin_mapping)
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/plugin.py", line 61, in _import_plugins
    module = __import__(path, fromlist=[module])
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/handlers.py", line 78, in <module>
    from awscli.customizations.s3.s3 import s3_plugin_initialize
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/customizations/s3/s3.py", line 15, in <module>
    from awscli.customizations.s3.subcommands import ListCommand, WebsiteCommand, \
  File "/home/circleci/.local/lib/python3.9/site-packages/awscli/customizations/s3/subcommands.py", line 18, in <module>
    from botocore.utils import is_s3express_bucket
ImportError: cannot import name 'is_s3express_bucket' from 'botocore.utils' (/home/circleci/.pyenv/versions/3.9.18/lib/python3.9/site-packages/botocore/utils.py)

Expected Behavior

Aws cli working and able to update lambda functions.

Current Behavior

Aws cli cannot update lambda functions.

Reproduction Steps

pip install --user awscli

Possible Solution

No response

Additional Information/Context

Collecting awscli
  Obtaining dependency information for awscli from https://files.pythonhosted.org/packages/57/86/532bb5246cda791d01895f0c576ef1e8e6ab9cc226ad26cb06767f97b0bb/awscli-1.32.25-py3-none-any.whl.metadata
  Downloading awscli-1.32.25-py3-none-any.whl.metadata (11 kB)
Collecting botocore==1.34.25 (from awscli)
  Obtaining dependency information for botocore==1.34.25 from https://files.pythonhosted.org/packages/12/90/d54466c673c4a3ab376b41dd88fcbfda30771650fbc5c5161884831392fa/botocore-1.34.25-py3-none-any.whl.metadata
  Downloading botocore-1.34.25-py3-none-any.whl.metadata (5.7 kB)
Collecting docutils<0.17,>=0.10 (from awscli)
  Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 548.2/548.2 kB 13.2 MB/s eta 0:00:00

Collecting s3transfer<0.11.0,>=0.10.0 (from awscli)
  Obtaining dependency information for s3transfer<0.11.0,>=0.10.0 from https://files.pythonhosted.org/packages/12/bb/7e7912e18cd558e7880d9b58ffc57300b2c28ffba9882b3a54ba5ce3ebc4/s3transfer-0.10.0-py3-none-any.whl.metadata
  Downloading s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB)
Collecting PyYAML<6.1,>=3.10 (from awscli)
  Obtaining dependency information for PyYAML<6.1,>=3.10 from https://files.pythonhosted.org/packages/7d/39/472f2554a0f1e825bd7c5afc11c817cd7a2f3657460f7159f691fbb37c51/PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting colorama<0.4.5,>=0.2.5 (from awscli)
  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting rsa<4.8,>=3.1.2 (from awscli)
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting jmespath<2.0.0,>=0.7.1 (from botocore==1.34.25->awscli)
  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.34.25->awscli)
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 38.9 MB/s eta 0:00:00

Collecting urllib3<1.27,>=1.25.4 (from botocore==1.34.25->awscli)
  Obtaining dependency information for urllib3<1.27,>=1.25.4 from https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl.metadata
  Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 11.5 MB/s eta 0:00:00

Collecting pyasn1>=0.1.3 (from rsa<4.8,>=3.1.2->awscli)
  Obtaining dependency information for pyasn1>=0.1.3 from https://files.pythonhosted.org/packages/d1/75/4686d2872bf2fc0b37917cbc8bbf0dd3a5cdb0990799be1b9cbf1e1eb733/pyasn1-0.5.1-py2.py3-none-any.whl.metadata
  Downloading pyasn1-0.5.1-py2.py3-none-any.whl.metadata (8.6 kB)
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.34.25->awscli)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading awscli-1.32.25-py3-none-any.whl (4.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 128.0 MB/s eta 0:00:00

Downloading botocore-1.34.25-py3-none-any.whl (11.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 125.5 MB/s eta 0:00:00

Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 738.9/738.9 kB 34.2 MB/s eta 0:00:00

Downloading s3transfer-0.10.0-py3-none-any.whl (82 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.1/82.1 kB 17.0 MB/s eta 0:00:00

Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.9/84.9 kB 19.9 MB/s eta 0:00:00

Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 28.7 MB/s eta 0:00:00

Installing collected packages: urllib3, six, PyYAML, pyasn1, jmespath, docutils, colorama, rsa, python-dateutil, botocore, s3transfer, awscli
Successfully installed PyYAML-6.0.1 awscli-1.32.25 botocore-1.34.25 colorama-0.4.4 docutils-0.16 jmespath-1.0.1 pyasn1-0.5.1 python-dateutil-2.8.2 rsa-4.7.2 s3transfer-0.10.0 six-1.16.0 urllib3-1.26.18

[notice] A new release of pip is available: 23.2.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip

CLI version used

1.32.25

Environment details (OS name and version, etc.)

CircleCI ubuntu-2204:2023.10.1

[kdaily]

Hi! We were not able to reproduce this installing the latest AWS CLI into a new virtual environment. It does look like you're installing the latest version of botocore (botocore-1.34.25), but for some reason it seems like you're using a version that doesn't have the change. It looks like this was added in botocore-1.33.2 (commit).

Could you:

1. Provide the output of the command with --debug?
2. Provide the output of pip freeze?

[yamachu]

Thank you for your quick reply.
I have investigated and it seems that pip install botocore==1.17.20 was done in the latter workflow and there was a conflict.

The following is how pip freeze was run.

circleci@7ba99a64421f:~/repo$  pip freeze
argcomplete==3.1.1
awscli==1.32.25
boto3==1.14.20
boto3-stubs==1.14.48.0
botocore==1.17.20
CacheControl==0.12.5
cachetools==3.0.0
certifi==2018.11.29
chardet==3.0.4
click==8.1.7
colorama==0.4.4
distlib==0.3.7
docutils==0.15.2
filelock==3.12.2
firebase-admin==2.14.0
google-api-core==1.21.0
google-api-python-client==1.9.3
google-auth==1.18.0
google-auth-httplib2==0.0.4
google-cloud-core==0.29.0
google-cloud-firestore==0.30.1
google-cloud-storage==1.13.1
google-resumable-media==0.3.1
googleapis-common-protos==1.52.0
grpcio==1.60.0
httplib2==0.18.1
idna==2.8
jmespath==0.10.0
msgpack==0.6.0
mypy-boto3==1.14.48.0
mypy-boto3-ce==1.14.48.0
mypy-boto3-s3==1.14.48.0
mypy-boto3-sqs==1.14.48.0
mysqlclient==1.3.14
packaging==23.1
pipenv==2023.8.23
pipx==1.2.0
platformdirs==3.10.0
protobuf==3.12.2
pyasn1==0.4.4
pyasn1-modules==0.2.2
python-dateutil==2.8.1
pytz==2018.7
PyYAML==6.0.1
requests==2.21.0
rsa==4.0
s3transfer==0.3.3
six==1.12.0
uritemplate==3.0.1
urllib3==1.24.1
userpath==1.9.0
virtualenv==20.24.3

Therefore, the problem was not caused by an upgrade of the awscli version.
Sorry about that.

[kdaily]

Thanks for confirming! Closing this out. Let us know if you encounter any issues.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.