You can use build arguments to specify the python version
https://docs.docker.com/reference/dockerfile/#arg
and place your different files in different folders containing the python version like:
./files/common/
./files/python-3.8/
./files/python-3.5/
Example:
ARG PYTHON_VERSION
FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
ARG PYTHON_VERSION
COPY ./files/common/ /files/
COPY ./files/python-${PYTHON_VERSION}/ /files/
RUN python${PYTHON_VERSION} ....
docker build . --build-arg PYTHON_VERSION=3.8 -t tag
ARG must be defined before and after FROM so you can use it in the FROM instruction and also after it.