I have elaborated the problem here so anyone can help me please stuck on it for quite a bit of time
you can try
RUN pip3 install opencv-python-headless
Though the title says FLASK API and the question is FastAPI
if you are looking for FLASK API below can be helpful.