RuntimeError: can't start new thread

Hi,

I’m trying to build a docker image via Jenkins and it is continuously getting failed saying that “RuntimeError: can’t start new thread” . Below are my docker file and requirements file content. Since I’m very new to docker, it will be great if someone can guide me on resolving this error.

I’m currently using docker version 24.0.6, build ed223bc

Docker file:

FROM python:3.9-slim
WORKDIR /app
COPY . .
RUN pip install -r ‘requirements.txt’
EXPOSE 8501
HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health
ENTRYPOINT [“streamlit”, “run”, “CustomerSupport8.py”, “–server.port=8501”, “–server.address=0.0.0.0”]

Requirements.txt

openai==0.28.1
streamlit==1.27.0
langchain==0.0.316
numpy==1.18.0
pandas==2.0.3
pypdf==3.7.0
streamlit-chat==0.1.1
tiktoken==0.3.2
faiss-cpu==1.7.1

Error:

Status: Downloaded newer image for python:3.9-slim
—> 0540301f15db
Step 2/7 : WORKDIR /app
—> Running in d788cbe7b39a
Removing intermediate container d788cbe7b39a
—> 935037a4877e
Step 3/7 : COPY . .
—> fd81d949066b
Step 4/7 : RUN pip3 install -r requirements.txt
—> Running in c75f005d7e95
Collecting streamlit
Downloading streamlit-1.27.2-py2.py3-none-any.whl (7.6 MB)
ERROR: Exception:
Traceback (most recent call last):
File “/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py”, line 160, in exc_logging_wrapper
status = run_func(*args)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py”, line 247, in wrapper
return func(self, options, args)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py”, line 419, in run
requirement_set = resolver.resolve(
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py”, line 92, in resolve
result = self._result = resolver.resolve(
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py”, line 481, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py”, line 348, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py”, line 172, in _add_to_criteria
if not criterion.candidates:
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py”, line 151, in bool
return bool(self._sequence)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py”, line 155, in bool
return any(self)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py”, line 143, in
return (c for c in iterator if id(c) not in self._incompatible_ids)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py”, line 47, in _iter_built
candidate = func()
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py”, line 206, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py”, line 297, in init
super().init(
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py”, line 162, in init
self.dist = self._prepare()
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py”, line 231, in _prepare
dist = self._prepare_distribution()
File “/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py”, line 308, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py”, line 491, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py”, line 536, in _prepare_linked_requirement
local_file = unpack_url(
File “/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py”, line 166, in unpack_url
file = get_http_url(
File “/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py”, line 107, in get_http_url
from_path, content_type = download(link, temp_dir.path)
File “/usr/local/lib/python3.9/site-packages/pip/_internal/network/download.py”, line 147, in call
for chunk in chunks:
File “/usr/local/lib/python3.9/site-packages/pip/_internal/cli/progress_bars.py”, line 52, in _rich_progress_bar
with progress:
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py”, line 1169, in enter
self.start()
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py”, line 1160, in start
self.live.start(refresh=True)
File “/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/live.py”, line 132, in start
self._refresh_thread.start()
File “/usr/local/lib/python3.9/threading.py”, line 899, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can’t start new thread

[notice] A new release of pip is available: 23.0.1 → 23.2.1
[notice] To update, run: pip install --upgrade pip

pip uses thread to show progress bar. Try to disable it:

RUN pip install --progress-bar off -r requirements.txt

1 Like

This helps me with pip install, thx!

Could you explain me pls, why i receive same error when i try to use threads\procs in my docker ( container with python).

from multiprocessing.dummy import Pool as ThreadPool

or

import concurrent.futures

or

import threading

All libs had same error:

RuntimeError: can't start new thread
1 Like

I also got this problem, and after I added --privileged when do docker run, the problem fixed, hope it can help you.

1 Like

Thx, it works for me! Could you tell me is that secure? What kind of problems i should expect after use that workaround?