Hi all,
I’ve struggled to install the apache-airflow-providers-apache-hdfs package into Airflow - Docker 2.5.3 container on Mac.
My docker version: Docker version 20.10.17, build 100c701
I tried to include the HDFS library in the requirements.txt, and it seems to install properly. However, it seems the HDFS library requires the Kerbedos library, and it causes this error.
I’m not sure what to do to fix this error. I appreciate any help and guidance.
This is the error message:
> [airflow-scs-airflow-worker 3/3] RUN pip install --no-cache-dir --user -r /requirements.txt;:
#0 1.691 Collecting gssapi==1.8.2
#0 1.825 Downloading gssapi-1.8.2.tar.gz (94 kB)
#0 1.860 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.3/94.3 kB 2.7 MB/s eta 0:00:00
#0 1.892 Installing build dependencies: started
#0 6.210 Installing build dependencies: finished with status 'done'
#0 6.212 Getting requirements to build wheel: started
#0 6.487 Getting requirements to build wheel: finished with status 'error'
#0 6.491 error: subprocess-exited-with-error
#0 6.491
#0 6.491 × Getting requirements to build wheel did not run successfully.
#0 6.491 │ exit code: 1
#0 6.491 ╰─> [21 lines of output]
#0 6.491 /bin/sh: 1: krb5-config: Permission denied
#0 6.491 Traceback (most recent call last):
#0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#0 6.491 main()
#0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#0 6.491 json_out['return_val'] = hook(**hook_input['kwargs'])
#0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
#0 6.491 return hook(config_settings)
#0 6.491 File "/tmp/pip-build-env-zzg3__cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
#0 6.491 return self._get_build_requires(config_settings, requirements=['wheel'])
#0 6.491 File "/tmp/pip-build-env-zzg3__cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
#0 6.491 self.run_setup()
#0 6.491 File "/tmp/pip-build-env-zzg3__cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
#0 6.491 exec(code, locals())
#0 6.491 File "<string>", line 109, in <module>
#0 6.491 File "<string>", line 22, in get_output
#0 6.491 File "/usr/local/lib/python3.10/subprocess.py", line 421, in check_output
#0 6.491 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
#0 6.491 File "/usr/local/lib/python3.10/subprocess.py", line 526, in run
#0 6.491 raise CalledProcessError(retcode, process.args,
#0 6.491 subprocess.CalledProcessError: Command 'krb5-config --libs gssapi' returned non-zero exit status 127.
#0 6.491 [end of output]