Docker run -it not displaying output logs on vscode using wsl2

Docker for AWS
, ,
1

Expected behavior

when running docker run -it with my params I’m expecting to output on bash terminal the execution of Spark session from my python code.

Actual behavior

when running docker run -it with my params it is displaing less outut logs than expected.

Additional Information

the source code was tested from another computer running exact same config on windws/wsl2/docker and output all logs.

the docker run command looks like this

docker run -it -e AWS_REGION="my_region" \
-v ~/.aws:/home/glue_user/.aws -v $WORKSPACE_LOCATION:/home/glue_user/workspace/ -e AWS_PROFILE=$PROFILE_NAME -e DISABLE_SSL=true --rm -p 4040:4040 -p 18080:18080 --name glue_spark_submit "image_name" \
spark-submit --jars /home/glue_user/workspace/glue-jars/jconn4.jar \
/home/glue_user/workspace/$SCRIPT_FILE_NAME \
--JOB_NAME glue-teste-bq-hist --job-conf-name conf-exemplo-carga-bronze-sqlserver --artifactory-bucket "my_bucket" --bookmark-table "my_table" --account-id "my_account_id" --aws-region "my_region" --environment dev --account-id-shared "my_account_shared"

wsl2 --version

WSL version: 2.2.4.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.4169

docker version

Docker version 27.2.1, build 9e34c9b

python --version

Python 3.11.6

.wslconfig

[wsl2]
memory=4GB  # Limita a memória a 4GB
processors=2  # Limita o uso de CPU a 2 núcleos
localhostForwarding=true  # Habilita o encaminhamento de localhost
swap=0  # Desabilita o uso de swap
debugShell=true  # Habilita o shell de depuração

Steps to reproduce the behavior

  1. clone repo
  2. export variables
  3. create dockerfile
  4. build docker image
  5. execure docker run -it with my params
2

I know python requires the env variable PYTHONUNBUFFERED=1 to not use buffers and show the message immediately depending on how you run the contaner. If you have problem with python, that is what you should try. If you jave problem with Java, I don’t know what that needs to work properly. I didn’t have problem with that yet.

One more small note: When you use the logger object in Python, that sends the output properly while printf by default doesn’t. I hope it at least give you some ideas even if your problem is with Java.

1 Like