Reading a CSV file in apache flink container filesystem from a java/apache flink application

Docker Desktop for Windows
1

I have an apache flink application created with maven that reads data from a csv file and creates a datastream. the code is below:

public static void main(String[] args) throws Exception {
        final ParameterTool params = ParameterTool.fromArgs(args);
        input = params.get("input");

"
"

FileReader filereader = new FileReader(input);

when I test it in IntelliJ everything works, but when I run the Jar file in the job manager using the command:

./bin/flink run -c test1.RollsCount ./examples/project1.jar --input ./examples/dataset.csv

I get the following error:

Caused by: java.lang.NullPointerException
        at java.base/java.io.FileInputStream.<init>(Unknown Source)

I tried to change the code writing the file path in the java code as follow:

 Reader filereader = new FileReader("/opt/flink/examples/dataset.csv");

but this time I have the following error:

Caused by: java.io.FileNotFoundException: /opt/flink/examples/dataset.csv (No such file or directory)

so it looks like I am not writing correctly the path but looking on internet it looks corect.

2

For some easy debugging: How about you output the parameter and list the files of your current and the target directory in your code?

3

Maybe I miss something, but can you explain how it is related to Docker? I only see shell commands and Java code.

4

thanks for the tip! I will try that.

5

The problem is that the application runs perfectly from intellij. when I run it on Docker container generated from apache flink image the file path is not recognized and I do not know what is wrong.

I did some more testing and it looks like the file path is recognized if the filereader is in the main method, but if it is in a separated class no.