I guess Dialogic HMP does not meant to be installed in a Docker container. Searching for “Dialogic hmp Docker” I found a document about installing Dialogix XMS (not HMP) in a Docker konténer but that document refers to a docker image which is 6 years old.
I don’t know what the install.sh does so I can’t tell you how to do it right. After a very quick research It seems to me the HMP installer uses an RPM package. Am I right? So you need to find out how to install a package like that non-interactively. I would help you more if I had more time but I am a little busy until the weekend.
Even if you can figure it out, I am not sure this software will work inside the container.