Request timeout in custom server

[Imtinan1996]

Hi, i have written a custom server using model_server.py as a reference, and I have another server deployed using the docker.

Previously i was using flask servers to serve my model. I decided to compare graphpipe and flask results, GP definitely performs better over flask, when I run 100 concurrent requests on the deployed docker, i can see that GP performs 3 times better than flask.

The issue is that when I try to run the same tests on my custom GP server the requests start to time out, why is that the docker will not time out for 100 requests, but the requests time out for my custom server, the input passed to both the docker and custom server consists of an image, but the custom server receives an extra array of 5 floats (bounding box). Is there any additional information that needs to be set when creating the httpd server?

my custom server serves well and is created by referring to the model_server.py in the examples folder