1

I am running an apache web server on debian 6.0 with kernel 2.6.32-5-amd64.

In certain times of the day, usually when more people are online, the server have very slow responses. I am sure, that is not load, it is not DB, it is not PHP, accualy this command:

time wget 127.0.0.1

is quite slow:

--2013-09-20 15:36:49--  http://127.0.0.1/
Connecting to 127.0.0.1:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2013-09-20 15:36:52 ERROR 404: Not Found.


real    0m2.663s
user    0m0.004s
sys     0m0.004s

Response time is up to 5sec. The point of waiting is after "HTTP request sent, awaiting response..." Page generating after this lag common for all http requests is fast, like 50ms.

I tested my web using http://tools.pingdom.com/fpt/, first response is always slow, most of content load is also slow but few are quite fast, but I cannot tell, if this tool have some sort of cache.

telnet 127.0.0.1 80

is fast, the response show without delay:

Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Where can be the problem? It must be something before the request goes to apache.

Please tell me at least about some debugging tools for this.

Maťo Urbašík
  • 11
  • You should make a wget on an existing site. In your case you get a 404. – René Höhle Sep 20 '13 at 14:14
  • How many people you have online ?, what is your apache configuration ? maybe your max apache request is to low. – Skamasle Sep 20 '13 at 14:18
  • You really need to provide more information. What sort of page is getting loaded, how many requests per page, how many people, web server logs, top etc – Drew Khoury Sep 20 '13 at 14:44
  • 404 error does not matter. When everything is OK, 404 response is fast, as should be:

    --2013-09-23 08:11:15-- http://localhost/

    Resolving localhost... 127.0.0.1 Connecting to localhost|127.0.0.1|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2013-09-23 08:11:15 ERROR 404: Not Found.

    real 0m0.003s user 0m0.004s sys 0m0.000s

     – Maťo Urbašík Sep 23 '13 at 06:12
  • @Skamasle Cca 200 people online according to google analytics realtime.

    I dont think the bottleneck is PHP. As I wrote, Im sure PHP scripts is fast, it also means DB queries. First thing in my PHPs is get microtime(), last is get microtime() again and write difference out. When request took 3 seconds, PHP generating with all the quesies is like 50-100ms.

    I also dont think the bottle neck is Apache. I added %T/%D to the access log, but in log, even when request is slow, it is always something like 0/42904

     – Maťo Urbašík Sep 23 '13 at 07:02
  • What about IO wait ? and disk speed ? and are you on dedicated server or VPS ? – Skamasle Sep 24 '13 at 03:23

1 Answers1

1

I don't think 2.6 seconds is terribly slow for your average site that wasn't created by a team of full time engineers (See: google.com).

Your problem is probably in I/O, either network or disk or memory or etc.

You're going to have to trace through the application stack. This involved looking into your web server software (apache httpd, ngnix, etc), the php stack, the php code, and any backend storage mechanisms like File systems and MySQL that support the stack.

The delay could be caused by a slow disk, an unindexed mysql query, a bottleneck at the network, or even an optimized loop or function in the code. I don't think we will be able to help you much in this unless you can provide details about your environment and ask specific questions.

Daniel Widrick
  • 3,488
  • 2
  • 13
  • 27
  • 2.6 seconds IS terribly slow, when the content itself is generated in 50-100ms.

    Can you give me some tips for performance monitoring tools on IO, network, HDD and memory?

    MySQL is not the issue, queries are fast. I'm monitoring them.

     – Maťo Urbašík Sep 23 '13 at 07:03
  • The server behavior makes it sound like there is a bit of caching going on. SSL session caching, content caching, MySQL query caching, Linux file caching, etc... All would cause new requests to be slow and subsequent ones quite quick. Without metrics on the whole stack, we can't diagnose it in a specific manor. – Daniel Widrick Sep 23 '13 at 07:26
  • What can I send here, that'll be relevant to the problem? Configs, logs, dmesg, ...? – Maťo Urbašík Sep 23 '13 at 20:19
  • You will likely have to find someone willing to audit your entire stack. Look at the code, the server, the back ends, the network etc. There is a problem.. But right now.. It could be anywhere. – Daniel Widrick Sep 23 '13 at 20:22