1
Skythe
Xoops overloading my Server
  • 2005/4/18 22:02

  • Skythe

  • Just popping in

  • Posts: 99

  • Since: 2004/7/2 2


Hi there

I got a problem with my Xoops driven website. Today it went down and I asked my Host about it. He said it was my fault for having more than 260 database processes running at certain times of the evening. (same thing with my previous hosts....)

Between 5 pm and midnight the site's got an average of 40,000 hits (or 160 pages) per hour. Now he's telling me to upgrade to a root or managed server which I'm not gonna do. I'm already paying a LOT and hardly use 50% of the 200 GB bandwidth.

Comments, help? :(

Modules I'm using are news & newbb, thats it. The rest is static galleries. They supposedly dont use the database at all.

2
Dave_L
Re: Xoops overloading my Server
  • 2005/4/18 23:08

  • Dave_L

  • XOOPS is my life!

  • Posts: 2277

  • Since: 2003/11/7


Check that you don't have persistent connections enabled in mainfile.php.

Caching modules and/or blocks can also reduce database queries.

If that doesn't help, here's a way of logging all the database queries. It may help if a specific client (such as a search robot), or a particular XOOPS module, is responsible for the overload.

https://xoops.org/modules/smartfaq/faq.php?faqid=462

3
Skythe
Re: Xoops overloading my Server
  • 2005/4/19 15:21

  • Skythe

  • Just popping in

  • Posts: 99

  • Since: 2004/7/2 2


Quote:

Dave_L wrote:
Check that you don't have persistent connections enabled in mainfile.php.

Caching modules and/or blocks can also reduce database queries.

If that doesn't help, here's a way of logging all the database queries. It may help if a specific client (such as a search robot), or a particular XOOPS module, is responsible for the overload.

https://xoops.org/modules/smartfaq/faq.php?faqid=462


Persistent connections and module-wide cache are off. The blocks I checked all had cache disabled too.

I was gonna try that hack but I dont quite get it. Whats that function XoopsMySQLDatabase::queryF part? Cant find that or logdir in my mysqldatabase.php.

4
Dave_L
Re: Xoops overloading my Server
  • 2005/4/19 18:46

  • Dave_L

  • XOOPS is my life!

  • Posts: 2277

  • Since: 2003/11/7


Turning caching on will reduce the number of database queries. Maybe I wasn't clear about that. I don't know if it will reduce the number sufficiently.

I edited that FAQ to clarify the instructions. But don't hack that file unless you're comfortable doing it.

5
Skythe
Re: Xoops overloading my Server
  • 2005/4/19 19:02

  • Skythe

  • Just popping in

  • Posts: 99

  • Since: 2004/7/2 2


Ok I'll try with Caching first and ask my Host if that reduced the server load. What time period should I set the modules to? 30s, 1h, 1 day, 1 week?

thanks for your help so far Dave



Edit: I hacked that file and got the first log just now. Lots of ips, browser ids, and stuff. Reminds me of counter stats. What should I be looking for in the logs?

6
Dave_L
Re: Xoops overloading my Server
  • 2005/4/19 19:40

  • Dave_L

  • XOOPS is my life!

  • Posts: 2277

  • Since: 2003/11/7


You may have to experiment with the cache settings. But be careful not to cache content that depends on the user's access level, e.g. a block that displays different content for users and admins.

In the log file, I would look for "patterns".

The IP address and user agent identify the client who's accessing the site. If you see 10000 consecutive lines with the same IP address and user agent, that could be a search bot or someone else "spidering" your site.

The script name is also logged, so that can show if a particular module or part of a module is responsible for most of the load.

7
brash
Re: Xoops overloading my Server
  • 2005/4/20 0:12

  • brash

  • Friend of XOOPS

  • Posts: 2206

  • Since: 2003/4/10


Something else that might help you Skythe is using the MySQL counter and script execution timer I created (with Dave's help I might add) which can be found here. Not as granular as the FAQ Dave pointed you to, but what it does is give you the total queries used to generate each page, as well as how long the script(s) took to generate that page. The number of MySQL queries you can effectively use is going to depend mainly on your available processing power and your site traffic. Execution time should help give you a good idea of where this sweet spot sits for your site.

8
iolai
Re: Xoops overloading my Server
  • 2005/4/20 0:35

  • iolai

  • Quite a regular

  • Posts: 206

  • Since: 2004/6/26


In newbb2 there is an option to allow caching for certain items. Try to enable it. It is NOT in the preferences menu of the main XOOPS admin panel, it is in forum->preferences->enable cache.

In addition, a good idea would be to disable the following features: who is online and HP/MP/EXP mod.

9
Skythe
Re: Xoops overloading my Server
  • 2005/4/20 17:01

  • Skythe

  • Just popping in

  • Posts: 99

  • Since: 2004/7/2 2


Quote:

Dave_L wrote:
You may have to experiment with the cache settings. But be careful not to cache content that depends on the user's access level, e.g. a block that displays different content for users and admins.

In the log file, I would look for "patterns".

The IP address and user agent identify the client who's accessing the site. If you see 10000 consecutive lines with the same IP address and user agent, that could be a search bot or someone else "spidering" your site.

The script name is also logged, so that can show if a particular module or part of a module is responsible for most of the load.


I set News, Polls, Forum and Members to 1 hours each. There should not be a problem with those and access levels right?

The logs are already adding up big time. 22 files and 71 megs. You did good in pointing out their size in your faq..

I went through one log to see how many occurences I could find for each IP: 54, 86, 102, 122, 126, 138, 158, 172, 181, 190, 192, 204, 285, 306, 312, 330, 726 and 732 times.

?



@brash Thanks. I installed it. On the front page (news) it read 4.523 Seconds | 374 Queries. O.o I reduced the amount of news entries to be displayed from 15 to 5 and it went down to 0.168 Seconds | 33 Queries. Gosh. Now its gone back up to an average of 2.9sec.
Every other page is around 0.13 sec, 30-45 queries.

Could that have been the problem??? Too many entries per page? But 15 dont seem like overkill to me. I mean most of them include images but that doesnt affect the number of querries, just loading speed.....

@iolai I'm still using newbb 1. ;)

10
seventhseal
Re: Xoops overloading my Server

What about the obvious - did you look at your access logs to find out how many people are linking directly to you, or your graphics, and then displaying them on their site? Or, what php pages are hit the most, are they coming in through a different page than the front page? I am a hosting company, and I can tell you, most of the time the type of traffic you are talking about shouldn't hurt you. Even with DB connections capped. There are other factors, like how much your current host caps you (200 Gig I guess), what your DB connection rate allowed, etc. The hosting company will never take the blame - you have top prove a configuration problem.

Just an opinion...


Login

Who's Online

292 user(s) are online (226 user(s) are browsing Support Forums)


Members: 0


Guests: 292


more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: Nov 30
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits