1
wizanda
Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/5 10:35

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok just trying something and seems to work, yet unsure...can this be done...you know with XOOPS Banners there are two index's named the same thing.

Well if Primary index is made multiple then the extra is no longer needed...as the call that are made from XOOPS to the MySQL are tables not index's...( is it

so by including it in the primary isn't that then faster, as all it then searches is one index...and that index is the main one it needs?

2
wizanda
Re: Help with Fixing the Xoops Tables in Mysql....
  • 2006/12/5 11:25

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Still not understanding this as the more I try the more it speeds up my home server....as in thing like Bid and mid adding to the primary index instead and remove the others...

Yet still unsure what trouble it may cause and is on my home testing site....
yet it seems to work better...why don't people use primary index...please explain if there is some major reason for having loads of extra index's
Understand on things like password and stuff...

Yet isn't the purpose of the database that it all sits together as one thing....
so the primary index is the said goal of any task, so labeling it with all bits...does this work or is there some thing else?

3
wizanda
Re: Help with Fixing the Xoops Tables in Mysql....
  • 2006/12/5 22:26

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok don't need help...well do, sort of now

Think me understand now....basically a primary index...can contain many parts...it is the first place MySQL looks and the cause of that table...

So by removing some and add other can double the speed of XOOPS as we did on home server and now our own...which some of the bits included are...cbb removing extra calls and adding some in the primary index instead of making extra index's as each index is search and stored for MySQL before hand...

so we done mylinks mydownloads...Xoops it's self as loads of missing template primay index = id and that speeds up things...removing double index and replacing it wiith primary...

now just stuck on how to let everyone else have it...as well as the new theme...we have just qaudrupled the speed of XOOPS over night with now java cached theme templates as well as Smarty filling info...for near ajax on some pages...

Anyways as soon as figure a way out to upgrade for all to do this will.....

4
wizanda
Re: Help with Fixing the Xoops Tables in Mysql....
  • 2006/12/7 14:54

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok we have a way of XOOPS being one of the fastest CMS systems with a few fixes, if what we have done is right…

There can be one primary index...and the rest reduced or modified......
The reason XOOPS sites are slowing is index's! we don't need so many….

Check on Wikipedia what a Primary index is for....and you see it is like any other index.....yet must contain certain criteria these include...:

it must be something new each time...
it can not have duplicates of its self....
fair enough everyone thinks lets use that and a ID and that will save time...correct and it does........

yet then it finds that primary index...and then says.....mmmmm now need another index for the information, as you also have a storage of that running?

So instead....what it needs to do and what we have done to our own site...is if an id and secondary id have been placed...these in fact are one, they count the same information twice that is all...

As a reference is this, when a category index....
links to category ID for one and then to topic also in primary...as the two are symbiotically linked you can't have one without the other....

So when calling the database of our own site, now it says....need this information from the category
It finds this through searching of the primary index's being the main purpose of that table and it’s index…so called primary....

PHP needs tables
Not index's
The index is a way of MySql questioning its self....and searching that information for important key points within it....

To save searching time...not to add it....

So also with anything small enough, also with a sequential number of it own that are there to link these have been added in one go as primary so as soon as the database looks for it, it has it.....(we added the title to the primary index also on some occasions, yet may make these text)...

so then now the database...in MySql is ready for the questioned being asked of it self...

not that it firsts search an id...then has to wait for the information to be processed of a secondary index....or if so, the primary containing the number of id and linking extra id so it is easily placed when looked at to begin with not after a second search.

some other CMS systems we looked at, to learn about this...had chosen a different route and to leave just id's used as primary and nothing else....

yet within the CMS in question, you see overload running…. as when there is a lot of questions………….it is finding the id's fine............yet then needs to look each time for anything it is asked question of…..i.e.…. user name ect or titles of things....

So in ours now the only time a secondary index is added is when it is something that is less important, yet still needs to be searched.....(else a text-index being there for all bodies of texts to be next to it’s id)

our site see how it runs and if we get any complaints from our server company....

yet considering we have had 7 host before e this one...and each is due to server load....

we ran this as written on our WAMP server being my laptop…. which at the XOOPS site over-powering normally; runs its fans so is audibly noticeable…. the moment XOOPS over-runs, trying to find extra search that make no sense...
Now though, there isn't anything; in fact my home server seems much happier without the banner...going oops someone made another index...we don't need?

So my question now follow all of this and if all goes well and nothing goes wrong; which doubt it will....


how on earth do we go about re-shaping XOOPS MySql....understand can have an update MySql....yet this also requires most of peoples modules being reformed also...


Yet only dropping index's which isn't something, we have had to do much other then manually....as.....can figure out how to drop a database and re-set it, yet that will drop everyone’s information....

So how to, just drop the key's and make them primary without replacing any other data?
How do we do this...as after as you can see on our site...it is nearing one of the fastest CMS systems going...

5
hooperman
Re: Help with Fixing the Xoops Tables in Mysql....
  • 2006/12/7 16:00

  • hooperman

  • Just popping in

  • Posts: 31

  • Since: 2004/9/20


Quote:

wizanda wrote:
How do we do this...as after as you can see on our site...it is nearing one of the fastest CMS systems going...


Thats cool
Would love to see this work on the database being made a priority

6
wizanda
Re: Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/7 18:52

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok on second thoughts....Text-index searching slows the server down and reduces the search speed; not increases it....

So they are now gone, in fact the id alone and linking files and things the system needs quickly are best....in the primary index in one go and miss the rest....more work for the server...as long as it gets the id...it can interlink with the tables much quicker....


still unsure on pure numerical...if any one knows what data is differently stored for a primary index in comparison with a normal index would be helpful....

So far the only difference that is noticeable is speed...the primary runs much faster....a second index should really not be used if possible, it takes longer...as you are searching the table twice; so slowing server time....

Keep you reported on what we find

7
wizanda
Re: Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/8 8:54

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok after almost losing the site, due to an overload somewhere; was about to trash all the work in the MySQL index...then realized it must be me....

So on second thoughts, what would cause a page to have trouble loading to begin.....mmmmm....oh yeh xoops_sessions....I set it on primary by mistake....

So currently my sessions is now on unique as each and every ip, id and update is unique, when put like that together...and each is called a lot....

makes me wonder; if it was not all these being linked, is what causes our log-out when posting problem in Xoops...?

Having the sessions all stored in one index together...or unique unsure which is best.....it makes the loading time even faster....

As in what ways shouldn't when the Server says here is an IP, there is it's update and there is its id within the same system, not a second index....

Else this is why we see a loading time after submission, as the system it's self has to recognize the person again from said update, that then links to an ip and the rest....

Now on this realized another problem exists....

Being the multiple calls for who is online?

In each and every database works along these lines,
Data storage = information
Index storage = references.....

Now if there is a lot of index references and especially extra index’s, yet not much happening....this is taking up valuable resources.....


So in our case.....for instance....we have installed xmbmemberstats + CBB + Xoops....

Each of these is taking a separate record to display who is online?

Unhappy about that.....really would be nice to just be one system that interlinks...

yet point being within the index scenario, these were doing far to much to begin with....

Especially if you turn off CBB’s who is online and then notice, that the index's are still running, it might not be adding data, yet the index remains and so is search continuously also....

So by limiting the number of searching of who is online in index’s (data remains) also speeds up our site loads ....as it is mentioned on many site how many resources who is online uses.....

well really on analyzing of this we feel it would be better if we could system link the who is online...as when the XOOPS core takes a unique id for every session and gives this an update time....that would be the quickest place to add who is online to the unique id that is already there ....

8
wizanda
Re: Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/8 10:33

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok with my new friend PhpMyAdmin.....

ALTER TABLE `xoops_sessionDROP PRIMARY KEY ,
ADD PRIMARY KEY ( `sess_id` , `sess_updated` , `sess_ip` )


That is what has just been done with our site the other day and is wrong!!
Last night after pages were found easy, yet the system clung up due to being primary for an index or system that is to be changing....so each and every item is too remain or be deleted out of the main list according to the primary rules.....it is something that is part of the main call of that table....

ALTER TABLE `xoops_sessionDROP PRIMARY KEY ,
ADD UNIQUE `sess` ( `sess_id` , `sess_updated` , `sess_ip` )


so that is what is done now and if you check our site, the time it takes to find you are there, and where you are going is much quicker

Also now realized PhpMyAdmin can show me what we need for a XOOPS update!
Help people, Please! like Dev and Mod's why doesn't anyone like me...
!!I know I talk to much !!!!

Yet please, if this all works we can make XOOPS run 3 times faster at least......as we have

9
wizanda
Re: Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/9 18:49

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


Ok after first trying setting one Index...not primary as that is a real no! No!...as when you make anything primary, it is said to be there from site beginning - until end....

So setting a Primary id for session as has been done is not good....

Reason is the session_Id is not a single number yet a big list...here is the info that will be stored once in the database and then indexed each time, an index is made...
sess_id     bf0fc13150e5e1da612a5ee2c9ac0cd1      
sess_updated 
1165580615      
sess_ip 
127.0.0.1      
sess_data 
xoopsUserId|s:1:"1";
xoopsUserGroups|a:2:{i:0;s:1:"1";i:1;s:1:"2";}
xoopsUserTheme|s:7:"default";
XOOPS_TOKEN_SESSION|a:2:{i:0;a:2:{s:2:"id";s:32:"e049272122b77fef18f477797af21fe7";s:6:"
expire"
;i:1165581502;}i:1;a:2:{s:2:"id";s:32:"efa39ace1bcf985e7201e24b6b7887b2";s:6:"
expire"
;i:1165581512;}}


So let’s analyze why any of that would want also indexing a second time, as well as data stored?

The Id to begin with, well we could make that an index or primary as has been done; yet the minute anyone leaves...that is no longer needed to be indexed....

Mysql takes longer to refresh then XOOPS does in terms of when it calculates the index's....

So in that being data first; it finds the id, it find all information with it....

What do we need out of that? The current operation being session data....can we interlink this with the ID without making two separate index's, no...Not without specifying a limit as it is text on the end one...so any text indexes reduce speed of the server....a normal index can also index or rather render small bits of text...as if a title for instance is linked to an id....

That will mean that the point of MySQL looking at the id it needs it also gets the title index with it....now since what had been done is for this often to go in second index...along with primary that takes longer....

So it is the same information that primary index's as anything else...yet when it is in a second index...first it will find primary with id it is looking for...then it goes and takes a second look for the other index....

Now MySQL is clever then this and the moment it finds that id, and index…. is a way of top scanning over the database for speed...so then it gets the information it needs....

So any calls that can save time; saves it looking within the data as well as an index...as both are stored and both can be looked at for use....

Yet when the shortest path is used this makes for a much better and faster system...like you expect out of modern technology....

If anyone reading this and would like to try any of it on their own site...fear not....as you drop an index, and rebuild an index...as long as you don't drop the actual tables or data....

So you can arrange play and mess depending what your sites purpose is....to arrange the index's to only calculate what is needed, and to include the points that will be most searched within the data and trying to use primary...as so far have found no difference between a primary and a normal index other then one must be unique and not contain replications, of any item within it...

So as long as something unique is placed also with something not unique within the fields the index will look at...then this has been ok for us and seems to vastly have improved the speed of Xoops.

Ok back to this XOOPS sessions we have now tried dropping all index’s in XOOPS sessions...as after accumulation time since last night, of it all being indexed in a single index. no longer primary and being a unique index, all stored as one….as all ip's and session id are unique....

Found a slowing of the system today...so now...though MySql doesn't like no index...and shows [!] no index specified....

it is far faster without even an index as then the from my reckoning...... first it gives you a session id...this is placed as data....the data is then passed on and told what is going on, this is then placed at the same point in our system...so making it near by....the only point of an index, really would be if I was interested in keeping a record or faster way, to see what all are doing across the site at once, yet they are not at once, they are all separate entries....

yet in the long run as we see, what do you need that indexing for...it is only needed as required data at said time of operation and after it is no longer used...as the next visit is another id...so in being no index's for sessions, it means there is no weight if more users...

As you would think if you indexed the users, as had been done in Xoops, that this would mean as more people arrived, the index gets bigger, the data gets bigger…yet all we really need is the data….as the moment they leave data goes…. index remains until next calculated…

Instead by removing all index’s as we are now trying…. it stores the data………. it registers the use and user….it changes….. it is dropped there and then…else we see these massive over-heads on sessions index’s………. that are no longer used the moment the person leaves the site…

10
wizanda
Re: Fixing the Xoops Tables in Mysql to Run Loads Faster
  • 2006/12/10 19:28

  • wizanda

  • Home away from home

  • Posts: 1585

  • Since: 2004/3/21


The quest continues and since no one will reply...here is a removal of some extra index's that are causing errors in MySql for 2.0.16...

ALTER TABLE `xoops_bannerDROP INDEX `idxbannercid`,  

ALTER TABLE `xoops_bannerDROP INDEX `idxbannerbidcid`,  

ALTER TABLE `xoops_bannerDROP PRIMARY KEY ,
ADD PRIMARY KEY ( `bid` , `cid` )


Here placing both index's in one primary index instead, means the moment it finds the id it finds where it goes...else it searches a second non need index that in fact slows it down with an error..

ALTER TABLE `xoops_newblocksDROP INDEX `mid

ALTER TABLE `xoops_newblocksDROP INDEX `visible

ALTER TABLE `xoops_newblocksDROP INDEX `isactive_visible_mid

ALTER TABLE `xoops_newblocksDROP INDEX `mid_funcnum

ALTER TABLE `xoops_newblocksDROP PRIMARY KEY ,
ADD PRIMARY KEY ( `bid` , `mid` , `title` , `weight` , `visible` )


Now if the moment any block is called it needs Block id, module id, a title if to see or not and weight...adding them in one index means it calls one index instead of multiple and when it finds the item, it find almost all needed data to show a block at once...else it will first search primary find the id, then search a second index....

ALTER TABLE `xoops_ranksDROP INDEX `rankminrankmaxranspecial

ALTER TABLE `xoops_ranksDROP INDEX `rank_max

ALTER TABLE `xoops_ranksDROP INDEX `rank_min

ALTER TABLE `xoops_ranksDROP PRIMARY KEY ,
ADD PRIMARY KEY ( `rank_id` , `rank_title` )


Since all you need with a Rank that shows on every page of a forum is a title...
The rest will be found from the id...and since there aren't 3 extra index's cross referencing it is faster we feel....

ALTER TABLE `xoops_priv_msgsDROP INDEX `to_userid

ALTER TABLE `xoops_priv_msgsDROP INDEX `msgidfromuserid

ALTER TABLE `xoops_priv_msgsDROP INDEX `touseridreadmsg`


They were private message containing when sent and who, since all of this is registered in data that can be found by the id....in the long run considering the amount of use messaging receives...it will find an id in less time if not cross configured....

ALTER TABLE `xoops_usersDROP INDEX `uiduname`

ALTER TABLE `xoops_usersDROP INDEX `uname

ALTER TABLE `xoops_usersDROP INDEX `email`


These are a big one as if every user is cross referenced again slowing time, why?

It finds one index. it find another which is best it asks?

So removing errors like these, does help….


Also since removing all XOOPS sessions index’s so far better will tell you when have a thousand people surfing...



Also if you need emails viewing and names these can be added to the primary index of users making loading faster on things like Xmemberstats, or the members list ect forum also
Anywhere you need an item can be indexed (primary is first index looked at) to increase the needed speed of things it is looking for....

Login

Who's Online

171 user(s) are online (105 user(s) are browsing Support Forums)


Members: 0


Guests: 171


more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: Apr 30
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits