Techsurvivors

Archives => 2008 => Topic started by: krissel on October 31, 2008, 06:18:59 PM

Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on October 31, 2008, 06:18:59 PM
We will try to move the whole shebang to a new server around midnight Saturday, EDT.

The site may be unreachable for a few hours so don't worry, that is if you are awake.   smile.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on October 31, 2008, 06:44:28 PM
I've just submitted the support ticket to ASO - we'll let you know exactly what time we'll be offline when I hear back from them. We've been advised to turn the forum off for about 4 hours to guard against lost data, as the site will be visible on both servers for that time.

We're hoping that this will solve some of the problems we've been experiencing lately, which we think are due to an overloaded server.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on October 31, 2008, 08:23:19 PM
Update: I'm to reopen the support ticket tomorrow night (Saturday) about 15 minutes before we want to move servers.

Didn't get much info back re: server load - the response was:
QUOTE
The server load was a little high, but nothing outrageous.  There wasn't much that could be done to lower it though.


Okaaay. Let's hope that whereever they move us it's better!!
Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on November 01, 2008, 01:54:29 AM
One more thing...actually the server's IP will change which may mean some of you will not be able to reach TS until the new number propagates through the DNS system. It may be a day or more. sad.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Highmac on November 01, 2008, 04:09:48 AM
Midnight EDT - around 5am UK, so Europeans may lose TeeEss for quite a chunk of Sunday.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: sandbox on November 01, 2008, 07:41:11 AM
Not to worry, I've done it a few times and it really isn't that big of deal. What server did they give you?
Title: IMPORTANT: TS move to new server POSTPONED
Post by: gunug on November 01, 2008, 07:49:50 AM
I'm sure everything will come out okay; what's the new IP address going to be?
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Xairbusdriver on November 01, 2008, 10:41:51 AM
I don't think you'll actually need the IP address. Just keep using the URL. That is used by the DNS servers to look up the actual IP numbers. That's the whole point of having human readable (and sometimes rememberable) URL's. I think what kriss is saying is that it sometimes takes hours for all those DNS servers to get updated, they are all over the country and some check for updates more often than others. So, if you can't get connected, just keep trying every hour or so. It would probably be best not to access TS during the swap to avoid confusing the database that keeps track of our posts, for one thing. Of course, kriss is usually the only one awake at that time...smile.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 01, 2008, 11:36:45 AM
You can access TS during the swap - we'll be turning the forum off during the swap though, for about 4 hours. The reason for that is that during that time, the site will be on both servers and the database could get a little confused and posts lost if something goes to the old server instead of the new one. After that, Kris will turn the board back on and we should be good to go.

SB, I don't know what server they're moving us to - I have asked as part of the whole request, but got no response. I will of course ask again when it comes time to actually reopen the ticket, and I will also ask to be given a few minutes to check out its load etc. BEFORE they move us. I was told us that the server load on Analucia was "high but not outrageous, but they couldn't do anything to lower it" - um, methinks a server load of 10.5 or so on a 2-CPU server is a mite outrageous. That happened yestereday morning. Last night it was 3.84 when I checked - STILL too high. It should be less than 2. And the Apache Server status was reporting up to 40% at times, which was higher than I'd seen on any other ASO server. The weird thing is - I just checked it now and it's down to 3.69% in the Apache Server Status and .88% in the Cpanel server status report - I've never seen it so low. I'm now wondering if our complaining somehow had some effect?

SB - can you tell me the difference between what is reported in the Apache Server Status ( http://analucia.asmallorange.com/server-status ) and what we see in the CPanel re: CPU load? It's something we've been puzzling over and trying to figure out - and trying to figure out what figures we should be concerned about.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 01, 2008, 10:31:24 PM
The plot thickens.

Today I looked at our server stats several times - the CPU load is now considerably LESS than some other servers at ASO; in fact the loads have dropped a lot on many of the servers. Curiouser and curiouser.

So, having a poke around again and seeing that the numbers had stayed low, I emailed ASO tech support and asked why the sudden change? And here's the response:

QUOTE
Hello,

I have looked into this for you and it appears that our system admins have performed an audit on this server and found a few users that were causing some higher then normal load but this abuse was not showing up in our process slight.  It looks like they finally tracked down the abuses and have stopped them or eliminated them from the server.  Our Admins are always watching the servers and we do our best to catch those causing issues for our Shared server clients.

Please let us know if you notice any more of these previous server issues as we will continue to improve our services for all of our clients.

Please let us know if we can further assist you with anything else.

Thanks,

Ryan


So it would appear that the squeeky wheel did (finally) get some grease! I suspect they may have had a good hard look at all the servers as a result. The highest load I could find in a brief perusal (A through E) was about 20% - much better than any stats I'd seen in the past few days. Analucia is less than 1%. smile.gif

So, I think we'll sit tight and see if things improve, rather than move right away. There may be no need now. smile.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: sandbox on November 02, 2008, 01:15:20 AM
hey paddy, i couldn't access the link.....
Now that they discovered the culprit and if the capacity is not too high you may want to stay put. I'm sure you've considered this as well but in my experience I've found that they will treat the wounded or abused server with a little more attention in the future and not burden it with a heavy load. wink.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on November 02, 2008, 01:57:18 AM
For now we are going to sit tight and watch things.  If it stays within reason and we don't have a bunch of outages we'll remain on Analucia.

Please keep posting to alert us to any time you haven't been able to get to TS.  salute.gif


Mark, I fixed the link. It doesn't have the www.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: sandbox on November 02, 2008, 11:56:58 AM
QUOTE
SB - can you tell me the difference between what is reported in the Apache Server Status ( http://analucia.asmallorange.com/server-status ) and what we see in the CPanel re: CPU load? It's something we've been puzzling over and trying to figure out - and trying to figure out what figures we should be concerned about.


Paddy, what I've always just took for granted, whether it's correct on not, is that the load range on cpanel was an average of the timeline indicators ?,1,5,10,30,? minute spread and the status page defines the specifics for the techs. I have used cpanel load range as a reference many times when writing tickets and they were mostly answered with a reference to the increase or decrease in the load indicator. I do recall once though, that my ticket seemed like it was behind the curve and that the issue was a spike rather than a trend but the indicator showed a high load over a long time period, which would indicate an average if the problem was resolved. I'm just assessing this from reconstituted memory so I'm not quite sure the answer is accurate.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Mrious_be on November 03, 2008, 02:37:51 AM
Way over my head here but I'm glad to see (read) that TS is doing fine without the (expensive?) move to another server.

smile.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 03, 2008, 09:57:25 AM
No expense involved, Marcel - it was just going to be our ISP moving us from one of their servers to another of their servers. smile.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Xairbusdriver on November 03, 2008, 01:58:01 PM
Good work! clap.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Jack W on November 03, 2008, 06:27:30 PM
Forums Dead in the water between 7:15pm and 7:25pm this evening.

- Jack
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 03, 2008, 07:16:34 PM
http://analucia.asmallorange.com/server-status confirms this - the server uptime is 46 minutes at 8:07 p.m. That's the second short outage today and the server load is way up again - not as high as before, but getting close. dry.gif We will continue to monitor this situation of course - please continue to post any outages you see and check the server status when it comes back up, particularly if it was down for more than a minute or two, so we can document the length of time of the outages (approximately).

There is one user in particular that seems to be driving things nuts (really long times to process requests) and I'm going to file a ticket on that to see if there is still some funny stuff going on.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on November 04, 2008, 02:00:41 AM
There is a discrepancy between server uptimes. The one link is more honest than the other. This one is accurate...

http://analucia.asmallorange.com/server-status


This one lies...

http://analucia.asmallorange.com/uptime.php


Server http was down tonight for about 25 minutes around 1:15 AM so the time on the first link is correct. The second is blatantly false unless the first notes any failure and the second records when all three aspects of the server go down. Just don't know. Thinking.gif

Anyway, server load shortly after it came back up was 19.35 out of 2 cpus.  eek2.gif

Just now it is .63.  rolleyes.gif  So there is some pretty serious spiking going on.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 04, 2008, 10:43:47 AM
Finally - after numerous exchanges, I got a cogent explanation of what all those numbers mean.

Apparently, Apache Server Status reports on Apache - not on whether the server itself is up or down. Of course, for our purposes, it's one and the same, since without Apache, we're dead in the water. Ditto the CPU load reporting in that status report - it's reporting on what percentage of the CPU horsepower is being used by Apache. Obviously, the higher it is, the higher the load, though if something is busy making all sorts of calls via MySql etc. the total CPU load could be high even if Apache is low(er).

The CPanel server load was 19.35?? (or the uptime.php link - same thing) If that's the case, that's insane and I need to add it to my ongoing conversation with Tech Support (I'm now talking to a Level 2 tech).

I'm fairly sure of where I want to migrate to: http://rose.asmallorange.com/uptime.php (it's the Northlea server) or maybe this one: http://isabel.asmallorange.com/uptime.php
Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on November 04, 2008, 01:39:35 PM
That CPU load was from CPanel.

[attachment=1101:server_l..._11.4.08.jpg]


Out to vote and to the folks... later.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Paddy on November 04, 2008, 02:35:10 PM
Well, I sent that on to ASO techs and got this response:

QUOTE
Hello,

We've been looking into this and have rearranged a few accounts within our servers to balance the load better.  This should resolve the long term issues you have been seeing lately.

Regards,
Dennis


Let's hope it does resolve the problems...we will be watching!
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Xairbusdriver on November 04, 2008, 04:05:38 PM
Well, if you say so. It might keep things fresher around here if we went to 'rose.' But it might be a thorny arrangement... rolleyes.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: sandbox on November 05, 2008, 03:08:32 AM
tamba2.org.uk on analucia seems to be busy with something? I can't see Wordpress instructions requiring that much attention.

Paddy try to get to Tim, he runs the place and should advise you on your best course of action. If you have any trouble drop me an email and I'll talk to him.
Title: IMPORTANT: TS move to new server POSTPONED
Post by: chriskleeman on November 05, 2008, 07:57:43 PM
FYI,

The server on my website, "Karen" was having her own set of troubles tonight earlier around 7-7:45 P.M. In and out, up and down...

Seems to be cleared up now... Thinking.gif

Chris K
Title: IMPORTANT: TS move to new server POSTPONED
Post by: Xairbusdriver on November 06, 2008, 09:25:52 AM
I just hope this thread doesn't degrade into a "sexist" thing...eek2.gif rofl.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: RHPConsult on November 07, 2008, 03:14:38 AM
Actually, I am quite concerned that when and if, as promised, "the whole shebang" . . . "propagates throughout the entire system" . . . where it can be observed by both the children and the horses, we may be asking for serious trouble!      whistling.gif Devilish2.gif  toothgrin.gif
Title: IMPORTANT: TS move to new server POSTPONED
Post by: krissel on November 07, 2008, 04:28:41 AM
Now, now... rolleyes.gif