| Author |
Topic |
CCP Whisper
 |
Posted - 2009.09.14 11:40:00 - [ 1]
At 18:31 UTC on the September 13 our Tranquility server started exhibiting unusual technical issues which required an emergency shutdown at 19:12 UTC. Tranquility was brought back up with throttled connections, disabled API service and forums at 19:38 UTC. Tranquility was back online and fully accepting connections at 20:52 UTC. As of 11:22 UTC today, September 14, the forums are back online. However, as a precaution during on-going analysis and troubleshooting we are keeping API service offline until after downtime on September 15.
Players who are encountering issues with gameplay as a result of the server reboot should file a petition with our customer support department. Please be advised that there is a high level of incoming petitions and that response times are being affected.
We apologize for any inconvenience caused by this unexpected downtime and thank you for your patience as we take steps to ensure this will not be repeated. |
Chribba
Otherworld Enterprises
Otherworld Empire |
Posted - 2009.09.14 11:45:00 - [ 2]
Glad to see the hamsters back. |
Cat o'Ninetails
Caldari
Rancer Defence League
|
Posted - 2009.09.14 11:46:00 - [ 3]
well at least we know that rancer was safe at 19:12
\o/ |
Atreus Tac
Blood Covenant
Pandemic Legion |
Posted - 2009.09.14 11:53:00 - [ 4]
Underneath Cat o and chibbra |
Hooch Flux
Caldari |
Posted - 2009.09.14 11:55:00 - [ 5]
Edited by: Hooch Flux on 14/09/2009 11:55:50
Right, it was all my fault. Both Sat and Sunday's outage was due to me! You see, at both times that the server went down I was experimenting with my System scanner and probes, must have killed it!
Coincidence or not? |
Southern Suzy
Minmatar |
Posted - 2009.09.14 12:00:00 - [ 6]
something tells me that latest patch broke something |
Captain HappyChap |
Posted - 2009.09.14 12:01:00 - [ 7]
Edited by: Captain HappyChap on 14/09/2009 12:01:29
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated. Although the usual "We cant do anything" rubbish would not go down particularly well. |
Louis deGuerre
Gallente
Malevolence.
|
Posted - 2009.09.14 12:01:00 - [ 8]
Edited by: Louis deGuerre on 14/09/2009 12:01:41Thank The [Insert Diety of Choice] I can post again !  Gosh, the withdrawal systems were worse than my regular failed stopping with smoking attempt.  Also, how can the forums affect the server ???  |
Muscaat
EVE Markets
|
Posted - 2009.09.14 12:03:00 - [ 9]
|
CCP Whisper
|
Posted - 2009.09.14 12:03:00 - [ 10]
 Originally by: Captain HappyChap
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated.
If you have experienced a loss which you attribute to server issues please file a petition with customer support. |
Coriolis Chandrasekhar |
Posted - 2009.09.14 12:18:00 - [ 11]
|
Epitrope
The Citadel Manufacturing and Trade Corporation |
Posted - 2009.09.14 12:37:00 - [ 12]
Originally by: CCP Lingorm
As mentioned at the fanfest we are looking into a new replication service that will allow us to move the API service from directly querying the Tranquility DB to having a dedicated database that is replicated with TQ. This will allow more freedom in what we code for the API service as we will not directly impact TQ performance. CCP Whisper and the good team in Operations are working hard on making this happen but it is still some way down the track yet.
( source, 2008.12.23) Did this ever happen? If so, why is the API down now? If not, this would seem to be a(nother) good reason to go ahead with those plans, and a good time to ask why they were delayed. |
Tiberius Adama
Amarr
Capital Banking and Investment
New Eden Research. |
Posted - 2009.09.14 12:38:00 - [ 13]
Originally by: Captain HappyChap
Edited by: Captain HappyChap on 14/09/2009 12:01:29
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated. Although the usual "We cant do anything" rubbish would not go down particularly well.
Would it kill you to speak something that resembles proper English? once? |
Tiberius Adama
Amarr
Capital Banking and Investment
New Eden Research. |
Posted - 2009.09.14 12:44:00 - [ 14]
Edited by: Tiberius Adama on 14/09/2009 12:44:06
DP |
Lui Kai
Better Than You
|
Posted - 2009.09.14 12:47:00 - [ 15]
Might as well edit that thread topic to today's date - she's tanking again already. |
CCP Whisper
|
Posted - 2009.09.14 12:52:00 - [ 16]
Originally by: Epitrope
Originally by: CCP Lingorm
As mentioned at the fanfest we are looking into a new replication service that will allow us to move the API service from directly querying the Tranquility DB to having a dedicated database that is replicated with TQ. This will allow more freedom in what we code for the API service as we will not directly impact TQ performance. CCP Whisper and the good team in Operations are working hard on making this happen but it is still some way down the track yet.
(source, 2008.12.23)
Did this ever happen? If so, why is the API down now? If not, this would seem to be a(nother) good reason to go ahead with those plans, and a good time to ask why they were delayed.
We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs. Believe it or not, TQ generates more traffic than some large financial systems. There's people working on a solution, but it's going to take time. Cue cries of soon™ etc etc... |
Some Advisor |
Posted - 2009.09.14 13:10:00 - [ 17]
cloud databasing, codename skynet
or whatever :P
eve is too powerful for the **** out there :P |
Von Jupiter |
Posted - 2009.09.14 13:18:00 - [ 18]
Update 20:52GMT: Tranquility is online and fully accepting connections. As a precaution we are keeping the forums and API service offline until after downtime tomorrow, 14th September. We sincerely apologize for this inconvenience.
... So, the monkeys were just too busy on raging to minmatar that they couldn't even separate API to following day? |
Scurvy Pestilential |
Posted - 2009.09.14 13:27:00 - [ 19]
Guess compeltely ruining the directional scanner was really worth it huh? |
FT Cold
Mission Runner Exclusive
|
Posted - 2009.09.14 13:51:00 - [ 20]
Originally by: Scurvy Pestilential
Guess compeltely ruining the directional scanner was really worth it huh?
Quoting for hilarity and truthfulness. /Thread tbfh |
Grez
Neo Spartans
Laconian Syndicate |
Posted - 2009.09.14 14:11:00 - [ 21]
ITT: Players who think they know what they're talking about, talking about what they know nothing about. |
Gehnster
Gallente
RED SUN RISING |
Posted - 2009.09.14 14:32:00 - [ 22]
Here is to hoping you resolve the problem and the API DOES come up tomorrow. I had an idea for a new app last night and would like to be able to test it. :) |
Aralieus
Amarr
Shadowbane Syndicate
|
Posted - 2009.09.14 14:43:00 - [ 23]
Welp, its good to see forums back up and GL with the issuse that are still present. |
anubislambda |
Posted - 2009.09.14 15:08:00 - [ 24]
oh bawwww still forums are up at least  |
Miesterio |
Posted - 2009.09.14 16:06:00 - [ 25]
this databse issue why some areas suck balls to plex in now? |
Arec Bardwin |
Posted - 2009.09.14 16:31:00 - [ 26]
Some people are happy as they got double of everything they bought just before the server nuked |
OwlManAtt
Gallente
Dreddit
Test Alliance Please Ignore |
Posted - 2009.09.14 16:56:00 - [ 27]
So what actually happened? Where was the fault?
Is this a database load issue? Seeing that the forums and API were pulled down, did you believe they were causing the problem? Having observed the load with the forums/API offline, have you been able to determine if they were causing/contributing to the problem? |
Guillame Herschel
Gallente
NME1
|
Posted - 2009.09.14 19:38:00 - [ 28]
Originally by: CCP Whisper
We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs.
Wow, who'd have guessed that Oracle replication is incompatible with MSSQL?  |
Capt Clavain |
Posted - 2009.09.14 20:24:00 - [ 29]
I feel your pain .. scalability work on large databases is quite challenging, specially when you're up against the wall of having systems in production and having to keep up with the growth (I've been there). I still think MSSQL was a pretty poor choice, but I guess at this point you're stuck with it anyway.
Sure the total ownership cost is lower than an Oracle or a DB2, but even starting with EnterpriseDB or MySQL might have been a better choice on there.
|
Fille Balle
Ballbreakers R us
|
Posted - 2009.09.14 22:12:00 - [ 30]
Originally by: CCP Whisper
We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs. Believe it or not, TQ generates more traffic than some large financial systems. There's people working on a solution, but it's going to take time. Cue cries of soon™ etc etc...
I believe you. Thanks for putting in all those hours to get the server up and running! We really appreciate it. Oh, and thanks for letting as all know. Not that most of us have a clue what you're talking about, but it's still nice to see that you make an effort. Keep up the good work!  PS: If you need moar hamster feed, I know a guy that knows a guy in Jita  |
All Channels
EVE General Discussion
