open All Channels
seplocked EVE General Discussion
blankseplocked Unexpected downtime at 19:12 UTC, September 13
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: [1] 2

Author Topic

CCP Whisper

Posted - 2009.09.14 11:40:00 - [1]
 

At 18:31 UTC on the September 13 our Tranquility server started exhibiting unusual technical issues which required an emergency shutdown at 19:12 UTC. Tranquility was brought back up with throttled connections, disabled API service and forums at 19:38 UTC. Tranquility was back online and fully accepting connections at 20:52 UTC. As of 11:22 UTC today, September 14, the forums are back online. However, as a precaution during on-going analysis and troubleshooting we are keeping API service offline until after downtime on September 15.

Players who are encountering issues with gameplay as a result of the server reboot should file a petition with our customer support department. Please be advised that there is a high level of incoming petitions and that response times are being affected.

We apologize for any inconvenience caused by this unexpected downtime and thank you for your patience as we take steps to ensure this will not be repeated.

Chribba
Otherworld Enterprises
Otherworld Empire
Posted - 2009.09.14 11:45:00 - [2]
 

Glad to see the hamsters back.

Cat o'Ninetails
Caldari
Rancer Defence League
Posted - 2009.09.14 11:46:00 - [3]
 

well at least we know that rancer was safe at 19:12

\o/

Atreus Tac
Blood Covenant
Pandemic Legion
Posted - 2009.09.14 11:53:00 - [4]
 

Underneath Cat o and chibbra

Hooch Flux
Caldari
Posted - 2009.09.14 11:55:00 - [5]
 

Edited by: Hooch Flux on 14/09/2009 11:55:50
Right, it was all my fault. Both Sat and Sunday's outage was due to me! You see, at both times that the server went down I was experimenting with my System scanner and probes, must have killed it!

Coincidence or not?

Southern Suzy
Minmatar
Posted - 2009.09.14 12:00:00 - [6]
 

something tells me that latest patch broke something

Captain HappyChap
Posted - 2009.09.14 12:01:00 - [7]
 

Edited by: Captain HappyChap on 14/09/2009 12:01:29
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated. Although the usual "We cant do anything" rubbish would not go down particularly well.

Louis deGuerre
Gallente
Malevolence.
Posted - 2009.09.14 12:01:00 - [8]
 

Edited by: Louis deGuerre on 14/09/2009 12:01:41
Thank The [Insert Diety of Choice] I can post again ! Razz

Gosh, the withdrawal systems were worse than my regular failed stopping with smoking attempt. Embarassed

Also, how can the forums affect the server ??? Shocked

Muscaat
EVE Markets
Posted - 2009.09.14 12:03:00 - [9]
 

Originally by: Louis deGuerre
Also, how can the forums affect the server ??? Shocked


As far as I'm aware... they share the database server... Sad

CCP Whisper

Posted - 2009.09.14 12:03:00 - [10]
 

Originally by: Captain HappyChap
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated.


If you have experienced a loss which you attribute to server issues please file a petition with customer support.

Coriolis Chandrasekhar
Posted - 2009.09.14 12:18:00 - [11]
 

Originally by: Captain HappyChap
Edited by: Captain HappyChap on 14/09/2009 12:01:29
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated. Although the usual "We cant do anything" rubbish would not go down particularly well.



uh, maybe time for a name change to Captain NotsoHappyChapQuestionVery Happy

Epitrope
The Citadel Manufacturing and Trade Corporation
Posted - 2009.09.14 12:37:00 - [12]
 

Originally by: CCP Lingorm
As mentioned at the fanfest we are looking into a new replication service that will allow us to move the API service from directly querying the Tranquility DB to having a dedicated database that is replicated with TQ. This will allow more freedom in what we code for the API service as we will not directly impact TQ performance. CCP Whisper and the good team in Operations are working hard on making this happen but it is still some way down the track yet.
(source, 2008.12.23)

Did this ever happen? If so, why is the API down now? If not, this would seem to be a(nother) good reason to go ahead with those plans, and a good time to ask why they were delayed.

Tiberius Adama
Amarr
Capital Banking and Investment
New Eden Research.
Posted - 2009.09.14 12:38:00 - [13]
 

Originally by: Captain HappyChap
Edited by: Captain HappyChap on 14/09/2009 12:01:29
I hope they're going to be refunding either a day for the numerous downtimes over the past weeks or alternatively return my ship i lost because i couldnt fire my guns because teh crystals got stuck cos of lag and i got blowns up either would be appreciated. Although the usual "We cant do anything" rubbish would not go down particularly well.


Would it kill you to speak something that resembles proper English?

once?

Tiberius Adama
Amarr
Capital Banking and Investment
New Eden Research.
Posted - 2009.09.14 12:44:00 - [14]
 

Edited by: Tiberius Adama on 14/09/2009 12:44:06
DP

Lui Kai
Better Than You
Posted - 2009.09.14 12:47:00 - [15]
 

Might as well edit that thread topic to today's date - she's tanking again already.

CCP Whisper

Posted - 2009.09.14 12:52:00 - [16]
 

Originally by: Epitrope
Originally by: CCP Lingorm
As mentioned at the fanfest we are looking into a new replication service that will allow us to move the API service from directly querying the Tranquility DB to having a dedicated database that is replicated with TQ. This will allow more freedom in what we code for the API service as we will not directly impact TQ performance. CCP Whisper and the good team in Operations are working hard on making this happen but it is still some way down the track yet.
(source, 2008.12.23)

Did this ever happen? If so, why is the API down now? If not, this would seem to be a(nother) good reason to go ahead with those plans, and a good time to ask why they were delayed.


We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs. Believe it or not, TQ generates more traffic than some large financial systems. There's people working on a solution, but it's going to take time. Cue cries of soon™ etc etc...

Some Advisor
Posted - 2009.09.14 13:10:00 - [17]
 

cloud databasing, codename skynet


or whatever :P

eve is too powerful for the **** out there :P

Von Jupiter
Posted - 2009.09.14 13:18:00 - [18]
 

Update 20:52GMT: Tranquility is online and fully accepting connections. As a precaution we are keeping the forums and API service offline until after downtime tomorrow, 14th September. We sincerely apologize for this inconvenience.
... So, the monkeys were just too busy on raging to minmatar that they couldn't even separate API to following day?

Scurvy Pestilential
Posted - 2009.09.14 13:27:00 - [19]
 

Guess compeltely ruining the directional scanner was really worth it huh?

FT Cold
Mission Runner Exclusive
Posted - 2009.09.14 13:51:00 - [20]
 

Originally by: Scurvy Pestilential
Guess compeltely ruining the directional scanner was really worth it huh?


Quoting for hilarity and truthfulness.

/Thread tbfh

Grez
Neo Spartans
Laconian Syndicate
Posted - 2009.09.14 14:11:00 - [21]
 

ITT: Players who think they know what they're talking about, talking about what they know nothing about.

Gehnster
Gallente
RED SUN RISING
Posted - 2009.09.14 14:32:00 - [22]
 

Here is to hoping you resolve the problem and the API DOES come up tomorrow. I had an idea for a new app last night and would like to be able to test it. :)

Aralieus
Amarr
Shadowbane Syndicate
Posted - 2009.09.14 14:43:00 - [23]
 

Welp, its good to see forums back up and GL with the issuse that are still present.

anubislambda
Posted - 2009.09.14 15:08:00 - [24]
 

oh bawwww

still

forums are up at least Smile

Miesterio
Posted - 2009.09.14 16:06:00 - [25]
 

this databse issue why some areas suck balls to plex in now?

Arec Bardwin
Posted - 2009.09.14 16:31:00 - [26]
 

Some people are happy as they got double of everything they bought just before the server nuked Razz

OwlManAtt
Gallente
Dreddit
Test Alliance Please Ignore
Posted - 2009.09.14 16:56:00 - [27]
 

So what actually happened? Where was the fault?

Is this a database load issue? Seeing that the forums and API were pulled down, did you believe they were causing the problem? Having observed the load with the forums/API offline, have you been able to determine if they were causing/contributing to the problem?

Guillame Herschel
Gallente
NME1
Posted - 2009.09.14 19:38:00 - [28]
 

Originally by: CCP Whisper
We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs.


Wow, who'd have guessed that Oracle replication is incompatible with MSSQL? Rolling Eyes

Capt Clavain
Posted - 2009.09.14 20:24:00 - [29]
 

I feel your pain .. scalability work on large databases is quite challenging, specially when you're up against the wall of having systems in production and having to keep up with the growth (I've been there). I still think MSSQL was a pretty poor choice, but I guess at this point you're stuck with it anyway.

Sure the total ownership cost is lower than an Oracle or a DB2, but even starting with EnterpriseDB or MySQL might have been a better choice on there.

Fille Balle
Ballbreakers R us
Posted - 2009.09.14 22:12:00 - [30]
 

Originally by: CCP Whisper
We require database replication for more reasons than just the API, but looking into some of the commercial applications used by other large databases such as banks and insurance companies revealed that these are just not suited to our needs. Believe it or not, TQ generates more traffic than some large financial systems. There's people working on a solution, but it's going to take time. Cue cries of soon™ etc etc...


I believe you. Thanks for putting in all those hours to get the server up and running! We really appreciate it. Oh, and thanks for letting as all know. Not that most of us have a clue what you're talking about, but it's still nice to see that you make an effort. Keep up the good work!Very Happy

PS: If you need moar hamster feed, I know a guy that knows a guy in JitaYARRRR!!


Pages: [1] 2

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only