open All Channels
seplocked EVE General Discussion
blankseplocked EVE Online service restored after unexpected downtime
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: first : previous : ... 7 8 9 10 [11] 12 13 14 15 ... : last (16)

Author Topic

Ryuuko Daiterra
Posted - 2007.10.20 13:56:00 - [301]
 

Hi! Yes yes, I'm late to the discussion... regardless

how about SP vouchers for downtime? Maybe building a skill queue is easier, maybe it isn't, but what if emergency down time could be compensated with some sort of admin assigned 'implant' that comes with an average, median, or base amount of skillpoints to be applied at the whim of the end user?? (I realize that any one user will get different SP rates depending on what they are training, but even if they can recoup 70 or 80% of lost time, they'll be mollified anyhow.) Very Happy

CCP Pete

Posted - 2007.10.20 13:57:00 - [302]
 

Originally by: Kyoto Rose
Edited by: Kyoto Rose on 19/10/2007 22:57:47
This was proven false by Wrangler a few lines before this post here. Still, the guy guessed close enough to the released information that we didn't know for sure it was false until Wrangler posted something. :)

Quote:
<&Sharkbait> one of the db admins got hit with keylogger
<&Sharkbait> used an authorized account to shift spawn tables by one row
<&Sharkbait> meta 6 gear was being dropped from standard spawns of the affected npc group
<&Sharkbait> maintenance pruning returned 6539 officer items on one account, so the servers were shut down, forums are integrated and are also shut down
<&Sharkbait> not that long, we have full weekly backups
<&Sharkbait> unfortunately tracing all damage done manually is impossible, we have no tools designed to read transactions when the db itself is skewed
<&Sharkbait> yes, no billing or account status was affected, thats part of the billing dept



as regards this information. you will never ever see me posting on any other public forums as a developer for CCP. nor would i give out such information via IRC.

in the event that i do or have to give out information, it will be on this site in the form of a forum, news post or dev blog.

basically unless it's here, i didn't say it.

CCP Pete

Posted - 2007.10.20 13:57:00 - [303]
 

posted with wrong character....sue me

mr bighelmet
Dirty Deeds Corp.
Posted - 2007.10.20 13:58:00 - [304]
 

Originally by: Kayna Eelai
Originally by: Morrow Disca

You sir/madam, need a big cup of STFU. I don't work for a big firm, or claim to be Bill Gates, but i know how DNS works and you clearly do not. Twisted Evil


and you sir, need a huge big CUBE of "get a clue"

what exactly happens when a primary dns server goes down? OH YEAH! secondary one starts acting...

and now, try figure the rest out by yourself, mister clueless.


damn your good, had me fool for a moment thinking that you are this dumb for real.

CCP Sharkbait

Posted - 2007.10.20 14:03:00 - [305]
 

Originally by: CCP Wrangler
During this downtime a lot of rumors were spread, this one among them. I can assure you that our Sharkbait did not post this, but he is very eager to find out who did...


if i ever find this person out, i will personally close his or her accounts. regardless of who it is.

Bruce McMoose
Posted - 2007.10.20 14:07:00 - [306]
 

To all the whiners saying that CCP did the wrong thing, we want reimbursing.... to quote the virgin mary... Shut up.

Read the EULA, Item 12 name no warrentys especially this part -


While CCP attempts to have the System available at most times, CCP does not guarantee that the System will always be available, or that the System will not become unavailable during Game play. The System may become unavailable for a number of reasons, including without limitation during the performance of maintenance to the System, for the implementation of new software, for emergency situations and due to equipment or telecommunications failures.

CCP dont have to do anything to reimburse you.


I think CCP did everything in they're power, and did the right thing. As soon as they saw something out of the ordinary, they jumped on it before it got out of control. So CCP, ignore the whiners :) If you didnt jump on the breach when you did, youd just get flamed for doing nothing.
So Cut CCP some slack, and a job well done to CCP. If im ever in Iceland ill buy all you guys a beer. And dont mention fanfest cause im flat out broke :)

Jonat Eken
T.A.N.S.T.A.A.F.L.
Posted - 2007.10.20 14:08:00 - [307]
 

Edited by: Jonat Eken on 20/10/2007 14:22:04
Originally by: Treher
I signed on 2 minutes before to switch a finishing skill. I looked again at eve and saw the message about the emergency shutdown.

I smashed and smashed to no avail onto a long skill, but it didn't switch.

Nearly half a day down the drain. At least I can say that I was one of the few people that experienced maximum skill loss from the outage.

To the non-empathetic tards posting "who cares" jabs about people that lost skill time, put down the cheese curls and find a woman.



Well, they say misery loves company, and I guess it's true, 'cause I feel a bit better knowing I wasn't the only one who had that experience. ugh Annoying, wasn't it?

However, I don't blame CCP for how they responded. UO and that ilk come to mind. I am quite ticked with whoever did this though... Don't ban them, let us know their char name instead. Twisted EvilTwisted EvilTwisted Evil

Adhar Khorin
Amarr
Imperial Shipment
Posted - 2007.10.20 14:09:00 - [308]
 

CCP, you made the right call. Thanks for the updates, and it gave me an excuse to get some yard work done.

Cheers,

Adhar Khorin

Anazeteo Euphoria
Euphoria Research
Posted - 2007.10.20 14:10:00 - [309]
 

A lot of companies either don't notice or fail to take proper action when they do notice exploiting going. They fear that if they bring down servers etc they will lose players etc. They don't realize that by not reacting properly and bringing everything down or doing whatever they need, they will lose even more players eventually because exploits can seriously kill a game.

So I'm glad CCP took such measures and hope they keep bringing everything down if that is what is needed to prevent exploits in game. Heck, roll back everything if there are dupes etc. Don't let the exploiters ever win!

CCP Sharkbait

Posted - 2007.10.20 14:14:00 - [310]
 

Originally by: velox
Bravo CCP on your quick thinking in forming a 'Taskforce' to handle the situation, however may i suggest that in future it is IMHO an idea to appoint someone within said future task force(s) to the role of community liason.

This person would be incharge of putting up a notice or two informing the community (us) what is going on. In doing so we could spend the downtime reflecting on other things instead of scouring the internet for eve conspiracy theories.

All you had to do this afternoon was say 'Guys/girls we have found a small database problem we need to D/T tranquility we will let you know as soon as we have an uptime eta.

Velox


alittle about the taskforce Very Happy

the taskforce was kinda created by me. if we find an exploit on TQ or serenity, it's checked and investigated by the GM's then if they think it's major i normally get a phone call. this taskforce is only for software really and has never had to deal with the network/hardware before.

the current member of this force are :

CCP Sharkbait
CCP Hammerhead
GM Guard
CCP Prisim X (he whines alot)
CCP Oveur

there is an internal mailing list that other people are on, but generally i get involved with all the exploit issues on TQ and i found out that the people listed are people i like to work with when shit hits the fan basically. once i found the exact problem (thx to server GM's alerting me to the problem) the whole of the operations team was included into this and they had to do stuff i not only don't have a clue about, but don't have access to.

the operations team are the 1's to thank for dealing with this and resolving this so soon.

mightymadmat
Amarr
Equity Corp
Dragon Spire
Posted - 2007.10.20 14:16:00 - [311]
 

Originally by: CCP Spielmann
At 10:25 GMT today we discovered an anomaly in the EVE Online Database indicating a potential exploit. Our policy in such cases is to mobilize a taskforce of internal and external experts to evaluate the situation. At 12:57 that group concluded that our best course of action was to go completely dark while an exhaustive scan of our entire infrastructure was executed.

What we discovered was an indication that one of our databases was being accessed through a security breach. While some may feel that such a drastic reaction was not warranted, it is always our approach to err on the side of caution in order to protect the players.

We of course understand the effect and disruption this has had for our players and apologize for not having been able to explain to the community what was going on. In these cases it can often be counterproductive to containment to give out information while we are in the process of evaluating the scope of the problem.

Our taskforce quickly found the security breach and prevented that from being used. We subsequently found three cases of database actions being performed through the security breach but none of those affected other users than the one doing the actions. We can also confirm that no personal details such as users’ credentials or credit card numbers were exposed through this incident.

The servers were brought back online at 22:00 GMT and we will of course continue to monitor the situation closely over the weekend and the following week.

Again we sincerely apologize for this disruption.

Regards,
-Jón Hörđdal
Chief Operating Officer, CCP


Horrah, I was wondering what happened to the game, as it was the first time I played eve for a few weeks.

Im just glad no permanent damage has been done, and life can continue as normal.

Ryuuko Daiterra
Posted - 2007.10.20 14:19:00 - [312]
 

Originally by: Bruce McMoose

CCP dont have to do anything to reimburse you.



No, of course they don't... that doesn't mean they are obligated NOT to. Very Happy If someone over at CCP is interested in improving the gaming experience, the ideas like short skill queues or SP compensation 'implants' are interesting and may have some potential. If anyone has made a post here I didn't bother to read whining about CCP, then to them I say 're-evaluate your priorities in life'... and then turn around to CCP and say 'here's a couple friendly suggestions that might soften the blow from events like this in the future.'

So what's your malfunction? Very Happy

NitroBlood
Roving Guns Inc.
RAZOR Alliance
Posted - 2007.10.20 14:20:00 - [313]
 

cheers Jon

Bruce McMoose
Posted - 2007.10.20 14:29:00 - [314]
 

Originally by: Ryuuko Daiterra
Originally by: Bruce McMoose

CCP dont have to do anything to reimburse you.



No, of course they don't... that doesn't mean they are obligated NOT to. Very Happy If someone over at CCP is interested in improving the gaming experience, the ideas like short skill queues or SP compensation 'implants' are interesting and may have some potential. If anyone has made a post here I didn't bother to read whining about CCP, then to them I say 're-evaluate your priorities in life'... and then turn around to CCP and say 'here's a couple friendly suggestions that might soften the blow from events like this in the future.'

So what's your malfunction? Very Happy


My post was aimed at the ones actually asking for time back and demanding CCP do stuff, (Honestly i only read upto page 3 and had enough smack for one day). They're we some good ideas, i like the skill queue idea. I believe its been suggested before, my favorite idea was the secondary skill idea, where you can train 2 skills on one account at once but at a reduced rate, when the primary finishes the secondary carrys on but at full speed. I would like to of seen some news aswell as to what was going on. But i understand CCP were busy. If anything, just to stop 'fake sharkbite' from spreading lies and making the situation worse.

So i have no malfunction. Im just one of the majority who are annoyed at people jumping at CCPs throats everytime something goes wrong with the servers. Im just saying that people should support CCP in they're actions rather then saying 'you should of done this instead' trying to sound like they know more then CCPs architecture then they do.

Kropotkin
Gallente
Center for Advanced Studies
Posted - 2007.10.20 14:31:00 - [315]
 

Seems to me CCP did exactly the right thing, taking the whole compromised complex down until they understood what was going on.

But I also think communications with the customers about unscheduled downtimes could be improved.

1. I wonder whether it'd be worthwhile to set up a simple, one-page web site on a cheap commercial hosting service, on a whole new domain name -- maybe eve-online-status.com ?

Putting it on a whole new domain name would avoid the agonies associated with trying to change the binding of eve-online.com. On the other hand, what customer would ever remember the name of the status domain, which customers would never use unless the main domain is down?

Maybe the Game-client could display the status URL? After all, the customer is most likely to discover the outage by trying to play, no? Simplest and cheapest would probably be to wire the status URL into the signon page as static text. Or there could be a clickable button like the "Account Management" button that already exists.

2. How about giving each user a profile setting, "EMail me when unscheduled downtimes end? (Yes/No)"?

Seems to me the main thing customers want to know, when there's an unscheduled downtime, is when they can play again.

Note that the "EVE's up!" mail barrage would only go out *after* normal operation resumed.

3. If you wanted to do a little more work, you could have the eve-online-status.com page offer a box in which each visitor could enter an EMail addr to be notified when normal operation resumes. That would both (a) restrict the "EVE's up!" mail barrage to people who've expressed interest in *this particular outage* and (b) give each player the opportunity to choose *during the outage* what EMail addr would receive the notice -- I can imagine a player who'd want notice to his main EMail account, if he expected to be at his desk, but would prefer notice to his wireless text-message device if he wanted to go shopping.

One caution about 3 above: there might be a problem of script-kiddies using bots to enter zillions of non-player addrs, and implementing countermeasures might cost more than the notification facility is worth. Or maybe one could just use one of those "prove you're human by typing in this box the obfuscated text displayed in this graphic" gizmos?

OSGOD
Posted - 2007.10.20 14:33:00 - [316]
 

if nothing happened why do i have to train an extra 9 hrs *snip* Keep it civil, please. -Rauth Kivaro([email protected])

Mickey Mouth
Posted - 2007.10.20 14:34:00 - [317]
 

Congrats on finding the exploit and fixing it. I hope the culprit is facing legal consequences.

I would strongly recommend you create a PSA site hosted by an independent ISP on another network. Nothing fancy but this will make you stand above the rest of the MMO crowd and build goodwill in the community.

Due to the unique nature of EVE's skilling system, having the servers unavailable for hours is very stressful. This will lead to people thinking the worst and speculating, allowing rumours to spread and so forth.

Like any business, being in the dark for hours is simply not viable.

Once again, good job in your efforts. Its not easy to make decisions in the heat of the moment and going dark was probably the best choice.

Best speed in keeping EVE safe and fun!


TomParad0x
Caldari
RogueNET
Posted - 2007.10.20 14:34:00 - [318]
 

Edited by: TomParad0x on 20/10/2007 14:35:41
Originally by: Harkwyth Mist
Edited by: Harkwyth Mist on 20/10/2007 01:36:13
Originally by: TomParad0x
You couldnt tell there was a problem?

Yes, but as I already stated my complaint is with the lack of communication, not with the nature of the technical problem.

Originally by: TomParad0x
So.. your comparing real life items

It's called an Analogy. It's not a too complicated subject for you is it ?



Oh, I know what its called, but your Analogy is flawed. You are comparing a game, which is fairly unimportant, to a means of travel for millions, that millions depend on daily to get places. Your also comparing a game to something you pay a hell of a lot more for - so yes, you will get better service through them.

Last I heard, missing the latest bit of news on whats wrong with EVE wont make you late for a job interview / wedding / etc / etc / etc.

Originally by: Harkwyth Mist

Originally by: TomParad0x
What do you think IRC is for?

I'm not allowed to use IRC at work



You shouldnt be browsing the EVE forums at work either, id think, or playing EVE at all... you should be doing work... so what does all this matter if you were working?

Originally by: Harkwyth Mist

Originally by: Madelchai
It's a lot easier to get info on a train or plane than it is to find and fix the cause of a database security issue, and a lot easier to predict a new schedule for it.

There was no information at all, hence my complaint.

I dont care whether it was database hacking/dead hamsters in the power-room or no donughts for tea-break, the nature of the problem doesn't concern me.

Returning to my analogy, If I'm catching the 12:45 to Paddington and it's going to be 6 hours late, I expect to be told of the delay around 12:45, I do not expect to be told of the 6 hour delay at 18:45.

(I've written my analogy out in full, instead of inferring it's meaning, as you both seemed incapable of understanding a fairly simple and straitforward part of the English language)



You obviously fail at basic reading comprehension skills if you honestly believe I don't know what an analogy is, and that I don't know the meaning behind yours - I got what your analogy was stating, or else my reply would not have been what it was, and furthermore, your pathetic attempts to insult my knowledge of the English language on a simple subject further reduces my respect for anything you have to say in this forum. As I said above, your analogy is flawed.

Ghonzhu
Posted - 2007.10.20 14:37:00 - [319]
 

Edited by: Ghonzhu on 20/10/2007 14:40:50
What is lost amongst everyone is that CCP has very poor security. If, indeed, it was a GM account being keylogged and had CCP had proper security (full tracking of GM activities ingame or otherwise), it would have been a simple fix. The fact that CCP had to resort to black indicates that CCP had no clue on containment (or prevention).

We may be thrilled of what CCP has said, but we should be equally disappointed at what CCP didn't say.

CCP needs to come clean as to why the CCP went black in totality. If not, it still demonstrates their inability to be truly candid, secured and proactive. (The length and method clearly demonstrate a lack of prudence and knowledge.)

Naqua'aht
Minmatar
N O M A D s u n
Posted - 2007.10.20 14:38:00 - [320]
 

Edited by: Naqua''aht on 20/10/2007 15:04:23
Edited by: Naqua''aht on 20/10/2007 14:54:44
Always good to hear of an exploit being stopped and an exploiter getting what's coming. Well done IMO!

Hopefully the security issues will be thouroughly reviewed after this breach but the trouble with hackers is that they always find something to exploit, some little crevice to lever open. No system is 100% secure, even with the best will in the world, you just cant mitigate for human nature.

Keeping the forums on a seperate server, so that information could be made available to the worried masses, would seem prudent in such circumstances (Im no expert though)... But im not sure it would necessarily prevent those suffering a bout of capsule deficit syndrome from having a bit of a panic attack altogether... Wink

Im rather inclined to place the blame for our inconvenience firmly at the door of the person(s) responsible for the hacking, rather than the guys who had to deal with it.


Personally, my accounts were not unduly troubled by the the downtime but i agree about the usefulness of some sort of skill queue system. Occasionally unforseen circumstances will force dramatic action and such an addition would go a long way towards minimizing the losss of training time. I know that this thread isnt really the place to discuss development issues, but it would be damn nifty...

Gargulous
Posted - 2007.10.20 14:40:00 - [321]
 

Well done CCP on a quick response.

I have two cents to put in regerding chatter on this forum.

First is regarding scepticism, expressed by by some naieve people, that the unscheduled downtime was a security response to espionage. Such things can be done, have been done, and will be done again...if you think that anything that happens in movies is not possible in real life then you have similar problems to those who believe everything that happens in movies can be done in real life.

The seccond cent is in relatoin to CCP not keeping us updated on a second by second basis. CCP Speilmann says

'We of course understand the effect and disruption this has had for our players and apologize for not having been able to explain to the community what was going on. In these cases it can often be counterproductive to containment to give out information while we are in the process of evaluating the scope of the problem. (my italics)'

This is not about being unable to inform us of what was happening, but about not alerting the nefarious individual that caused the problem to the fact they were on to them.

I feel the CCP team has demonstrated a high level of professionalism in dealing with this security breach. As mentioned earlier, all systems not turned off and locked away are potentially succeptible to such attacks. I would prefer to be a mushroom for half a day, with a few hours of skill training time wasted, than have either my RL details in the hands of skum, and/or have the Eve universe destroyed, or completely unbalanced.

Congrats and thanks CCP. Cool

Calvin Firenze
Minmatar
Caeli Veredictum
Posted - 2007.10.20 14:40:00 - [322]
 

Edited by: Calvin Firenze on 20/10/2007 14:41:39
cba reading 10 pages of tinfoilhattery and *****ing...made it to page 3

The unexpected DT caused me to finally get on coldfront, been meaning to forever. I got all the explanation I needed, and Wrangler even scolded us by saying Oveur was not a clown to be brought out to dance for us...sadly. Razz

No, I didn't lose any training time on any of my accounts, so I won't gripe about that. However, I agree a skill queue would be nice so I don't have to stop a skill at 90% completion to start another when I need to go do things IRL (work, date, sleep etc etc). The best restriction IMO would be to limit queued skill to 9 hours or less. This would allow plenty of time for the first skill to finish and you to get home from work/wake up and add/start another skill. Why 9 hours? Typical work day is 8 hours and 9 would allow for some commute time on top of work schedule.

A communications blackout is perfect in a security breach tbh. It doesn't alert the responsible individual(s) that they were discovered. I'm kinda glad Wrangler was nice enough to even be on irc, he didn't have to be(well maybe he did, I dunno)...all he had to do was make the announcement in the topic and say "cya later guys"

Good job on CCP's part.

P.S. Whoever was talking about pizza on irc and made me want one really bad...I ordered one but there was no delivery driver :( no pizza for mehEvil or Very Mad

Edit: I guess its 11 pages now.

Voltron
Caldari
Black Lance
Executive Outcomes
Posted - 2007.10.20 14:46:00 - [323]
 

Edited by: Voltron on 20/10/2007 14:46:45
Did what you had to do to get the problem solved, Thanks!

I'm thrilled that you guys know well enough to not communicate to the community what was going on at the time, Kudos to proper investigative techniques!

Edit:

I don't know how long the downtime was, but however long it was shouldn't matter to anyone, all that matters is the issue was fixed, no reimbursement should be necessary.

Volt

bustilong
Posted - 2007.10.20 14:48:00 - [324]
 

I think all of you addicts need to stop crying and complaining like babies out there and let ccp handle the problems as thy do so well with as it is

Asulox
Caldari Faction Wolfpack
Posted - 2007.10.20 14:48:00 - [325]
 

Edited by: Asulox on 20/10/2007 15:13:42
Edited by: Asulox on 20/10/2007 14:50:43
Damn, does this mean i cant keep my extra 300m SP?

Ventaran
Posted - 2007.10.20 14:51:00 - [326]
 

Well all seems to be good though I checked the bounty office just now and found that the top 1 guy is:

A: 35 years in his/her corp (time travel?)
B: The only faction he/she is related to is CONCORD (-10) and nothing else

Must be me though as I'm still new to EVE.

EdRush
Caldari
Caldari Provisions
Posted - 2007.10.20 14:52:00 - [327]
 

Since yesterday i have been unable to log in my second account ,I have sent a petition,but no answer yet ..User "characters name " has been disabled last login 2007:10:19 22.05 number visits :1199 reason none ..This happens both when i try to log in to the game and my account details ShockedShockedShocked

SoldierOfJustice
Infortunatus Eventus
Posted - 2007.10.20 14:55:00 - [328]
 

Originally by: CCP Sharkbait

the operations team are the 1's to thank for dealing with this and resolving this so soon.


They are also the ones to blame if ppl arent satisfied then Laughing.

Treher
Minmatar
Infinity Enterprises
Interstellar Alcohol Conglomerate
Posted - 2007.10.20 14:55:00 - [329]
 

Edited by: Treher on 20/10/2007 14:57:05

Originally by: Jonat Eken

Originally by: Treher
I signed on 2 minutes before to switch a finishing skill. I looked again at eve and saw the message about the emergency shutdown.

I smashed and smashed to no avail onto a long skill, but it didn't switch.

Nearly half a day down the drain. At least I can say that I was one of the few people that experienced maximum skill loss from the outage.

To the non-empathetic tards posting "who cares" jabs about people that lost skill time, put down the cheese curls and find a woman.



Well, they say misery loves company, and I guess it's true, 'cause I feel a bit better knowing I wasn't the only one who had that experience. ugh Annoying, wasn't it?

However, I don't blame CCP for how they responded. UO and that ilk come to mind. I am quite ticked with whoever did this though... Don't ban them, let us know their char name instead. Twisted EvilTwisted EvilTwisted Evil



SP are very important in EVE. Whether we mess up or something else happens, it's always a disappointment. These are game forums; it's an appropriate place to vent about something happening in the game. Most people are not blaming CCP or demanding SP back, but this is the appropiate thread to vent/complain because this is the only place where our comrades can empathize (other than in game).

Now, if went out to the store today and ran into a colleague and told them about my woes in EVE, then I would be a loser.

CCP Wrangler

Posted - 2007.10.20 15:05:00 - [330]
 

Originally by: Vorok
Edited by: Vorok on 20/10/2007 05:34:55
Let's look at the real victim here: Dark Shikari. Since he is directly connected to the forums via a neural interface, his VERY ESSENCE is constantly browsing them. He was at serious risk of mental scarring or even death when the connection was severed while he was still inside. CCP is facing the risk of a potentially crippling law suit as a result of their negligent actions. I urge Dark Shikari to consider immediately retaining legal counsel. Pick me I'll take only 10% of the settlement!
I apologise on behalf of CCP for this, though we will still crush DS in court if he tries anything. Laughing
Originally by: Ventaran
Well all seems to be good though I checked the bounty office just now and found that the top 1 guy is:

A: 35 years in his/her corp (time travel?)
B: The only faction he/she is related to is CONCORD (-10) and nothing else

Must be me though as I'm still new to EVE.
Before AURORA closed we had events by them in game and to make it more credible some of those characters were modified. It could be that one of those characters is what you're seeing. Wink
Originally by: EdRush
Since yesterday i have been unable to log in my second account ,I have sent a petition,but no answer yet ..User "characters name " has been disabled last login 2007:10:19 22.05 number visits :1199 reason none ..This happens both when i try to log in to the game and my account details ShockedShockedShocked

Try to use password retrieval. Smile


Pages: first : previous : ... 7 8 9 10 [11] 12 13 14 15 ... : last (16)

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only