open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: TQ Level Up
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: first : previous : ... 3 4 5 6 [7] 8 9 10 11 : last (11)

Author Topic

Freelancer117
Posted - 2010.06.18 11:08:00 - [181]
 

1 THz processor powah FTW eh ?!

dont make the npc's too intelligent plz Laughing

Punx Evangeline
Veto.
Veto Corp
Posted - 2010.06.18 11:49:00 - [182]
 

Count me as one who likes to read about the system that runs New Eden. Very well done CCP Yokai.

-Punx

IngarNT
Minmatar
Sebiestor Tribe
Posted - 2010.06.18 15:07:00 - [183]
 

Originally by: CCP Yokai


Hawk TT - Why not "Boot from SAN"?

Answer - Right now the ease of management is not a big issue. It's a fairly small number of servers, the data on the local disk is nothing important, and honestly we are using the high end (lots of MTBF)Hard drives so, failures are few and far between. Not that it isn't easier/better booting from HBA/LUN0 but just quite a few items down the “to do” list.





I assume its the same for that 4.n Tb of 'internal disk' - I still think you should SAN it, maby not the boot sector if you cant be bothered mucking about with veritas/vioboot but a good 'SAN guy' say, for example.... ME, could get you 5ms responce times on reeds from disk (not that you do meny) - given all that tier zero tin you have on the flore, i suspect with tuning actual read responce time is in the microseconds.
get some Brocade HBA's in, a pair of DCX's, implimernt QOS so the database traffic goes down trunked ISLs, two fabrics for redundence...
god, im actually salavating.

Liorah
Posted - 2010.06.18 16:15:00 - [184]
 

Originally by: Lord XSiV
No respected professional out there in the industry would ever take the risk of going with a Cisco solution when the implications of an outage were so great.

No respected entity would place lives in the care of equipment not certified for that purpose, regardless of vendor. And no respected entity would deploy any important system without redundancy in place, because no matter how good or bad your vendor is, equipment will fail. (Before you ask, cluster nodes are considered disposable)

Bring the things you've learned from Eve back to RL. Everything is a cost-analysis and risk-assessment. You weigh cost vs risk, and choose the right option based on your requrements and that evaluation. Corporate deals (legal or otherwise) change the playing field, so that a solution for one entity is not as good as a different solution for another entity.

Originally by: Lord XSiV
The discussion is really about how much better CCP could be doing things if they elected to go with a better solution.

Wonderful!

Now, based on your expert knowledge of, and experience with, CCP's network needs, usage patterns, utilization, environmental and power needs, and corporate contracts, agreements, and policies, what is the best solution that they should be using?

nesdaq
Posted - 2010.06.18 17:31:00 - [185]
 

Originally by: CCP Yokai

[...]
Answer - The list today is well... from today. We are looking for good reasons to make changes, but they have to make significant impact. Since peak capacity on a node is so important for fights, 3.33GHz even on an older generation is still very high end. Give me some 10GHz CPU's and I'd be all over it.
[...]


That was the idea of intel/amd to? more Ghz, more, more, MOAR!!!!
But then they heading another approach by introducing dual/quad/hexa core cpu's. So how is CCP handling this technique.
Regular applications are getting there to support naive multi-core processing, hows CCP doing at this? Are we going to see a dedicated rack to support jita soon? Very Happy

CeneUJiti
Posted - 2010.06.18 18:51:00 - [186]
 

Originally by: nesdaq
Originally by: CCP Yokai

[...]
Answer - The list today is well... from today. We are looking for good reasons to make changes, but they have to make significant impact. Since peak capacity on a node is so important for fights, 3.33GHz even on an older generation is still very high end. Give me some 10GHz CPU's and I'd be all over it.
[...]


That was the idea of intel/amd to? more Ghz, more, more, MOAR!!!!
But then they heading another approach by introducing dual/quad/hexa core cpu's. So how is CCP handling this technique.
Regular applications are getting there to support naive multi-core processing, hows CCP doing at this? Are we going to see a dedicated rack to support jita soon? Very Happy


Funny person speaks funny thins. Laughing

CCP would need to rewrite most of server code to achieve multi-cpu support. CCP already has very little earnings from a tiny customer base and can't or doesn't want to afford a major code rewrite like that.

Ethan Kaiser
Deep Core Mining Inc.
Posted - 2010.06.19 00:29:00 - [187]
 

Grats TQ Laughing

Lanu
0utbreak
Outbreak.
Posted - 2010.06.19 01:05:00 - [188]
 

Your my new favorite dev if you keep up with the blogging (+photo's and vids!!)! Cool

ps:

Sorry Eris.. Embarassed

Doppleganger
Minmatar
Band of Builders Inc.
Sodalitas XX
Posted - 2010.06.19 01:21:00 - [189]
 

With the $$ CCP has made over the last few yrs I'm suprised their server room doesnt look more like this.

If you want to wait until 2011 I know Blue Gene/Q will be water-cooled and they are getting rid of those silly looking sloped sides that were done to maximize the air cooling. I could leak a few scale model previews but only if we could make a deal for some eve based perks. Laughing

Koronos
Interstellar eXodus
BricK sQuAD.
Posted - 2010.06.19 04:21:00 - [190]
 

As a seven year eve player (as of this week yarrr!) I can't tell you how much I appreciate the in depth info and the fact that you continue to read and respond in detail to this thread Yokai. It gives me hope for Eve. So, it made me hesitate to wave a negative flag, but there is a very real and disturbing point that somehow doesn't sink home to enough people, and since you've shown willingness to read and respond I'll try to say it here.

For a long time we had lots of lag in fleet battles, and then miraculously after brackets and so on a few major patches ago, we could actually play them. Then at some point after a major patch they stopped working again. And the latest release did nothing to help. It wasn't a gradual decrease due to more players, it was a patch broken.

As a network and datacenter administrator and all-around nerd myself I am quite geeked up by these improvements, but I'm pretty sure they won't resolve the horrible fleet lag issues we are experiencing now on TQ.

That said, again bravo and I still have hope.

<3 Koronos

Mstislav Mkrtchyan
Posted - 2010.06.19 04:56:00 - [191]
 

Are we going to get a refund on game time for the 10 days the server will probably be down for?

Damion Rayne
Gallente
Dark-Rising
Executive Outcomes
Posted - 2010.06.19 05:18:00 - [192]
 

Originally by: Memorya
/Fail

Upgrade network framework and you migh actualy see 2k fleet battles, but until then... dream on....


Obvious Troll is Obvious.

Seriously, how big of a raging idiot are you?

Koronos
Interstellar eXodus
BricK sQuAD.
Posted - 2010.06.19 06:29:00 - [193]
 

Originally by: Lord XSiV
<snip>The discussion is really about how much better CCP could be doing things if they elected to go with a better solution. The one that they are going to is far too costly, under performs (for proof just look at large fleet battles) which results in poor efficiency and a bad customer experience.


If you really know as much as your peenwaving would lead us to believe you'd know that no amount of hardware can compensate for poor code.

Protip: cisco is not the bottleneck for fleet fights (for proof just look at pre/post dominion).


The Lynxpardinus
Caldari
Wolfsbrigade
Posted - 2010.06.19 07:25:00 - [194]
 

Originally by: Lord XSiV
(or cared about since I am now retired)

This is funny, you try to make yourself look like an experienced professional while you sound like a teen troll that has no idea of how real life business works. You talk about "being the architect of some critical systems", guess what, in real life 99.9 percent of the time, we managers do not architect, we inherit mission critical systems that are outdated, tied to service contracts with companies we don't like that are prohibitively expensive to break, with a shrinking budget and a projected usage of the system that is going to overload it before the end of the fiscal year. Manage to keep a system like that online and then I'll take you seriously.

Your recommendation that they start with a new architecture from scratch is laughable to anybody that manages or is responsible for any type and/or size of ongoing IT operation and shows how naive and misinformed you are of how the real world works.

Originally by: Lord XSiV
Or even worse, put lives at risk.

Really? we are talking about a game here, this is not a system linked to an ER department. In business this is called "risk assessment", you may want to look it up. I'll make it easy on you and give you the Cliff Notes: CCP has probably already calculated the likelihood of an outage of x hours using their current equipment, how much revenue they stand to lose from it and they probably decided that it did not warrant spending money on "better non-Cisco" hardware.

And while on the topic of Cisco, why do you keep blaming them even when CCP is acknowledging that the bottleneck is CPU, not network? Is this what you used to do with your "customers"? not listen to their problems and then recommend expensive hardware overhauls? I wonder how much of a commission you got from Juniper every time you snookered another client into buying stuff they did not need. Word of advice to everybody out there: Beware of any contractor that is too dogmatic when it comes to brand names. (That is if I buy that you are an actual professional, I still think you are just a troll)

To put it in perspective it would be like if you recommended Pizza Hut to buy a Ferrari California to each driver to improve delivery time when the problem is that it takes too long to make the pizza. Ridiculous, right? just as ridiculous as your posts have been so far.

Zendoren
Aktaeon Industries
The Black Armada
Posted - 2010.06.19 08:45:00 - [195]
 

Originally by: Koronos
Originally by: Lord XSiV
<snip>The discussion is really about how much better CCP could be doing things if they elected to go with a better solution. The one that they are going to is far too costly, under performs (for proof just look at large fleet battles) which results in poor efficiency and a bad customer experience.


If you really know as much as your peenwaving would lead us to believe you'd know that no amount of hardware can compensate for poor code.

Protip: cisco is not the bottleneck for fleet fights (for proof just look at pre/post dominion).




I agree

Assuming that CCP did not change hardware since dominion; I would have to say that the problem is in the coding.

Every Tech junkie from scriptkitty to CCNA's knows that if something brakes and you have only changed one thing since it was working (in CCP's case, the code) than the problem is in what you changed.

Makes sense to me.

CCP Yokai

Posted - 2010.06.19 09:35:00 - [196]
 

Edited by: CCP Yokai on 19/06/2010 09:59:08
Koronos - As a network and datacenter administrator and all-around nerd myself I am quite geeked up by these improvements, but I'm pretty sure they won't resolve the horrible fleet lag issues we are experiencing now on TQ.

Response - Iagree, and I hope I was not unclear here at all. We are going to improve everything we can to give the software that runs Eve the breathing room it needs. This move will make sure TQ has the space, power and cooling as well as a big step up in switching. It will not single handedly fix all lag forever. CCP is not singled handed either. There are tons of people spending all day working on lag, tuning, etc. The ops guys are just doing our small part to help.


Edited to add: I'm also an Eve Player and have been for 5+ years myself. I've been in 1200+ battles (more than 1600 in H-W just a few weeks back was the biggest I have ever seen), I've seen the long wait times for a screen to load etc. But, I can tell you... no other game can get this close to that kind of combat. No excuses... just saying we are pushing the edge of what massive PVP has ever been in a game.

CCP Yokai

Posted - 2010.06.19 09:48:00 - [197]
 

Lanu - Your my new favorite dev if you keep up with the blogging (+photo's and vids!!)!


Answer - Camera is already packed in the bags for London. The new space is dead sexy. We already have alot of pics of the space empty... we'll get a few out with TQ actually alive in there.

Koronos
Interstellar eXodus
BricK sQuAD.
Posted - 2010.06.19 14:58:00 - [198]
 

Awesome. Thanks Yokai.

Originally by: CCP Yokai
Edited to add: I'm also an Eve Player and have been for 5+ years myself. I've been in 1200+ battles (more than 1600 in H-W just a few weeks back was the biggest I have ever seen), I've seen the long wait times for a screen to load etc. But, I can tell you... no other game can get this close to that kind of combat. No excuses... just saying we are pushing the edge of what massive PVP has ever been in a game.


No question.

Xelpm
Posted - 2010.06.19 15:02:00 - [199]
 

Edited by: Xelpm on 19/06/2010 15:03:10
Being in the hosting biz over the last 12 years, even in a small way with 1 Cab w/several Servers, the CCP Cluster is still way cool given that all of this stuff is for our use in a GAME!

My biz partner and I personally moved my lone Blade Server from NYC to Philly about 5.5 Years ago! No matter what size, a job like this, is always a lot of hard work and then you're glad to see it when it’s done then you move on to the next big thing! If you never have done something like this you can't imagine how much work this is going to be!

Hat's off to CCP plus good luck on Wednesday! You want to complain about Cisco this or Foundry that go right ahead! ugh It's still an awesome cluster! Very Happy

PS: I Vote for a time elapsed Video -- one from a frontside cab angle and one from the hot isle!!!

Batolemaeus
Caldari
Free-Space-Ranger
Morsus Mihi
Posted - 2010.06.19 16:07:00 - [200]
 

Originally by: CCP Yokai

Edited to add: I'm also an Eve Player and have been for 5+ years myself. I've been in 1200+ battles (more than 1600 in H-W just a few weeks back was the biggest I have ever seen), I've seen the long wait times for a screen to load etc. But, I can tell you... no other game can get this close to that kind of combat. No excuses... just saying we are pushing the edge of what massive PVP has ever been in a game.


Yeah, but remember that it was a _lot_ better before Dominion. People just want back to the time where you could have lag free battles with 300 people.

CCP Yokai

Posted - 2010.06.19 16:30:00 - [201]
 

Originally by: Batolemaeus


Yeah, but remember that it was a _lot_ better before Dominion. People just want back to the time where you could have lag free battles with 300 people.


There are things we are hoping to do to help smaller battles get better access to dedicated hardware quicker...

Part of where we are going with predicting hot spots and remapping blogs will talk about the plans in summer cycles.



Lord TGR
Bat Country
Goonswarm Federation
Posted - 2010.06.19 16:52:00 - [202]
 

Out of curiosity, were there any hardware changes in conjunction with the dominion expansion, or was it a pure software change?

Caladain Barton
Navy of Xoc
The Remnant Legion
Posted - 2010.06.19 19:45:00 - [203]
 

Edited by: Caladain Barton on 19/06/2010 19:55:51
Edited by: Caladain Barton on 19/06/2010 19:52:55
Edited by: Caladain Barton on 19/06/2010 19:51:56
Originally by: CCP Yokai
Originally by: Batolemaeus


Yeah, but remember that it was a _lot_ better before Dominion. People just want back to the time where you could have lag free battles with 300 people.


There are things we are hoping to do to help smaller battles get better access to dedicated hardware quicker...

Part of where we are going with predicting hot spots and remapping blogs will talk about the plans in summer cycles.





Throwing hardware at a problem that was caused by software won't fix the issue. More hardware is always nicer, but the trend in computing is to put more cores on a proc instead of faster procs. The Eve code is locked to a single core..so ultimately performance will lag significantly behind.

That said, the 1600 person battle in H-W would have been perfectly playable pre-dominion. Dominion was a software patch..the hardware didn't change out with the patch. Dominion made the game unstable..why not look at why it's unstable in *addition* to throwing more hardware at the problem?

More hardware is always good, don't get me wrong. When the trend was to shove more power into a single core, things looked pretty for eve's future. But now...i dunno man, i can't help but look at all the cool things you guys have done in the past, and then look at the software team's work as of the last couple patches (100 man fleet jumping into an empty system blackscreening is a nice feature of the latest patch)..they really need to get their act together.

Change the core code to span multiple cores and then you'll see massive performance gains from throwing more hardware at the problem.

By the way, thanks for sticking around and responding throughout the thread. The eve community misses Devblogs where dev's do this :-)

Budsin Adar
Posted - 2010.06.19 21:28:00 - [204]
 

whats moving a cluster going to do to help with eve??ShockedI thought by now we would be walking in stations as Promised back in april or may. Also mining planetary rings and comets? So where is all this in the 2nd part or how is all this going to play out and when can we use the stuff from planets as of now? Peace all and fly safe. just questions my friends have been asking me anyways. AS I to wish to know Peace all :) 07YARRRR!!

TheLostPenguin
Posted - 2010.06.20 10:56:00 - [205]
 

Edited by: TheLostPenguin on 20/06/2010 11:02:29
Moving it is going to do a few things, notably improving cooling and allowing more space for the server, dont know how great this cooling gain will be but if it allows them to turn up clock speeds a bit then that would help performance, guessing that would be a possibility since there's no need for additional cooling to counter stability issues (not like we see the server routinely crashing due to load, it just isn't fast enough). Extra space means space for the new hardware that would be required for dust514 to run on, don't be too surprised if a few more bits get bolted onto tq in the near future too since they'll have the space to do so after this (this is just a move of existing hardware, nothing new actually added), more nodes to reinforce more systems concurrently or just to give fewer systems/node on average would result in slight performance gains (yeah I know the main breakage with big fleets fights is code, but more power overall will help general performance) PLUS they're consolidating most if not all of their hardware for website/forums/testservers into one location, again just because the space is avalible to do so, which apparently will "provide better network connectivity, fewer intermediary devices and increased capacity.".
Since it's not posible to go out and just buy uber-faster gear every little that can be done to help on the hardware front should help....

As for "I thought by now we would be walking in stations as Promised back in april or may." ROFLMAO, you forgot to mention what year, WiS/Ambulation/Incarna/Whatever it gets called in another 2 years has been Coming Soon™ pretty much as long as EVE has been around, don't hold your breath waiting for that, "mining planetary rings and comets?" will be a "groundbreaking never-before considered and wholly innovative" expansion in a couple of years, and will probaly suck just as bad as mining other rocks does :)

*edited to fix several glaring typos as yet to train Proofreading I

CCP Yokai

Posted - 2010.06.20 12:23:00 - [206]
 

Originally by: Budsin Adar
whats moving a cluster going to do to help with eve??Shocked


TheLostPenguin said it better than I did... "Moving it is going to do a few things, notably improving cooling and allowing more space for the server"


Dhalya
Posted - 2010.06.20 13:26:00 - [207]
 

Most probaly you are running your 60+ AppServers in A-A cluster, having your 8853's fitted in 8677's (and usually) FC interconected either with 4 or 8 Gbps modules through SAN Fabric switches.
From the same fabric switches you are interconnecting your clustered 7233's DB servers with a whopping 256GB of RAM to avoid frequent HDD access, and your DS (if IBM) SAN storage.
And finally you are routing you heavy IP-traffic through your 7609/7613 Cisco utilzing the 720Mbps modules (route processors) & DFC's with multiple WAN links.
Although the size of your WAN links as a whole can impose a deadly factor in lag issue, your running hardware is most probably your main issue (appart from coding issues).

Would like to thank you for the heads up of our favorite virtual world's hardware.

Warnings
Crimson Empire.
Nulli Secunda
Posted - 2010.06.20 14:26:00 - [208]
 

i'm sad that ccp don't thinked to use a cool thing like this :

http://www.theregister.co.uk/2009/11/27/ibm_power7_hpc_server/page3.html

It's a joke, it's a bit heavy price i think and ccp can use only x64 servers.
I would prefer That CCP use 4P opterons with 48 cores and 256Go of memory. It's far stronger that they'll use soon. And opterons are a lot faster in virtualisation mode.

And the thing i would love is 2 servers like this, for blobs. Our stupids FC don't want to attack in 4-5 systems in same time. So 1000+ systems are unplayable.

And kill the gameplay of big alliances.

And ****ings titans that got reimbursed that shouldn't ...

i hope ccp plan servers like this, with huge amount of ram. For databases this kind of servers are crazy fast, and eve is mostly a huge dynamic database.

And without good code, hardware is nothing. I hope ccp is doing some great multithreaded code. A reinforced node will be able to use more than one single core, may be twice faster, even more on quad+ cores. ( and would be awesome on a 48 cores ^^ )

And sorry to say that to CCP, but for moment sov 2.0 sucks.

I hope you'll do better for 2000+ blob playable with big hardware ^^

Hawk TT
Caldari
Bulgarian Experienced Crackers
Posted - 2010.06.20 22:09:00 - [209]
 

CCP YOKAI REALLY DEMONSTRATES WHAT "COMMUNITY INTERACTION MEANS"!
GREAT & FRESH ATTITUDE!!!


I just hope that he would not get bored by the tons of irrelevant and "shallow" comments on the forums...

Anyway, I DO HAVE some more questions, giving the fact the CCP has a special relationship both with IBM and NVIDIA:

Off-loading Server Side SOL calculations to GPGPU?

Have you @ CCP evaluated and/or experimented with CUDA/OpenCL, apart from the APEX Clothing for Incarna?

Some facts:
a) IBM has released recentely their iDataPlex dx360 M3 node, supporting up to 2 x NVIDIA GPGPUs. BladeCenter is OK for Databases, but iDataPlex is for HPC ;-) That reminded me of the following things...
b) CUDA supports double-precision FP, you get a magnitude better FP performance w/ a magnitude faster local RAM buffer
c) Off-loading space simulation calculations to a massively parallel GPGPU architecture should make a big difference in terms of off-loading the CPU, which has to do all the rest and especially I/O
d) There are lots of projects for Phyton warpers/libs for OpenCL / CUDA.

I could see a couple of possible problems, porbably there are more:
a) CPU to GPGPU link goes through PCIe - that means "bandwidth bottleneck" and relatively high latency. This could be a problem for moving the data from the Host RAM to the GPGPU buffer, all the time, in real time...?
b) GPGPU memory buffers are small, probably not big enough to do all the SOL math?
c) The programming effort is prohibitevly expensive due to lack of GPGPU programmers on the market?
d) IBM iDataPlex is a different monster, not-quite-mature etc.?

With CUDA/OpenCL on the server side you could do even more stuff - things like "precise collision-detection", more realistic mass/agility related simulation, ship-model desintegration (chopping off parts of a ship) etc.


Jim Luc
Caldari
Rule of Five
Vera Cruz Alliance
Posted - 2010.06.20 23:31:00 - [210]
 

Originally by: Hawk TT
CCP YOKAI REALLY DEMONSTRATES WHAT "COMMUNITY INTERACTION MEANS"!
GREAT & FRESH ATTITUDE!!!


I just hope that he would not get bored by the tons of irrelevant and "shallow" comments on the forums...

Anyway, I DO HAVE some more questions, giving the fact the CCP has a special relationship both with IBM and NVIDIA:

Off-loading Server Side SOL calculations to GPGPU?

Have you @ CCP evaluated and/or experimented with CUDA/OpenCL, apart from the APEX Clothing for Incarna?

Some facts:
a) IBM has released recentely their iDataPlex dx360 M3 node, supporting up to 2 x NVIDIA GPGPUs. BladeCenter is OK for Databases, but iDataPlex is for HPC ;-) That reminded me of the following things...
b) CUDA supports double-precision FP, you get a magnitude better FP performance w/ a magnitude faster local RAM buffer
c) Off-loading space simulation calculations to a massively parallel GPGPU architecture should make a big difference in terms of off-loading the CPU, which has to do all the rest and especially I/O
d) There are lots of projects for Phyton warpers/libs for OpenCL / CUDA.

I could see a couple of possible problems, porbably there are more:
a) CPU to GPGPU link goes through PCIe - that means "bandwidth bottleneck" and relatively high latency. This could be a problem for moving the data from the Host RAM to the GPGPU buffer, all the time, in real time...?
b) GPGPU memory buffers are small, probably not big enough to do all the SOL math?
c) The programming effort is prohibitevly expensive due to lack of GPGPU programmers on the market?
d) IBM iDataPlex is a different monster, not-quite-mature etc.?

With CUDA/OpenCL on the server side you could do even more stuff - things like "precise collision-detection", more realistic mass/agility related simulation, ship-model desintegration (chopping off parts of a ship) etc.




I like this. I don't quite understand it, but I like this. Precise collision detection and ship model disintegration and debris would be so cool!

So, in layman's terms, how will this improve the Eve experience? If it won't fix lag, what improvements are going to be made to fix it? More hamsters?


Pages: first : previous : ... 3 4 5 6 [7] 8 9 10 11 : last (11)

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only