open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: The Long Lag
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 2 3 [4] 5 6 7

Author Topic

Liang Nuren
Posted - 2010.08.18 08:24:00 - [91]
 

DISCLAIMER: I had like a lot of beer while writing this post. I hope it is coherent.

Hey Warlock, I'd like to thank you for putting out such a detailed dev blog. You're quickly topping my list of favorite people at CCP. :)

My particular realm of expertise is scaling data warehouses - so for this particular problem I'm nothing more than an extremely interested armchair programmer from another company. I also want to say that I appreciate your depiction of the nature of jump lag, and I offer my sympathies and most heartfelt encouragement. Try not to hit your head too hard when you see what the problem was. :)

The rest of the dev blog was a little complicated, and I'd like to make sure that I adequately understand it. To put it in the simplest terms, the fundamental problem that you're facing is that as more objects begin interacting in a generic system*, the amount of processing and communication between these objects rapidly increases (N squared?).

It seems like there's naturally two things you can do about this fundamental "N-Body"*** problem:
- Get better/more processing bandwidth. This would cover such things as a fully distributable system, but because CCP cannot possibly have unlimited hardware all you're really doing is moving the number of reasonable/allowable objects in the system* up. Given the nature of players, they will immediately nudge this new barrier.
- Limit the number of objects in the system*.

Here are some thoughts on the situation, some of which were collected from your dev blog:
- Remove bugs and inefficient code. This would operate exactly the same way increasing available processing bandwidth (though depending on the fixes it could be pretty substantial!). It obviously cannot be a permanent solution.
- Artificially limit the number of objects in a system*. This might happen via instancing or via game mechanics like requiring an attacker to hold many different objectives in different systems* simultaneously.
- Artificially compress the number of objects in a system*. This might happen via allowing a FC direct control of fleet members (similar to "grouped guns", but much expanded).
- Artificially limit the number of actions of objects in a system*. This might happen via some kind of "siege mode" (which would consequently disallow movement, for example). It might also happen by removing/limiting system** dependencies (logically or via the "Battle Mode" mentioned earlier).

* Not an eve system. A generic system - a container or set of objects if you will. I suspect that currently for Eve, this probably is actually a system. Razz
** A software system, such as local chat, the market, evemail, and your combat. In the dev blog, you mentioned how Eve is really a whole set of interconnecting distributed software systems.
*** If it isn't really an N-body problem, feel free to correct me. It sounds and feels like it is, though.

So it seems like really, the dev blog can be summed up as:
"We're working on stomping bugs and fixing inefficient code, which will improve processing bandwidth/throughput. This is an incredibly frustrating process too! Once we finish that, we don't have a choice but to implement a game design fix if we want to permanently fix this"

-Liang

Una Achura
Posted - 2010.08.18 08:30:00 - [92]
 

Edited by: Una Achura on 18/08/2010 08:33:46
Edited by: Una Achura on 18/08/2010 08:33:19
Originally by: AeonOfTime
The solution is right under your noses, and CCP Warlock even mentioned it in his excellent post:

Quote:
There can also be Schrodinger effects, where examining the state of the system changes its behaviour enough that the issue doesn't manifest itself.


Laughing


Actually looks like a Heisenbug, not a Schrödingbug (But probably really a Mandelbug)

Freyya
Advanced Planetary Exports
Intergalactic Exports Group
Posted - 2010.08.18 09:07:00 - [93]
 

I'm just going to throw it out there for the specialists to ponder and iterate on (noob tech speak alert);
Most of us are aware of the "ability" to insert "spatial anomalies" into the local system.
Is it not possible for the code monkeys to devise a script that injects code into the client with a tracker/(genetic) marker which allows you to follow it through the paths it takes to/in the server and see where it dies off or what causes it?

See it like this; You know there's a massive fleetfight that's going to happen. Create a character with one of the involved alliances/corps as ticker (c'mon we all know you guys can do that :P) and join their fleet that's going to jump in. Insert the script and let it run while experiencing the long lag. don't fight or do anything, just let it sit there gathering data and die.

Possible that I'm thinking stupid and just spewing idiot and fool type nonsense but mehh.
On the other hand that might have been tried already or something similar.

MissyDark
Posted - 2010.08.18 09:10:00 - [94]
 

Originally by: CCP Atropos
Originally by: Bomberlocks
Originally by: Trebor Daehdoow
Edited by: Trebor Daehdoow on 17/08/2010 21:41:14
Great blog, Jackie. As soon as you finished your informal presentation at the summit, I knew it would be a hit with the players.

If you could go into some more detail about the session-change issue, I think that would be much appreciated.

BTW, everyone should be sure to read the presentation that is attached to the blog. It goes into things in more detail, and provides important context for understanding this and other technical devblogs. And it is basically all shiny graphs.
The ironic thing, as far as I see it, is that although the eve server might be distributed, the client server relationship is still absolutely hierarchical. I honestly wonder if offloading some of the computation to clients wouldn't be a good thing.

But then you end up with the client being authoritative which is generally a bad idea.


Sure. Unless you make connected clients parts of your server infrastructure. You know, skype meets [email protected] style.

CCP Atropos

Posted - 2010.08.18 09:35:00 - [95]
 

Originally by: MissyDark
Originally by: CCP Atropos
Originally by: Bomberlocks
Originally by: Trebor Daehdoow
Edited by: Trebor Daehdoow on 17/08/2010 21:41:14
Great blog, Jackie. As soon as you finished your informal presentation at the summit, I knew it would be a hit with the players.

If you could go into some more detail about the session-change issue, I think that would be much appreciated.

BTW, everyone should be sure to read the presentation that is attached to the blog. It goes into things in more detail, and provides important context for understanding this and other technical devblogs. And it is basically all shiny graphs.
The ironic thing, as far as I see it, is that although the eve server might be distributed, the client server relationship is still absolutely hierarchical. I honestly wonder if offloading some of the computation to clients wouldn't be a good thing.

But then you end up with the client being authoritative which is generally a bad idea.


Sure. Unless you make connected clients parts of your server infrastructure. You know, skype meets [email protected] style.

For those applications it doesn't matter that the client is authoritative, since you're sending data that's less important (read: valuable) to the system. If we do this with EVE, all it takes is one unscrupulous person to massively ruin the value of the entire system.

Simply put, if you want to retain the value of the entire system for all users, you must operate under a paradigm where the terminal (client) is considered hostile, whether it is as a result of deliberate intervention or simply lag or corruption.

Of course there will be compromises, where you can offload some data transactions, and run simultaneous simulations, only transmiting the changes (deltas), much as the physics system (Destiny) does.

Alain Kinsella
Minmatar
Posted - 2010.08.18 10:03:00 - [96]
 

Edited by: Alain Kinsella on 18/08/2010 10:11:48
Edited by: Alain Kinsella on 18/08/2010 10:06:09
Not sure what was scarier - the fact that I read all that (minus the thesis), or that I understood most of it. Shocked

@ Warlock - I only read the thesis abstract, but was enough to bring up one interesting question. Has a way been devised for taking a 'node' currently running in the 'Mesh' model, and re-architect it locally (and dynamically) as a Hierarchy? Your thesis appears to imply an emergent topology is possible, both in theory *and* practice.

---

As someone who has knowledge of the SL Grid model, I did find it interesting (even amusing) that the two of you are actually attacking the same problem - but due to different scaling directions. [SL is severely over-provisioned in hardware, for very little load on an average server - requiring them to scale *down*, not up, to stay solvent.]

In both formats, the ability to dynamically re-provision will probably be an important key in the scaling fix. I think CCP may have the edge here due to, ironically, running in a Windows server farm. From what I know of another game's backend model, the ability to move a running TCP socket between servers was a key reason for migrating *their* cluster from Unix to Windows (the client needs to support this, though, and Wine can't emulate this particular trick yet).

---

Regarding 'Physics completely in client' - Myst Online (Uru) did this, due to the original single-player component. It was found to cause severe latency issues, due to multiple clients sending 'kickable' position updates to the server - sometimes in conflict with each other! So its important to be careful with this one.

@ Trebor - I also do not believe that certain things should be client-only, even if you have the Server pre-sign the external files. There's many example links I can supply from SL-space (some of it recent, some of it old, and in one instance I heard it live from Philip). [EDIT - Atropos has summarized this quite well in the post above mine.]

Bomberlocks
Minmatar
CTRL-Q
Posted - 2010.08.18 10:04:00 - [97]
 

Originally by: CCP Warlock
Originally by: Bomberlocks
...<snip>...

I'm afraid if you want to protest idiocy, you're going to have to stop asking such good questions Smile

I guess the short answer is, clearly not fault tolerant enough at the moment. Although to be fair the system in general is remarkably fault tolerant. We do have tics on the servers, but they aren't hard, real time tics. That approach can work well for some constrained problems, but system wide synchronization introduces its own scaling issues for distributed systems, especially over WAN's, not to mention a really nasty impossibility proof.

TCP is the transport mechanism for all cluster communication, we then put our own message handling system above that, and then that's encapsulated in RPC semantics for the game developers. One fairly obvious explanation for the long lag issue is that its a deadlock problem, and a client getting rate adapted would provide an explanation for how it could get triggered, under a couple of peculiar to fleet fight situations we could think of.

Upshot was, a couple of our long suffering QA engineers had some fun with Netem, to no good result unfortunately. It was such a nice theory too, *sigh*
Thank you very much for your reply. I'm beginning to get a vague picture of the run time processes and also, more importantly, I'm beginning to see how vast the problem is.

In a post I made further up, after reading your presentation, I saw that although the server processes of the eve engine are distributed, the client-server architecture seems to be very much an absolute hierarchy. I wondered whether there was a possibility of offloading some of the load to the clients.

Obviously, in a general game sense, this is considered a bad idea, as CCP Atropos posted above, due to the fairly obvious problem of running into a lot of hacked client problems, this necessitating some kind of DRM solution (itself a veritable minefield) and itself possibly running into even harder to spot problems on another.

However, certain things, such as the eve channel voice communications could possibly be offloaded to the clients, couldn't they? The voice communication (although not much used in big fleet fights as most prefer external 3rd party providers like Teamspeak or Ventrilo) might add a significant amount of network load and server processing overhead, I would think. Given that the voice functions have no other effect on the game aside from, well, not having voice if it doesn't work, wouldn't putting this into a p2p kind of architecture, where the server provides tracking of participants but the actual voice processing and network communication is directly from client to client, drop some load from the server?

Trebor Daehdoow
Gallente
Sane Industries Inc.
Posted - 2010.08.18 10:23:00 - [98]
 

Originally by: Estimated Prophet
If you like sci fi I recommend Halting State by Charles Stross.

Stross made it onto my coveted "Buy in Hardback" list in record time. Highly recommended. Anyone who works in IT and does not love the Laundry stories needs counseling.

CCP Oveur

Posted - 2010.08.18 10:44:00 - [99]
 

Originally by: Nareg Maxence
Originally by: CCP Oveur
Originally by: Mynxee
Thrilled to see this dev blog, thrilled to check it off as Delivered on the CSM's "CCP Deliverables List" from the Summit, and now actually gonna get a nice cup of coffee and read it. Thanks, Warlock.



Why isn't that list in the Evelopedia?

Why isn't there a link that says Evelopedia with big friendly letters on the main EVE website?

(Not counting the Item Database link.)


Look further down, it's between EVE Gate and EVE Store.

Yeay Fritg
Caldari
Confrerie de Kaedri
Cluster Of Rebirth
Posted - 2010.08.18 11:04:00 - [100]
 

Originally by: CCP Warlock


If we knew we would tell you (honest, gov!). At first we thought it was introduced with Dominion, but after some player comments in the forums, and some detective work by the GM's, the balance of suspicion moved to it being a pre-existing bug that something in Dominion made worse. These really are the nightmare kind of bugs, because as you say, the first question everybody asks when something like this happens, and usually the easiest way to track it down, is 'what changed?'.

Partly too I think, there has been a certain degree of confusion about 'lag', between general performance degradation - and agreed there's also some work we need to do there too - and the specific long lag bug where the jump never fully completes, yet the player is a valid target in the destination system.


Good at least to see non corrected Bug annoy the Dev as the Players !

When, please, a true Dev Blog on bugs ?

Sorry, but what's the reason not to solve Dominion identified bugs ?

Cause you don't know if it will create new ones somewhere ?

I would like to understand why so many builds in the last Tyrannis 1.0.4 for so few bugs solved in.

Please help us to know if you care about bug or if you just let them there till it impact the 'Saint Lag'.

Yeay

Venkul Mul
Gallente
Posted - 2010.08.18 11:16:00 - [101]
 

Originally by: Hamish Nuwen
Originally by: CCP Oveur
Originally by: Hamish Nuwen
In the long run, you should stop thinking.

This. Bliss.

Meanwhile, you can continue thinking about:

  • No cargohold (=database) dependencies in combat (ammunition bays, preloaded scripts, ...)

  • Maybe a explicit combat mode (you can't open assets, or market or fitting, any not related with combat) with messaging priority (obviously only those related to combat)

  • [/list]


    Define combat for you "suggestions". Have you considered some scenario like:
    - Ooops a Gurista frigate is attacking my hulk, I can't touch my cargohold or switch crystals until it is dead;
    - No looting during a mission, you can't open your cargohold;
    and so on.

    PvP fleet combat is important in the game but bending every other part of it to the needs of fleet combat is very bad.

    Nareg Maxence
    Gallente
    Posted - 2010.08.18 11:30:00 - [102]
     

    Originally by: CCP Oveur
    Originally by: Nareg Maxence
    Originally by: CCP Oveur
    Originally by: Mynxee
    Thrilled to see this dev blog, thrilled to check it off as Delivered on the CSM's "CCP Deliverables List" from the Summit, and now actually gonna get a nice cup of coffee and read it. Thanks, Warlock.



    Why isn't that list in the Evelopedia?

    Why isn't there a link that says Evelopedia with big friendly letters on the main EVE website?

    (Not counting the Item Database link.)


    Look further down, it's between EVE Gate and EVE Store.

    Here. Have an upside down bottle of water.

    Falkrich Swifthand
    Caldari
    eNinjas Incorporated
    Posted - 2010.08.18 11:34:00 - [103]
     

    Originally by: Una Achura
    Actually looks like a Heisenbug, not a Schrödingbug

    I've seen a schroedinbug! There was a memcopy call that had the source and dest pointers reversed, we never did figure out how it used to work...

    Aineko Macx
    Posted - 2010.08.18 11:38:00 - [104]
     

    Not so much new stuff in the presentation for somebody that already has an M.Sc. in CS, but glancing at the thesis it looks interesting Cool

    Originally by: CCP Oveur
    But the first step, and greatest future potential, is to get things to fully capitalize on multi-processor and multi-core. Like spanning a solar system over multiple cores.

    I thought I'd never see the day. Maybe I will after all.

    Opposed to my usual pessimism regarding CCPs competence and the state of eve, I really wish you guys good luck in succeeding.

    Originally by: Trebor Daehdoow
    Originally by: Estimated Prophet
    If you like sci fi I recommend Halting State by Charles Stross.

    Stross made it onto my coveted "Buy in Hardback" list in record time. Highly recommended. Anyone who works in IT and does not love the Laundry stories needs counseling.

    Confirming this. His freely available Accelerando got me hooked.

    MissyDark
    Posted - 2010.08.18 12:04:00 - [105]
     

    Originally by: CCP Atropos
    Originally by: MissyDark

    Sure. Unless you make connected clients parts of your server infrastructure. You know, skype meets [email protected] style.

    For those applications it doesn't matter that the client is authoritative, since you're sending data that's less important (read: valuable) to the system. If we do this with EVE, all it takes is one unscrupulous person to massively ruin the value of the entire system.



    You can solve that by redundancy - assign the same calucations to lets say 10 random clients instead of one. Of course I realize that the bottleneck moves to synchronization however it is worth to think about - you get access to more computing power than you can need.

    Cryuji Eterna
    Minmatar
    The New Order.
    Posted - 2010.08.18 12:30:00 - [106]
     

    Originally by: MissyDark
    Originally by: CCP Atropos
    Originally by: MissyDark

    Sure. Unless you make connected clients parts of your server infrastructure. You know, skype meets [email protected] style.

    For those applications it doesn't matter that the client is authoritative, since you're sending data that's less important (read: valuable) to the system. If we do this with EVE, all it takes is one unscrupulous person to massively ruin the value of the entire system.



    You can solve that by redundancy - assign the same calucations to lets say 10 random clients instead of one. Of course I realize that the bottleneck moves to synchronization however it is worth to think about - you get access to more computing power than you can need.


    I don't think the issue is with computing power (if so it would be an easy get faster/more powerful processor fix). From my understanding it is a bandwidth issue between the different aspects of the eve system. To fix they can either increase the bandwidth capacity between the system components, or refactor and/or redesign the system so it uses less bandwidth overall.

    Falkrich Swifthand
    Caldari
    eNinjas Incorporated
    Posted - 2010.08.18 12:49:00 - [107]
     

    Originally by: MissyDark
    Originally by: CCP Atropos
    Originally by: MissyDark

    Sure. Unless you make connected clients parts of your server infrastructure. You know, skype meets [email protected] style.

    For those applications it doesn't matter that the client is authoritative, since you're sending data that's less important (read: valuable) to the system. If we do this with EVE, all it takes is one unscrupulous person to massively ruin the value of the entire system.



    You can solve that by redundancy - assign the same calucations to lets say 10 random clients instead of one. Of course I realize that the bottleneck moves to synchronization however it is worth to think about - you get access to more computing power than you can need.


    There's already plenty of spare computing power in the form of extra cores in the servers. It should be easier to use that than to farm calculations out to multiple clients, and there's much less latency to worry about.

    Peter Tjordenskiold
    The Executives
    Executive Outcomes
    Posted - 2010.08.18 13:13:00 - [108]
     

    Edited by: Peter Tjordenskiold on 18/08/2010 13:13:33
    Awesome blog. I don't doubt about this frustrating situation and wish you all luck to solve this issue.

    I worked for 2 years ago self on a distributed system on a much smaller scale. One small mistake in the start and your hole software design is blowing up. I'm a little bit pessimistic. Maybe less is more? To many features maybe inducing to much communication and demanding an another design approach?

    This game is awesome and I play it since 2006 and despite with these problems I'm don't considering to abandon my account. I'm confident that the devs are able to solve the problems.

    ghosttr
    Amarr
    ARK-CORP
    Intrepid Crossing
    Posted - 2010.08.18 14:04:00 - [109]
     

    I think sisi could use alot of work on being more friendly to get going, just adding something to automate the current sisi setup process isnt enough imo.


    - The eve installer should have an option to install sisi to a predetermined folder. (ticked off by default?)

    - There should be a sisi link in the start menu folder, if the files are not already copied it should offer to create the directory (predetermined, with an option for the user to specify a custom folder) and copy the files.

    - There should be a option to have the tq client download sisi updates. (this way sisi is kept up to date, and you dont have to download all the patches at once to login to sisi)

    - Maybe a sisi installer/patcher for steam? (so steam manages it and keeps it up to date for youRazz)

    Louis deGuerre
    Gallente
    Malevolence.
    Posted - 2010.08.18 15:16:00 - [110]
     

    Good blog keep 'm coming.

    Apologies are nice but they buy us nothing (as we say in Holland). I'd much rather see CCP gets its priorities straight and assigns more resources to EVE, and paradigm shifts from 20% bugfixing to 80% bugfixing.

    I wish you much luck on finding and solving the problem. As a programmer I realize it's a hard task for you guys and you should all get massive pay rises, better lunches and working hours, and lots of free stuff.

    * hopes his boss spy software will pick this up *

    SovietShield
    Amarr
    British Legion
    Posted - 2010.08.18 15:39:00 - [111]
     

    the problem being single world idea has become too deluded, I dont want to play with 50,000 concurrent players.

    electrostatus
    Center for Advanced Studies
    Posted - 2010.08.18 15:57:00 - [112]
     

    Originally by: CCP Oveur

    You seriously do not want me to think about solving technical issues. Last time that happened unicorns died, dolphins left earth and war broke out in a galaxy far far away.


    So wait, are you saying that the last time you thought about solving such issues led to the works of 'The Last Unicorn', 'The Hitchhiker's Guide to the Galaxy' and the whole of 'Star Wars'? I'm suspicious of this claim.

    Mirjam Starborn
    Amarr
    Posted - 2010.08.18 16:31:00 - [113]
     

    Debugging distributed real time systems is a little different from dealing with sequential programs.

    "little" - this is probably the biggest understatement ever made so far by CCP officials.

    Anyhow, CCP Warlock, many thanks for the detailed dev blog and thanks for sharing the pdf presentation. It is an impressive system CCP brought together so far but as you say there is room for improvement as well.

    When you look at issues as tracking down massive lag I feel sympathy with the dev tem having to deal with these problems. These are really, really, hard stuff to deal with. Question is, can they even be solved? A empirical derived postulate says that all available resource will be used up effectively removing any optimization done before and then it will be as-if one where back at square one again, despite the fact the system performance is better than ever before (does it sounds familiar ;).

    I do remember the time when the server suffered global lag, but those issues has been fixed, and the performance increased dramatically over night with new storage devices. If removing these improvements I am confident in that EVE would freeze to death with the current load of today.

    The performance achievement, making way for new content and features to be added and then again slow down the system, are most appreciated work.

    You have my sympathies – and deepest respect.

    CCP Warlock

    Posted - 2010.08.18 17:59:00 - [114]
     

    Originally by: Lord Zim
    Getting the server code to span multiple cores is probably going to lift the limits by which we're ... erm, limited by. However, I'm not so sure that's what should be the main focus to start with, although once the cause for long session changes is found then getting the code multicored should probably be next on the list to fix module lag.




    We have quite a few activities going on in parallel, some of them have a longer time frame than others of course.

    Originally by: Lord Zim

    Having said the obvious, I have a few questions.

    1) Does SiSi have multiple nodes? I've been able to jump into systems with 500+ easily, and load grid within a minute, and I've jumped into systems with 50 (not fighting) and not loading. I've also been kept in the old system for 20-30 minutes while jumping out, while watching everyone else at the gate (hostiles and friendly) warp around and shoot eachother just fine, only to load almost instantly once I'm "out of" the old system. So my theory is that the problem may be hard to reproduce on SiSi because you're jumping from one system to the next within the same physical node, and thus not triggering the specific issue that was introduced (or worsened, I really don't care about the distinction) in Dominion. I assume this has been thought of, but I just want to throw this in there just in case it hasn't, because I remember seeing it mentioned somewhere that sisi and tq aren't quite as equal as might be desired for testing complex issues like we're looking at now.



    Yes it does. SiSi isn't always completely identical to TQ, at the moment it has slightly slower hardware for example. Generally though that's an advantage, especially for work on load related issues.

    The large variation in situations where this problem can occur, and also when it doesn't, is decidedly strange. There has been a lot of debate about this, and there are various theories, but no good answers at the moment at least. The within the same physical server or between servers issue has also been explored, and as far as we are aware it's not a factor. It has also been a consideration in mass test design.

    Originally by: Lord Zim

    2) I haven't kept up 100% on what has been fixed, but the NC vs white noise fight in 6NJ (I think it was) where the titans were rubberbanded back into the system, has that problem been isolated and fixed?




    Yes, this was fixed in Tyrannis 1.0.2.

    Vincent Athena
    Posted - 2010.08.18 19:37:00 - [115]
     

    I have been writing programs for decades, and over that time have learned several tricks to getting code to run faster. Many of these are specific to the task the program is doing, but some are more general. They tend to be methods that modify one or a couple of lines of code. I wonder how much effort CCP puts into this sort of tuning of the code in its quest for speed.

    As an example (and forgive me, I do engineering programming, and hence use fortran):

    Line 1
    A = B / C / D

    Line 2
    A = B / (C * D)

    Line 2 runs faster than line 1, even though they both do the same task, because the floating point processor takes several times as long to do a divide as a multiply.

    On that theme, you should never divide by 5, but multiply by 0.2. If you store a constant to be used in an equation, do not divide by that constant, instead store its reciprocal and multiply.

    Another example: Lets say you have a block of code that executes only when a unlikely event happens, like 2 ships bumping, and several conditions have to be true before that code executes. You could use

    If (condition1 .and. condition2 .and. condition3) then
    [code]
    endif

    But its better to do

    If (condition1) then
    If (condition2 .and. condition3) then
    [code]
    endif
    endif

    because more than likely condition 1 will be false and the other conditions will not even have to be evaluated. I had one program which this single change made run 10 times faster.

    Another example: lets say you want to see if 2 objects are closer together than some set Distance. You could use

    If ( sqrt( xdif^2 + ydif^2 + zdif^2) < Distance ) then

    But if you did this:

    If ( xdif^2 + ydif^2 + zdif^2 < Distance^2 ) then

    Its faster because doing a square is faster than a square root. Even faster is:

    If ( abs(xdif) < Distance ) then
    If ( xdif^2 + ydif^2 + zdif^2 < Distance^2 ) then

    Because if the difference in the x dimension is large, then there is no reason to do the full 3D distance calculation because the 2 objects cannot be close.

    Anyway, you get the idea: going through code line by line and making changes that do not change what the code does, but how it does it, can have big impacts on speed. Obviously on a code as huge as EVE you want to concentrate on code that executes a lot, to get the biggest impact from your programming effort. But you also want to keep methods like this in mind when writing new code, so its fast from the beginning.

    CCP Warlock

    Posted - 2010.08.18 19:40:00 - [116]
     

    Originally by: Bomberlocks
    <snip>...
    In a post I made further up, after reading your presentation, I saw that although the server processes of the eve engine are distributed, the client-server architecture seems to be very much an absolute hierarchy. I wondered whether there was a possibility of offloading some of the load to the clients.

    Obviously, in a general game sense, this is considered a bad idea, as CCP Atropos posted above, due to the fairly obvious problem of running into a lot of hacked client problems, this necessitating some kind of DRM solution (itself a veritable minefield) and itself possibly running into even harder to spot problems on another.



    In distributed computing we're generally trained to think very carefully indeed about system integrity and ensuring that protocols can't be cracked, or otherwise exploited. I will say that MMO's in general, and EVE's player base in particular, raise that aspect of the job to an entirely new level. So yes, there are a lot of things we could do on the client that are ruled out for those reasons. Even the multiple client solutions - such as Cryuji Eterna suggested can be exploited with a little effort and some assistants - and we're creating a game that encourages people to form groups. If we can think of ways around them, then we have to assume that you guys can too.


    Originally by: Bomberlocks

    However, certain things, such as the eve channel voice communications could possibly be offloaded to the clients, couldn't they? The voice communication (although not much used in big fleet fights as most prefer external 3rd party providers like Teamspeak or Ventrilo) might add a significant amount of network load and server processing overhead, I would think. Given that the voice functions have no other effect on the game aside from, well, not having voice if it doesn't work, wouldn't putting this into a p2p kind of architecture, where the server provides tracking of participants but the actual voice processing and network communication is directly from client to client, drop some load from the server?


    We do offload some services like voice from the cluster exactly as you suggest, we use a third party supplier called Vivox, and there's no real load profile on the cluster from that service.

    P2P is another avenue we occasionally discuss. It's not a panacea, and there can be non-technical issues with it especially with most European countries where bandwidth usage is usually metered by the month, but if we had the right use case it would certainly be an option.

    Smurphy1
    Silver Snake Enterprise
    Against ALL Authorities
    Posted - 2010.08.18 19:52:00 - [117]
     

    Edited by: Smurphy1 on 18/08/2010 20:01:32
    I'd be very much inclined to hear more about CCP's plans for moving the servers to a multi core system and would hope that there will be moar blogs about it in the future. I have two questions though.

    1. I remember reading somewhere that when a player jumps to another system that the receiving process has to basically rebuild the object that represents the player/ship instead of getting passed a reference to the original object or a copy. Is this still true?

    2. What part of a fleet fight generates most of the server load under normal circumstances? Between things like updating position and velocity, collision detection, module activation, grid load, damage calculations, etc.

    Edit: I had a thought about distributing some load to clients was about calculating the distance to objects. Im not sure if the server currently calculates the distance from all objects to all other objects but if it did one way that the load could be reduced is if the clients calculated the distance from all objects to themselves. Then if they initiated an action that required them to be within a certain distance the server would verify the distance between them to insure that they are correct. This would distribute the N^2 load across N clients and the server would only have to do a distance calculation when it needed to verify which would only be on the order of N.

    Mirjam Starborn
    Amarr
    Posted - 2010.08.18 20:42:00 - [118]
     

    Originally by: Freyya
    Possible that I'm thinking stupid and just spewing idiot and fool type nonsense but mehh.
    On the other hand that might have been tried already or something similar.


    In fact they seams to do something similar, reported in this recent dev blog

    Mirjam Starborn
    Amarr
    Posted - 2010.08.18 21:13:00 - [119]
     

    Originally by: Vincent Athena
    Another example: Lets say you have a block of code that executes only when a unlikely event happens, like 2 ships bumping, and several conditions have to be true before that code executes. You could use



    Most languages will do a left to right evaluation and will stop the evaluation when a logical conjunction is false in the expression and verse versa with disjointed expression.

    Originally by: Vincent Athena

    Line 1
    A = B / C / D

    Line 2
    A = B / (C * D)



    When it comes to optimizing mathematical expression (or even the order they are performed in) modern complied languages has optimization options for speed and any decent complier should be handle this automatically - the reason for this is that the complier in most case knows more about the underlying hardware than a application developer ever will do.

    A am a bit surprised Fortan does not do these things in the background.

    Nareg Maxence
    Gallente
    Posted - 2010.08.18 21:23:00 - [120]
     

    Originally by: Vincent Athena
    I have been writing programs for decades, and over that time have learned several tricks to getting code to run faster. Many of these are specific to the task the program is doing, but some are more general. They tend to be methods that modify one or a couple of lines of code. I wonder how much effort CCP puts into this sort of tuning of the code in its quest for speed.


    A decent compiler should be able to optimize those code examples to the point where there is no difference in execution speed.


    Pages: 1 2 3 [4] 5 6 7

    This thread is older than 90 days and has been locked due to inactivity.


     


    The new forums are live

    Please adjust your bookmarks to https://forums.eveonline.com

    These forums are archived and read-only