open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Fixing Lag: Drakes of Destiny (Part 2)
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 2 3 [4] 5 6

Author Topic

CCP Masterplan


C C P Alliance
Posted - 2010.12.12 14:18:00 - [91]
 

Edited by: CCP Masterplan on 12/12/2010 14:25:04
Originally by: Khenar
This leads me to think that you are still performing individual serialization operations for each cloaked observer so that their client updates include the update information of their cloaked ship. How does this affect the performance of the node if a significant number of cloaked observers were introduced into a grid?

You are correct, there are a number of edge-cases that I simplified out of the explanation for the sake of space. There is still a small part of each update that can't be pre-serialised (which is why SendToClients doesn't collapse down to a very small time). Cloaking and a few others details are handled here.

Originally by: FzZZy
Thanks for a good devblog. I realy like when you at CCP share your small eureka moments. You mention that the increase in inventory and explosions was due to not maxing the preformence. Is it the same case why evolve increased with 1/2 (>10ms)?

The serialization of a ball that happen every tick, can't it be cached to the next tick whare the ones that is updated is reserialized? Or does the a ball update to often?

Evolve actually varies a fair bit between frames (In that particular test, it was in the range 25-40 ms if I recall) That is why I recorded the first graph over 100 seconds, to smooth out this variation.
As for re-using the serialisation across ticks - it is impossible to determine if something else modified some ball state outside of the Destiny loop. I suppose we could do something with dirty-flags, but I'm not sure the saving would be too much.


Originally by: Neo160
question about missiles. Pre-Tyrannis (in Dominion i think), if a missile was flying slow enough, and far enough away, i could click the missile and actually track it with my camera (pretty cool), this ability was removed after Tyrannis released. was this a lag fighting decision, and if so, whats the CPU impact from being able to track missiles with the in-game camera? just curious Very Happy

great blog, very informative, as I'm in school for this kinda stuff, it helps to be able to try to wrap my head around programing code and logic without worrying about deadlines or grades Laughing, DeVry goes through stuff really fast, and they only touch on the basics, if at allNeutral.

I'm not sure about the 'why' of that change - I wasn't involved in it. However camera tracking is purely client-side, so it wouldn't have changed anything as far as the server is concerned.

Originally by: Typhado3
very good to hear about this stuff being done. Also nice to see explanations and graphs.... however that first graph in part 2 the label on the y axis is seconds?? thought it would be cpu

It is actually seconds spent inside timer areas, which I grouped into the six high-level categories. However you could also consider it as CPU allocation if you like - in this case they are pretty much equivalent.

zcar300
Gallente
Posted - 2010.12.12 17:31:00 - [92]
 

Interesting. Keep up the good work.

MotherMoon
Huang Yinglong
Posted - 2010.12.12 19:06:00 - [93]
 

Edited by: MotherMoon on 12/12/2010 19:15:17
Originally by: CCP Explorer
Originally by: Siena Petrucis
Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.


ok lets be honest, It doesn't feel like an intended feature to have smartbombs randomly take down missiles. As it stands they are better at it than defender missiles. Shouldn't smartbombs be for killing drones? Does anyone even use Smartbombs for killing missiles? I've seen in used in the alliance tournament years go, our team even used it, but it doesn't work, it doesn't stop ****.

Missiles seriously need to be removed from being effected by AOE weapons and replaced with something that doesn't kill the servers for the sake of a feature that is hit miss at best (as far it working to kill a missile in a laggy environment or not)


Maybe even simpler. Get the best of both worlds. It's a mid slot (or low?) module that targets a ship, and slows down it's rate of fire and/or missile speed (scripts).

Now you can "protect" your buddies from missiles, and with the removal of missile updates for everyone and not just the person being shot at.

Quote:
You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement. You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.


BUT WHY do they interact with AOE weapons. Whats the design here to accomplish? I'm not saying it would be easy to do, but removing them gives you a lot more options.

What makes keeping missiles effected by AOE so important that it justifies the load it causes?

I'm not saying make them work like turrets, just make them local, so if you turn off effects it won't bother you.

I mean we already got drones. and defender missiles can't hit missiles shooting people other than yourself. So if the weapon put in the game to stop missiles can't target and defend your comrades, please just tell us *me* why smartbombs do?


Max Kolonko
Caldari
Worm Nation
Ash Alliance
Posted - 2010.12.12 20:02:00 - [94]
 

Originally by: MotherMoon
Edited by: MotherMoon on 12/12/2010 19:15:17
Originally by: CCP Explorer
Originally by: Siena Petrucis
Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.


ok lets be honest, It doesn't feel like an intended feature to have smartbombs randomly take down missiles. As it stands they are better at it than defender missiles. Shouldn't smartbombs be for killing drones? Does anyone even use Smartbombs for killing missiles? I've seen in used in the alliance tournament years go, our team even used it, but it doesn't work, it doesn't stop ****.

Missiles seriously need to be removed from being effected by AOE weapons and replaced with something that doesn't kill the servers for the sake of a feature that is hit miss at best (as far it working to kill a missile in a laggy environment or not)


Maybe even simpler. Get the best of both worlds. It's a mid slot (or low?) module that targets a ship, and slows down it's rate of fire and/or missile speed (scripts).

Now you can "protect" your buddies from missiles, and with the removal of missile updates for everyone and not just the person being shot at.

Quote:
You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement. You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.


BUT WHY do they interact with AOE weapons. Whats the design here to accomplish? I'm not saying it would be easy to do, but removing them gives you a lot more options.

What makes keeping missiles effected by AOE so important that it justifies the load it causes?

I'm not saying make them work like turrets, just make them local, so if you turn off effects it won't bother you.

I mean we already got drones. and defender missiles can't hit missiles shooting people other than yourself. So if the weapon put in the game to stop missiles can't target and defend your comrades, please just tell us *me* why smartbombs do?





I agree. Frankly, I wasn't even aware that one can do that with AOE weapons like SB. If there are some very precise reason for this AOE-missile-affection-thingie to be there, please tell us, but if not: let's just get rid of it.

Princess Morenta
Minmatar
Republic Military School
Posted - 2010.12.12 21:00:00 - [95]
 

Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p

Pre torp nerf :(

Clansworth
Good Rock Materials
Posted - 2010.12.12 21:26:00 - [96]
 

For the love of god, could you PLEASE upgrade the graphics and sound instead of wasting time fighting LAGGZ?

MotherMoon
Huang Yinglong
Posted - 2010.12.12 21:40:00 - [97]
 

Originally by: Princess Morenta
Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p

Pre torp nerf :(


new? well EXCUSE me :P

I'm far from new lol. also torps can't be shot down by SM anymore.
And you still could of one with defenders instead of they were useful. :P

r they could make it so smartbombs can't shoot down missiles that aren't shooting you.

Either way I've played a lot of eve, and smartbombs are not an effective counter, so lets get a real one, and decrease lag.

Chaos Incarnate
Faceless Logistics
Posted - 2010.12.13 01:54:00 - [98]
 

firstly, excellent devblog duet. I was pretty awful at computer science in college, and it's been written with care enough so i have an idea what's going on, so props++ to you. Anyhow, couple questions:

1) Is the CPU the main bottleneck of server performance at the moment? Has it always been the main bottleneck of server performance, or is this something that has emerged recently (or perhaps gained in importance as other limits on server performance have been pushed back)?

The reason i ask is that i've seen a lot of lag over the years from various things, databases and proxy servers and stackless IOs, but apart from the drone nerf however many years ago there hasn't been much mention of CPU issues that I remember; at least not in terms of optimizations like these.

2) assuming that the CPU is the main bottleneck now...as you guys keep working on these code optimizations and the max stable fleet fight numbers increase, is there anything that you guys are concerned will emerge as a new limiting factor? or is everything else (hopefully) scalable enough that it's just a question of processing time?

oh, also, bonus question: You guys call them bubbles, we call them grids (presumably because we guess that they're cubic?). Are they actually shaped one way or another on the server side? Laughing

Debir Achen
Caldari
Posted - 2010.12.13 04:16:00 - [99]
 

Originally by: Princess Morenta
Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?

Liang Nuren
Posted - 2010.12.13 05:23:00 - [100]
 

Hey, pretty sweet dev blog there - all the new dev blogs are letting you guys get good practice. Soon, you will be holding seminars telling other game companies how to write dev blogs. Twisted Evil

Seems like a good optimization to me. I really thought you guys were going to announce a Drake nerf with this dev blog series - fantastic job on fixing the problem instead of just nerfing the Drake.

-Liang

Tric Starless
Posted - 2010.12.13 07:02:00 - [101]
 

Great Dev Blog. Thanks.

Now having profiled/optimized a lot in the distant past... and knowing how programmers under deadline like to copy/paste code, I'm almost willing to bet a dollar that you've got that same (or similar) innefficient loop scattered throughout your codebase. You're probably already looking into that though aren't you.

Regards

Jorn Dwer
Gallente
Center for Advanced Studies
Posted - 2010.12.13 07:06:00 - [102]
 

Originally by: Debir Achen
Originally by: Princess Morenta
Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?
I'll just leave this here http://www.youtube.com/watch?v=1E9uVtP7IQ4&feature=related

Mr Garo
Posted - 2010.12.13 09:24:00 - [103]
 

Thank you for an excellent dev blog. As a programmer I really appreciated the level of technical detail here. Keep these coming!

- Garo

Gretcherl
Posted - 2010.12.13 09:28:00 - [104]
 

I'm really pleased with the optimisations. It also demonstrates well the attention to detail required in the programming profession.

I trust that CCP are making use of the more `conventional' optimisations, such as eliminating unnecessary computations and input/output, using intrinsic functions rather than loops, xrange() instead of range(), loading functions in local memory, etc.

:)

SirHarryPierce
Posted - 2010.12.13 13:31:00 - [105]
 

great read! keep the graphs comming.

tehSiner
Abnormal Experience
Posted - 2010.12.13 14:29:00 - [106]
 

since Evolve() is so "light" can we perhaps expect boxes in addition to spheres?

Siena Petrucis
Caldari
Jelly Kings
BricK sQuAD.
Posted - 2010.12.13 14:32:00 - [107]
 

Originally by: Debir Achen
Originally by: Princess Morenta
Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?


Yes its possible. But there is virtually no practical use. The smartbomb ships need to be in exact line between the missile ships and the other ships they are protecting. They must keep a good formation without gaps and without much overlap (otherwise they kill each other). Its better to put these pilots into logistics or jammers - that will give you more and also more reliable damage mitigation.

Siena Petrucis
Caldari
Jelly Kings
BricK sQuAD.
Posted - 2010.12.13 14:38:00 - [108]
 

Originally by: CCP Explorer
Originally by: Siena Petrucis
Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.


Just remove that. As stated above, there is no impact on the useful gameplay.

I guess another problem is missile flight time.Remove that, too. Even with missile flight time eliminated, there are still working sufficiently different from turrets.

Lili Lu
Posted - 2010.12.13 15:46:00 - [109]
 

Originally by: Jorn Dwer
Originally by: Debir Achen
Originally by: Princess Morenta
Not our fault you new guys don't know or read up on the mechanics..

Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?
I'll just leave this here http://www.youtube.com/watch?v=1E9uVtP7IQ4&feature=related


Rolling Eyes I'm so tired of people who had nothing to do with this trotting it out over and over as if it still works. I can criticize you for this because that's me sitting in the middle of that fleet. It worked pretty damn good the first time. However, there are lots of problems with getting it to work, addressing counters and such, which I don't want to go into here because it could still work for INIT again.

But, if you were folowing the INIT killboard over the weeks since that shiney video you'd notice that Firewall fleets dropped out of favor. And what is INIT flying now? . . Drakes ugh So, please everyone who is tempted to do so stop being simpletons and posting this video as proof that Drakes don't need to be nerfed, or whatever you are posting it for. It was a fun moment in time in EVE tactical fleet development that may reemerge but for now is largely abandonned.

Nice blogs. Missiles deserve to be in the game, server CPU load or whatever. It's great you are fixing some of the problems with them. However, it still won't address the changes and other problems in game that have brought the Drake, a single BC out of 8 let alone 12 BSs, into the top spot in the serious fleet battle ship choice position. Even if you fix missiles here you will still have a Drake plague which might even get worse. So instead of Drakes figuring on 3 times the kills v the next closest competitor you will soon have 4 5 or 6x representation of Drakes over the next competitor. Other threads have discussed nerfing Drakes, which I hope you devs continued to read. Nerf the ****ing Drake as you have hinted at. You know you've got a problem with that ship as you jokingly included them in the title for these two blogs. Thanks.

Thirler
The Arrow Project
Morsus Mihi
Posted - 2010.12.13 16:36:00 - [110]
 

Very good blog about optimization. I'd love to see even more of those.

Tres Farmer
Gallente Federation Intelligence Service
Posted - 2010.12.13 17:36:00 - [111]
 

Very nice read.
If there would be a vote for the Dev Blog of the year, this one would be my favourite!
Keep em coming!!!!

totalamd5
Posted - 2010.12.13 23:05:00 - [112]
 

Edited by: totalamd5 on 13/12/2010 23:09:41
So... Why it starts to run much faster (more fps) after switching off brackets (not the effects)?

Cresalle
Posted - 2010.12.14 01:56:00 - [113]
 

Missiles are 'items' because they behave like other 'items'. They have a location and a vector and a very simple AI for target tracking. Making them 'non-item' entities would have a greater cost than benefit. For instance, you can outrun a missile, depending on range, ship speed, missile speed, etc. You can smartbomb a missile. If CCP ever decides to fix them, you can use defenders against a missile. Changing missiles to be 'non-item' entities would only be a noticable improvement in situations where the missile lag is creating a significant percentage of the lag. Even in such cases the improvement would only be a percentage of the percentage of cpu load caused by the missiles. The change being proposed would require a lot of work on the part of the devs that could be better spent on other things. At the moment I really think drones deserve a LOT more attention than missiles since a couple supercaps can quickly drop a node's performance into the toilet.

As far as this fix, I'm glad it was done. To be completely honest quadratic loops would be the very first thing I'd be looking at (with a very large microscope) if I were working on optimizing this kind of thing.

There is one thing that I cannot let stand, and that is the facetious comment about python vs cpp. Saying that you can write crappy code in cpp is no kind of justification for using a less-optimal language. I'm sure you already know this, but allow me to make a gratuitously graphic allegory.

Common sense: Eating apples is better than eating poop.
CCP: No way.
Common sense: Yes.
CCP: No, because if the apple is rotten then you can get sick.
Common sense: *old-fashioned look*

Debir Achen
Caldari
Posted - 2010.12.14 05:56:00 - [114]
 

Originally by: Cresalle
There is one thing that I cannot let stand, and that is the facetious comment about python vs cpp. Saying that you can write crappy code in cpp is no kind of justification for using a less-optimal language.
Depends on where you're taking the speed hit. If you spend lots of time context switching across syscalls or blocking on DB access, then low-level vs high-level won't make much difference. And don't forget that while an interpreted language will be slow the first time the code is executed, running the same code thousands of times quickly removes the compile hit.

It's also possible to write more complicated code better in a higher level language, since you can reach a higher level of abstraction without overflowing your cognition with mundane implementation details.

Sometimes, a higher-level compiler can do better than hand-tuning over a larger problem space. And sometimes it can't. But "use a lower-level language" should normally be a last resort for hardware-specific optimisation. As a general solution, it might actually make things worse.

Iamien
Democracy of Klingon Brothers
Posted - 2010.12.14 13:04:00 - [115]
 

Originally by: CCP Masterplan

Secondly, when it is laid out in several lines of psuedocode, it becomes very easy to see what is wrong. However those few lines represent many hundres of lines of real code, spread across multiple modules, that have evolved over time to do many things. It was quite a lot of work to understand the low-level elements, and then to see how they piece together, finally forming the full picture.
It is entirely possible that initially it was written in a different way altogether. Piece-by-piece it was changed over time, eventually arriving at where it was before I began looking into it.



Does CCP still employ any of the original developers responsible for any of the crucial low-level code? Even if they moved on after release to different projects, you would think after EVE started to see success that an effort would be made to re-acquire some of these people as they would provide invaluable insight on how stuff was put together. Every programmer has a different style and without vigorous documentation, you are in for a lot of agonizing work finding what your code does.

De'Veldrin
Minmatar
Norse'Storm Battle Group
Intrepid Crossing
Posted - 2010.12.14 14:42:00 - [116]
 

My favorite part is this:

Originally by: CCP Masterplan graph

Old :( New :)



Confirming once and for all that New > Old.

Also, loved the info on Destiny. Awesome sauce for Team Gridlock.

Connor Kent
Caldari
Icelandic Oppression Haven
Posted - 2010.12.14 19:21:00 - [117]
 

So, what happens when you add enough ships & observers to make the CPU peg at 95% again? What's the new bottleneck?

Vincent Athena
Posted - 2010.12.14 20:02:00 - [118]
 

Edited by: Vincent Athena on 14/12/2010 20:03:19
Fun read! Nice work.

I'm wondering if it would work to have every missile launcher come with a pre-allocated ball for its missile, rather than creating and destroying the ball every time the missile fires. The clients would all know that this ball would jump back to the firing ship once the missile hits or is destroyed. Maybe less cpu would be needed to handle missiles.

Longer time ticks: Two different methods have been suggested, and CCP has responded to only one (unless I missed something).

Method 1: do 1 second of simulation in 2 seconds, time slows. CCP response is we are considering it, there are issues with having 2 times scales going on at once.

Method 2: do 2 seconds of simulation in 2 seconds. Would this help in any way?

Also: you can view outstanding calls by hitting shift control alt m. Maybe this number should automatically be displayed somewhere, like above the time, whenever it is greater than zero for more than a few seconds. It would tell the players that things are getting laggy, stop pounding the keyboard and give the server a chance to catch up.

Edit for cut paste oopsie

Mzee Machado
Posted - 2010.12.14 20:45:00 - [119]
 

These blogs are great. It is a joy to see pseudocode after 20 years of no programming. Great solving skills applied.

Jim Luc
Caldari
Rule of Five
Vera Cruz Alliance
Posted - 2010.12.15 01:01:00 - [120]
 

CCP Masterplan, I'd love to learn more about how the graphics are decided upon, for the lasers & missiles - how much of each relies on the actual math and physics.

One of the reasons is because when firing a missile, it seems that all missiles connect with my ship even if it says it misses. Same goes for lasers. Shouldn't some missiles explode around the ship, and lasers overshoot the ship, noticeably missing the vessel? It would be more realistic looking.

Also, the points on the ship that the lasers & missiles hit all seem to be the same, ie not randomized. If you watch a Sansha battle, you can see that about 50 ships are all hitting the exact same point on a ship, and when I get hit by enemy missiles, it's all hitting the exact same location. How hard is it to modify the client to display misses and randomize the point on the ship that it hits? Thanks!


Pages: 1 2 3 [4] 5 6

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only