open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Missiles Hate My Hamsters
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 [2] 3 4 5

Author Topic

Axemaster
Posted - 2011.02.07 20:58:00 - [31]
 

Edited by: Axemaster on 07/02/2011 21:03:33

To be 100% honest, I am starting to wonder why CCP doesn't just commission a company to make a chip physically optimized to process these events (guns and missiles). I have a friend who works in the lab with me who designed a signal processor that can integrate pulses and find signal peaks in 50 nanoseconds per input event. It can do that because it's a custom chip.

Imagine doing that with these silly missiles. You could probably get the whole process down to less than 100ns in total, and remove weapon based lag forever.

In other words, you could probably make the computational process 1000 to 10000 times faster.

Hell, you could probably put the entire Dogma system onto a custom chip once you finalized the code reasonably well. Then the only worry would be the networking issues.

Taedrin
Gallente
Kushan Industrial
Posted - 2011.02.07 21:05:00 - [32]
 

Has CCP considered attacking the "drake problem" from both angles? IE, not only by optimizing missiles, but also by adjusting game mechanics such that missiles are no longer something that a player spams? i.e. missiles would no longer be the Caldari's primary weapon system and instead used as a tactical weapon similar to how bombs are used?

Of course, this would involve finding a replacement for the Caldari primary weapon system and would likely involve re-balancing railguns (which suck atm, partly because nobody snipes anymore)

OblivionDawn
Posted - 2011.02.07 21:10:00 - [33]
 

Here's a design related idea that probably sounds stupid, but hey.

Instead of making the missile a physics object to simulate flight time/velocity/ect, why not do something like this:

Make missiles a particle effect. When a launcher fires a missile, it slowly goes out a few meters from the launching ship, and hovers near it for a certain amount of time (to simulate flight time), and then shoot towards the target almost instantly at the end of the timer (through use of mwd, or whatever EVE lore can come up with.) The time that missile hovers near launching ship could be considered a "targeting period" or something during which the missile in question is locking on to its target with its advanced guidance system (more technobabble reserved for whatever EVE lore can come up with). Defender missiles (should they be included in this concept), could shoot toward the attacking missile while they float near the ship. Smartbombs could disrupt the targeting system of the attacking missile, making them miss or something. Max distance for the missile can be represented by a fuse timer or something, (i.e. the missiles explode in midflight if the enemy is too far away, even after the missile targets it). And for a ship to outrun a missile, this could be represented by the targetting system failing to get a lock, in which case the missiles just explode/disintegrate/whatever.

Presumably no physics ball required, unless I overlooked something.

CCP Veritas

Posted - 2011.02.07 21:10:00 - [34]
 

Originally by: Bartholomeus Crane
Method words


It was really just a trap so I could make sure that you, Bartholomeus Crane, read the entire thing before replying.

Originally by: Bartholomeus Crane
It looks though as if missiles will always use up substantially more resources. Is this about as far as you can go?


It's as far as I'm willing to go with specifically missiles at this point, so practically speaking the answer's yes. I'll be monitoring load from high load areas after these changes go in to see if it makes objective sense to continue micro-optimizing missiles specifically or if some other system sticks its neck out as the Next Big Problem.

If missile processing remains high on the load profile, we won't have too much choice but to change them to use fake physics of some kind as many other posters here are suggesting. I've been trying to avoid that switch-over 'cause I think it's neat to be able to outrun the buggers or smartbomb them; losing those abilities would make the universe seem less "real" to me. That and rebalancing an entire weapons system from the ground up is a ton of work.

Cergorach
Amarr
The Helix Foundation
Posted - 2011.02.07 21:15:00 - [35]
 

Originally by: Axemaster
To be 100% honest, I am starting to wonder why CCP doesn't just commission a company to make a chip physically optimized to process these events (guns and missiles). I have a friend who works in the lab with me who designed a signal processor that can integrate pulses and find signal peaks in 50 nanoseconds per input event. It can do that because it's a custom chip.

Imagine doing that with these silly missiles. You could probably get the whole process down to less than 100ns in total, and remove weapon based lag forever.

Besides the custom made hardware (which the hamsters generally don't like) I suspect that it is very expensive, not only that, I suspect that the whole codebase needs to change. There are programmable processors out there that would fit in a standard CPU socket, those are not cheap either but I suspect cheaper then custom chips. Another option is using Nvidia Qaudro boards with Cuda. But the enormous task of rewriting the code is huge and probably the most expensive thing CCP has done.

Good work Dev team CCP, very impressive improvements with relatively simple (sounding) solutions. I'm not very well at home in Python, you can shoot me with some missiles if I say something really stupid. But how difficult is it to move the calculations to a separate treads or cores? I understand that the that wouldn't speed up the individual missiles, but generally speaking folks don't shoot a single missile and often many missiles are in the air at one time. Would the syncing of the data be a problem that the version of Phyton you currently use doesn't provide for?

How is the server currently handling missile weapon groups? As separate missiles or a group of missiles? If single missiles, why not as a group? Same trajectory, same target, and has all the same characteristics. The only problem I could see is the use of defender missiles, maybe you could add a stat 'number of missiles', if hit by a defender you subtract the number of defender missiles in the defender group, from the other missile group. If the group hits you multiply the damage by the number of missiles. Or would this generate other unwanted behavior?

Trebor Daehdoow
Gallente
Sane Industries Inc.
Posted - 2011.02.07 21:19:00 - [36]
 

Excellent work. Indeed, in dimensio veritas!

(Apologies to my grammar school latin teacher, who is no doubt spinning in his grave)

LHA Tarawa
Posted - 2011.02.07 22:01:00 - [37]
 

I am a computer programmer, and work on a high performance, large data application.

I read these blogs and am blown away by the excellent information in them. Seriously, I think CCP could scim through these blogs and come up with the bases of an AWESOME text book on how to create high performance systems.

Do: aviod doing the same work twice. (Examples, if you already have a value here, copy it there rather than recalculating it)

Do: Allocate the smallest memory foot print possible. Alloc and dealloc are expensive, not to mention creating memory fragmentation that will later have to be cleaned up.

Do: Use link list based data srtructures if you will be inserting and deleting from the middle of the list.

Do NOT: use array based data structures if will will be inserting and deleting from the middle of the list.

Do NOT: use link list based data structures if you will just be walking the data structure.


One of the systems we wrote, we actually never dealloc objects once we create them, until the system is shut down. We have a linked list stack we push them to when we no longer need them, then pop them back off when we need another. Only when the queue is empty do we alloc more objects. WAY faster to update a couple pointers than to alloc, dealloc, or garbage collect.


LHA Tarawa
Posted - 2011.02.07 22:11:00 - [38]
 

Originally by: Cergorach
But how difficult is it to move the calculations to a separate treads or cores? I understand that the that wouldn't speed up the individual missiles, but generally speaking folks don't shoot a single missile and often many missiles are in the air at one time. Would the syncing of the data be a problem that the version of Phyton you currently use doesn't provide for?


I think the key point of this article is that missles are "objects is space" just like all the ships, wrecks, gates, bases, POSes, etc, etc. Basically, this can't be moved to its own system because EVERYTHING that is not objects in space already has been moved to other systems.

They used to treat missles like full ships. Since most ship attributes do not apply to missiles (resists, shield, armor, cap, recharge, etc, etc), they were allocating a much larger chunk of memory than needed. So, now they create a much smaller object that just has the set of attributes needed for missiles. AND instead of recalculating the max velocity, flight time, etc, for each missile fired, they just copy this info from the launcher, since it was already set there when you loaded the ammo.

And, instead of getting the list of objects as an array based data set, now they are getting it as a data structure that is based on a linked list so that it is much faster to add and remove objects form the middle of the set.

LHA Tarawa
Posted - 2011.02.07 22:15:00 - [39]
 

Originally by: Taedrin
Has CCP considered attacking the "drake problem" from both angles? IE, not only by optimizing missiles, but also by adjusting game mechanics such that missiles are no longer something that a player spams?


Then they will also need to attack drones.

While they are at it, how about some kind of mechanic that punishes large fleets... Make it so that 100 can beat 200 and 200 can beat 400. Otherwise, every battle will become 1000 vs 1000, and no matter how many improvements they make, the increasing number of ships will still overwhelm the ability of the systems to handle them.

Better to attack the slow performing code before attacking the game design. Game design is an UGLY rabbit hole to go down.

CCP Veritas

Posted - 2011.02.07 22:18:00 - [40]
 

Originally by: LHA Tarawa
One of the systems we wrote, we actually never dealloc objects once we create them, until the system is shut down. We have a linked list stack we push them to when we no longer need them, then pop them back off when we need another. Only when the queue is empty do we alloc more objects. WAY faster to update a couple pointers than to alloc, dealloc, or garbage collect.


This goes a step further in the console world (where I started in the industry). With the addition of fixed memory, alloc is bad neighbor to have. Many systems work under a strict memory budget and allocate their objects from a fixed buffer at initialization time, never to add or remove from that list. This has the same benefits as what you're describing, but with the added benefit of cache coherency if you're iterating through all of the same objects.

You just, ya know, have to deal with running out. If you've ever seen something vanish from existence while playing a console game, you likely know what the root cause was now~

Manfred Rickenbocker
Pan Galactic Gargle Blasters
Important Internet Spaceship League
Posted - 2011.02.07 22:19:00 - [41]
 

Originally by: Axemaster
Edited by: Axemaster on 07/02/2011 21:03:33
To be 100% honest, I am starting to wonder why CCP doesn't just commission a company to make a chip physically optimized to process these events (guns and missiles). I have a friend who works in the lab with me who designed a signal processor that can integrate pulses and find signal peaks in 50 nanoseconds per input event. It can do that because it's a custom chip.


Ideally, yes, perfectly, no. Your friend's chip is doing signal processing which scans a given data stream provided to it. DSP chips are very specific and typically operate on real time data. What CCP has is a database issue. As you will notice from the blog, the time lag comes from getting and setting data values for objects as well as scanning lists, both of which require fetches from database memory. Fetches from memory is incredibly more time consuming than reading from a buffer in a chip (not to mention they are different kinds of memory with different distances from the chip etc). Besides, peaks can be an incredibly easy thing to find in digital signals.

Originally by: Cergorach

Good work Dev team CCP, very impressive improvements with relatively simple (sounding) solutions. I'm not very well at home in Python, you can shoot me with some missiles if I say something really stupid. But how difficult is it to move the calculations to a separate treads or cores? I understand that the that wouldn't speed up the individual missiles, but generally speaking folks don't shoot a single missile and often many missiles are in the air at one time. Would the syncing of the data be a problem that the version of Phyton you currently use doesn't provide for?



If you do this, you can run into blocking conditions. You want to make sure that individual memory addresses aren't being read as they are written etc. Plus, the thread that they are writing is better off sequential as they have it: you want to make sure that all entities and graphics are complete before updating. Sending them through various threads wouldn't always help. Also thread overhead can be a bit of a ***** and they would probably have to recode from scratch a good portion of their code. That is conjecture of course, but having a database update take only .08ms though to begin with, that's impressive in and of itself!

Personally, I, too, would love to see updates on how the drone mechanics are being furthered. Hopefully if they can hop those up on caffeine like they did with missiles, they might be able to squeeze some decent and responsive AI on board as well. Well behaved drones = best drones.

Xailia
Unsteady Corporation
Posted - 2011.02.07 22:19:00 - [42]
 

Yay! Tech blog! \o/

If missiles were moved to a fake physics simulation, could outrunning still be possible if some outrunning probability calculation was made? And assume the target ship kept its current velocity and vector, or checking those values a few times during the missile flight.

CCP Veritas

Posted - 2011.02.07 22:20:00 - [43]
 

Originally by: LHA Tarawa
And, instead of getting the list of objects as an array based data set, now they are getting it as a data structure that is based on a linked list so that it is much faster to add and remove objects form the middle of the set.


It's a minor point but I'd be remiss to let it stand. Python "set" is a hash table, not a linked-list.

LHA Tarawa
Posted - 2011.02.07 22:28:00 - [44]
 

Originally by: CCP Veritas

It's a minor point but I'd be remiss to let it stand. Python "set" is a hash table, not a linked-list.


Oh hash table... Array of arraies. Great if each hash value has few members. If each value starts to have lots of members, then you are right back to the array based problem. But avoids the ugly alloc and dealloc of a true linked list data sructure.


Louis deGuerre
Gallente
Malevolence.
Posted - 2011.02.07 22:32:00 - [45]
 

Excellent blog and lol @ defenders Razz

I shall continue to group my missiles religiously. Wink

MailDeadDrop
Rage and Terror
Posted - 2011.02.07 22:33:00 - [46]
 

Speaking from the "there's more than one way to skin a cat" vein... You could solve, or at least greatly lessen the impact of, missile hamster killing by simply buffing guns. Just sayin'.

MDD

Debrie
Posted - 2011.02.07 22:36:00 - [47]
 

Edited by: Debrie on 07/02/2011 22:37:28
tend to agree with a few posters

had you taken this time and energy to FIX DEFENDERS and to project there protection to everyone your fleeted with

there would be less "drake blobs" spamming missles lagging the servers to death in the 1st place :p

Batolemaeus
Caldari
Free-Space-Ranger
Morsus Mihi
Posted - 2011.02.07 22:40:00 - [48]
 

Originally by: MollyPuss
great work me old fruit :)Quick question? When I'm in a fleet and all the drones are out like a nest of drunken ants, my FPS drops to a stutter. Is this a sign I need to get a more expensive computer or do we need to make drones thinner too?


Do you have brackets on?
You should disable both drone models and drone brackets. The bottleneck here is the UI code that is horribly inefficient combined with an extreme amount of elements being rendered.

Be glad you're not flying a carrier or mothership. Launching / Scooping drones into a dronebay with ~200+ drones makes your client drop a few frames for every drone being moved..

River Oemhii
Posted - 2011.02.07 22:59:00 - [49]
 

Originally by: Cergorach
Good work Dev team CCP, very impressive improvements with relatively simple (sounding) solutions. I'm not very well at home in Python, you can shoot me with some missiles if I say something really stupid. But how difficult is it to move the calculations to a separate treads or cores? I understand that the that wouldn't speed up the individual missiles, but generally speaking folks don't shoot a single missile and often many missiles are in the air at one time. Would the syncing of the data be a problem that the version of Phyton you currently use doesn't provide for?


TL;DR: For Eve to move in to the Multithreaded World would require CCP to build a new Stackless Python Virtual Machine that does not use the GIL, no simple task. There may be better uses of staff time in the short-term?

I can give some insight on Python, threading, stackless python, and while we'll be stuck with the current system for a bit.

Some background, Python, is an interpreted language, meaning its code is not 'compiled' to machine assembler/byte code but read and dynamically executed by a Virtual Machine. In this case the most common interpretor is CPython.

Python's implementation requires using something called the GIL, Global Interpreter Lock. This prevents Python code from executing multiple threads at the same time. The reason is because some of CPython's methods and systems are not 'Thread Safe' meaning that if executed across multiple cores at once you may run in to issues such as race conditions that will kill your program.

At one point the authors of CPython did attempt to remove the GIL and setup context Locks on the methods that required it, this actually led to a decrease in performance and the changes were rolled back.

Here's a good presentation on the GIL: http://python.mirocommunity.org/video/1101/mindblowing-python-gil

Eve however, doesn't strictly use Python. It uses another implementation called Stackless Python. Stackless Python adds an important feature, mainly Green Threads. These are extremely lightweight threads that the program manages, versus the OS in most other cases.

Eve uses these extensively for object handling (I believe, fuzzy here, Dev?) and transactions. These Green Threads allow 1,000's, or 10,000's of threads to run at once. Each green thread (tasklet) is given a slice of cpu time to execute then yield. Returning back to the manager to let the next thread to run.

This is a Coroutine paradigm meaning that the entire state of the function stack is stored, so the function, even in the middle can be stopped, and yield back allowing the program to come back later and pick up execution where it left off.

Ok, so what does this have to with Eve and multicore/multithreading? Well there are a large number of Python implementations that do not use the GIL, for .Net/Mono: IronPython, Boo, JavaVM: Jython, but there are no alternate implementation of Stackless Python.

Unfortunately, one of the main useful features (Green Threads) of Stackless Python also makes it rather hard to create other Virtual Machines for.

For instance, .Net only supports limited Coroutines using Enumerables and the Yield keyword. There's no way in the framework to natively store the call/function stack and local values to restore the thread later. That means that IronPython/Boo would have to create System.Thread objects which are expensive and managed by the OS. The alternative is to re-write the entire tasklet/green thread system in the new platform: C/C++/.Net/Mono/Java etc.

So, for Eve to move in to the Multithreaded World would require CCP to build a new Stackless Python Virtual Machine that does not use the GIL, no simple task.

w1ndstrike
Legio Geminatus
Gentlemen's Agreement
Posted - 2011.02.07 23:01:00 - [50]
 

Originally by: PTang
Edited by: PTang on 07/02/2011 20:30:10
Thanks for all the work guys!

but... I have one question... now that you've gotten to the repeats per second to such a level, why not...

Double missile damage.
lower rate of fire.
keep dps the same



i mean, sure it would have some impact on gameplay, making missles an alpha damage weapon, but why not? That would get them down to the same cpu cost as gun fire?

or am I thinking of this the wrong way?


the big problem with this is the same reason the rocket nerf happened in the first place. missiles are unqiue in that they are balanced not by "miss chance" but by Rof/Speed/Damage/Damageappllied

if you change any one of the 4 factors globally you have to rebalance the other 3 to compensate, which in some cases just doesn't work (as we saw with what happened to rockets when torps got rebalanced)

its a nice theoretical idea to reduce load, but the impelentation of it would be more of a nightmare than the optimization route


all that said, Good Job Veritas!

Ender Flagrante
Gallente
The Scope
Posted - 2011.02.08 00:58:00 - [51]
 

Originally by: PTang
Edited by: PTang on 07/02/2011 20:30:10
Thanks for all the work guys!

but... I have one question... now that you've gotten to the repeats per second to such a level, why not...

Double missile damage.
lower rate of fire.
keep dps the same


i mean, sure it would have some impact on gameplay, making missles an alpha damage weapon, but why not? That would get them down to the same cpu cost as gun fire?

or am I thinking of this the wrong way?


The problem with this solution is that missiles are already an alpha damage weapon. A Drake with kinetic missiles already has alpha damage and range on par with a 720mm Artillery Hurricane using Tremor, but with much higher DPS. Doubling alpha would give Drakes an alpha strike capability much greater than that of 720mm Artillery Hurricane using EMP/PP/Fusion with 3x the range.

Bartholomeus Crane
Gallente
The Crane Family
Posted - 2011.02.08 01:01:00 - [52]
 

Originally by: CCP Veritas
Originally by: Bartholomeus Crane
Method words


It was really just a trap so I could make sure that you, Bartholomeus Crane, read the entire thing before replying.

Originally by: Bartholomeus Crane
It looks though as if missiles will always use up substantially more resources. Is this about as far as you can go?



But, but, you know I read very last word of you blogs Veritas. You know I do!

Originally by: CCP Veritas
It's as far as I'm willing to go with specifically missiles at this point, so practically speaking the answer's yes. I'll be monitoring load from high load areas after these changes go in to see if it makes objective sense to continue micro-optimizing missiles specifically or if some other system sticks its neck out as the Next Big Problem.

If missile processing remains high on the load profile, we won't have too much choice but to change them to use fake physics of some kind as many other posters here are suggesting. I've been trying to avoid that switch-over 'cause I think it's neat to be able to outrun the buggers or smartbomb them; losing those abilities would make the universe seem less "real" to me. That and rebalancing an entire weapons system from the ground up is a ton of work.


Non Vincit Nisi Veritas!

Sidrat Flush
Caldari
Eve Industrial Corp
Posted - 2011.02.08 01:32:00 - [53]
 

So defenders work but not as well as smart bombs versus missiles.

Turrets are kinder on the server and have a much higher dps.

Does eve really need missiles and defenders? I don't want to discourage the work done in order to optimise missiles themselves, however it seems it would be far easier to rip missiles out of the game altogether and stick with lasers/hybrids/projectiles/drones - four races four weapon types. Although Gallente pilots that use hybrids may baulk at this suggestion there comes a time when everyone should train logistic skills and add on drone support.

Oh dear, recursion inevitable - drones are producing more lag than ever.

Maybe all I want is my missile skill points reimbursed and not have to wait a maximum of 15 seconds ish - to receive the damage report.

arambe fuddled
The Commonwealth Federation
Posted - 2011.02.08 01:43:00 - [54]
 

As a computer science student I find these blogs to be a great read. I love the detail, keep it up.

Suncats Shadow
Caldari
PuttieTat Lab Rats
Posted - 2011.02.08 01:43:00 - [55]
 

Quote:
Methodology note: For these numbers, I had 200 ships shooting on a devhaxed Drake. Rate of fire was hacked to be high enough to


Sounds great, since I am a missile type.

Would also like to know when I could get one of the 'devhaxed' Drakes for my mission running and wormhole exploring?

CoolLaughing

Lili Lu
Posted - 2011.02.08 02:04:00 - [56]
 

Smarties as the anti-missile mod is stupid. It doesn't work well and has drawbacks that no other anti-weapon system does. If you could get some point defense gun/launcher introduced to the game it would be better. And, I'm sure its rate of success against any missile in range whether targetted at your own ship or not could be set at something that wouldn't negate missiles as a viable weapon system. Hell, we can do much more now with tracking disrupters and there is no issue of turrets not being viable.

Also, "Fly safe, my Drake-obsessed friends" Really? Thing's been imbalanced since it entered the game. Just took a while for pvp people, who didn't think in terms of ridiculous passive shield regen for pve, to realize it and for some other game changes to bring it to the fore. You guys know and have stated it's not good to have one BC to rule them all. This change to missiles while welcome if it does reduce lag and other game performance problems does nothing to address the use imbalance with drakes. Still waiting for your promised drake nerf to hit the test server.

Thagomizer
Posted - 2011.02.08 02:31:00 - [57]
 

Very interesting blog, thank you CCP Veritas.

I had one question about this (emphasis mine):
Quote:
Since in-launcher missiles don't really have an identity, it comes down to querying "those missiles in that launcher", which will remain the same between launches, so we'll only do the calculations once and use them until you switch ammo types.


Would this cause any problems if those attributes change in some way while a ship is in space, but the ammo isn't switched? For example, adding/removing a ballistic control while at a POS/carrier maintenance bay? What about boosters whose penalties might impact something like explosion velocity/radius? I might be reading what you wrote a bit literally, but I was just curious.

Internet Knight
The Kobayashi Maru
Posted - 2011.02.08 02:40:00 - [58]
 

Originally by: CCP Veritas
Whenever we do a query to our database, we get the results back in a data structure derived from Python's "list" data structure, which implementation-side is an array of pointers to the elements' values. This is ideal for nearly all cases, since we get back an ordered list of rows from the database that we don't mess with at all.

Inventories are a completely different use case however. We get back that initial query from the database, but then we mess with it heavily and don't particularly care about order. Adding to a list is cheap, you just tack onto the end (and grow if need be), but search and removal are horrendous. Since list doesn't have indexing nor sorting, a search for inclusion is exhaustively linear. Removing an item from a list involves doing a linear search of the list for the item in question followed by copying the remaining entries in the array one slot up. O(n) is usually a sign you're done, but in this case it's a sign of a major problem.
Just curious... the inventory viewing system on the client side (oh look at my items in my hangar! oh look at my items in my cargo hold! etc), is that affected by the inventory system about which you're talking? Because, honestly, I feel quite awful with the thousands of different types of items in some of my hangars...

PC l0adletter
Posted - 2011.02.08 03:15:00 - [59]
 

e-hugs

keep it up

VonKolroth
Gallente
Dreddit
Test Alliance Please Ignore
Posted - 2011.02.08 04:17:00 - [60]
 

I read this Dev Blog while drunk, and lo and behold, all became clear to me. Technical blogs only alleviate what is a completely intoxicated guess of 2-4% of those of us who play eve in-person. About 23% (Also a drunken estimate) of us would be happy enough for the the banned portion if you just erased Caldari from the game.


Pages: 1 [2] 3 4 5

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only