open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Well, that was a few people
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 2 [3]

Author Topic

Vyktor Abyss
The Abyss Corporation
Posted - 2010.11.10 16:32:00 - [61]
 

Top bloggin'!

Pity all the blogs tend to get released at the same time though, this deserved more than a few hours as "the latest blog", and because you'd get a lot more suggestions and feedback if it was "latest" for a bit longer.

Cheers.

ivar R'dhak
Minmatar
Posted - 2010.11.10 16:33:00 - [62]
 

Edited by: ivar R''dhak on 10/11/2010 16:34:07
Originally by: Shanky McStabber
Edited by: Shanky McStabber on 10/11/2010 16:00:14
Originally by: Ms Freak


The idea around pre-calculating the dropped-items is a nice one - that would almost entirely remove the need for that calculation mid-fight.



The problem with this idea is the server would then have to recalculate dropped items every time you fired a missile/gun, launched a drone, fired a bubble. You would quickly lose all the gains by the sheer number of small "adjustments" that occur during the fight.

Thus my suggestion to "cheat" and mark all consumables as destroyed by default. Would even make kinda sense as the small stuff should have a very high chance to go kaput anyway.

Only time when the list would get queried on the server would be when you hit the jettison command.


Gravemind GER
Caldari
Fnord Works
The Initiative.
Posted - 2010.11.10 16:42:00 - [63]
 

@ccp

are you refering to: http://www.youtube.com/watch?v=taDBLIqPJw0 with the one ring to rule them all!

one napfest to rule them all!

TornSoul
BIG
Gentlemen's Agreement
Posted - 2010.11.10 17:25:00 - [64]
 

Originally by: Ms Freak
Originally by: TornSoul
Edited by: TornSoul on 10/11/2010 15:33:58
Originally by: Ms Freak
LXQ was unplayable

FWIW - During LXQ I was pretty much roaming in what we now jokingly call God Mode.
Barely any lag (2-6 secs) for the entire duration (until that last SBU did some wonky stuff to everyone there)

Every major battle since, it's been the reverse...

Just goes to show...



Don't get me wrong - the client was working fine, cameras, moving, wapring (to some extent anyway) but in terms of actually getting anything done (like pew pew) it was broke.

And yes, it's quite amazing how it can let 3000+ players fly around a system as long you don't shoot each other yet the second pew pew happens the server tries to hang it's self but is so stressed by the thought it actually keeps plugging away!! Very HappyVery Happy


Don't get me wrong either Razz

*Everything* worked just fine (only the slightest hint of lag)
Targeting, shooting, activating/deactivating modules - you name it.
I've never racked up that many kills in one fight ever (or since...)

As said - "God Mode" Razz

It was truly remarkable while everyone else (that I know of) *****ed about the lag - I basically had none.

(Gave rise to a few conspiracy theories about chars with low charID's being favored and what not Razz)


Abdiel Kavash
Caldari
Paladin Order
Fidelas Constans
Posted - 2010.11.10 21:19:00 - [65]
 

Originally by: TornSoul
It was truly remarkable while everyone else (that I know of) *****ed about the lag - I basically had none.

(Gave rise to a few conspiracy theories about chars with low charID's being favored and what not Razz)



Now that you mention it, I do get younger players in my corp complaining about severe lag when I am having only slight delays... Shocked

/me tinfoil hat

Hawk TT
Caldari
Bulgarian Experienced Crackers
Posted - 2010.11.10 23:34:00 - [66]
 

Hmm, I guess Destiny actually does some heavy math, especially solving gazillion differential equations per second Shocked

At least according to this GDC 2008 presentation (PPT Donwload)...

Using a GPGPU for off-loading math could be a possible future, but a distant one IMHO.
The "cost" and complexity of transfering data between the CPU and the GPGPU in a real-time simulation is just too high. Every ns counts...

Sooo, until CCP invent a workaround (or a solution) to Stackless Python's inability to schedule micro-threads to multiple OS threads (thus multiple CPU cores), the only viable brute-force hardware solution comprises of a combination of the following:
1. New CPUs with better single-thread performance (higher IPC, frequency, better & bigger caches etc.)
2. New platforms (chipset/CPU combos) with lower memory latency / higher bandwidth
3. May be, only may be, utilizing some of the advanced SIMD CPU instruction sets. The upcoming Intel AVX instruction set is the most promising in terms of speeding up wide FP intesive calculations...(normalizing 3D vectors 2-2.9x speedup, solving ODE 2x speed up, etc.).

Last, but not least, there is one big RISK Embarassed regarding Option 3 or any form of it (using SIMD like AVX or GPGPU code on the server). In order to gain performance, the math libraries used by Destiny should be optimized (manually) for AVX / GPGPU (you choose). This means different branch of code...

So, what's the problem? It's only on the EVE server side, right?

Nope, Destiny is the physics engine that runs both on the EVE server and the EVE client side. It's just that the client side works with a sub-set of the data (only with the player's causuality bubble), while the server takes care for the whole system.

Because computers are deterministic, using the exact same executable code & data on both sides (server & client) should produce (supposedly) the same results. Remember, the EVE server is authorative, the EVE Client should be in-sync and adjusts to the server. OK, so CCP controls what kind of CPUs are used for the SOL Nodes running destiny, but CCP could not control what kind of CPUs with what kind of SSE/AVX instructions available are used by the players.

Ok, so what?

This basically means, that using manually (and highly) optimized server code for destiny and using "standard" code on the client (for compatibility if AVX / GPGPU is not avaialble) could (and most probably will) produce big issues...I might be wrong, but the current hard-to-find-and-kill bugs would become just nice memories...Confused

Right now

Originally by: CCP Veritas

Originally by: Imhothar Xarodit
Does Destiny make use of SSE instructions? This might sound silly, but it can speed things up quite a bit.


You're assuming that the bulk of the Destiny load is doing heavy math. I haven't been looking deep at Destiny personally lately (CCP Masterplan has), but I don't think that's a safe assumption.

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2010.11.11 00:17:00 - [67]
 

Edited by: Tonto Auri on 11/11/2010 00:19:14
Nice blog, but honestly, why not take the controversal approach? When you see a blob aggregating and started to move, don't move the blob, move systems to it and move systems out of it.
Every single system in null is fairly low-populated, it should be safe and quick to move such low-packed system to a next node. This way, you could dynamically achieve the results you are creating by hands per request: Blob will eventually end up on a node that is running single solar system. Just because all the other systems have so far escaped the blobbed node.

EDIT: The ideal approach would be to create a separate environment for every grid, reducing "a-handful-of-thousands-of-men-in-the-basket" to bearable amount of people fighting in the actual engagement.

Blazde
4S Corporation
Morsus Mihi
Posted - 2010.11.11 00:28:00 - [68]
 

Edited by: Blazde on 11/11/2010 00:29:28
Originally by: Hawk TT
Hmm, I guess Destiny actually does some heavy math

It uses some pretty heavy math but we're not talking solid blocks of FP instructions in a tight loop, it's more like:
- iterate over this
- iterate over that
- shuffle some memory around
- make temporary copies of various vectors
- math
- check stuff
- copy something
- log something
* do the above >10million times a second with constant cache misses due to the >3200 huge datastructures involved

If Destiny is really a bottlekneck they'd be looking at algorithmic and architectural changes before squeezing minor 2x performance increases out of the math. I'm kinda surprised to hear it is because it already seems very smart in what it does, it's one of the few C++ bits of game logic and as you say it operates on all clients and the client never seems stressed by it. Clients only simulate one grid of course, though in LXQ I was on the busiest grid with more than half the players and had no noticeable performance drop. But hey, it's different scales, and not like my client was trying to run a node at the same time *shrug*

There was a 10+ page discussion on SHC a few months back where a load of us err.. 'armchair devs' speculated about the viability of making Destiny run in a seperate thread. One of the arguments being that it would be *relatively* easy to seperate it because it's not python code and it already has a well-defined interface, but it would still require some important non-transparent changes in the way game-logic worked. Of course no actual devs weighed in so we were as clueless at the end as when we started but it's interesting to see it might have been a more relevant discussion than some of us realised.

HeliosGal
Caldari
Posted - 2010.11.11 11:50:00 - [69]
 

the inner dev mind is a mystery to all

Joseph SaintJohn
Posted - 2010.11.12 03:10:00 - [70]
 

I have read all the "CCP dev blog" "we're fighting lag " "blah blah blah" until I'm sick and tired of it. All the graphs and "spin" is increasingly unable to hide the fact that the entire IT staff at CCP has reached it's competency limit. Ultimately , CCP needs to realize it's current IT staff is just not smart enough and does not have the training to make the computing infrastructure of EVE better. "slick" presntations can only mask fundamental incompetance for so long. Either CCP spends the money for "brilliant and inspired" IT staffing or EVE will fail. The superb and outstanding people are out there , Google found the ones it needed. I urge the management at CCP to stop buying into the "spin" you hear from your current IT staff. Get on the phone and call a few people at MIT or Cal Tech , and see if they think your IT staff are "the best and the brightest".

I believe it has become "intuitively obvious to the most casual observer" that the problem , going forward , is not good intentions ; but the need for a level of competence not currently present in the CCP IT staff.

I also know a post like this will never be seen by CCP management. It will be buried by incompetent staff that want to keep their jobs. Shame on you all.


Joe

Hawk TT
Caldari
Bulgarian Experienced Crackers
Posted - 2010.11.12 12:14:00 - [71]
 

Dear Sir,

Your statement shows only your own ignorance and incompetence. This.

CCP's staff breaks new grounds in creating the biggest single-shard virtual universe. There is no such thing as "training" in regards to the challanges they face. Calling "MIT boffins" could not be a "silver bullet", because there are known possible solutions in theory, but in practice, CCP has to deal with millions of lines of legacy Python code, before they could start implementing long-term remedies.

As many people asked "What CCP's Devs have been doing the last 6 months?", IMHO it would be safe to answer like this - "Refactoring old code & algorithms, going to 64-bit code & data structures, corification of services, implementation of HPC management tools etc.".

I am talking about hard, nasty and dirty behind-the-scenes jobs, having no visible effects to the player community, apart from old bugs being discovered and new bugs being created!

THIS IS THE NECESSARY EVIL IN ORDER TO LAY GROUND FOR THE STUFF THAT WILL BE VISIBLE!


I don't expect that my statements will be comprehended and/or accepted by more than 5% of the EVE Players, just because understanding the complexity of the problem requires more than just "common sense" or "IT background" - it requires real-world experience with HPC / Real-Time Systems challanges, things not quite common in the IT area...

KEEP BREAKING GROUNDS CCP! TAKE YOUR TIME, YOU ARE MAKING HISTORY!

Originally by: Joseph SaintJohn
I have read all the "CCP dev blog" "we're fighting lag " "blah blah blah" until I'm sick and tired of it. All the graphs and "spin" is increasingly unable to hide the fact that the entire IT staff at CCP has reached it's competency limit. Ultimately , CCP needs to realize it's current IT staff is just not smart enough and does not have the training to make the computing infrastructure of EVE better. "slick" presntations can only mask fundamental incompetance for so long. Either CCP spends the money for "brilliant and inspired" IT staffing or EVE will fail. The superb and outstanding people are out there , Google found the ones it needed. I urge the management at CCP to stop buying into the "spin" you hear from your current IT staff. Get on the phone and call a few people at MIT or Cal Tech , and see if they think your IT staff are "the best and the brightest".

I believe it has become "intuitively obvious to the most casual observer" that the problem , going forward , is not good intentions ; but the need for a level of competence not currently present in the CCP IT staff.

I also know a post like this will never be seen by CCP management. It will be buried by incompetent staff that want to keep their jobs. Shame on you all.


Joe

Altor Quon
Gallente
Posted - 2010.11.12 12:44:00 - [72]
 

Originally by: Hawk TT

I am talking about hard, nasty and dirty behind-the-scenes jobs, having no visible effects to the player community, apart from old bugs being discovered and new bugs being created!

THIS IS THE NECESSARY EVIL IN ORDER TO LAY GROUND FOR THE STUFF THAT WILL BE VISIBLE!


I don't expect that my statements will be comprehended and/or accepted by more than 5% of the EVE Players, just because understanding the complexity of the problem requires more than just "common sense" or "IT background" - it requires real-world experience with HPC / Real-Time Systems challanges, things not quite common in the IT area...



Heh, been there, done that :p
"It's working fine, why do you need to change it?" "Btw, can you make it a bit faster, we got performance problems" "Heh, why do you need all that budget to speed things up a bit?"
Then try to explain that all the skimping on budgets and quick & dirty solutions they pushed on the IT people the past couple of years are the cause of their problems...

Ah well, as you say, most people don't even know how IT departments work (and especially the management and clients involved, who decide about the budgets)

I think CCP is on the good way; first pay off the technical debt, prepare for later, and I guess we'll see new exciting stuff soon after :)

Hawk TT
Caldari
Bulgarian Experienced Crackers
Posted - 2010.11.12 23:50:00 - [73]
 

Edited by: Hawk TT on 12/11/2010 23:51:03
Originally by: Blazde
Edited by: Blazde on 11/11/2010 00:29:28
Originally by: Hawk TT
Hmm, I guess Destiny actually does some heavy math

It uses some pretty heavy math but we're not talking solid blocks of FP instructions in a tight loop, it's more like:
- iterate over this
- iterate over that
- shuffle some memory around
- make temporary copies of various vectors
- math
- check stuff
- copy something
- log something
* do the above >10million times a second with constant cache misses due to the >3200 huge datastructures involved

If Destiny is really a bottlekneck they'd be looking at algorithmic and architectural changes before squeezing minor 2x performance increases out of the math. I'm kinda surprised to hear it is because it already seems very smart in what it does, it's one of the few C++ bits of game logic and as you say it operates on all clients and the client never seems stressed by it. Clients only simulate one grid of course, though in LXQ I was on the busiest grid with more than half the players and had no noticeable performance drop. But hey, it's different scales, and not like my client was trying to run a node at the same time *shrug*

There was a 10+ page discussion on SHC a few months back where a load of us err.. 'armchair devs' speculated about the viability of making Destiny run in a seperate thread. One of the arguments being that it would be *relatively* easy to seperate it because it's not python code and it already has a well-defined interface, but it would still require some important non-transparent changes in the way game-logic worked. Of course no actual devs weighed in so we were as clueless at the end as when we started but it's interesting to see it might have been a more relevant discussion than some of us realised.


Hmmm, check those:
1) CCP Veritas (Penrif on Reddit) - 1
2) CCP Veritas (Penrif on Reddit) - 2
3) Intel MKL optimized for AVX

If I read CCP Veritas comments + the Intel MKL/AVC Whitepaper, I think that linking and using the Intel MKL in the C++ code of Destiny and the weapons subsystem could bring some tangible effects. 2x performance improvement is a dream, now the Devs are trying to squeeze every possible CPU cylce Very Happy

Choupette666
Posted - 2010.11.13 19:10:00 - [74]
 

And what about a REAL factional warfare ? What about security status changes, real empire factions wars, real interactivity ?
Apocrypha = ok...
Dominion = mmmh... ok after all
Tyrannis = pfff... ok. Just useless but ok
Incursion = ?! Wtf ? Just... why ? Faction ship marketing and... oh ! My toon now got boobs ! I'm a happy player Evil or Very Mad
Nobody will listen, my reply will not change nothing, money is law, wow is money, eve will copy wow. Welcome in a new Eden.
Fs all.

Debir Achen
Caldari
Posted - 2010.12.07 02:35:00 - [75]
 

As someone who has never particularly experienced Eve lag due to over-loaded servers, can I ask a little more about how it manifests?

There are basically two manifestations of under-performance:

(1) Everything runs slower. In this manifestation, all events remain in sync with each other, but trail behind real-time. It's like playing the game in slow-motion.

(2) Synchronisation fails. In this manifestation, the user's view of the world and the "canonical" view maintained by the server get out of sync - what the user sees happen is not the same as what the server sees happen.

In general, #2 is a lot worse experience than #1. When the lag is client-end (client's machine, network links), #2 is unavoidable. But when the lag is server-end, it's much better to slow everything down than to send the client infrequent snapshots or to react "late" to client events.

Example: My worst lag experience to date was in the single player game Wing Commander 3. Occasionally, the graphics processing would have trouble keeping up and the game would freeze a moment. This was tolerable. What was painful is that the underlying engine would keep its clock running while the interface was frozen, and when the interface caught up it would jump ahead in time, often denying me the opportunity to fire or dodge. The "correct" behaviour is to pause or slow everything, thus keeping the interface and the game model in sync. (Yes, I know this isn't as easy in multi-player networked games, but the principle is still sound).

Back to Eve - how does server lag manifest - is it just "slow-mo", or do things start jumping around and acting unpredictably as the server processes events out-of-apparent-order? To what extent is it possible to "slow the game down" to provide more time for the server to process and communicate events?

Aineko Macx
Posted - 2011.02.13 10:40:00 - [76]
 

Quote:
The EVE server does have one system that keeps up with real time - the physics simulation, Destiny. It will do everything it can to keep the game world moving at the same time as the clock on the wall. Every other system in the game shares the rest of the available time to do their work.

Re-reading the blog, I stumbled over this statement. I can see reasons for running Destiny with real-time requirements, but it seems this creates a lot of other problems. I'd be interested in knowing why it wasn't decided to drop the priority down to the same level as the other subsystems, which would make the game run at a slower speed in load situations, but with less problems for the user experience. Ofc the timescale of the slaved simulation on the client would have to be kept in sync with the skewed timescale on the server, but that shouldn't be too hard to implement...

Aineko Macx
Posted - 2011.02.13 11:16:00 - [77]
 

Edited by: Aineko Macx on 13/02/2011 11:17:16
Originally by: Debir Achen
As someone who has never particularly experienced Eve lag due to over-loaded servers, can I ask a little more about how it manifests?
...
(2) Synchronisation fails.

That became much rarer, after CPP fixed a bug which could make the physics simulation on the client reach a different state than on the server. In that case you would stop seeing simulation balls on your grid (ships and stuff), just static things like stations and gates. You could still move, warp around and get killed, but not initiate interactions (since you also stopped having anything on your overview).

Quote:
(1) Everything runs slower.

Yes, but the slowdown is not equal among all systems (see my previous post).
The main effects are basically that all user input takes longer to result in effects and it also seems your client receives updates less frequently. Ship commands, changing ammo, locking, broadcasts, dieing, jumping, grid updates, even chat posts, they all take longer.
Then there are side-effects like modules/guns getting stuck, killmails being generated with wrong info or not generated at all, bombs not exploding after the nominal flight time/distance, lock requests being ignored, player ships not despawning and getting killed 45 minutes after logging off or jumping out, and ofc the dreaded grid load issue, where the pilots jumping/logging into a loaded system would never load the new grid while they would be visible and killable for the pilots already in system. I'm definitely forgetting some other effects...
CCP has made progress on most of those issues.


Pages: 1 2 [3]

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only