open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Fostering meaningful human interaction, through testing
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 2 [3] 4 5 6

Author Topic

wr3cks
Reliables Inc
BricK sQuAD.
Posted - 2010.08.16 22:56:00 - [61]
 

Good stuff.

I was at the last test, though, and it seemed very well attended. I think there were at least 350 to 500 in local, no? Isn't that within your stated goal? Are you trying to get it up to 1k+?

Also, as I suggested in the feedback thread, please post info about the test (the moveme channel, that the market in syndicate is seeded with 100 isk stuff, etc) in the Help default channel's MOTD. I didn't know about any of that stuff and was autopiloting 38 jumps to jita to go shopping until I caught an alliancemate, but there were 20ish odd guys who hopped into the Help channel looking for the same info.

Ben Derindar
Dirty Deeds Corp.
Posted - 2010.08.16 23:02:00 - [62]
 

Good blog, and good follow-up presence in this thread from CCP. One question though:

Originally by: CCP Tanis
The "herding mentality" and how that affects EVE is something I've put a quite bit of thought into over the years. Though I cannot say with any authority what the game designers are thinking on that front currently, as I'm in QA. I can say that it will never be solved by a single thing, it's simply too ingrained of a part of how humans work in large numbers.

Does game design not agree that the ease with which players can currently form up from anywhere across the entire galaxy at a day's notice through a combination of travel mechanics, is a major catalyst for this?

/Ben

Deliceous
Lone Star Exploration
Lone Star Partners
Posted - 2010.08.16 23:08:00 - [63]
 

Nice to see the inclusion of the CSM

I guess I will have to bribe my corp members to participate in the next Mass test

raukosen
Posted - 2010.08.16 23:59:00 - [64]
 

On the subject of lag and modules not cycling I want to mention something:

Everyone who's been in a very laggy fight knows that the best way to keep your guns from getting stuck is deactivating them by clicking the icon next to the target and not F1, F2 etc. (or the equivalent buttons in the UI). If your guns / reps are still cycling when the target dies then you can basically forget about getting them to stop without warping out. For some reason spamming F1, F2 etc. doesn't seem to actively deactivate modules the same way spamming the icon next to the target does. If that was rectified it would be a huge help in laggy fights

CCP Habakuk

Posted - 2010.08.17 00:39:00 - [65]
 

Originally by: wr3cks
Good stuff.

I was at the last test, though, and it seemed very well attended. I think there were at least 350 to 500 in local, no? Isn't that within your stated goal? Are you trying to get it up to 1k+?

Also, as I suggested in the feedback thread, please post info about the test (the moveme channel, that the market in syndicate is seeded with 100 isk stuff, etc) in the Help default channel's MOTD. I didn't know about any of that stuff and was autopiloting 38 jumps to jita to go shopping until I caught an alliancemate, but there were 20ish odd guys who hopped into the Help channel looking for the same info.


In the last test we had about 550 characters participating (the numbers dropped a bit at the end, as the test was quite long). This was awesome, but it would be great if we could keep this number for the next tests or even increase it a bit (with the improved hardware we might see that we have less lag with the same amount of participants).

Regarding Help channel MOTD on Singularity: I changed it a few days ago to give hints to the other channels on Singularity and I'll add a note to our checklist to change the MOTD after a new mirror. Thanks for the hint!

Dodgy Past
Amarr
Digital Fury Corporation
Posted - 2010.08.17 00:42:00 - [66]
 

Edited by: Dodgy Past on 17/08/2010 00:55:58
This shows some real understanding of the problems at hand and demonstrates the plans to address many of the issues that people have been screaming and shouting about.

I'd like to congratulate those who've been working on this for showing that they can see things from both sides of the fence and the effort they're putting into sustaining and growing the mass testing program.

Simpler SISI setup will be so wonderful \o/

Jim Luc
Caldari
Rule of Five
Vera Cruz Alliance
Posted - 2010.08.17 01:25:00 - [67]
 

I really don't understand the backend server technology, but latency requests makes sense to me. If it takes .5ms to receive something, then whatever time the logic takes, then another .5ms to send back info to the client, it seems pretty limiting and this will just get worse as we get more people playing Eve.

How can Eve be scaled for future expansions? I'm sure this may show my ignorance, but why aren't all systems given a separate physical server? Is the issue cost? You don't fire on people in other systems, so it's only an issue when going from system to system, but the latency for 1500 people all jumping into a system I would think is small if there's only one command processed (jump from point server a to server b).

How much would 3000 blades cost? ugh


Janitor I
Posted - 2010.08.17 01:35:00 - [68]
 

In summary .. more blah blah blah and no solution .. ugh

Obsidian Hawk
RONA Corporation
RONA Directorate
Posted - 2010.08.17 02:04:00 - [69]
 

Wow lots of angry troll alts in here but question:


CCP TANIS and his team.


For thursdays test: could you simulate a small fleet of 50 ships hotdropping in mind combat using a titan bridge? I know this has been done before, but with an expected 500 people Im sure it would yield more results.

OR

Have fighting take place outside a pos tower with full active modules using the same bridging as above. It might cause some cool errors.

Caladain Barton
Navy of Xoc
The Remnant Legion
Posted - 2010.08.17 03:44:00 - [70]
 

Edited by: Caladain Barton on 17/08/2010 03:44:51
Have you guys tried working with Coalition leaders in 0.0 to get a mass turnout?

We can pull 1000+ with experienced FC's.

Or you can toy around with half that...Good effort thus far though.

Herschel Yamamoto
Agent-Orange
Nabaal Syndicate
Posted - 2010.08.17 03:50:00 - [71]
 

Originally by: Caladain Barton
Edited by: Caladain Barton on 17/08/2010 03:44:51
Have you guys tried working with Coalition leaders in 0.0 to get a mass turnout?

We can pull 1000+ with experienced FC's.

Or you can toy around with half that...Good effort thus far though.


They've repeatedly asked for large groups to participate, in forums that alliance leaders are known to read. They haven't pestered them directly, but they are not ignorant of the possibilities.

Caladain Barton
Navy of Xoc
The Remnant Legion
Posted - 2010.08.17 04:45:00 - [72]
 

Quote:
They've repeatedly asked for large groups to participate, in forums that alliance leaders are known to read. They haven't pestered them directly, but they are not ignorant of the possibilities.


No argument..but at the same time, no dialog :-) I was just curious if they had opened up an evemail/im/vent convo and contacted some of the major fleets.

Hakaru Ishiwara
Minmatar
Republic Military School
Posted - 2010.08.17 04:56:00 - [73]
 

Originally by: CCP Warlock
Originally by: something somethingdark
XBOX HUGE wall of text
Actual content thats new and previously unknown : 4% (much like recent eve expansions)

anyways
here is an idea for a mass test
Load up an old Apocrypha client/server on sissi and then go compare logs


Unfortunately if logs could solve this problem it would be fixed. My eyes bleed from looking at logs so much the last few months.
Putting aside the 'how' of measuring stuff and capturing data(logs, etc.), may I suggest that the point lost in this dialogue is as follows:

The version of Apocrypha in-place prior to the Dominion expansion performed better in some high-load situations as compared to Dominion with similar scenarios.

For the past eight and a half months countless forum posters have inquired into the difference between the two code bases (a herculean task, perhaps) and why hasn't that angle been explored in the midst of these 'investigations.'

From my perspective, CCP has been working with blinders on by measuring, prodding and hoping to discover some magical data set from an inherently broken Dominion and now Tyrannis code base.

So my question is this: what aspects of the game environment and code base changed between Apocrypha and Dominion to create such a dramatic drop in true and perceived performance?

Sure, I have a pretty good idea on some of the answers from a client perspective (fleet finder window updates, overview updates, etc.), but CCP has yet to address these changes in their entirety.

Enough with the tales of mystery and investigations. The execution and testing of Dominion blackened CCP's eyes (both of them) and has created quite the game experience and marketing conundrum. Massive fleet fights -- once a highly used marketing tool enjoyed by thousands of existing customers and used to bring in new customers -- can no longer be truthfully advertised as a working part of the game.

Also, /signed on the 'CSM is only one means of communicating with your customers' side conversation in this thread. The CSM has its place, but that is not to be the exclusive conduit for information and dialog between customer and company.

Genya Arikaido
Posted - 2010.08.17 05:21:00 - [74]
 

Originally by: Obsidian Hawk
Wow lots of angry troll alts in here...


Where? 1 or 2 in 3 pages of posts? That's below par for any topic with CCP participation...

I'm actually enjoying the candor from Warlock and the others.

Warlock: I believe that breaking that '1 node: 1 star system' limit can be done effectively and efficiently. I just wish I knew more about the particulars rather than the theoretics. I suppose I'd have to work at CCP to find out that level of detail though. *sigh* Too bad your HR dept sucked ass a year or so ago and took 8 months to get to my app. Missed all the programmer vacancies, and I CBA to reapply if that's what is likely to happen again. Point out HR guys at the coming Fanfest for me so I can emorage. Laughing

Darth Vapour
Posted - 2010.08.17 05:42:00 - [75]
 

Originally by: CCP Tanis
We've just started allowing bombs during mass-tests (as of the last test on Aug 5), for exactly this reason. It adds complications with people being incidentally podded, but clones work around that well enough.


If only there was a type of bomb which does very little damage.

Ban Doga
Posted - 2010.08.17 06:18:00 - [76]
 

Edited by: Ban Doga on 17/08/2010 06:17:43
Have you considered running one of those tests with an old Apocrypha build to be able to compare them directly?

When you say the new module handling improves the behaviour so that "modules appeared to by cycling much more often than before." when will this be deployed on TQ?

As a general impression: not much new, especially not in the improvements section.

Rhok Relztem
Caldari
CGMA Synergist Syndicate
Posted - 2010.08.17 06:19:00 - [77]
 

Edited by: Rhok Relztem on 17/08/2010 06:32:40
Fantastic Dev Blog! While I won't pretend to fully understand everything in there, I did get the gist of what is transpiring. Kudos to you CCP Tanis for the level of detail in that blog and for getting those lines of communication opened again.

As for getting more players onto SISI and into the mass tests...

Making it easier to actually get onto SISI is a huge step in the right direction. I actually started to get SISI installed a few times with the thought of attending the mass tests but the procedure was so convoluted and time consuming that I just gave up, and I consider myself fairly knowledgeable computerwise and quite skilled at installing difficult programs and software.

Once you actually make it easier to get onto SISI, enticing people to attend the mass testing is another matter entirely. I noticed a few suggestions about people in highsec who have no fleet experience and about 0.0 players who have their territory to worry about as well as players in general who don't feel it's worth their time and hence the idea of rewards of some sort. Sooo, I have a few suggestions (surprize! Razz)...

  • For the highsec crowd (and lowsec/pirates, or anyone for that matter, who is unfamiliar with large fleet action) - A tutorial - Fleet Pilot - along with the Career Path tutorials that takes the pilot through a series of fleet maneuvers in a mission style path. Upon completion of the tutorial path, the pilot is rewarded with a certificate of completion (perhaps a new certificate tree).

  • For the 0.0 corp/alliance crowd - I'm basicly just seconding the suggestion made to lock down POVs for one hour before and one hour after a mass test thus allowing significant numbers of fleet players to attend without the fear of attack while testing on SISI.

  • Rewards for attending mass tests:
    1. 1mil SP for use on SISI per mass test attended with a bonus of 5mil SP for every three mass tests attended.

    2. A Mass-Tester badge on Tranquility for each test attended. For each five badges collected, an upgraded badge is given as well as a certificate in the new certificate tree (see highsec tutorial certificate above), and the five singles are traded in for a total of 500,000 SP for use on the Tranquility server.

    3. After each mass test, all attendees are added to a drawing, with one player receiving 2 PLEX and five players receiving 1 PLEX.

  • For everyone (and especially the FW crowd) - I like the idea of the Blue-vs-Red teams too. Perhaps ranks of some sort could be earned and rewarded, but it's getting late and I'm tired so I'll let someone else expand on that.

Luke S
Zeta Corp.
Posted - 2010.08.17 06:54:00 - [78]
 

GOOD LORD! that's a lot to read. Yes, I read It all. CCP Thanks for all your hard work.
Here is one for you.
When it comes to testing the Client. How are the tests when it comes to graphically? With the new Video card hardware, Some people like to test out their rig and see how good it really is. Has your Dev team or teams made a Graphic benchmark that is eve related?

Having a Graphic benchmark can allow most (if not all) of the player base to test their clients on their own free time. then upload the results to CCP's Cloud (server). This also could find bottlenecks with in the code on the client side.

Then again. this just could be more work for you :P

Sered Woollahra
No Fixed Abode
LEGIO ASTARTES ARCANUM
Posted - 2010.08.17 07:34:00 - [79]
 

Quote:
Highsec residents, while they could easily participate, may be hesitant to join large fleets, even on the test server, without having some experience in fleets.


To be honest, some of us are using the mass tests to *get* some fleet experience Very Happy

ReddSky
Posted - 2010.08.17 11:16:00 - [80]
 

CCP Tanis, good blog, nice high level info, cool graphs, keep up this kind of communication plz. Very Happy

Yeay Fritg
Caldari
Confrerie de Kaedri
Cluster Of Rebirth
Posted - 2010.08.17 11:37:00 - [81]
 

Edited by: Yeay Fritg on 17/08/2010 12:38:56
Edited by: Yeay Fritg on 17/08/2010 11:38:02
Ok, you fight Lag since a momemt and it will give us results in TQ.

But may we have a Dev Blog about Bug Hunting ?

Even if the game can work a normal speed how are your progress on Dominon Bug e.g. ?

When a Lag Free & Bug Free game ?

CCP, truly engage you on the resolution of the Dev Bug BackLog and I will bring you my guys on the Test Event during the week ! Trust is all want we should have. But sorry you work for me the opposite way.

Or we can work your way : please comes come on help us to solve the bug ! We nned you ! We have identified them we just now need to find where are the developers ! If you don't come we will not be able too. then we will saw you statistics ! And wait you to be happy !

Sorry, but on my side I will now only judge what I see in the gate, not what I read on the forum from CCP thread ! Even if I'm sure your are working on, the problem is still there since...Dominion.

Cheers,
Yeay

Vincent Gaines
Macabre Votum
Morsus Mihi
Posted - 2010.08.17 12:09:00 - [82]
 

Originally by: Caladain Barton
Edited by: Caladain Barton on 17/08/2010 03:44:51
Have you guys tried working with Coalition leaders in 0.0 to get a mass turnout?

We can pull 1000+ with experienced FC's.

Or you can toy around with half that...Good effort thus far though.


I can't speak for other alliances, but our illustrious leader in MM has encouraged us to be in the tests.

Many involved in wars (like the NC was during earlier tests) have had those times used by the enemy to take hold of systems knowing that many players are on SISI.

MissyDark
Posted - 2010.08.17 12:22:00 - [83]
 

Guys, do you remember you are not fixing the servers or lag, but player experience associated with the lag perception? That is an essential difference.

Keeping that in mind, I'd like to suggest couple of things (this coming from professional programmer).

1. To look at player experience I believe you need to observe what happens on the client in correlation to what information reaches the client and what is being sent.

To do that you could release a plugin/debug version of the test server client with a permanent additional windows or set of windows, showing more or less detailed information sent and received by the client. Sort of like a packet dump. Other windows with the state of what's going on in key memory objects and variables, in real time. Then have a player fraps it and send it to you. This way a dev can get his/her hands on almost real experience with debug variables on screen.

Believe me, that leads to a lot of "oh crap, it's so obvious!" moments.

Now to rewards:
"14 day plex for everyone participating in the sunday test" and you have yourself a lot of random candidates.

To get organized force, approach entire alliances, pvp alliances and offer rewards to the alliance members (not alliance). Everyone getting a special ship? Like one of those broken, never used ones, because they miss something. Add that something to special issue ship, offer it and there you have organized groups of players ready to participate.

CCP Warlock

Posted - 2010.08.17 12:34:00 - [84]
 

Originally by: Bartholomeus Crane
Edited by: Bartholomeus Crane on 16/08/2010 21:36:08
Originally by: CCP Warlock
Originally by: Genya Arikaido
Edited by: Genya Arikaido on 16/08/2010 20:10:24
Right, and since you gave that nasty little variable a name, what's being done to reduce the data center delay to the point that running a single star system over multiple nodes becomes possible? What is that delay at now (on avg)? What would be needed to make it feasible for multi-node star systems to where it doesn't interrupt the player experience?

If this preempts your devblog for tomorrow, just say so and I'll wait for the blog. Laughing


In the limit, this is always going to be a constraint, for reasons that go back to Shannon's Law. With current technology that limit is the speed of light. (Negotiating a change there might be a little iffy, given the fallout effects on other physical systems.)

However, let's say hypothetically that quantum communication worked and communication between two nodes really was instantaneous. In that case, the problem would be moved to the computational time it took to process messages and respond, and all the associated queuing and consequent buffer handling. There would still be a limit on the total amount of communication the cluster could support, and that is what ultimately limits cluster scaling.


Very well, but stepping back from the world of quantum communication and quickly scurrying back to the world where the Shannon-Hartley theorem still applies: I do hope you realise that running the bottleneck processes pseudo-sequentially on a single core will ultimately - or rather eventually - throw up quite a hard threshold, beyond which you will have to dip your toe into the wild waters that are distributed computing proper.



Oh absolutely. In the long run we have to be able to scale down (running multi-core single SOL systems), as well as up - running more and more machines in the cluster. Especially when we look ahead at the larger (in terms of processor density) multi-core systems that will be coming out over the next few years.

Originally by: Bartholomeus Crane

Even pseudo-dynamically conjuring away all other non-related processes will eventually still only leave you with a single core to run your process on. The limits of which can only be stretched so far with more efficient, or in this case, actually working code. Moore's law is all well and good but the vagrancies of real life (in this case CPU power dissipation) mean that the world has gone multi-core and speed-up through instruction-level parallelism or clock-rate increases is basically an end. Eventually you'll have to, as it were, cut the cake, and multi-thread/process over multiple cores with all that that entails.

I won't deny that a balance will have to be struck between communication and computational bottlenecks but there are many solutions available for this problem. Solutions that are theoretically and practically proven and that do not necessarily rely on expensive or esoteric hardware. Has CCP spend any time thinking or working towards such a proper distributed architecture in the long run? Or are you still fully engaged in plugging the holes of the kiddy pool you're currently swimming in? As it were ...


I'm not sure quite what you mean by the "wild waters of distributed computing proper"? Eve runs as a large set of inter-communicating processes across over 200 nodes, and ~50,000 clients. That's real distributed computing, and some pretty wild waters even by today's standards.

If you have particular models or papers in mind for your view of what a proper distributed architecture would be, it might be easier to discuss what you mean there.

CCP Warlock

Posted - 2010.08.17 12:42:00 - [85]
 

Originally by: Jim Luc
I really don't understand the backend server technology, but latency requests makes sense to me. If it takes .5ms to receive something, then whatever time the logic takes, then another .5ms to send back info to the client, it seems pretty limiting and this will just get worse as we get more people playing Eve.

How can Eve be scaled for future expansions? I'm sure this may show my ignorance, but why aren't all systems given a separate physical server? Is the issue cost? You don't fire on people in other systems, so it's only an issue when going from system to system, but the latency for 1500 people all jumping into a system I would think is small if there's only one command processed (jump from point server a to server b).

How much would 3000 blades cost? ugh




We don't give all systems a separate physical server because they don't need one. Were the player base to expand to the kind of numbers when they did, then the existing architecture could be scaled to that number with some modifications to internal routing. For a lot of players simultaneously jumping into a single system, the problem is that that is essentially a set of requests for each player, so the total number of messages that the server has to deal with is a multiple of the number of players. So induced computational load (the amount of processing on the server that has to be done in response to a message's arrival), and the associated queuing is one of the issues there.

Yeay Fritg
Caldari
Confrerie de Kaedri
Cluster Of Rebirth
Posted - 2010.08.17 12:44:00 - [86]
 

Edited by: Yeay Fritg on 17/08/2010 12:44:34
Originally by: CCP Warlock

I'm not sure quite what you mean by the "wild waters of distributed computing proper"? Eve runs as a large set of inter-communicating processes across over 200 nodes, and ~50,000 clients. That's real distributed computing, and some pretty wild waters even by today's standards.

If you have particular models or papers in mind for your view of what a proper distributed architecture would be, it might be easier to discuss what you mean there.


Sorry it's sounds technically interesting but well do you think we are here to discuss about your architecture choices ?

If yes I feel we are on a very bad way if Architecture is discussed on forums...

Help me..where is a thread about : CCP solve our issue this one is a pure Hardware one ?

MissyDark
Posted - 2010.08.17 12:50:00 - [87]
 

Originally by: CCP Warlock
For a lot of players simultaneously jumping into a single system, the problem is that that is essentially a set of requests for each player, so the total number of messages that the server has to deal with is a multiple of the number of players. So induced computational load (the amount of processing on the server that has to be done in response to a message's arrival), and the associated queuing is one of the issues there.


Have you thought of changing server architecture so it would allow for dynamic allocation of resources across multiple physical machines? For example, have 10 blades idle and re-route to them any superfluous load from other servers? I realize how huge that change would be, but it would probably solve the hardware-related lags of any kind.

lylaal
Caldari
Posted - 2010.08.17 13:13:00 - [88]
 

its good to see you lot are working on a decent fix for a workable test server client.

lets hope it wont take too long for it to released and that would already save alot of problems with people being able to attend the tests or just testing themselves on sisi

Daedalus II
Helios Research
Posted - 2010.08.17 13:42:00 - [89]
 

Question:

I have never been in any large fleet fights so I base this question from what I've heard. But from what I've heard there was much less lag in Apochrypha and then it sort of spiked in the next expansion, from one update to the next. This to me sounds like one or very few large bugs came into play here.

However from what I read you mostly try to fix many minor problems. Not that I don't think that isn't worth fixing, all bugs should be eliminated, but isn't it likely that module cycle lag and such existed already in Apochrypha? or was this somehow introduced in the following expansion? Because if it did already exist in Apochrypha, then fixing these problems will still not fix whatever got messed up later and will only marginally improve the lag.

What I'm trying to say is that I find it hard to believe that every one of these small problems you are looking at that you believe build up to this huge lag problem somehow must have been inserted in the expansion following Apochrypha. I find this very strange. How can so many small things go wrong in a single expansion? I feel it more probable that there is one or a very small number of huge bugs instead, somewhere. Some stupid thing where all accounts are logged in twice or all ticks are calculated three times more than needed or whatever.

CCP Warlock

Posted - 2010.08.17 14:05:00 - [90]
 

Originally by: MissyDark
Originally by: CCP Warlock
For a lot of players simultaneously jumping into a single system, the problem is that that is essentially a set of requests for each player, so the total number of messages that the server has to deal with is a multiple of the number of players. So induced computational load (the amount of processing on the server that has to be done in response to a message's arrival), and the associated queuing is one of the issues there.


Have you thought of changing server architecture so it would allow for dynamic allocation of resources across multiple physical machines? For example, have 10 blades idle and re-route to them any superfluous load from other servers? I realize how huge that change would be, but it would probably solve the hardware-related lags of any kind.


Being able to do this automatically, and much more transparently to the players, is one of the things we are actively looking at.


Pages: 1 2 [3] 4 5 6

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only