open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Introducing steps to improve network performance of EVE
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 [2] 3

Author Topic

Levitikon
Constructive Influence
Northern Associates.
Posted - 2011.03.23 15:38:00 - [31]
 

Originally by: Helothane
Edited by: Helothane on 23/03/2011 13:44:48
I'd have to look up the product literature, but it sounds like FCP dynamically decides which networks to announce via what interfaces. As long as the interval that it can choose to change announcements is large enough, I think it sounds like a good solution. BGP does not do quick, dynamic changes well.

NB: The exception to the rule of "Tier 1 ISPs don't pay for connections, others pay them" is when they are connecting to other tier 1 ISPs. The assumption there is that the traffic from another tier 1 will be more or less balanced with the traffic coming from the other side of the connection that tier 1 ISPs don't charge each other. The problem comes in when a tier 1 ISP decides that another company is no longer a tier 1 ISP and wants to charge them...

Quote:
I assume it's not possible for the end-user (or the game client) to force a specific route?


No, you cannot. Routing between ASs is controlled by the AS path, and as an end user, you have no control of that. There are only 65000 unique AS numbers for the entire Internet (65000 - 65535 are like the 192.168.x.y IP addresses, they are not supposed to be publically routable), so for CCP to have its own is quite something.

(Former network engineer for a US Tier 1 ISP, back when there were the 'Big 7')


You're operating on information that is quite oooooooooooold.

AS numbering have been upgraded to 32bit 6 years ago. Now there are over 4.2B ASN available for taking.

Valeroth Kyarmentari
Posted - 2011.03.23 15:55:00 - [32]
 

Originally by: Levitikon

AS numbering have been upgraded to 32bit 6 years ago. Now there are over 4.2B ASN available for taking.


Where are you getting this information? I've been doing Network Engineering for about 13 years now... and just got my CCIP about 18 months ago and hard time finding any reference of that.

Okay... Check that, I see that in Jan 2007 the IANA allows for the possibility of 32 bit AS numbers, but as of IOS 15, Cisco has yet to implement it (and since a good portion of the worlds network runs on Cisco I'd wager few other company have also). Okay wait... a little more research, and I see Cisco did work it into IOS starting around Jan 2009. It's available, but not readily apparant unless you need it. I've worked mainly for old large companies though which must be why I've never encountered this.

Regardless, the rest of Helothane's statement seems accurate to me.

I'm quite curious how FCP does what it does, and it may be something I'll do some reading on.

I'm a bit curious now, though how many network engineers spread accross the world we have playing eve. Might be cool to have an informal group of EVE Network Engineers.

BearUkraine
Gallente
Light Style
Posted - 2011.03.23 16:32:00 - [33]
 

Originally by: Estimated Prophet
Edited by: Estimated Prophet on 23/03/2011 12:56:42
Edited by: Estimated Prophet on 23/03/2011 12:55:40
Originally by: Joe SMASH
I DEMAND A GRAPH! And make it pretty! Cool


Here's my before, from Australia: http://i.imgur.com/vEeyH.png

Awesome) Laughing
Hope, CCP will fix it.

ihcn
Posted - 2011.03.23 17:11:00 - [34]
 

Isn't ipv6 supposed to alleviate this issue? Referring to your example, if a specific subnet is assigned to the UK, another assigned to USA and another assigned to germany, routers can make more intelligent decisions about where to send packets based on the destination.

Valeroth Kyarmentari
Posted - 2011.03.23 17:34:00 - [35]
 

Edited by: Valeroth Kyarmentari on 23/03/2011 17:36:06
Edited by: Valeroth Kyarmentari on 23/03/2011 17:34:57
Originally by: ihcn
Isn't ipv6 supposed to alleviate this issue? Referring to your example, if a specific subnet is assigned to the UK, another assigned to USA and another assigned to germany, routers can make more intelligent decisions about where to send packets based on the destination.


IPV6 does provide some optimiaztions not found in IPV4 (which might help you locally), but those generally are not going to affect BGP. BGP will still gernerally take the route with the shortest AS_PATH, the fewest number of Autonomous Systems between you and the destination. It doesn't really look that what's going on in those AS's to see if a better path would be preferred. My guess (just a guess) is that FCP analayzes all of the paths, and then inserts commands (probably route-maps) into the routers to affecting outgoing and to a lesser extent incoming traffic. There's a number of ways it could do this, but my brain is a bit rusty on it since I've not had to modify any in a few years.

Hexxx
Minmatar
Posted - 2011.03.23 17:50:00 - [36]
 

Originally by: Lord's Prophet
This devblog is just another step towards the EVE cluster becoming self-aware and killing us all.


QFT

AntISGhey
Posted - 2011.03.23 18:48:00 - [37]
 

"The FCP has now been in monitor mode for few weeks and we can see clearly that he can improve greatly on the default BGP routing policy - So we have decided to put him live very soon."

Can the FCP ever be a hot chick?

Genocide Machine
Posted - 2011.03.23 19:01:00 - [38]
 

i barely understood a word of it, but Hell yeah!

J Kunjeh
Gallente
Posted - 2011.03.23 19:12:00 - [39]
 

Great dev blog! I too am a big fan of the technically oriented dev blogs I've been seeing of late. Keep em' coming CCP!

Dr Sheepbringer
Gallente
Halinallen veroparatiisi
Inglorious Carebears
Posted - 2011.03.23 19:54:00 - [40]
 

The brainhamster is starting to sound like Skynet!! Shocked

Justasii's JitaBitch
Posted - 2011.03.23 20:21:00 - [41]
 

"The FCP is not a total fix of an imperfect system, we will still be affected on routing decisions made by network operators far from our reaches, we do our best to help with those cases. So please don't stop letting us know of issues with your connection, we look at them case by case. Hopefully the FCP will help out."

The system may be imperfect, but "perfect" in a complex system will be hard to measure. In some ways, it's like golf: You shoot for a lower score but you can never win the game. It will be interesting to see if this system improves everyone or only improves users stuck behind the "black holes" and other anomalies this system is supposed to detect. My off the hip opinion is that most users will not be affected by this. Latency is not a solvable problem in many areas and CCP controls, what 4, out of hundreds and thousands of variables? I guess I'm having a hard time grasping how the system will holistically improve the server performance.

Helothane
Posted - 2011.03.23 20:38:00 - [42]
 

Edited by: Helothane on 23/03/2011 20:45:20
No doubt I am behind the times in terms of developments, as I haven't had to deal with BGP in seven years. However, according to this site, only 6144 32 bit AS numbers have been allocated as of today, compared to 58366 16 bit. They are out there, but they represent less than 10% of the AS numbers allocated, and even then that doesn't tell us how many are actually in use.

edit: Helps if I read further down the page in that report. 927 32 bit AS numbers are being advertized presently, compared to 35968 16 bit. Less than 3% of the AS numbers in use.

Soldarius
Caldari
Peek-A-Boo Bombers
Posted - 2011.03.23 21:55:00 - [43]
 

Very cool. Keep up the good work.

Lors Dornick
Caldari
Posted - 2011.03.23 22:37:00 - [44]
 

Originally by: Lord's Prophet
This devblog is just another step towards the EVE cluster becoming self-aware and killing us all.


Nah, it's the MCP we should fear, not the FCP.

Klandi
Consortium of stella Technologies
Posted - 2011.03.23 23:48:00 - [45]
 

Please tell me you made these decisions AFTER you applied an analyser to the appropriate port(s) and looked at the TCP traffic to measure the resets, retransmissions and zero window counts. Those are the figures I would be interested in!!!

I can't do it because the EULA states I am not allowed to attach an analyser in-line and capture traffic while running Eve!

Decus Daga
Black Thorne Alliance
Posted - 2011.03.24 01:26:00 - [46]
 

Great!

Turn it on already, i just filed a bug report about random disconnects and desyncs lol.

Great dev blog, me likes the techy ones.

Richard Royce
Caldari
Abh Empire
Posted - 2011.03.24 02:38:00 - [47]
 

Thanks for keep us informed. This sort of post really connects us end users to the game. We love this game and see the potential, post like this keeping the end-user in the loop really help us to hang on for future gaming goodness.

.

Magnius Ion
Posted - 2011.03.24 03:02:00 - [48]
 

Admittedly, I have not read through this discussion, or much on lag either, but was thinking about a possible solution to lag. What if the user had a spacebar effect of stopping and starting overview updates and special effects. I don't pretend to know much about the issues at hand, but imagine fleets able to issue orders to pilots to freeze overview updates. Pilots could still pick currently displayed targets and issue commands like orbit or approach and lock, but they would basically fly blind until they issue a toggle (spacebar) to update (or to start receiveing updates to the overview. Would that not make battle response better and still give the pilot the option to get updates when they choose to. Maybe I need to goto a different forum for this idea or maybe I'm just stupid to the whole issue. At any rate, I thought the hardcore pvper's could sort out the effects that could be stopped and mention this as a method to put more control into the pilots hands. I'm afraid to post this cause I'm sure it will be shot down... here goes nothing.

EdwardNardella
Capital Construction Research
Posted - 2011.03.24 03:20:00 - [49]
 

Edited by: EdwardNardella on 24/03/2011 03:20:37
WANT GRAPH NOW

Also, CCP you guys are really lucky getting to invest in such cool things!

Simo Orgas
Posted - 2011.03.24 03:36:00 - [50]
 

Edited by: Simo Orgas on 24/03/2011 03:39:15
FCP? Really?

You do realize that there is a reason why RouteScience went bankrupt - its because, while FCP worked, it didn't work well enough. If you have competent network engineers, FCP is pretty much worthless unless you have some really suboptimal routing (due to a fiber cut, for example, making traffic hairpin)

There is a reasonable cost optimization reason to use FCP, but the declining cost of transit has made this pretty much immaterial.

I think people who expect a big performance improvement from this will be let down, except in corner cases.

Since CCP hired Derek Wise, I had hoped you guys might start peering more aggressively on the LINX, AMSIX, east coast of US (111 8th, Ashburn, etc)

big fluf
Posted - 2011.03.24 03:49:00 - [51]
 

yea, but did I read ti is not live yet????

when you unleaching it on us?

Abdiel Kavash
Caldari
Paladin Order
Fidelas Constans
Posted - 2011.03.24 08:08:00 - [52]
 

Originally by: Lord's Prophet
This devblog is just another step towards the EVE cluster becoming self-aware and killing us all.


I, for one, welcome our new internet spaceships overlords.

Toshiro GreyHawk
Posted - 2011.03.24 10:23:00 - [53]
 


There's another way ...



All kidding aside ... we had a way of telling what the optimal path was ... but I wasn't a router guy and ... sad to say ... in the years since my retirement - I've forgotten the details. Hmmm ... OSPF ... Open Shortest Path First ...

But - yes - that was an IGP rather than a BGP protocol ... OK ... never mind ...

Ha! Ha! I'm impressed I remembered that much ... *sigh* ...



I thought about just deleting this post but decided to leave it up for the nerd links ... Ha! Ha!

.

Korerin Mayul
Amarr
Posted - 2011.03.24 11:46:00 - [54]
 

Edited by: Korerin Mayul on 24/03/2011 11:48:52
Came hoping for a post on Fibre Channel Protocall, left dissapointed :(

That said, in the extreame example, FC would suck balls aswell, FSPF dosent need to care about latencey as its not for WAN, Christ, we have to manually calculate the buffer credits needed to fill a long link.

CCP Mort

Posted - 2011.03.24 14:32:00 - [55]
 

Hi

Hi all, thank you for the comments.

The FCP has been live since 10th of March, we where bit late with the blog.

I will follow up with a detailed blog (with graphs) when the fanfest frenzy is over ;)

I have seen sites like Easynews etc. that provide users with control to chose the next hop provider - that is pretty cool actually, but there are issues like how long should we allow a route change to be in place, DOS prevention and users must not be able to change routes for other players.

for the BGP nerds: The best way to override a route is to inject a new route into BGP with a different next-hop attribute and use metrics like local pref to force it to be used. If the FCP would "fail" the route drops out and we go back to default BGP routing policy as configured in the edge routers.

We are of course not a Tier1 provider as we buy transit, we only buy transit from Tier1's. Then we have a pseudo "direct" connection to some 250 end user DSL providers - This is provided by our main provider (PacketExchange) so these 250 providers appear as one hop away with high bandwidth circuits.

LINX and AMSIX: very likely sooon!

- CCP Mort

Jake S
Posted - 2011.03.24 19:12:00 - [56]
 

In my day job, I've been using an Internap FCP to optimize my BGP network traffic for over 5 years (back when it was NetVMG).

I've been happy with it as a network performance enhancing tool and as a troubleshooting tool. The FCP reconstructs TCP sessions from either a packet capture, or via a built-in NetFlow collector. It aggregates these sessions into like networks (usually /24s) then monitors these networks for packet loss, latency, and jitter. If it notices a change in the performance characteristics for a network, it will probe out all available external paths to see if an alternate path will improve performance for the impacted network. If an alternate path will improve the performance, it will inject an iBGP route into the route table with a high local preference, the injected route will now become the best iBGP route, and traffic will flow out that path. To prevent route thrashing, it has a hold timer, so a network has to perform poorly for 10 minutes to be eligible for route optimization.

In the real world, it will improve the performance for the worst performing networks, that have alternate paths available. Mine performs about 15,000 route changes a day and improves the latency for the impacted networks by an average of 33%, and reduces packet loss by 20%.


Diomedes Calypso
Aetolian Armada
Posted - 2011.03.24 20:13:00 - [57]
 

More great stuff...

As consumers we're going to be forced more and more to play watch-dog on our isp's based on their move towards preferencial routing due to a business model where they create content of their own or .(ehem) collect a toll from content providing companies to give those companies prefferencial connection speeds to customers.

Is there any way we can tell if our particular ISP is giving eve a low priority at a more local level? This is beyond my area of knowledge so i might get the question wrong, but is there a tool that will log spikes and troughs during a few hour period of play so we can go back and get an idea where things got sticky and where they didn't?

Terrorina
Posted - 2011.03.24 22:12:00 - [58]
 

very happy to see that ccp is actively trying to reduce the lag instead of just blowing smoke. obviously this is not a magic 'fix lag' button but it's a great step in the right direction. so coming from a guy who is very quick to jump on the 'ccp doesnt know wtf its doing due to this horrible lag' bandwagon, thank you ccp.. this is a big step in the right direction.

Cool

XT 0023
Posted - 2011.03.24 22:15:00 - [59]
 

Originally by: Diomedes Calypso
More great stuff...

As consumers we're going to be forced more and more to play watch-dog on our isp's based on their move towards preferencial routing due to a business model where they create content of their own or .(ehem) collect a toll from content providing companies to give those companies prefferencial connection speeds to customers.

Is there any way we can tell if our particular ISP is giving eve a low priority at a more local level? This is beyond my area of knowledge so i might get the question wrong, but is there a tool that will log spikes and troughs during a few hour period of play so we can go back and get an idea where things got sticky and where they didn't?


will pay $10 extra /month for less lag so I can shoot all the frozen supers :P

Warwick Bentley
Posted - 2011.03.25 02:38:00 - [60]
 

What I think is ironic is that I was just sitting down to study for my upcoming Cisco BGP test and wanted to quickly check and see if the latest QEN was out and found this instead.

This is doublely great because when my wife yelled at me asking if I was playing Eve instead of studing I said, nope I am studying.



Pages: 1 [2] 3

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only