open All Channels
seplocked EVE Technology Lab
blankseplocked EVE API - HTTP compression
 
This thread is older than 90 days and has been locked due to inactivity.


 
Author Topic

Ydnari
Gallente
Estrale Frontiers
Posted - 2011.07.29 19:45:00 - [1]
 

Some of the API calls produce relatively large XML output, such as the Assets list, this can easily end up being a megabyte or more. There's others that you might call more frequently that add up too.

This could be greatly reduced by enabling compression in the response (if requested and supported by the client in the usual way, standard HTTP stuff). XML compresses well due to the large amount of repeated text in it; around 10:1 compression for the assets API response.

Obviously this trades off decreased bandwidth with increased processing at the server.

The size of the response doesn't really matter when it's being run on a desktop or from a server, but when you're writing a mobile app you have to account for these things - 3G connections aren't particularly fast, and tend to have monthly caps on them.

Since there seems to be a little movement in the API area with the Contracts API, thought I'd raise this issue and see if CCP will consider it.

Zeta Zhul
Caldari
Preemptive Paranoia
Posted - 2011.07.30 20:11:00 - [2]
 

IMO if CCP really wanted to reduce the overall amount of bandwidth spent on the API calls and server resources how about letting us grab API data based on a >= datetime value? If already know that I've got everything older than a specific datetime value then letting me grab everything newer and equal to that datetime value ensures that most of what I'm getting isn't duplicated.

It's like the AssetList data; most of that doesn't change very often.

Johnathan Roark
Caldari
The Graduates
Morsus Mihi
Posted - 2011.07.30 21:31:00 - [3]
 

Originally by: Zeta Zhul
IMO if CCP really wanted to reduce the overall amount of bandwidth spent on the API calls and server resources how about letting us grab API data based on a >= datetime value? If already know that I've got everything older than a specific datetime value then letting me grab everything newer and equal to that datetime value ensures that most of what I'm getting isn't duplicated.

It's like the AssetList data; most of that doesn't change very often.


That wouldn't help with ccp's sever resources, its more likely to hurt. It requires additional sorting by ccp's database server. I really do not think ccp is tracking when items appeared in a station, just if they are their now. As far as bandwidth, as much as ccp is buying, the api server is probably the least of there concern.

Assuming there is not much impact on the api severs, I am all for enabling http compression.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2011.07.31 00:00:00 - [4]
 

I've had it enabled in Yapeal for a while hoping they would decide to turn it on and they have said that with some of the changes they have made the servers themselves aren't heavily loaded any more if I understand right they are mostly limited by the database bandwidth so as not to cause actual issues in game. They had no problems going to HTTPS which cause extra load and they don't seem to think it was a problem. I would say it could save a lot on the long cache times APIs like /eve/CertificateTree.xml.aspx, /eve/ErrorList.xml.aspx, and /eve/SkillTree.xml.aspx etc where the data rarely changes and it could even be pre-compressed and cached. It would save bandwidth on several of the other larger APIs but you have to weigh if it makes sense to use it with ones that are basically going to be used once if people are obeying the cachedUntil times. It might not make sense to use it with something that is only used once or changes almost every time it's sent.

I understand where using it would be a great benefit for mobile application like you outline there would also be some big benefits for applications that are trying to keep all the APIs updated for a larger user base. When you start working with a few hundred or even thousands of accounts often the limiting factor can be network bandwidth.

Quarantan
Caldari
Burning Napalm
Northern Coalition.
Posted - 2011.07.31 13:29:00 - [5]
 

if you want http compression for your mobile app you can always just write yourself a api proxy and compress it there. or poke one of the current api proxy providers to enable compression, perhaps you can get them to do it :)


 

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only