open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: The API Dev Blog Trilogy - Volume One
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: [1] 2

Author Topic

CCP Fallout

Posted - 2010.09.20 17:35:00 - [1]
 

CCP Stillman's newest dev blog includes some of the work that we've be doing on the API, and includes important information for API developers. Read all about it here.

Update: the date now correctly reads Monday September 27, 2010.

Aineko Macx
Posted - 2010.09.20 17:46:00 - [2]
 

First!

/happy to hear about API development Cool

Selene D'Celeste
Caldari
The D'Celeste Trading Company
ISK Six
Posted - 2010.09.20 17:54:00 - [3]
 

Edited by: Selene D''Celeste on 20/09/2010 18:01:19
Whoo API love.

Edit: How will site owners be contacted if they are causing issues via the API?

Lucyna
Interstellar Killer Bee Enterprises
Posted - 2010.09.20 17:59:00 - [4]
 

When are we going to see contracts in the API?

Silk75
Bad Kitty Inc.
Wildly Inappropriate.
Posted - 2010.09.20 18:01:00 - [5]
 

Nice to hear the improvements, although my calendar says Monday is the 27th September.

o/

Majestic Moose
Posted - 2010.09.20 18:05:00 - [6]
 

Actually september 26 is a monday in 2011.

Jason Edwards
Internet Tough Guy
Spreadsheets Online
Posted - 2010.09.20 18:09:00 - [7]
 

Database optimizations

We also found Microsoft SQL Server to be at fault. So we will be going to Postgresql 9.0: It has a lot of new stuff, including hot standby support, streaming replication, better permissions management, and a lot more.

Open Source is the future of CCP.

Delagos Almondis
Posted - 2010.09.20 18:15:00 - [8]
 

Originally by: Lucyna
When are we going to see contracts in the API?

^this.

Caiman Graystock
Caldari
Cornelius Starship and Computer Design
Posted - 2010.09.20 18:24:00 - [9]
 

In before Chribba. Thanks for the blog guys. I love seeing all of these blogs you're putting out lately!

CCP Stillman

Posted - 2010.09.20 18:25:00 - [10]
 

Originally by: Selene D'Celeste
Edited by: Selene D''Celeste on 20/09/2010 18:01:19
Whoo API love.

Edit: How will site owners be contacted if they are causing issues via the API?

In most cases, we've been able to find the owner of a server by simply opening the site that is located on the server, or simply by the IP itself.

So please make sure that a domain is associated with your IP, so that we can figure out who owns the site. Having contact details on the site is even better.

Indeterminacy
THORN Syndicate
BricK sQuAD.
Posted - 2010.09.20 18:27:00 - [11]
 

Originally by: Jason Edwards
Database optimizations

We also found Microsoft SQL Server to be at fault. So we will be going to Postgresql 9.0: It has a lot of new stuff, including hot standby support, streaming replication, better permissions management, and a lot more.

Open Source is the future of CCP.


loool. even after all these years no open source database server can match any commercial product for supported performance and reliability.

Ideological Purity be damned.

Jason Edwards
Internet Tough Guy
Spreadsheets Online
Posted - 2010.09.20 18:33:00 - [12]
 

Originally by: Indeterminacy
Originally by: Jason Edwards
Database optimizations

We also found Microsoft SQL Server to be at fault. So we will be going to Postgresql 9.0: It has a lot of new stuff, including hot standby support, streaming replication, better permissions management, and a lot more.

Open Source is the future of CCP.


loool. even after all these years no open source database server can match any commercial product for supported performance and reliability.

Ideological Purity be damned.

Actually features and performance wise. Postgresql defeats all the major databases. It even competes with the databases like mongodb in their niche market. The only reason postgresql isnt popular is the entire lack of documentation and suck a stickler for the sql standard.

Aranial
Gallente
Empyrean Warriors
The Obsidian Front
Posted - 2010.09.20 19:17:00 - [13]
 

Edited by: Aranial on 20/09/2010 19:19:44
Edited by: Aranial on 20/09/2010 19:18:19
From a person who knows next to nothing about the API system this was a highly interesting and informative dev blog.

Keep up the good work,

Aranial.

Edit: 'RAID 10 8-disk setup utilizing 15,000 RPM fiber channel disks' dribble...dribbleLaughing.

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2010.09.20 19:21:00 - [14]
 

Soo, can we expect DTD's for API feeds and normal (for HTTP) authentication mechanics?

ElfeGER
Deep Core Mining Inc.
Posted - 2010.09.20 19:23:00 - [15]
 

Edited by: ElfeGER on 20/09/2010 19:47:40
hmm I would just use a view for the logging table issues

Lost Hamster
Hamster Holding Corp
Posted - 2010.09.20 19:27:00 - [16]
 

How can we figure out what the error code means in the API log?

I found some 117 error in my log file, however in the
http://www.eveonline.com/api/doc/errors.asp list there is no error 117 listed.


Fake edit, as I found the answer later:
Would be nice to update the above page with the error list from here:
http://api.eve-online.com/eve/errorlist.xml.aspx

T'Amber
Garoun Investment Bank
Posted - 2010.09.20 19:40:00 - [17]
 

I'm just glad Mynxee isn't sitting next to me

EmbarassedEmbarassedEmbarassed

Scetrov
Gallente
Gallente Textiles and Manufacturing
Posted - 2010.09.20 19:55:00 - [18]
 

I appreciate that from the language you are using that it is mostly requests coming from one location rather from requests from applications installed on users machines. However if there is anything I can do with EVEMon that will help do let me know.

Also analytics would be really handy to know how our users are using the API, f.ex. how many users are on the latest version of EVEMon, how many use which APIs, etc.

Dusty Meg
Posted - 2010.09.20 20:25:00 - [19]
 

Always good to hear how CCP are making eve even better.

And giving love to the third party devs, Very Happy

Zagdul
Gallente
Clan Shadow Wolf
Fatal Ascension
Posted - 2010.09.20 21:26:00 - [20]
 

The reason a lot of our software is getting bad requests for API keys is not because of a fault in our software, it's because of a fault in the API system.

My forums have an API check. This check is run via chron job to validate API's in my alliance. If someone has changed their API, it'll spit out an invalid request then deny them access to my forums. I then manually delete the members so that request is no longer made.

There's nothing I as a site admin can do against this. If someone changes their API, I can't control them. The best I can do is limit the amount of bad requests.

But blacklisting someone like me who may make 10-20 bad requests a month?




Arous Drephius
Posted - 2010.09.20 22:01:00 - [21]
 

Originally by: Zagdul
The reason a lot of our software is getting bad requests for API keys is not because of a fault in our software, it's because of a fault in the API system.

My forums have an API check. This check is run via chron job to validate API's in my alliance. If someone has changed their API, it'll spit out an invalid request then deny them access to my forums. I then manually delete the members so that request is no longer made.

There's nothing I as a site admin can do against this. If someone changes their API, I can't control them. The best I can do is limit the amount of bad requests.

But blacklisting someone like me who may make 10-20 bad requests a month?



I doubt CCP cares about 10-20 bad requests per month, however you could reduce it to just one invalid request per person that changes their key.

As soon as your cronjob gets the 'authentication failure' message back from the API, mark the user account as invalid, and have the job check to see if each user is invalid before sending the request to the API next time.

CCP Explorer

Posted - 2010.09.20 22:32:00 - [22]
 

Originally by: Arous Drephius
Originally by: Zagdul
The reason a lot of our software is getting bad requests for API keys is not because of a fault in our software, it's because of a fault in the API system.

My forums have an API check. This check is run via chron job to validate API's in my alliance. If someone has changed their API, it'll spit out an invalid request then deny them access to my forums. I then manually delete the members so that request is no longer made.

There's nothing I as a site admin can do against this. If someone changes their API, I can't control them. The best I can do is limit the amount of bad requests.

But blacklisting someone like me who may make 10-20 bad requests a month?
I doubt CCP cares about 10-20 bad requests per month, however you could reduce it to just one invalid request per person that changes their key.

As soon as your cronjob gets the 'authentication failure' message back from the API, mark the user account as invalid, and have the job check to see if each user is invalid before sending the request to the API next time.
Exactly.

The sites we have already contacted have issued thousands of requests with invalid keys in a matter of a few days, with repeated requests with the same invalid keys again and again.

Muscaat
EVE Markets
Posted - 2010.09.20 22:36:00 - [23]
 

Originally by: Tonto Auri
Soo, can we expect DTD's for API feeds and normal (for HTTP) authentication mechanics?


DTDs are horrid... why not just go for XML schemas?

(My IRL company provides a service of designing and managing XML schemas. If any devs want to get in touch we'd be happy to discuss the possibilities Smile)

ChrisIsherwood
Posted - 2010.09.21 00:25:00 - [24]
 

Thanks for the API efforts!!!

Is there any way we will ever get the cacheing at the server side? My concern is that even if all my apps are well behaved, I could still request a cache many times in an hour. Locally I might have jEVEAssets, EFT, EVEHQ, etc. running and each could do an API call once an hour. Even if I were to run a local cache, there are all the various sites like eveboard, evemetrics, evecommander, api validated forums and voice servers, capsuleer and iclone running on ipads as well as iphone. Nothing that relies on the client to know about caching will ever be overly successful. Hopefully, as the technology grows, the number of API applications will continue to explode, especially on mobile devices. (Thanks again for the API efforts.)

Web sites don't really rely on the browser to decide what content to cache; there is at least an Apache cache mod if not separate caching cluster. If i access a URI from different browsers or computers, the data gets cached because it happens at the server not the client. Even easier if the API were segregated by dates since the API for 20100919 for idkey 123 will never change or need to be recomputed; that is just fixed XML to serve up.




Originally by: Indeterminacy

loool. even after all these years no open source database server can match any commercial product for supported performance and reliability.



If we were talking about Oracle or DB2, there's valid arguments. But this is lolMSSQL we are talking about.

Wollari
Phoenix Industries
Wicked Nation
Posted - 2010.09.21 00:45:00 - [25]
 

Edited by: Wollari on 21/09/2010 00:57:34
\o/ yeah more api improvements, go ccp go :-)

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2010.09.21 04:05:00 - [26]
 

Originally by: Muscaat
Originally by: Tonto Auri
Soo, can we expect DTD's for API feeds and normal (for HTTP) authentication mechanics?


DTDs are horrid... why not just go for XML schemas?

(My IRL company provides a service of designing and managing XML schemas. If any devs want to get in touch we'd be happy to discuss the possibilities Smile)

You know, it not really that different? Any data definition would help speed up the development.
I'm more interested in removing fifth legs and second tails from API, really.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2010.09.21 04:26:00 - [27]
 

Scetrov -
Quote:
Also analytics would be really handy to know how our users are using the API, f.ex. how many users are on the latest version of EVEMon, how many use which APIs, etc.
These stats would be nice but even just some general stats on how many requests CCP is seeing per API I think would be very useful and interesting to know for all the third party developers. I'm sure now that CCP have got people working on this they've started collecting that info if they weren't already before. Might also be useful to know which APIs are receiving the most bad requests so everyone can double check their code for those first just to make sure there aren't any issues before anyone has to contact them about it.

I know that as a library developer info along those lines would be most useful to me in deciding where I might put more checking/monitoring in my code and maybe even prompt me to add stuff to make it easier from anyone using my library track their API usage so they might be able to ID any problems with their applications before scaling up past the testing stage into full production.

Keiko Kobayashi
Amarr
Celestial Janissaries
Curatores Veritatis Alliance
Posted - 2010.09.21 04:40:00 - [28]
 

I still donít get why you donít use HTTP caching with Expired headers. Then you could easily insert caching proxies between your servers and the outgoing pipe, and tools would be able to use their platformís native HTTP caching abilities as well. You would have more control over the caching overall.

Jamina Galandel
Posted - 2010.09.21 06:53:00 - [29]
 

Sadly the last year or so, not a lot of news concerning new content is being published. I know that the technical parts may be interesting for one or the other, but as a gamer, I would like to know more about the upcoming content development. The aim for better performance and less bugs is what I see as a necessity - generally called "software maintenance" in a software development lifecycle. I know that some gamers might look at it differently, but this is just my view of things.

Is there any dev blog planned in the near future about new upcoming player content? What about decisions that were made and why? What options were there and why did you choose solution A rather than solution B?

I hope I don't sound disrespectful, as this is not my intent. I would just like to see dev blogs that suit everyone's taste now and again.

Thanx in advance for your understanding!Very Happy

Xavier Linx
Posted - 2010.09.21 07:23:00 - [30]
 

Originally by: Tonto Auri
Originally by: Muscaat
Originally by: Tonto Auri
Soo, can we expect DTD's for API feeds and normal (for HTTP) authentication mechanics?


DTDs are horrid... why not just go for XML schemas?

(My IRL company provides a service of designing and managing XML schemas. If any devs want to get in touch we'd be happy to discuss the possibilities Smile)

You know, it not really that different? Any data definition would help speed up the development.
I'm more interested in removing fifth legs and second tails from API, really.


My sentiments exactly. Personally I hate working with DTDs but they are much preferred to "nothing" which is the case now.
On the other hand... A well documented SOAP interface with WSDL would be reaaaaally nice. Very Happy



Pages: [1] 2

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only