open All Channels
seplocked EVE Technology Lab
blankseplocked PLEASE MAKE API SERVER RFC2616 COMPLIANT
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: [1] 2

Author Topic

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.09.06 21:04:00 - [1]
 

according to the RFC:

4.2 Message Headers

HTTP header fields, which include general-header (section 4.5),
request-header (section 5.3), response-header (section 6.2), and
entity-header (section 7.1) fields, follow the same generic format as
that given in Section 3.1 of RFC 822 [9]. Each header field consists
of a name followed by a colon (":") and the field value. Field names
are case-insensitive. The field value MAY be preceded by any amount
of LWS, though a single SP is preferred. Header fields can be
extended over multiple lines by preceding each extra line with at
least one SP or HT. Applications ought to follow "common form", where
one is known or indicated, when generating HTTP constructs, since
there might exist some implementations that fail to accept anything

beyond the common forms.


...but the API server doesn't recognize "host" or "HOST" (only "Host"). I don't know if it has problems with other header fields or not.

Is it possible to correct this problem so that I can complete my Qt classes for the Eve API without reimplementing the entire HTTP protocol class?

Immersive
Immersive Technology Solutions
Posted - 2008.09.07 05:40:00 - [2]
 

Bug Report, imho Smile

Chibisuke
Gallente
Children of Avalon
Posted - 2008.09.08 18:14:00 - [3]
 

Edited by: Chibisuke on 08/09/2008 18:15:01
Seams you need to bug report to microsoft then.

Date: Mon, 08 Sep 2008 18:10:55 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727

I wouldn't dare using IIS in any production enviornment anyway, but it seams the admins of CCP don't have any security concerns, so...

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.09.08 18:48:00 - [4]
 

Originally by: Chibisuke
Edited by: Chibisuke on 08/09/2008 18:15:01
Seams you need to bug report to microsoft then.

Date: Mon, 08 Sep 2008 18:10:55 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727

I wouldn't dare using IIS in any production enviornment anyway, but it seams the admins of CCP don't have any security concerns, so...


Bug report to Microsoft - LMAO

I'm not familiar with this ASP.NET. I figured it was some part of that visual studio dot net crap and possibly something CCP could fix on the API server. I didn't see the "server: IIS" part in the header.

My problem here is that I can use curl and it will work fine (on *nix) but I was hoping to make API classes that would work on any platform, using the Qt classes. The Qt classes rely on standards compliant http servers, though, which the API is not.

I'm kind of lost as to what to do next. The MS bug report sounds like a waste of time, and the CCP bug report obviously won't end in IIS being fixed. I can't really see them switching to apache or something either...

Does anyone have any idea how to proceed?

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.09.08 19:06:00 - [5]
 

After I stopped laughing at the 'file a bug report to microsoft' comment, I decided that even though I was almost sure it was a joke I would check into the possibility...

As I suspected, filing a bug report with Microsoft is nearly impossible, if possible at all. And since I don't own any Microsoft products, it would probably cost me hundreds of dollars just to attempt to file one that nobody would ever do anything with...

So, unless the problem can be corrected from the API server's scripts, it sounds like maybe the only way it will be corrected is for CCP to use a different web server for the API scripts.

(at least based on the limited information I have)

So, what are the odds?


Scoutette
Ki Tech Industries
Posted - 2008.09.08 23:59:00 - [6]
 

File a regular bug report with CCP.

If needed they will file a bug report with Microsoft, if that is infact the source of the problem.

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.09.09 00:51:00 - [7]
 

Edited by: Gilbert T on 09/09/2008 00:56:23
Edited by: Gilbert T on 09/09/2008 00:52:34
For those who care, I have now filed a bug report. I don't know if anyone else can see it, but here is the URL to it:

(link removed - others can't view it apparently)



CCP Lingorm


C C P
Posted - 2008.09.09 09:29:00 - [8]
 

We are aware of this, but have no plans to work round the issue.

Longer term we are actually looking to migrate the API to a Web Service (fully standards compliant) as it provides us a number of benefits and development improvements.

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.09.10 18:58:00 - [9]
 

Originally by: CCP Lingorm
We are aware of this, but have no plans to work round the issue.

Longer term we are actually looking to migrate the API to a Web Service (fully standards compliant) as it provides us a number of benefits and development improvements.



I'm curious of what "longer term" might mean in this instance...

I could spend a fair amount of time reimplementing the QHttp classes to deal with the server, or just wait it out and finish my QEVEAPI classes once the server is fixed.

Just a general idea, like "maybe not for a few years" or "probably in the next month or 2" would be fine.


Immersive
Immersive Technology Solutions
Posted - 2008.09.11 06:52:00 - [10]
 

I could all but gaurantee you that CCP's "Longer-Term Plan" will not take effect within the next 6 months, probably not even 12.

CCP Lingorm


C C P
Posted - 2008.09.11 09:01:00 - [11]
 

6-12 would be a good 'guesstimate'. I do know it is being seriously considered.

We would still maintain the existing XML interface, but it would not be upgraded and all new functionality would go to the Web Service.


Hel O'Ween
Men On A Mission
EVE Trade Consortium
Posted - 2008.09.11 12:22:00 - [12]
 

Originally by: Chibisuke

I wouldn't dare using IIS in any production enviornment anyway, but it seams the admins of CCP don't have any security concerns, so...


Ah, c'mon. This is a unprofessional statement.

When was the last time a security leak was found in IIS? I mean, you don't blame Apache for security wholes in PHP or Wordpress or the like or the stupidity of its admin, for that matter.

Thoughtlessly repeating stereotypes that were true some 10-15 years ago (NT4 + IIS), but have long changed meanwhile don't do your credibility any good.

Sky Grunthor
Minmatar
Conflagration.
Wildly Inappropriate.
Posted - 2008.09.17 17:45:00 - [13]
 

Standard web services would of been the better choice to begin with. Why didn't CCP go with it or some other kind off standardized protocol?

Mildrena
Posted - 2008.09.19 15:07:00 - [14]
 

Eve was released in 2003.

Web Service standards were still very new in 2003. (I was working with them for my master's thesis at the time.)

Nobody was sure which standards would win.

It was the right move not to gamble on them in 2003.

Cesar Malari
Dark-Rising
IT Alliance
Posted - 2008.09.19 15:44:00 - [15]
 

Edited by: Cesar Malari on 19/09/2008 15:45:23
Edited by: Cesar Malari on 19/09/2008 15:44:26
Originally by: Mildrena
Eve was released in 2003.

Web Service standards were still very new in 2003.

Except that the API wasn't released in 2003. It was released in mid-2007. At that time, I think we can safely say that a web-services API would have been an easy choice, and not too hard to develop on CCP's chosen web technology (ASP.NET). In fact, on .NET, I think it'd have been *easier* to write a web services API than the route they ended up going, but I'm not going to complain too much -- it's not like it's that hard to use what they gave us.

CCP Lingorm


C C P
Posted - 2008.09.19 16:07:00 - [16]
 

The choice at the time was made because it was believed that XML would prove more adaptable to most of the programmers that would be using it.

Now, given the rise of Web Services and the better support for them in most programming languages we have revisited the idea and think that web services are a better long term strategic choice for this part of EVE.

Amida Ta
German Mining and Manufacture Corp.
Posted - 2008.10.21 12:58:00 - [17]
 

I don't know HOW many posts I have written to please use WebServices instead of that dreadful REST XML. And NOW where we have API-Interface Projects you think about using WebServices?
You sure know how to keep your community busy *g*.

CCP Lingorm


C C P
Posted - 2008.10.21 13:57:00 - [18]
 

hehehehe ... sorry, but the API was made before myself and Elerhino started at CCP, we have been picked to carry it on and to make it what we want it to be long term (with better caching and stuff) web services are a better choice.

But as I said we are keeping the old functionality so you do not have to go out and completely rewrite your applications.

We will also have some other news about the API at the Fanfest (will be posted here the week after) that I am sure you will all like.

Ambo
I've Got Nothing
Posted - 2008.10.21 16:16:00 - [19]
 

Originally by: CCP Lingorm
hehehehe ... sorry, but the API was made before myself and Elerhino started at CCP, we have been picked to carry it on and to make it what we want it to be long term (with better caching and stuff) web services are a better choice.

But as I said we are keeping the old functionality so you do not have to go out and completely rewrite your applications.

We will also have some other news about the API at the Fanfest (will be posted here the week after) that I am sure you will all like.



Awesome. Can't wait to hear what the news will be and VERY glad that I won't have to re-write all my XML parsing stuff unless I decide I want to! Smile

Gilbert T
Gallente
Gladiators of Rage
Posted - 2008.10.22 15:58:00 - [20]
 

I think the XML is great, I just wish the web server would function as it should. It seems that libcurl is equipped to deal with its handicaps, though, so I just made a Qt libcurl wrapper and I'm doing fine. It just sucks that my library will now be libcurl dependent when Qt already has HTTP protocol classes.

To just add to the trivia, RFC2616 was published in June 1999. I guess the guys at Microsoft must have forgotten to read it...


Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.10.22 17:09:00 - [21]
 

I don't know what's so dreadful about RESTful XML. It is as I understand it one way of doing Web Services and work across all platforms and programming languages. There may be some issues with the software CCP choose to use in their implementation but it's very easy to use and other than making it more standards compliant and adding more APIs I would hate to see it change for something new especially if it's going to make it a Microsoft only thing. Even going to some from of SOAP I would expect would add more server load for CCP and is why you're seeing many other companies now moving back to RESTful designs because of the much lower overhead on both the server and client.

Maybe I've missed something but other than being better implemented for less server load, more standard compliance, and easier management I see no reason to change it.

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2008.10.28 13:37:00 - [22]
 

Originally by: CCP Lingorm
The choice at the time was made because it was believed that XML would prove more adaptable to most of the programmers that would be using it.

Now, given the rise of Web Services and the better support for them in most programming languages we have revisited the idea and think that web services are a better long term strategic choice for this part of EVE.



But I easily recall the discussion about REST and authentication methods used in API access.
Why make it complex if HTTP standards already implies authorisation of many kind? Why not use what already exist?
You (devs, as a whole, sorry I don't know anyone in person so I refer you as a group) have done such work for EVE itself (StacklessIO utilising native signaling of IO operation completion foe instance). Why not streamline API in the same way?

Ix Forres
Caldari
Righteous Chaps
Posted - 2008.10.29 10:59:00 - [23]
 

Edited by: Ix Forres on 29/10/2008 11:01:00
I really hope CCP not only maintain the old RESTful XML API but continue to keep it up-to-date; it may not be optimal, but has the distinction of working well on many platforms with ease and being integrated into pretty much anything. Give it a year and my toaster will support XML and be able to pull my training skills down so it can remind me to change skills while I make breakfast; the same can't be said of web services (What exact interface to 'web service' are you thinking of, anyway? There's plenty of standards and 'standards' about).

I would also point out that the available libraries all function using the XML interface, and cover Java, PHP, Ruby, Lisp, RealBASIC, .NET, Perl and Python; most of those are very simple libraries that don't even need to know what the data looks like to work, and thus can be kept with any XML upgrades. Make it a web service and bang goes support for half the languages for a year, and you then get to make everyone running current apps upgrade all their code to another API/library just to get new features.

Doesn't make much sense to me. There are better things to spend time on, like making said new features. What we have works very well and as the old adage goes, don't fix what isn't broken. Spend some more time making QA better, optimising things for lower cache times, writing new features and improving documentation in my opinion.

edited: forgot perl. easy mistake to make.

Amida Ta
German Mining and Manufacture Corp.
Posted - 2008.10.29 21:42:00 - [24]
 

Sorry, but I'm of the complete opposite opionion. Retrieving XML files via HTTP sure is available for basically every platform in existance. But other than that there are absolutely no other reasons to have it.

Lets look at what we got:
A (meanwhile) quite large sets of APIs all producing completely non-standard and custom XML documents. Also some seem to have some sort of structuring that structure is not standardized either and differs slightly from file to file.
In consequence if you want to use the API today (and you do not just want to look at XML) your only chance is to HAND-Parse every type of XML-Response. This is completely insane.
My library does that and it's quite large. And there are still more problems. There is no versioning for the XML-Files. Not long ago one of my apps broke because the XML was changed in a way that could not be detected by the parser...
And parsing XML is not a simple thing, even with libraries it requires substantial amount of boilerplate code.

With WebServices you can autogenerate ALL of that boilerplate code. All responses are strictly standardized. It is 'impossible' to have structural error (like the ones we have now). Granted, not every development environment offers full support for that, but those that do make the usage of one of the APIs a matter of *seconds* even for the huge API Interfaces (like Charactersheet).

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.10.30 20:59:00 - [25]
 

Quote:
Sorry, but I'm of the complete opposite opionion. Retrieving XML files via HTTP sure is available for basically every platform in existance. But other than that there are absolutely no other reasons to have it.

What better reason could there be then it works everywhere so everyone can use whatever programming language or libraries they want?

Quote:
A (meanwhile) quite large sets of APIs all producing completely non-standard and custom XML documents. Also some seem to have some sort of structuring that structure is not standardized either and differs slightly from file to file.


Good luck finding a 'standard' way of show what's in the API but from what I can tell this is the 'standard' way that MSSQL does XML but since I don't use it myself I can't say for sure. I will agree that they could make some of the APIs more consistent in some areas. If CCP would make DTD for them many of the tools people are using could use them to help auto-gen the code or even a XSL could be useful.

Quote:
In consequence if you want to use the API today (and you do not just want to look at XML) your only chance is to HAND-Parse every type of XML-Response. This is completely insane.
My library does that and it's quite large.

Not really I made a library for PHP and it only needs a couple functions to parse the two ways they put information in APIs. One to work with stuff in tags and their values and one for when info is in attributes of a tag. I don't think it's that big for what it does so if you feel yours is maybe it time to do some re-factoring I know I've done that several times as I've learned more about the API and how to factor out the differences. If your not using them you might look at DOM and XPath which almost every language has libraries for. I've found them very useful with the APIs.

Quote:
There is no versioning for the XML-Files.

<eveapi version="2">
That looks like versioning to me but might not be what you're looking for I don't know.

Quote:
Granted, not every development environment offers full support for that, but those that do make the usage of one of the APIs a matter of *seconds* even for the huge API Interfaces

So they should change it to make it easier for some people but harder for others to use? Somehow that doesn't seem like a good thing to me.

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2008.10.31 05:22:00 - [26]
 

Originally by: Amida Ta
Sorry, but I'm of the complete opposite opionion. Retrieving XML files via HTTP sure is available for basically every platform in existance. But other than that there are absolutely no other reasons to have it.


There's absolutely no reasons to not have it.
XML HTTP feeds supported widely, in many ways, you could even directly embed your character sheet in Excel book.

Quote:
Lets look at what we got:
A (meanwhile) quite large sets of APIs all producing completely non-standard and custom XML documents.


XML itself is a standard, and it is VERY strict one.

Quote:
Also some seem to have some sort of structuring that structure is not standardized either and differs slightly from file to file.


Minor and arguable issue.

Quote:
In consequence if you want to use the API today (and you do not just want to look at XML) your only chance is to HAND-Parse every type of XML-Response. This is completely insane.


There's tons of programs that doing it for you, either MS IE offers XML structure browser.
Moot point, especially with "insane" exaggeration.

Quote:
My library does that and it's quite large. And there are still more problems. There is no versioning for the XML-Files. Not long ago one of my apps broke because the XML was changed in a way that could not be detected by the parser...


If that was change in API format and you were not updated your program, it is your fault.

Quote:
And parsing XML is not a simple thing, even with libraries it requires substantial amount of boilerplate code.


Depends on library you are using, it could be as simple as one visible page of code to break it all into usable structure.
Or tons of unreadable crap of poorly connected chunks of hand magic.

Quote:
With WebServices you can autogenerate ALL of that boilerplate code. All responses are strictly standardized. It is 'impossible' to have structural error (like the ones we have now). Granted, not every development environment offers full support for that, but those that do make the usage of one of the APIs a matter of *seconds* even for the huge API Interfaces (like Charactersheet).


These "WebServices" and other known schematics adding their own overheads to the data, so I can't see much disserence between these two ways.
You either using your own overhead to parse "raw" data, or you use someone else's overhead to parse data specially bloated for that overhead. All-in-all, result is the same.

Amida Ta
German Mining and Manufacture Corp.
Posted - 2008.10.31 15:18:00 - [27]
 

Originally by: Tonto Auri
Originally by: Amida Ta
Sorry, but I'm of the complete opposite opionion. Retrieving XML files via HTTP sure is available for basically every platform in existance. But other than that there are absolutely no other reasons to have it.


There's absolutely no reasons to not have it.
XML HTTP feeds supported widely, in many ways, you could even directly embed your character sheet in Excel book.


No I couldn't. The reason is very simple: Plain XML does not offer enough information to correctly do it (It might for some but in general it doesn't). If I tried to put it in Excel e.g. all decimal values would be wrong because by german Excel expects a ',' as decimal separator and would just ignore the existing '.'.

Originally by: Tonto Auri

Quote:
Lets look at what we got:
A (meanwhile) quite large sets of APIs all producing completely non-standard and custom XML documents.


XML itself is a standard, and it is VERY strict one.


Sure but the structure of the concrete APIs is completely non-standardized. By default in XML everything is text, but for a lot of reasons (e.g. see above) that is not sufficient. Sure having a XSD/DTD for the current XML-files would help a lot, too. But with a web-service you get that guaranteed - and for free.

Originally by: Tonto Auri

Quote:
Also some seem to have some sort of structuring that structure is not standardized either and differs slightly from file to file.


Minor and arguable issue.


If you want to parse that for an application it is surely neither minor nor arguable. For SOME usecases you are of course correct.

Originally by: Tonto Auri

Quote:
In consequence if you want to use the API today (and you do not just want to look at XML) your only chance is to HAND-Parse every type of XML-Response. This is completely insane.


There's tons of programs that doing it for you, either MS IE offers XML structure browser.
Moot point, especially with "insane" exaggeration.


Imho it's not really an option for most people to look at XML in plane text. Even when node-structured plain text.

Originally by: Tonto Auri

Quote:
My library does that and it's quite large. And there are still more problems. There is no versioning for the XML-Files. Not long ago one of my apps broke because the XML was changed in a way that could not be detected by the parser...


If that was change in API format and you were not updated your program, it is your fault.


And why? If the API had a standard way to tell that It had changed then my app could just have detected that and told that the current API is incompatible. THEN it would have been my fault. But in the current situation with no versioning there is just no viable programmatic way to detect any possible changes.

Originally by: Tonto Auri

Quote:
And parsing XML is not a simple thing, even with libraries it requires substantial amount of boilerplate code.


Depends on library you are using, it could be as simple as one visible page of code to break it all into usable structure.
Or tons of unreadable crap of poorly connected chunks of hand magic.


Well I have been thinking about OO languages here. Obviously just getting the structure of an XML document is not really much of an efford (thats what XML is good for). Connecting to other backends (like OO) however is.

Ix Forres
Caldari
Righteous Chaps
Posted - 2008.11.01 03:34:00 - [28]
 

Originally by: Amida Ta
words


Sorry, but A) Your point on XML/SOAP human-readability is completely moot. SOAP is a far more complex format to read as a human compared to XML. B) your point on OO backends somehow being 'incompatible' with XML? What? Seriously? Working great over here in an entirely object-oriented manner. Each API section is an object, with objects (rows in a transaction log for example) within it. XML can be easily abstracted as such without needing to do anything fancy in the vast majority of OO languages; Python, Ruby, Perl to name a few. More complex selection can be done easily with Xpath. Abstraction of datatypes is not an insurmountable issue and can easily be done through a variety of methods. Again, not a lot of extra code needed, and can be done library-level. See Reve for an example of an excellent API library which does just this.

Yes, you get some extra 'free code' with SOAP/WSDL. By adding in a whole crapload of cruft that is unneeded by an XML-bourne solution and which prevents use and interoperability of the API by certain consumers, which to my mind just confuses the smeg out of me. Why on earth would limiting the API to certain languages which happen to have SOAP libraries be a good idea?

You make one or two solid points; yes, the API needs some versioning, and yes, some more regularity in the formatting of the EVE API XML files would be lovely (Bonus points for DTD/XSDs, CCP!). However, reimplementing the API from scratch as a web service is a huge mistake, an unneeded move, one which will only lead to lessened adoption of the API by developers and further alienation of developers. I threw this thread at #eve-dev and the response was a unanimous 'wtf' from those awake at the time; imho CCP need to be working on improving the API, not doing yet more things that give no concrete benefit to developers on either side of the fence and which, in their creation, push 3rd party devs further away from the CCP development team working on the API. Real suggestions for real feature additions, bugfixes, and so on are rife; as Amida says, there's a lot of the API which doesn't meld to one specific standard. How 'bout fixing that for starters? It'd be a lot more useful, in my opinion, than a web service replacement.

- Summer Glau

Earl Comstock
Science and Trade Institute
Posted - 2008.11.02 03:35:00 - [29]
 

I coded up an API puller that can read, parse, and import into a database nearly every XML file that CCP has made available -- in an afternoon. (Postgres, perl, XML::Twig, LWP::UserAgent). I'm not sure what all the hubbub is about, but more to the point..

What the heck is this "Web Services" you're going on about? That's an awfully vague term that doesn't imply any sort of technology or transport, and every web services protocol stack that I've ever used has used XML as the message container.. please explain?

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.11.02 06:04:00 - [30]
 

Here's a link to wikipedia which seems to cover it:
Web_service

And yes if you read that you'll notice that REST is one type like I pointed out in one of my posts above. The others that new comers to web programming usually think of as web services are SOAP/WSDL, and XML-RPC because some new web libraries promote their use. They both have more overhead and take more code either in the library you use or your own code to use than REST. Personally it's never made sense to use http just to transport a bunch of xml format requests and errors back and forth when you can just POST request and get what you want in the response or an HTTP error code which can have a custom meaning if you want like CCP did with API. Just don't see the point of changing from something everyone understands and already has code for to something new myself with more overhead for both the server and users. As it says in the wikipedia:

Quote:
More recently, RESTful Web services have been regaining popularity, particularly with Internet companies.


One of the reason is lower server loading with less traffic between the servers and hosts which also makes me wonder why they want to change it other than improve the code and what we all seem to want which is some clean up of the output and maybe a dtd which would help them and us.


Pages: [1] 2

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only