open All Channels
seplocked EVE Information Portal
blankseplocked New Dev Blog: Power to the People...
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: 1 2 3 4 5 6 [7]

Author Topic

Vaerah Vahrokha
Minmatar
Vahrokh Consulting
Posted - 2011.09.01 06:36:00 - [181]
 

Edited by: Vaerah Vahrokha on 01/09/2011 06:39:56
Edited by: Vaerah Vahrokha on 01/09/2011 06:38:32
Originally by: Somatic Neuron
Just a few things here....first and foremost...this would have helped me a lot more several years ago...but, it's about time ;)

Second, XML is the way to go on this. Not CSV nor SQL. Standard XML rowsets would be fine and dandy.

Third, API calls, yes, please. Previous day's generated during downtime....all others available on a per call basis (they should be cached results, so cheap and easy...) pass in a date, and optionally a region. Probably the best would be another API call, that you could pass in an itemID, date range and optionally a regionID, to obtain only the items you are interested in....just brainstorming here...


The whole world back ends (those that really HAVE to work) are still running on good ole comma / tab delimited files. XML is bloated, requires more work before extracting the data and is more useful for frequently changing schemas than a dump that is the same since 8 years ago.

Also, to the guy who said MS SQL has a free edition: yes but it pumps so MUCH garbage in the computer it's awful. It's so awfully bloated and large for a task (importing EvE files) you could easily do with SQLite, that happens to be faster, smaller, portable AND really free.

As 3rd party EvE app developer, I will never ever accept to install MS SQL or to write an EvE application requiring such software to work. The guy who did so (EMMA EvE trading software) got so many belly aches you have just to read its troubled history on MD forum to learn how bad it is. I also maintained EMMA for a short while and indeed I basically had to reformat the computer that got infected with those many gigabytes of partially uninstallable garbage.

Kane Molou
Posted - 2011.09.01 08:38:00 - [182]
 

While I find this all nice et etc.. It gets me why CCP can't do something simular to what real life trading groups do both in and OUT of game...

In real life if i'm trading stock, shares, vast quanities of grain etc etc etc.. I go look at the current 'market' this market doesn't list off every purchase currently being done.... instead it shows me the various major market regions on our planet and the rolling average for the current trade price.. now this is typically delayed by about 15 - 30minutes.. no more..

Why is it delayed? Because it takes a little bit of time to run the data through a average(a1-a999999) to get done... But the point is that this data is then made avalible FREE to anyone in the Public Market.. be it Forex, Cattle trading, Grain Trading what ever and it's made avalible no matter WHERE in the world you are.

All this new data for eve is great yes but at the same time it's not it's majorly out dated and only helpful if you want to see the same info that a certain some one is meant to do for us anyway, you know the reason CCP employees him in part.. which is to give us historical market trends. In other words useless for the average trader, and actually even more useless really for the long term trader.. why? Well lets go and look at a few things:

first off we the players can only easily access our small regions market at any time so unless you have a bunch of alts etc or your using websites which may not even be up-2-date anyway.. your unable to do any real long term planning as a 'casual' trader, the ones which do offer you great instant access to data etc want isk or the like for it..

In this respect it's not like real life, which sucks.. I can't just go and get the data from the foreign exchange or the local bank or the local wheat board etc for say 4 regions away.. which is again stuipid heck apparently a region can't even talk to it's neighbour! Historical data is not going to help me here, because we all know that while Megathrons might have been selling for 100mil in Jita 3 months ago, they could be selling for 60mil now and have been for months anyway..

then there is the other reason this data is normally made public for historical viewing.. which is called investor relations and the like.. basically in real life I have this thing called the 'stock exchange' something eve obviously ONCE had in mind because we have the ability to make shares but then of course like everything it apparently was meant to be player run or something but we never got the tools to do it 'fairly' or the like.. and no one relly has the power to buy up enough stock to start one either.. but i'm degrading from the ponit ..

In real life, before a 'large' investor goes and buys stock in a company they would go and do their homework.. what homework is this? well normally they would look at what the company they are buying stock in produces or does.. so lets say Mining High Sec 4 Us. Now this company mines High Sec ores in *thinks* Genesis primarly Trit, Pyrite, Mex and Iso selling it to the market in this region.. I would as an investor look both at the 'market' for what they are selling, so i'd see if those minerals are going up, down or level in sales, then i would obviously look at the annual reports from the company and see if they made or lost money etc.. then based on all of this I'd go buy shares..

For something like this 3 month old data heck even day old data is FINE..

For Trading however, it is not.

As Dr EyjoG likes to point out Eve is a living Breathing Entity that tends to follow real life market trends.. it has orders being filled every minute of every day (minus down time).. so unless the data is avalible with only a 'small' delay like real life there is no point to it.

of course now i expect no answer to this forums die tonight again.

Meissa Anunthiel
Redshift Industrial
Rooks and Kings
Posted - 2011.09.01 08:50:00 - [183]
 

Stillman, Eyjo,

After much thought on the subject, while I still think the concerns I voiced are valid, I believe there will still be plenty of venues for casual traders to thrive in, mostly in the low-volume items and dangerous locations, and the advantages of "current" data outweigh the downsides. So go on and implement it with as short a delay as possible, even 1 hour :p, as long as there's no individual orders exposed and only average/median/volume/etc.

Hel O'Ween
Men On A Mission
EVE Trade Consortium
Posted - 2011.09.01 10:50:00 - [184]
 

Originally by: Monstress

You do know that databases that do support importing CSV files also support specifying custom delimiters, line endings, quotation and escape characters right?


Yes, I do know. But the typical user trying to open it in a spreadsheed not.

Bah, why do I still care at all?. My sub expires in December and that's it for me.

Lukas Rox
Torchwood Archive
Posted - 2011.09.01 11:37:00 - [185]
 

I have restored the backup and tried to link it to existing data from the usual data dump.

Unfortunately the values from column "stationID" in the Marketdata db don't match with values in the "staStations" table from Data Dump: JOIN between tables on stationID results in 0 records selected

The values from Marketdata.stationID are however similar to values from column staStations.corporationID

Is this a mistake in the backup or am I wrong here?

Pyus
Hand Of Midas
Posted - 2011.09.01 11:45:00 - [186]
 

Originally by: Lukas Rox
I have restored the backup and tried to link it to existing data from the usual data dump.

Unfortunately the values from column "stationID" in the Marketdata db don't match with values in the "staStations" table from Data Dump: JOIN between tables on stationID results in 0 records selected

The values from Marketdata.stationID are however similar to values from column staStations.corporationID

Is this a mistake in the backup or am I wrong here?


It's actually a region ID so do a join on tha and you should be good. All the data is per region info just like the in-game History tab when you're viewing an item.

Lukas Rox
Torchwood Archive
Posted - 2011.09.01 11:49:00 - [187]
 

Originally by: Pyus
Originally by: Lukas Rox
I have restored the backup and tried to link it to existing data from the usual data dump.

Unfortunately the values from column "stationID" in the Marketdata db don't match with values in the "staStations" table from Data Dump: JOIN between tables on stationID results in 0 records selected

The values from Marketdata.stationID are however similar to values from column staStations.corporationID

Is this a mistake in the backup or am I wrong here?


It's actually a region ID so do a join on tha and you should be good. All the data is per region info just like the in-game History tab when you're viewing an item.

yup, that works just fine. Column names are misleading :P

CCP Stillman

Posted - 2011.09.01 11:54:00 - [188]
 

Originally by: Lukas Rox
Originally by: Pyus
Originally by: Lukas Rox
I have restored the backup and tried to link it to existing data from the usual data dump.

Unfortunately the values from column "stationID" in the Marketdata db don't match with values in the "staStations" table from Data Dump: JOIN between tables on stationID results in 0 records selected

The values from Marketdata.stationID are however similar to values from column staStations.corporationID

Is this a mistake in the backup or am I wrong here?


It's actually a region ID so do a join on tha and you should be good. All the data is per region info just like the in-game History tab when you're viewing an item.

yup, that works just fine. Column names are misleading :P

Yep, I messed up. Blame me :D

Haguu
Caldari
TLA Ltd
Posted - 2011.09.01 14:15:00 - [189]
 

This is very exciting news and TYVM.

Originally by: Meissa Anunthiel
Stillman, Eyjo,

After much thought on the subject, while I still think the concerns I voiced are valid, I believe there will still be plenty of venues for casual traders to thrive in, mostly in the low-volume items and dangerous locations, and the advantages of "current" data outweigh the downsides. So go on and implement it with as short a delay as possible, even 1 hour :p, as long as there's no individual orders exposed and only average/median/volume/etc.


I am not as clear on the problems this would cause the casual traders. The professional traders, organizations, and b*ts currently have this information, and then some, via the cache. I thought the objection is that these players would prefer to continue to have an advantage over the masses? Once you read the cache, then you have complete control over it - upload to a website, send a TCP/mail message to another program/client to process, etc. My understanding is that you could have your child/secretary have their own account, they periodically create a cache your software reads and then contacts you IG or OOG that your attention is required. I could know I don't need to log in to babysit my orders because if there were an issue, the last time my secretary updated the cache my software would analyses the data, sense there is an exception condition, and contact me. Thankfully, this is EVE and nobody would ever violate the TOS and automate something based upon these cache values. Am I overlooking something? As soon as cache reading was allowed, it seems like the options the pros have now dwarf giving the casual player some sales data a few hour old?

-----

I know it is different data, but I really like eve-central and I think a lot of people rely on it, especially after the EM shutdown. If I were a CCP employee, it would worry me a bit that something a lot of my customers use is provided by one single individual that no longer plays the game. I.e., I think more data coming from CCP is a Good Thing.

-----

My opinion is that XML is probably the "proper" answer and CSV would be smaller, faster and more convenient. Obviously, JSON is absolutely forbidden since that is what Blizzard used for their market data downloads. But again, no amount of free MSSQL tools are going to help those of use not using windows or those against proprietary, and probably version-dependent, formats.

Zedia Zhane
Posted - 2011.09.01 15:31:00 - [190]
 

Edited by: Zedia Zhane on 01/09/2011 15:41:04
There are (or have been) several sites that give access to market data on a 24-hour delay. These sites provide APIs to allow people to download the data into their own applications for analysis.

I'm talking about Eve Central, Eve HQ, Eve Marketdata, and the now-defunct Eve Metrics sites.

What would be best is to stay as close as possible to existing data formats, so the existing applications can be re-used with minimal changes. What is needed is accurate data - those sites are somewhat notorious for occasionally having bad data in them. Actually... What would be best is providing an API so those sites can get accurate data and publish it via their already-exiting APIs, which the player base already uses.

As a side note, those site provide functionality for all Eve regions, with a 24-hour delay. I really don't see why CCP can't do the same. After all, those sites are written by players in their spare time, while CCP programmers are being paid to do this stuff.

EDIT: The big problem with those sites is that they currently rely on user-uploaded data. This is susceptible to numerous problems from corruption of individual data files to deliberate attempts at market manipulation by insertion of bad data. Having a common, reliable, CCP-provided source of accurate data would massively simplify the development and maintenance of such sites while making them more accurate for the end users.

Morar Santee
Posted - 2011.09.02 00:43:00 - [191]
 

Edited by: Morar Santee on 03/09/2011 04:12:17
Originally by: CCP Dr.Eyjog
Knowledge is power, and we want to give EVE Online pod pilots more power.

No **** bro? Then how about you give us the ability back to see other players' NPC standings? Just a thought.

hero honda
Posted - 2011.09.02 11:30:00 - [192]
 

Originally by: Morar Santee
Originally by: CCP Dr.Eyjog
Knowledge is power, and we want to give EVE Online pod pilots more power.

No **** bro? Then how about you give us the ability back to see character's standings? Just a thought.


Character sheet > standings.
Wink

Alice Cane
Posted - 2011.09.02 12:23:00 - [193]
 

Is there no way possible that we can get just plain old OHCL data?

The open and close could be just before and after downtime.

If we could get that and feed it into our own charting programs, well.

That would be awesome.

DrTritanium Merc
Posted - 2011.09.03 01:10:00 - [194]
 

Well they say nothing gets turned off. lol Right a few of my ships shields or armor stuff repairs or boosters were off line I had to play it out lucky in the field but okay if you think its fixed well?? Also going through most star gates the bubble goes backwards why?? let me know thanks. everyone else fly safe 0/

hero honda
Posted - 2011.09.03 04:09:00 - [195]
 

Originally by: Alice Cane
Is there no way possible that we can get just plain old OHCL data?

The open and close could be just before and after downtime.

If we could get that and feed it into our own charting programs, well.

That would be awesome.


Down time would be a nice time to compile a days worth of data, if there is time and space for it.

Besides I would not mind having candle stick charts for eve markets and you need the open and close for that.

Morar Santee
Posted - 2011.09.03 04:09:00 - [196]
 

Edited by: Morar Santee on 03/09/2011 04:09:28
Originally by: hero honda
Originally by: Morar Santee
Originally by: CCP Dr.Eyjog
Knowledge is power, and we want to give EVE Online pod pilots more power.

No **** bro? Then how about you give us the ability back to see character's standings? Just a thought.


Character sheet > standings.
Wink


Updated for "special" people.
Wink

hero honda
Posted - 2011.09.03 15:50:00 - [197]
 

Originally by: Morar Santee
Edited by: Morar Santee on 03/09/2011 04:09:28
Originally by: hero honda
Originally by: Morar Santee
Originally by: CCP Dr.Eyjog
Knowledge is power, and we want to give EVE Online pod pilots more power.

No **** bro? Then how about you give us the ability back to see character's standings? Just a thought.


Character sheet > standings.
Wink


Updated for "special" people.
Wink


Updated? that is where is was in 2004 and still there.

Morar Santee
Posted - 2011.09.04 04:02:00 - [198]
 

Seriously: Get a clue.

Evan183
Gallente
EVE University
Ivy League
Posted - 2011.09.06 01:42:00 - [199]
 

I believe that a MySQL data backup (basically standard SQL with a few differences) would work best for many people, as MySQL is very, very popular.

Dierdra Vaal
Caldari
Veto.
Veto Corp
Posted - 2011.09.06 13:02:00 - [200]
 

Edited by: Dierdra Vaal on 06/09/2011 13:01:58
do we discuss here or on the new forums?

https://forums.eveonline.com/default.aspx?g=posts&t=6288


Diomedes Calypso
Aetolian Armada
Posted - 2011.09.08 16:31:00 - [201]
 

Just to help me understand.

The figures will be fore average and median TRANSACTION prices I hope and absolutely ignore orders.

Orders are the price which no-one has yet sold or bought at, and are very frequently going to be innacurate both from manipulation and standard practice scam/oportunism market strategy (i.e shuttles for 8,888,888 , regional buys for 20 million implants at 1 isk etc).

Almost every item will have sell orders at 100 x the price just so people at the leading edge of the market will sometimes get a few hundred million windfall.

Simlar issues many items where the sellers maybe more patient than the buyers.. the buyers being retail customers, the seller's manufacturors .... I know from experience that the total volume of many items it far higher at one side of the bid ask spread than the other, and with wide bid ask spreads where that information is interesting.

tldr -- the information will be for actual transactions and ignore unexecuted buy and sell orders, won't it?


Pages: 1 2 3 4 5 6 [7]

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only