open All Channels
seplocked EVE Technology Lab
blankseplocked Asset CacheTime
 
This thread is older than 90 days and has been locked due to inactivity.


 
Author Topic

Myles Kantor
Posted - 2008.12.06 12:03:00 - [1]
 

hi,

i'm building a site for assets and just saw that i can only recive data once a day ( 23h )
i think this should be reduced to 1hour to bring it to a better useabillity.

i think there are other lists that can be set higher to keep the perfomence stable.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.06 21:01:00 - [2]
 

Somebody from CCP please correctly if I'm wrong but the reason behind once a day on AssetList is it's done only during DT because you can't get accurate one when character is buying, selling and move stuff around without a lot of server load. Even in game it only updates it once every 5 mins or so if I remember the warning it gives right. With their current server structure as I understand it to find out the information it may having to poll every system node in eve to gather the information Shocked That not something they can be doing for the API all the time. I would like to see the cache time set shorter myself even if it would just serve same old data say once an hour. That would make working out caching and other issues much easier. Also would help people use it when they have multiple apps trying to get same API data (Bad idea BTW) without any proxy.

Myles Kantor
Posted - 2008.12.08 19:08:00 - [3]
 

I don't know exactly how the API Interface is working on CCP's Server, but i don't think that CCP will generate the data for all charakter during the DT.
I think there is a script that will activated when i make a request and collect the data.
If the Assets are listed on every node you only have an reactiontime from some ms that will not make an effect on the server perfomance. It the same routine when you load your assets ingame and thats fast.

and btw i don't need the alliance list updated every hour ;)

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.08 20:46:00 - [4]
 

No sure they don't either but could be extracting it from the backup they are making at that time or something and not the live data.

And it wasn't that long ago that it would often take a couple minutes for you're assets in-game to appear. They seem to have improved it a lot but I'll still bet it's an expense query to run on the database plus they may have other reasons for not making it available more often.

Also wasn't that long ago that 1000 people trying to be in same system talking on local and doing a few market transactions could bring eve to it's knees let alone what 1000s (10000s) of outside apps trying to access every stray unit of velspar a character or corporation has floating around in a forgotten hanger somewhere could have on Eve. One thing with the API is it's not as self limiting like people playing Eve where when lag starts getting bad some people usually decide to log off and do something else and most of them are on for only few hours a day anyway. Most API programs are on 24/7 so there isn't a slow time for the API.

Myles Kantor
Posted - 2008.12.08 22:37:00 - [5]
 

would be interesting if a dev can tell something about it ;)

and btw. i don't think that 1000 people are using the assets outside from eve very often, but when they use it they want actually data ;)

Tonto Auri
Vhero' Multipurpose Corp
Posted - 2008.12.08 22:52:00 - [6]
 

Originally by: Myles Kantor
I don't know exactly how the API Interface is working on CCP's Server, but i don't think that CCP will generate the data for all charakter during the DT.


It is made exactly that way... unfortunatelly :/
All data generated by cron jobs and pushed out of live servers to the web.
Or, like Assets data, generated during server startup.

CCP Lingorm


C C P
Posted - 2008.12.09 12:47:00 - [7]
 

When you make your first call for you assets via the API Server, it queries the Tranquility DB for your Current Asset list and retrieves it, the API Server then records the Date and Time of the request and stores that locally.

If you make another request to your asset list via the API you will be told that your Cache is exhausted and nothing will be returned until after 23hours have passed since your last request.

The information is not generated at Downtime and then used for the rest of the day.

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.

Once this is in place we will be able to look at reducing the Cache timers on some of the API's. Until that point we will not be reducing the Cache timers due to load considerations.

Verite Rendition
Caldari
F.R.E.E. Explorer
EVE Animal Control
Posted - 2008.12.09 15:18:00 - [8]
 

Originally by: CCP Lingorm

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.
I can haz sovereignty.xml sooner?Shocked

CCP Lingorm


C C P
Posted - 2008.12.09 15:33:00 - [9]
 

Originally by: Verite Rendition
Originally by: CCP Lingorm

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.
I can haz sovereignty.xml sooner?Shocked


If we get it working correctly ... then maybe ....

Ki Tarra
Ki Tech Industries
Posted - 2008.12.09 16:56:00 - [10]
 

Originally by: CCP Lingorm
Originally by: Verite Rendition
Originally by: CCP Lingorm

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.
I can haz sovereignty.xml sooner?Shocked


If we get it working correctly ... then maybe ....

Isn't sovereignty information only updated/calculated on TQ at downtime: how could it make sense to get it "sooner" than immeadiately after downtime, or have it update more than once per day? Confused

Obviously most of the API's would benefit from the improved caching: looking forward to seeing it implemented. Cool

Myles Kantor
Posted - 2008.12.09 21:42:00 - [11]
 

Originally by: CCP Lingorm
When you make your first call for you assets via the API Server, it queries the Tranquility DB for your Current Asset list and retrieves it, the API Server then records the Date and Time of the request and stores that locally.

If you make another request to your asset list via the API you will be told that your Cache is exhausted and nothing will be returned until after 23hours have passed since your last request.

The information is not generated at Downtime and then used for the rest of the day.

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.

Once this is in place we will be able to look at reducing the Cache timers on some of the API's. Until that point we will not be reducing the Cache timers due to load considerations.



thanks for these information ...
but i don't know where the problem is.... the request time for the query is the same as ingame and could not need that many ressources.
and what is the different between ingame and outgame? i can only retrieve data if i have paid for the account.
And when i'm ingame i only need that outgame feature if i need it while i have better functions outgame that i need for ingame.
so it is my right to retrieve nearly ingame values ;)
or another way is to bring more functions to it ingame (search and sort) ;)
oh and plz don't see it as flaming :D


to the end could you tell us eta to this new technolgy? 1 month, 1 year or whatever? ;)

greetings
Myles

Verite Rendition
Caldari
F.R.E.E. Explorer
EVE Animal Control
Posted - 2008.12.10 01:53:00 - [12]
 

Edited by: Verite Rendition on 10/12/2008 01:54:43
Originally by: Ki Tarra
Originally by: CCP Lingorm
Originally by: Verite Rendition
Originally by: CCP Lingorm

As mentioned at Fanfest we are currently examining Databse replication technology that will allow us to replicate parts of the TQ DB to a read only API db so that we can take the request load away from the TQ DB Server.
I can haz sovereignty.xml sooner?Shocked


If we get it working correctly ... then maybe ....

Isn't sovereignty information only updated/calculated on TQ at downtime: how could it make sense to get it "sooner" than immeadiately after downtime, or have it update more than once per day? Confused
Sovereignty.xml is on a 6 hour cache timer. Depending on how the cache mechanism is feeling when you hit it up, it quite often delivers the cache data from between 9:00 and 11:00 when you query it after downtime. I effectively have to wait 1 whole cache timer window (6 hours) before I can get the appropriate data to draw the day's map.

If Lingorm can make it so that I can access the new data immediately after DT, he'll be my new hero.Smile

CCP Lingorm


C C P
Posted - 2008.12.10 08:47:00 - [13]
 

Well it would not be me that fixes it, as I am finishing up my involvement in the API project before I head off back to New Zealand.

But I have put all your requests onto the Product Owner for the API (CCP Whisper) and to the Developer for it (CCP Elerhino) and they have plans to keep the API a big part of EVE.

Verite Rendition
Caldari
F.R.E.E. Explorer
EVE Animal Control
Posted - 2008.12.10 15:53:00 - [14]
 

Originally by: CCP Lingorm
Well it would not be me that fixes it, as I am finishing up my involvement in the API project before I head off back to New Zealand.

But I have put all your requests onto the Product Owner for the API (CCP Whisper) and to the Developer for it (CCP Elerhino) and they have plans to keep the API a big part of EVE.

Aww, not again. What is it about the API that drives away people after less than a year?Sad

CCP Lingorm


C C P
Posted - 2008.12.11 09:24:00 - [15]
 

Sorry, but needs must.

Elerhino is a good Guy and Whisper has a better chance of driving the API direction through, I just provided techy architecture direction.



 

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only