open All Channels
seplocked EVE Technology Lab
blankseplocked Yapeal PHP API library (revision 1203 branches/keys/)
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: first : previous : 1 2 [3] 4 5 6 7 8 9 ... : last (24)

Author Topic

Iskandara Cho
Posted - 2008.12.29 21:35:00 - [61]
 

Edited by: Iskandara Cho on 29/12/2008 21:54:37
If these don't work I will resort to carrier pigeon...

First pass at documentation

Sample single db .ini

I pulled the latest files from subversion, reuploaded them, and again tried running the script from the command line with the same result.

The line in question (line 101 of the current file) reads:

$cachetypes = array('tableName' => 'C', 'ownerID' => 'I', 'cachedUntil' => 'T');

is it possible that something in my .ini file is causing this error?

For that matter, I am not sure I am calling the script properly.. what should my command line read for proper execution?

Thanks!

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.29 22:59:00 - [62]
 

Ok I now can see the files Very Happy and I'm looking through them as I'm typing here.

You can use the lines from my crontab examples in the CrontabSetup wiki page by just not having the '*'s in front of it. So
/path/to/yapeal/backend/eve-api-pull.php
should work on most systems. What I like to do is
cd /path/to/yapeal/
then run it with
backend/eve-api-pull.php
as that also let's me do things like
dir cache/log/,
vim -R cache/log/Yapeal_error.log,
vim config/yapeal.ini,
etc which makes my life easier during debugging.

So far the files look good though might want to check them with spell check. I also noticed you are setting the [DATABASE-UTIL] section to use another DB in your single DB sample which isn't a true single DB setup. After I've looked through them a bit more I'll probably add them to the subversion somewhere and make any changes to them I think of so you can take a look and see what you think.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.29 23:38:00 - [63]
 

Ok I added the yepeal_ini_docs.txt to trunk in config directory for now. Have a look and see what you think.

I missed where you asked if a bad ini file could cause errors and it could but most of them should be caught as it is read by inc/common_backend.inc

Mintoko
Gallente
Taedium In Perpetuam
Posted - 2008.12.30 01:46:00 - [64]
 

Originally by: Dragonaire
Mintoko - I've been looking at adding some kind of prefix to all the table names because some of the API re-use the same names in the rowsets but with different columns (fields) Razz An example would be 'skills' in both the CharacterSheet and SkillTree. To add any kind of prefix though will require editing most the SQL queries in Yapeal since most places the table names are hard coded into the SQL Razz Probably a bit of lack of foresight on my part but something that often gets overlooked when try to get something working.

I'm working on some changes to Yapeal that should make adding more APIs in the future easier and as part of that I plan on adding some kind of prefixing system so it's less of a problem. I'm not sure when I'll release these changes since it's very much in an early stage of development and I haven't even got any running code yet so best I can say now is it's on the list of things I plan to do Smile

If you decide you can't wait and go ahead on your own and make something hopefully you'll share it back with us and maybe I'll be able to add it into Yapeal for everyone else that might have the same need.


I certainly will....I almost had it completely working, but made a change that crapped up the whole thing. I made two fresh attempts and think that I may be successful on the third attempt. I'll let you know.

Iskandara Cho
Posted - 2008.12.30 07:21:00 - [65]
 

Glad you found the docs useful.

I've pulled the backend file again, checked the ini file thrice, and I'm still getting the error about an unexpected { on line 101...

Fishweasel
The Council
Posted - 2008.12.30 13:52:00 - [66]
 

I was hoping to get a look at this, however i cant find any files on the yapeal pages, am i missing something ?

Fishweasel

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.30 16:08:00 - [67]
 

Fishweasel - To download the code you need a Subversion client program which there's a link to a list of them on the Source tab. I use TortoiseSVN for Windows myself as I like the way it integrate with Windows Explorer but there are several others to chose from. I just use the normal svn commands with Linux and sometime on Windows too when it's easier then TortoiseSVN.

If you just want to have a look at the code on-line you can click on the Browse link in the upper left corner and have a look around.

At some point I may start releasing Yapeal as zip or tar.bz archives but it's changing sometimes several times a day (360+ revisions in last 2 months) that it's not worth the extra effort right now when getting started with Subversion isn't that hard even for someone that hasn't seen it before.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.30 16:58:00 - [68]
 

Iskandara Cho - Either still have an unintentional edit somewhere or a file got delete somehow. At that point Yapeal have only ran the stuff in inc/common_backend.inc and a few require_once on some of the error/exception code from there and I've double check all of them both in my editor and by running it on my test server over night and I'm not seeing any errors Sad You might try backing up your config/yapeal.ini file and delete everything where you have it installed and try re-installing the whole thing again. If that doesn't fix it it'll have to be something in your yapeal.ini and we'll have to figure out how you managed to get that error from a bad config file Shocked and see what can be done to catch it before it can do that.

Iskandara Cho
Posted - 2008.12.31 11:58:00 - [69]
 

No joy Sad

I've completely deleted the installation on my web server, updated from SVN, deleted the databases, ans started from scratch.

1) Uploaded files to server
2) edited ini file to new db info
3) ran script (php eve-api-pull.php)
4) Get an unexpected { on line 101 error

Question

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2008.12.31 15:36:00 - [70]
 

Very strange because I just did same thing and have no problems Sad
I'll sent you an e-mail via eve-mail and have you send me you're config file and I'll see if there's something there that I see and maybe try to get with you in eve or on IM too.

Ragga daPhunk
Order Of Lyon
Posted - 2009.01.04 17:13:00 - [71]
 

Edited by: Ragga daPhunk on 04/01/2009 17:56:03
greetings!

just discovered your project and i really like it.. but it doesn't work for me :( the only thing that gets updated in the database is the "CachedUntil" table but nothing else.
i enabled all logging and tracing but it doesn't come up with an error.

trace.log says:
[2009-01-04 17:06:13.530]
API: Before connect for eve-api-pull in eve-api-pull.php
DATABASE: Before connect for mysqli://eveapi:narf@localhost/eveapi?new in common_db.inc
API: Before dontWait
DATABASE: Before connect for mysqli://eveapi:narf@localhost/eveapi?new in common_db.inc


notice.log says:
[2009-01-04 17:06:22.042] Tired of waiting! Getting eve-api-pull for 0
[2009-01-04 17:06:22.048] Tired of waiting! Getting CharacterSheet for 8xxxxx


database connection seems ok since the cacheduntil table gets updated and stuff..
when it fetches the api it takes some seconds.. so somethings is done.. but what?
i enabled caching of the xml files - but the cache folders are empty.
curl support is compiled in btw.

PoshaX
Posted - 2009.01.04 23:34:00 - [72]
 

I haven't been able to figure out why, but I continue to get this exception right after it says it is updating the character information.

[2009-01-04 23:25:55.289] Uncaught exception in eve-api-pull.php
[2009-01-04 23:25:55.289] EXCEPTION:
Code: 1
Message: Missing required fields (balance, bloodLine, characterID, cloneName, cloneSkillPoints, corporationID, corporationName, gender, name, race) found while making upsert for CharacterSheet
File: /apps/eve/yapeal/inc/common_db.inc
Line: 272
Backtrace:
#0 /apps/eve/yapeal/inc/common_db.inc(430): makeMultiUpsert(Array, Array, 'CharacterSheet', 'mysqli://root:w...')
#1 /apps/eve/yapeal/inc/pulls_char.inc(106): upsert(Array, Array, 'CharacterSheet', 'mysqli://root:w...')
#2 /apps/eve/yapeal/backend/eve-api-pull.php(158): require('/apps/eve/yapea...')
#3 {main}
--- END TRACE ---


Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.05 00:25:00 - [73]
 

Okay here's some tips on figuring out problems with Yapeal. First have a look at GettingStarted on the Yapeal wiki then check out the new TroubleShooting page I just made. It also might be useful to look at the other wiki pages while you're there. Hopefully after going through the wiki you'll be able to find the problem and it'll be something you can fix yourself, if not then you should have the information and log files to make my life and yours a bit easier while we try to figure out what's going wrong Smile

revision 398 now out.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.05 00:50:00 - [74]
 

I made a couple small changes to config/yapeal.ini so anyone using Yapeal should check out the example file.

For anyone having problems with Yapeal please make sure you're using the latest version and go through the pages I link in the post above hopefully by doing that you'll be able to find the problem yourself and not have to wait on me to check the forums.

Ragga daPhunk - Not sure what you're problem is if you'll try out troubleshooting page from wiki it should help you track it down if not I'll need you're log files with trace_level=2 you'll find the instructions about this in wiki.

PoshaX - Make sure to double check your configuration and the required stuff in all the RegisteredXxx tables I think there's something missing there. If after check those things you're still getting an exception I'll need the trace_level=2 log files from you too and try to figure out what's going on.

Iskandara Cho
Posted - 2009.01.05 02:20:00 - [75]
 

Still no joy. I've contacted my server provider to see if they can shed any light on the problem. I'll keep you posted.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.05 02:48:00 - [76]
 

Edited by: Dragonaire on 05/01/2009 02:50:37
Might also try sending me your logs Iskandara Cho. I never saw any from you after you made contact with me via E-mail etc. If you have any that is Razz

Iskandara Cho
Posted - 2009.01.05 09:21:00 - [77]
 

Nope, no logs, it never gets to the point of actually executing anything.

I've heard back from support, and they said there is nothing they can do with 3rd party scripts. Not unusual I guess, but still kinda frustrating. I've sent you an email with the url to the phpinfo for my server.

Thanks for the help.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.06 18:50:00 - [78]
 

Okay been working with Iskandara Cho on her problem and found something that might be useful for anyone else that having problem getting Yapeal running to check.

Assuming a Linux system from a command-line try:

php --version

If the output reads:

PHP 4.X.X blah blah blah
OR
PHP 5.1.X
OR anything else

Except:

PHP 5.2.X blah blah blah

where X is a number that's the reason it's not working for you. Some of you may have looked at the output from phpinfo as a webpage and seen that it says PHP 5.2.X but the Command-Line Interface version of PHP that Yapeal uses is a different file and on some hosting sites you end up with the php4 CLI and php5 for webpages ugh
In Iskandara Cho's case on DreamHost that was what was happening Rolling Eyes

If you are hosted at DreamHost and you know Linux command-line a little try their instructions Advanced_PHP_configuration to fix the problem by following the ones for php5. You don't need the stuff for FastCGI for Yapeal but you might for the other stuff you're doing on your site I can't help you decide on that part Yapeal should work with or without.

The other way to fix it is to make a small change to backend/eve-api-pull.php file but you will have to do this every time you upgrade Yapeal.

For DreamHost change the very first line of backend/eve-api-pull/php from:

#!/usr/bin/php

To:

#!/usr/local/php5/bin/php

I'm going to add a page to the Yapeal wiki where I can add links and/or instructions for others but until then if you have this problem on another hosting service and figure out how to fix it please post it here and I'll add it to the wiki when I get page setup.

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.07 08:40:00 - [79]
 

New revision 421 out it should fix bug where Yapeal wouldn't run if trace=FALSE in [Tracing] section of config/yapeal.ini Embarassed

Yapeal project home http://code.google.com/p/yapeal/

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.07 19:13:00 - [80]
 

Ok decided to get some more input from people using Yapeal or other interested parties on things they'd like to see changed or added to it.

Here's a list of things people have already told me about:

  • Some kind of install/setup script

  • Some thing to help with managing the RegisteredUser/RegisteredCharacter/RegisteredCorporation stuff

  • Better documentation



Anything else? Also what other APIs would people like to see added to it next?
I'd like to get a little input on this and then I might even try making/adding some kind of development roadmap on wiki so people have something else to complain about on the forums when I don't always follow it Twisted Evil

I've already got one person that offered to take a stab at making an install script while I keep working on other stuff but I sure he wouldn't mind some help with it. For that matter if anyone out there has a programming itch and you know PHP or want to learn it I'd be happy to have the help that how Yapeal got started for me.Smile

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.08 20:17:00 - [81]
 

Hi all been working with PoshaX? about the missing required fields problem in CharacterSheet and seem to have tracked it down to a bug in old versions of php5 on some Linux systems. In this case happens to be Debian Etch.

-- Rant On --
Etch was released on December 18th, 2008 but bug was fixed in PHP Version 5.2.1 back in 8-Feb-2007 but apparently nearly 2 years isn't enough lead time for them to include itRazz or a couple months isn't enough time to release upgrade to even Version 5.2.6 which came out in 1-May-2008 which fixed several security problems etc Razz For those interested the newest release is 5.2.8 which fix a bug that slipped into 5.2.7 Embarassed or if you're a little more daring they have PHP 5.3 out which backports some stuff from PHP6 as I understand it. I've already had an official update to 5.2.8 available for my old OpenSuSE 10.3 for a while now Rolling Eyes
-- Rant Off --

Anyone using version 5.2.1 or newer of PHP shouldn't see the problem since that's when they fixed the bug. I've got a work around for the problem ready to be tested as soon as I get a chance to get back with PoshaX that I'll release if it works. BTW there's is a similar problem with CorporationSheet that my work around should fix also Razz

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.09 01:25:00 - [82]
 

Hi all me again been doing some cleanup up and move stuff around in Yapeal to streamline it a bit and make it easier for people to use. Some of the things I changed:

  • Moved the Docs directory out of the project directories in trunk and all the branches and put them into their own top level directory. I never seem to have any time to update them so no need confusing people with having outdated information being included. Also makes Yapeal a little faster to download and easier to integrate for people that don't want or need to include them with their projects.

  • Added new admin top level directory. I'm looking at maybe having a download/installer setup for people that don't or can't use subversion and I need to have a place outside trunk or the branches to keep the config files for it plus a few other misc stuff that isn't needed to use Yapeal but I need while working on it. So you might notice a few more things migrating over there as I run across them.

  • Finally I found some nice little graphics again that someone made for me to use with Yapeal a while back and I decided might be nice for me to include them for people to use in their own projects etc. If you want to include one of them on your website with link back to Yapeal's website that would be great but only do it if you feel like it. I'm happy if you just let me know with post on forum or an E-mail that you like it and are using it.


That's it for this post I think fly safe.

ThyShiva
Posted - 2009.01.09 20:09:00 - [83]
 

Hello,

I have not posted in a while as I have been intergrating Yapeal with my project. I would like to second Mintoko's request for a table prefix as many of us have database limits or just like to group tables within databases.

Keep up the good Dragonaire.

Shiva.

Satis Iqulenax
Amarr
Free Imperial Vikings
Posted - 2009.01.10 12:36:00 - [84]
 

Edited by: Satis Iqulenax on 10/01/2009 18:10:13
Edited by: Satis Iqulenax on 10/01/2009 18:09:41
Hi Dragonaire
I have been using another API function for retrieving all my corp info and character info.

After have installed this script for testing purpos, I can now allready say that it will make my site mutch faster since it don't need to pulle the XML files each time a page need to be showen and putting it in a database just make it faster and mutch easyer to sort the the content. :)

I will start to implement this into my website now.

One thing I was wondering about.
Once you updating an assetlist, will this script delete those info that have been used or sold or related to that ??

EDIT:
In the svn folder.... it's the trunc you need to download right ??
Just to be sure


Kind Regards
Satis

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.10 18:15:00 - [85]
 

Hi Satis Iqulenax glad you like Yapeal.

Quote:
Once you updating an assetlist, will this script delete those info that have been used or sold or related to that ??

No it doesn't try to make any changes to the AssetList or any of the other API data really for that matter. I don't see that being part of the job it's trying to do. Yapeal tries to just take what it gets from the Eve API and put it into the DB in what I hope is a sensible way.Wink To do that and be usable by multiple characters and corporations it does add a few extra columns like ownerID, accountKey, etc to the tables but I never take stuff away I leave that up to you as the application developers to decide what you want to do.

Just a couple things to think about if you try to update the AssetList yourself is that the itemIDs aren't constant in many cases. For example if you split a stack of ammo to use, sell, or just move part of a stack the old itemID is dropped and Eve makes 2 new IDs for the new stacks. The same thing happens if you repackage/assemble something it gets a new itemID. There are other times when it can change also I believe but I think you can see the problems with trying to update stuff as things change. Add to that that the next time Yapeal does get data from the API any of your change to Yapeal's AssetList table will be wiped and update with the data from the API. Hope that helps you if you try to do it yourself.

Also for anyone else looking at Yapeal as a way to have a full archive of all of the Eve API data keep in mind that things like ServerStatus, AssetList, etc are thrown away and replaced in effect. I did this because I felt having it act a snapshot of the in game data was what most people would be interested in and what the APIs were made to allow. I am looking at adding an ability to directly cache the XML into the DB as received and may add an ability to have it archive that data but it would be done as a single large text column not as processed data with multiple columns. I'm also looking at having all the log files optionally/additionally put into the DB but have to be careful not to get into infinite loops doing that Wink Note that by their nature the WalletJournals and WalletTransactions are an archive and the IndustryJobs and MarketOrders form a history of old records because they are updated with the API data until it stops sending each record.

Satis Iqulenax
Amarr
Free Imperial Vikings
Posted - 2009.01.10 20:36:00 - [86]
 

Originally by: Dragonaire
Hi Satis Iqulenax glad you like Yapeal.

Quote:
Once you updating an assetlist, will this script delete those info that have been used or sold or related to that ??

No it doesn't try to make any changes to the AssetList or any of the other API data really for that matter. I don't see that being part of the job it's trying to do. Yapeal tries to just take what it gets from the Eve API and put it into the DB in what I hope is a sensible way.Wink To do that and be usable by multiple characters and corporations it does add a few extra columns like ownerID, accountKey, etc to the tables but I never take stuff away I leave that up to you as the application developers to decide what you want to do.


So what you are saying is:
If you have an Strip Miner in your posession and Yapeal is catching it and addind that on to the database, then if you sell the strip miner so it's not in your possession anymore, Yapeal just ignor this and let that item stay in the database ??

If it does that, you will constantly adding items to the database and never clean up those items that an user don't have in hes/hers posession anymore and can still be showen on the webpage forever.
This will allso overfloat the DB with data it does not need.

Look at it in my way.
If I as a webdeveloper need to check those tables for expired items I would need to do somthing simular as you do. Catch the XML and check to see if this item is stille there. that will only make you need to get the XML file 2 times. one to add old new data as Yapeal and then one to delete the old data. since Yapeal don't do this at the same time.

One solution that I can think of is when the backend is doing it's work, then for each character or corp, it should delete all the data from that character or corp and then insert the new data. that should not take anylonger that the methode you are doing now.

Hope you understand what I'm saying.Very Happy
I'm just a littel confused by this data dump Wink

Kind Regards
Satis

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.10 22:15:00 - [87]
 

Quote:
Also for anyone else looking at Yapeal as a way to have a full archive of all of the Eve API data keep in mind that things like ServerStatus, AssetList, etc are thrown away and replaced in effect. I did this because I felt having it act a snapshot of the in game data was what most people would be interested in and what the APIs were made to allow.


You need to read the full post. But the way I worded it could be misunderstood so I'll try to rephrase it to be more clear. For all the APIs under eve, and server, they will have all their records truncated and then replaced by the new data pulled from the API. In tables like AssetList all records on a per ownerID basis well be deleted and replaced with the new data from API. For tables like WalletJournal, WalletTransactions, etc because of their nature they will just be added to. In MarketOrders and IndustryJobs they will be update by the new data until it stops changing then they do the same as WalletJournal, etc and keep a record of the last update received. This let's you look up old records if you want something that you can't do in game.
Here an example for a market order. You make a sell order for some ammo as people buy it from you the record is update with the amount left to sell in volRemaining. Now the API may return a record where orderState=2 meaning it has either expired or someone bought it all up before it ran out. You'll have to check if volRemaining=0 to tell if it was bought out. We will continue to receive it in more updates from the API for a while because of how the API cache works but since our order doesn't change anymore it's basically ignored at this point. just like updates to unchanging records for WalletJournal would be. The record for your order will stay around until you decide to delete it.

Hopeful this makes it more clear to you how Yapeal does things.

Ramov Tinoga
Tinoga Enterprises
Posted - 2009.01.11 00:00:00 - [88]
 

Originally by: Ragga daPhunk
the only thing that gets updated in the database is the "CachedUntil" table but nothing else.


When I run the script manually once a day, everything is updated quite fine.

When its run by cron every five minutes (*/5), I noticed that only the CachedUntil table is updated, but none of the other tables.
My server is running at GMT+1, so the times in the table and the local time are at different time zones. Could this cause the problem?

Satis Iqulenax
Amarr
Free Imperial Vikings
Posted - 2009.01.11 00:22:00 - [89]
 

Originally by: Dragonaire
You need to read the full post. But the way I worded it could be misunderstood so I'll try to rephrase it to be more clear. For all the APIs under eve, and server, they will have all their records truncated and then replaced by the new data pulled from the API. In tables like AssetList all records on a per ownerID basis well be deleted and replaced with the new data from API. For tables like WalletJournal, WalletTransactions, etc because of their nature they will just be added to. In MarketOrders and IndustryJobs they will be update by the new data until it stops changing then they do the same as WalletJournal, etc and keep a record of the last update received. This let's you look up old records if you want something that you can't do in game.
Here an example for a market order. You make a sell order for some ammo as people buy it from you the record is update with the amount left to sell in volRemaining. Now the API may return a record where orderState=2 meaning it has either expired or someone bought it all up before it ran out. You'll have to check if volRemaining=0 to tell if it was bought out. We will continue to receive it in more updates from the API for a while because of how the API cache works but since our order doesn't change anymore it's basically ignored at this point. just like updates to unchanging records for WalletJournal would be. The record for your order will stay around until you decide to delete it.

Hopeful this makes it more clear to you how Yapeal does things.


Yes it did. I allso did read the full post of yours and I did misunderstand it Embarassed

But now that this is cleared I feal happy since it's working as I was hoping it would Very Happy

New Question:
From: http://games.chruker.dk/eve_online/test_eve_api.php
there is an list of all the API data you can collect.
Are you planing to implement all of them ??

Kind Regards
Satis

Dragonaire
Caldari
Corax.
PURgE Alliance
Posted - 2009.01.11 01:07:00 - [90]
 

Glad to hear I was clearer on my second try. Smile

At some point I would like to see Yapeal do all the APIs and I plan on making some changes to how the code works so it's easier then it is now to add more APIs. I'm hoping to get it easy enough that I'm not the only one that can seems to understand it well enough to add more Wink

BTW I like what Chruker has done on his site and sometimes use it but I find the tables at http://wiki.eve-id.net/APIv2_Page_Index more useful for what I do.


Pages: first : previous : 1 2 [3] 4 5 6 7 8 9 ... : last (24)

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only