open All Channels
seplocked EVE Information Portal
blankseplocked QA Engineering Dev Blog - Empyrean Age
 
This thread is older than 90 days and has been locked due to inactivity.


 
Pages: [1] 2

Author Topic

CCP Taera

Posted - 2008.07.24 16:46:00 - [1]
 

With the addition of factional warfare and Black Rise, the first new region since the launch of EVE, Empyrean Age meant a ton of new content requiring in-depth testing from our QA team. Luckily, the guys in QA Engineering came up with some innovative methods to rise to the challenge. Read CCP Lingorm's new blog, QA Engineering - Empyrean Age, to learn about the new QA tools and how this will affect future expansions.

Matalino
Posted - 2008.07.24 17:01:00 - [2]
 

Nice blog.

It is kind of cool to hear about the bugs that we didn't need to find and report our selves. Razz

Batolemaeus
Caldari
Free-Space-Ranger
Morsus Mihi
Posted - 2008.07.24 17:14:00 - [3]
 

Very nice indeed, and much apreciated.
I love technical devblogs!

Kazuo Ishiguro
House of Marbles
Posted - 2008.07.24 17:21:00 - [4]
 

The load client sounds interesting; talking about that sort of thing, are there any plans to develop some sort of flight recorder, so that people who want to make videos can just record motions and actions the time and render everything afterwards, free of lag?

Chruker
Posted - 2008.07.24 17:31:00 - [5]
 

nice devblog

Did you really not have any testing scripts before?

Now when can regular players could get their hands on a version of this lightweight client (with the scripting removed)? ;-)

Also a suggestion for a QA test script:
I have this page http://games.chruker.dk/eve_online/character_creation.php which is generated by a simple script using the information in the datadumps. It even highlights errors like the level 6 that new amarr engineers got after one of the recent patches. Also if the combined skillpoint total is too far away from the 800k mark. Anyway that script would be very simple to expand to also make sure that each skill have its prequistics satisfied.

King Leopold
Congo Free State
Posted - 2008.07.24 18:04:00 - [6]
 

Slightly interested in your graphic modeling testing approach. Did you render these in some sim and then test the attributes? I've created similar harnesses before to test 3d images rendered to find oil and gas deposits. I was wondering if you uses DirectX or OpenGL api's to get inside those models.

Considering this is what i do for a living, I was wondering how CCP addressed the automation.

CCP Atropos

Posted - 2008.07.24 18:07:00 - [7]
 

Originally by: Chruker
nice devblog

Did you really not have any testing scripts before?

Now when can regular players could get their hands on a version of this lightweight client (with the scripting removed)? ;-)

Also a suggestion for a QA test script:
I have this page http://games.chruker.dk/eve_online/character_creation.php which is generated by a simple script using the information in the datadumps. It even highlights errors like the level 6 that new amarr engineers got after one of the recent patches. Also if the combined skillpoint total is too far away from the 800k mark. Anyway that script would be very simple to expand to also make sure that each skill have its prequistics satisfied.

The QA department have many regression tests and scripts that must be run through when testing at the various levels, but most of the game changes that occur are generally on a much smaller level; tweaks and balances to specific ships or features, rather than huge amounts of similar features. Who wants to be the unlucky one who jumps through 500 gates? Or goes to each asteroid belt in a region to check the ore there? These automated tests cover very needed areas of game play in a very simple manner.

Your suggestion for a script would be covered more by unit testing, which is also being expanded, since it's more about data validation and balancing than 'does it work' (although that too is part of the test).

Miyamoto Uroki
Caldari
Sarum Industries
Posted - 2008.07.24 18:23:00 - [8]
 

Quote:
We have created stubbed version of our audio and graphics engines so that we can now replace these in the client and significantly reduce the amount of CPU and memory used by the client

Hehe, I guess there are lots of players out there who would kill to get their hands on such a client version.

Very interesting dev-blog..oh and I would love to hear more about future API options and what you are planning for it

CCP Lingorm


C C P
Posted - 2008.07.24 18:48:00 - [9]
 

We have a dev blog coming soon(tm) on new API features and testing processes.

CCP Sputnik


C C P
C C P Alliance
Posted - 2008.07.24 18:50:00 - [10]
 

Edited by: CCP Sputnik on 24/07/2008 18:51:48
Originally by: King Leopold
Slightly interested in your graphic modeling testing approach. Did you render these in some sim and then test the attributes? I've created similar harnesses before to test 3d images rendered to find oil and gas deposits. I was wondering if you uses DirectX or OpenGL api's to get inside those models.

Considering this is what i do for a living, I was wondering how CCP addressed the automation.


We are not testing on the model itself (at least for now), the tests we are doing make sure that wrapped graphics object has the right attributes within the limits of our graphics engine.

For example we are testing if the textures of a ship are not bigger than a certain size, that the turret pairings of a ship are right, that the effects which are used are actually existing or that the bounding sphere (which is the physical representation of the object in the physics engine) are set to the right value.

All these tests are on a really low level and just should make sure that all the graphical assets we have are working with our graphics engine and have additionally no bad influence on the performance of the game.

We also have tests which are similar to the stuff you did, they are not in the automated framework yet but we are testing the placement of textures by replacing them with a one colored texture and checking with a script for gaps between textures. We are still working on a special framework for those tests which should be finished soon.

Unfortunately most of the "real" graphical tests still have to be done by a Tester, because "has the right attributes" is not the same as "looks GOOD" we don't expect the computer to see if the players like something or not. Therefore the major thing we want to achieve is that all our graphic assets have a certain standard and when a tester looks over it we are sure that its processed the right way by the graphics engine and the performance is as good as possible.

Edit: And about the load client ... thats my toy I'm not sharing it with anybody else YARRRR!!

PC5
Against Polish Authorities
AAA Citizens
Posted - 2008.07.24 19:14:00 - [11]
 

Edited by: PC5 on 24/07/2008 19:20:33

Some ppl call it nice devblog but its BORING. Whats interesting about testing stargates, asteroid belts, loot tables? Normaly i wouldnt complain but i want to read about something interesting and last 10 displayed devblogs are crap for me. Im not interested in Evelopedia, E-ON devblog nr X, Customer Support, PvE Content blog which lacks content or others.

Im sure ppl are more interested in such things as:
- planet colonization,
- comets,
- solar-system wide asteroid belts,
- atmospheric fights near planets,
- capital ships possible changes,
- fleet battles issues and CCP thoughts about that,
- future game development,
and many many more..

Some development projects failed - like anti blob warfare (ECM, bombs) and we have only crap like this recent devblog to read? Why its happening so? There are much more important things in game which need devblog but instead we get something like those last 10.

And other thing is when will we see updated "In Development" page? To lazy to change it?

Give us something good, something fresh, something we WANT to read about and help you somehow. Maybe its only temporary and as 'older' player i will see something interesting here soon. Lets hope so.

PS. Sorry for my english, it will improve i promise ;) I hope CCP will improve too, because 1-2 years ago it was interesting to read devblogs, now it isint. Not only for me but for serval ppl i play with and i know ingame.


LaVista Vista
Conservative Shenanigans Party
Posted - 2008.07.24 19:53:00 - [12]
 

Excellent blog.

Serenity Steele
Dynamic Data Distribution
Posted - 2008.07.24 19:53:00 - [13]
 

Great to see, the awesome work and tools you guys have developed, explained to the EVE Community. That's a hell of tool suite you have there!

Serenity Steele
Dynamic Data Distribution
Posted - 2008.07.24 20:00:00 - [14]
 

Edited by: Serenity Steele on 24/07/2008 20:00:47
Originally by: PC5
Edited by: PC5 on 24/07/2008 19:20:33
...
And other thing is when will we see updated "In Development" page? To lazy to change it?



Lucita, you might want to swing by the Assembly Hall to post your suggestions before calling others lazy.

Aelin Dao
GoonFleet
GoonSwarm
Posted - 2008.07.24 20:25:00 - [15]
 

Edited by: Aelin Dao on 24/07/2008 20:26:43
You mentioned a new framework for unit testing in the WiS codebase, and integrating it back into the existing code later. Does this mean that the main codebase doesn't have any unit testing at the moment, or was the worry about the new and old frameworks playing well alongside one another?

What about pyUnit prompted you to roll your own system? How does it compare to something like Nose, or even just doctests?

Also, thanks, it's nice to see technical blog posts. Keep 'em coming.

PC5
Against Polish Authorities
AAA Citizens
Posted - 2008.07.24 21:44:00 - [16]
 

Edited by: PC5 on 24/07/2008 21:47:10
Originally by: Serenity Steele
Edited by: Serenity Steele on 24/07/2008 20:00:47
Originally by: PC5
Edited by: PC5 on 24/07/2008 19:20:33
...
And other thing is when will we see updated "In Development" page? To lazy to change it?



Lucita, you might want to swing by the Assembly Hall to post your suggestions before calling others lazy.

Youre in CSM and say such things? Aparently you dont have any idea for how long that list is there and nobody cares to change it. Im not the only one who wants this page updated, so stop making idiot of me - sending me to post on other board. They know about that list, they havent done anything and i wont post another spam post on Assembly Hall - because its long term issue, maybe one year passed since last update.

Serenity Steele - please go read sections "The Drawingboard", "In Testing" in Patch Notes - thats same thing like "In Development". And finaly my point is if they have time to write 10 crap devblogs why the hell cant they update something REALLY interesting and important for many players? Maybe thats issue for you as CSM member i dont know. All i know that there are dozens of players who think same like me, ive talked to them, played with them, but they wont post here, because they dont care much for forums and they dont belive they can change anything. Well i had to say what i think here and i call those people lazy for not doing something which is important for me and many people and whatmore - its quite simple!

And i ask myself sometimes why is it so happening. Maybe they dont have time, maybe exprienced staff left for other projects, we see a lot of new faces, names, its whole new CCP for me - but i dont accept situations like those described, because its my favorite game after all ;) They can work on other games, projects as company, whatever, but i still call them lazy because they have time to write crap devblogs and they dont care about other more interesting things. Lets hope they are just lazy, it would be really bad if they wouldnt care.

Hehulk
Black Sea Industries
Posted - 2008.07.24 22:00:00 - [17]
 

Did your look into loot tables not pick up on the broken loot tables for some of the DED complex overseers? Like serpentis 3/10s still droping gisti modules when they should be droping coreli

Four Degrees
Caldari
Alternative End
Phobos Alliance
Posted - 2008.07.25 00:37:00 - [18]
 

Edited by: Four Degrees on 25/07/2008 00:37:51
Originally by: PC5
Lets hope they are just lazy, it would be really bad if they wouldnt care.


You managed to hit a nerve, well done.

Since you've been playing EVE you've taken for granted that CCP as a company are largely transparent about their activities, within reason, which is oh-so uncommon in todays climate in regards to large development companies.

The reason they are posting this information is to inspire confidence in their customers that things are improving, what they're doing to instigate improvement and also to "entertain" those interested in the inner-workings. (Your judgement on this Dev Blog being "crap" is lame, given the only reason it's crap to you is that you're not interested in the content.)

They don't necessarily need to update one single page on the website when they already have Live Dev Blogs and a large forum presence, plans are often widely reported if you're not living in a cave.

Furthermore, the "I know that there are dozens of players who think same like me," is not justification for anything and is a tired statement. All I can hope is that you'll be slightly more considerate in throwing words like "lazy" around, but my breath is not baited.

I was quite surprised regarding this Dev Blog, it was great to see that a large amount of manual "labour" was being put to rest efficiently. The Dev Blogs related to actual development of the content would no doubt be written whether or not these Dev Blogs existed, so they'll come when they'll come.

(p.s. inb4 CCP fanboi)
/waitsforitamirite?
invalid command

Kerfira
Kerfira Corp
Posted - 2008.07.25 01:05:00 - [19]
 

Originally by: PC5
Some ppl call it nice devblog but its BORING.

Speak for yourself Razz
For a software engineer with a keen interest in software testing, this is one of the better blogs I've seen.
Of.c. what I REALLY would like to see was examples of test scripts, or maybe even see a complete test suite and how it was run Cool

That being said, blog content has been a bit sparse lately, but I suppose CCP is off on vacation like the rest of the northern hemisphere at the moment....

Frankschtaldt
legion industries ltd
Posted - 2008.07.25 02:20:00 - [20]
 

Nice blog. Though I have to say "Ambulation" is a better name that "Walking in Stations".

Hugh Ruka
Exploratio et Industria Morispatia
Posted - 2008.07.25 07:43:00 - [21]
 

a Dev blog about QA testing. nice ... given the track record of CCPs QA, it's about time they figured out how to do some things properly :-)

Btw Lingorm, the Linux/MAC crowd is still waiting on that Dev blog about the alternate clients and stuff ...

CCP Lingorm


C C P
Posted - 2008.07.25 09:05:00 - [22]
 

Originally by: Aelin Dao
Edited by: Aelin Dao on 24/07/2008 20:26:43
You mentioned a new framework for unit testing in the WiS codebase, and integrating it back into the existing code later. Does this mean that the main codebase doesn't have any unit testing at the moment, or was the worry about the new and old frameworks playing well alongside one another?

What about pyUnit prompted you to roll your own system? How does it compare to something like Nose, or even just doctests?

Also, thanks, it's nice to see technical blog posts. Keep 'em coming.

The EVE code base is rather old and has never had a good unit Testing system applied. Yes we Update, refactor and maintain it butunit testing never seemed to get in.

pyUnit is the core of the framework that has been implemented, we have just had to add some additional features to support stackless, a service mocking system and support for on the fly recompile (we so when we are developing if we save changes to a file, it recompiles it, reruns any unit tests associated with it and carries on so that we do not have to restart our instances of EVE ,,, which is rather nifty.

Lake
The Praxis Initiative
Gentlemen's Agreement
Posted - 2008.07.25 11:04:00 - [23]
 

Edited by: Lake on 25/07/2008 11:06:35
Originally by: Miyamoto Uroki
Quote:
We have created stubbed version of our audio and graphics engines so that we can now replace these in the client and significantly reduce the amount of CPU and memory used by the client

Hehe, I guess there are lots of players out there who would kill to get their hands on such a client version.

Very interesting dev-blog..oh and I would love to hear more about future API options and what you are planning for it


By 'stubbed' they likely mean that other components of the EVE client can make calls to the audio and video engines and those respond in sane manners without actually doing anything. Not doing anything reduces client strain, but means the 'user' doesn't hear or see anything, so probably not all that useful for 'players'.

Unless you meant eula-breaking-macro-bot-scum when you said 'players' =p

Edit: lol, s-c-u-m is apparently a four letter word to the censor code. Not only does it mask it out, it also thinks it's more than 3 letters and catches the following space too.

CCP Lingorm


C C P
Posted - 2008.07.25 11:21:00 - [24]
 

Originally by: Lake
Edited by: Lake on 25/07/2008 11:06:35
By 'stubbed' they likely mean that other components of the EVE client can make calls to the audio and video engines and those respond in sane manners without actually doing anything. Not doing anything reduces client strain, but means the 'user' doesn't hear or see anything, so probably not all that useful for 'players'.

Unless you meant eula-breaking-macro-bot-scum when you said 'players' =p

Edit: lol, s-c-u-m is apparently a four letter word to the censor code. Not only does it mask it out, it also thinks it's more than 3 letters and catches the following space too.

That is exactly what we mean ...

For example the Audio 'Stub' get the command to play a sound ... it replies to the game code 'Done. And no errors reported.' While in actual fact it does absolutely nothing. This reduces CPU use for the client.

The Graphic Stub does something similar but because you still need to create and maintain false pointers and such it actually still does some work. But it makes use of the DirectX Null Device to create the graphics environment against so it can maintain the needed fake pointers and such, while not actually doing anything with them ... it is a pretty cool piece of code. We still have some areas of improvement that we can make to it, but all in all we are rather happy with it.

MynChu NicAtoch
Eternity INC.
Goonswarm Federation
Posted - 2008.07.25 13:45:00 - [25]
 

Edited by: MynChu NicAtoch on 25/07/2008 13:49:07
Edited by: MynChu NicAtoch on 25/07/2008 13:48:05
Originally by: CCP Lingorm
... the Audio 'Stub' get the command to play a sound ... it replies to the game code 'Done. And no errors reported.' While in actual fact it does absolutely nothing. This reduces CPU use for the client.


EVE has sound? WTF?!?!?

Seriously, I'd like to load my client without sound code and overhead. Like almost everyone who plays in a fleet I have sound disabled, 'cause I'm on a TS server (worse - with multiple command channels). Less CPU load is teh win.

Reduced graphics would also be terrific. I know its not how CCP attract new customers - EVE looks great and for some people that is probably important. However, I don't play EVE on a PC that does the graphics artists work justice to it, and have no plans to do so soon. I'd much rather you focussed your design on doing LESS graphics with LESS GPU and CPU utilization but more INFORMATION being conveyed to me in a usable format. My screen real estate is invaluable, and EVE wastes too much of it just to look pretty when I don't need it to. I can spend as much time in EVE working with spreadsheets, maps, emails, forums and instant messaging as I do with the client. I often have to close copies of the client to reduce load.

I'm one of the minority of players who make up for our unusual longevity in the game (having played it for more than the average seven months) by running multiple accounts (five in my case). Far from considering adding MORE to CCP's monthly income stream, I'm considering reducing this, mainly because the current graphics load doesn't allow me to get the gameplay out of them that I once could.

Actually .. I feel a CSM Assembly Hall post coming on.

Sgt Napalm
Veto.
Veto Corp
Posted - 2008.07.25 15:11:00 - [26]
 

That blog is really nerdy but I award it 10/10


CCP Sputnik


C C P
C C P Alliance
Posted - 2008.07.25 16:12:00 - [27]
 

Edited by: CCP Sputnik on 25/07/2008 16:12:09
Originally by: MynChu NicAtoch

EVE has sound? WTF?!?!?

Seriously, I'd like to load my client without sound code and overhead. Like almost everyone who plays in a fleet I have sound disabled, 'cause I'm on a TS server (worse - with multiple command channels). Less CPU load is teh win.

Reduced graphics would also be terrific. I know its not how CCP attract new customers - EVE looks great and for some people that is probably important. However, I don't play EVE on a PC that does the graphics artists work justice to it, and have no plans to do so soon. I'd much rather you focussed your design on doing LESS graphics with LESS GPU and CPU utilization but more INFORMATION being conveyed to me in a usable format. My screen real estate is invaluable, and EVE wastes too much of it just to look pretty when I don't need it to. I can spend as much time in EVE working with spreadsheets, maps, emails, forums and instant messaging as I do with the client. I often have to close copies of the client to reduce load.

I'm one of the minority of players who make up for our unusual longevity in the game (having played it for more than the average seven months) by running multiple accounts (five in my case). Far from considering adding MORE to CCP's monthly income stream, I'm considering reducing this, mainly because the current graphics load doesn't allow me to get the gameplay out of them that I once could.

Actually .. I feel a CSM Assembly Hall post coming on.


... just to make this clear these changes are not made to play with the client after wards, not even a macro miner could use it because its optimized for a special framework which allows us to simulate player behavior to test specific aspects of server code and hardware.
We are only able to reduce the cpu and ram usage of the client so far because we know they will be remote controlled.

Over all these Load Clients will improve our possibility's to test server performance in a way which is normally only possible for simple things like websites or SQL server.

Additionally a code made for performance is not really the same like a stable player game client, a Load Client doesn't complain if he is losing a ship to a crashing client - you know what I mean Wink

Chainsaw Plankton
IDLE GUNS
IDLE EMPIRE
Posted - 2008.07.25 17:22:00 - [28]
 

automated testing is win!

PC5
Against Polish Authorities
AAA Citizens
Posted - 2008.07.25 19:55:00 - [29]
 

Removed off-topic reference to Speed Rebalanced blog. Navigator

Elementatia
Morsus Mihi
Posted - 2008.07.25 20:00:00 - [30]
 

Very nice !

Thank you for the long and good devblog !


Pages: [1] 2

This thread is older than 90 days and has been locked due to inactivity.


 


The new forums are live

Please adjust your bookmarks to https://forums.eveonline.com

These forums are archived and read-only