AtlasFile Class Reference [Atlas Core]

#include <atlasFile.h>

Inheritance diagram for AtlasFile:

Inheritance graph
[legend]
List of all members.

Detailed Description

Represents a single .atlas file; provides an interface for performing threaded reads and writes against that file, as well as tracking the TOCs in that file.

It also deals with getting the resource system to play nice with Atlas.


Public Member Functions

 AtlasFile ()
 ~AtlasFile ()
void registerTOC (AtlasTOC *)
 Add a TOC to the file.
template<class T>
bool getTocBySlot (U32 slot, T *&toc)
 Get TOC by slot.
bool open (const char *filename)
 Open file, read in TOCs, and get ready for threads to be started.
bool createNew (const char *filename)
 Create a new Atlas file with the currently loaded TOCs.
bool optimize (const char *filename)
 Optimize to given path.
void startLoaderThreads ()
void stopLoaderThreads ()
void syncThreads ()
 Once loader threads have been started, make sure to call syncThreads every so often.
AtlasDeferredFilegetDeferredFile ()
 Get direct access to the AtlasDeferredFile, useful for directly manipulating the Stream.
const bool hasStream () const
 Does our AtlasDeferredFile have a stream associated with it? Mostly used for sanity checking.
void queuePendingDeserialize (AtlasReadNote *arn)
 Internal callback from AtlasResourceTOC<>.
void waitForPendingWrites ()
 Helper function - after you've done a bunch of writes, BEFORE you destroy the AtlasFile, make sure to call this so everything gets flushed to disk.
void precache ()
 Spin until all of our load queues are empty and we've processed all pending items.
void dumpLoadQueue ()
 Dump our current load priorities to the console, in the order we'll try to load them.
const U32 getTOCCount () const
AtlasTOCgetTOC (U32 index)
IO Statistics
void getIOStatus (IOStatus &status)
 Return the current file I/O status.

Static Public Member Functions

static ResourceOld< AtlasFileload (const char *filename)
 Do all loads through THIS method, not the ResourceManager.

Static Public Attributes

static U32 smDeserializerMaxLoadLimit
 The total amount of requests we allow to pile up in deserializer queue.
static U32 smPipelineThrottleLimit
 If the desializer has fewer pending requests than this and the load queue is empty, we allow new requests to enter the pipeline.
static bool smLogStubLoadStatus

Protected Member Functions

void enqueueNextPendingLoad (bool waitTillNext=true)
 Helper function to make sure we've got another note in flight.
void deserializerThread ()

Static Protected Member Functions

static void deserializerThunk (void *d)

Protected Attributes

AtlasDeferredFile mFile
 We actually manage the raw file IO with a subclass.
Vector< AtlasTOC * > mTOCs
 List of the TOCs which are present in this file, in order.
ThreadSafeQueue< AtlasReadNote * > mLoadingNotes
 Queue of notes that have had processing started on them by the IO thread.
ThreadSafeQueue< AtlasReadNote * > mPendingLoadQueue
 Queue of notes that are awaiting processing by the IO thread.
ThreadSafeQueue< AtlasReadNote * > mPendingDeserializeQueue
 Queue of notes that are done with IO and awaiting deserialization.
ThreadSafeQueue< AtlasReadNote * > mPendingProcessQueue
 Queue of notes that are ready for processing by the main thread.
ThreadmDeserializerThread
 The deserialization thread.
volatile bool mDeserializerThreadActive
 Control flag for the deserialization thread.
SemaphoremDeserializerSemaphore
U32 mLastSyncTime
 Track time of last sync() call.
U32 mLastProcessedTOC

Private Types

typedef ResourceInstance Parent

Friends

class AtlasTOC

Classes

struct  IOStatus
 Information about the current I/O system status. More...


Member Typedef Documentation


Constructor & Destructor Documentation

AtlasFile::AtlasFile (  ) 

AtlasFile::~AtlasFile (  ) 


Member Function Documentation

void AtlasFile::enqueueNextPendingLoad ( bool  waitTillNext = true  )  [protected]

Helper function to make sure we've got another note in flight.

void AtlasFile::deserializerThread (  )  [protected]

static void AtlasFile::deserializerThunk ( void d  )  [static, protected]

void AtlasFile::getIOStatus ( IOStatus status  ) 

Return the current file I/O status.

static ResourceOld<AtlasFile> AtlasFile::load ( const char *  filename  )  [static]

Do all loads through THIS method, not the ResourceManager.

We rely on the ResourceManager to track active Atlas instances and load/unload the AtlasFile as needed, but in order to properly bind the AtlasFile instance to the actual file, we have to bypass some parts what the ResourceManager does.

void AtlasFile::registerTOC ( AtlasTOC  ) 

Add a TOC to the file.

This causes undefined (bad) results if you run it on a pre-existing file, as it can't insert new data into an active Atlas file. (It would have to move all the chunks down, which would be both tremendously slow and also difficult to orchestrate with loading threads active.)

template<class T>
bool AtlasFile::getTocBySlot ( U32  slot,
T *&  toc 
) [inline]

Get TOC by slot.

This is typed so you always get the right sort of TOC. Slot order is important, and reflects the order the TOCs are written.

Basically, the slot-th toc of a given type is the slot-th TOC of that type which was written to the file. So a file might have a slot 0 geometry TOC, and then texture TOCs in slots 0 and 1 (for opacity maps and a lightmap). (In this scenario, their write order might be opacity map, geometry, lightmap - but we only consider TOCs of the same type when identifying slot order.)

bool AtlasFile::open ( const char *  filename  ) 

Open file, read in TOCs, and get ready for threads to be started.

bool AtlasFile::createNew ( const char *  filename  ) 

Create a new Atlas file with the currently loaded TOCs.

Atlas file construction works off of this pattern: 1. Allocate a new AtlasFile. 2. Create all TOCs. 3. Call registerTOC on them. 4. createNew().

bool AtlasFile::optimize ( const char *  filename  ) 

Optimize to given path.

Optimization entails writing the file from scratch, respecting the optimizeOrder of the stubs. This has two results. First, you get rid of any old chunk data. Second, it lets you optimize chunk order to reflect file IO patterns.

void AtlasFile::startLoaderThreads (  ) 

void AtlasFile::stopLoaderThreads (  ) 

void AtlasFile::syncThreads (  ) 

Once loader threads have been started, make sure to call syncThreads every so often.

This synchronizes with the AtlasDeferredFile, with the various TOCs in this file, and does final loading steps.

But as it happens you DON'T have to call this if all you're doing is writing.

You want to call this about once a frame in a runtime situation.

AtlasDeferredFile& AtlasFile::getDeferredFile (  )  [inline]

Get direct access to the AtlasDeferredFile, useful for directly manipulating the Stream.

const bool AtlasFile::hasStream (  )  const [inline]

Does our AtlasDeferredFile have a stream associated with it? Mostly used for sanity checking.

void AtlasFile::queuePendingDeserialize ( AtlasReadNote arn  )  [inline]

Internal callback from AtlasResourceTOC<>.

void AtlasFile::waitForPendingWrites (  ) 

Helper function - after you've done a bunch of writes, BEFORE you destroy the AtlasFile, make sure to call this so everything gets flushed to disk.

Otherwise you're likely to kill any in-flight writes.

The Atlas file should stay uncorrupted (unless you manage to kill it in the middle of a write, which is fairly unlikely since all writes are atomic OS IO calls), but not calling this guy when you're done writing to an Atlas file is a good way to have missing data.

void AtlasFile::precache (  ) 

Spin until all of our load queues are empty and we've processed all pending items.

void AtlasFile::dumpLoadQueue (  ) 

Dump our current load priorities to the console, in the order we'll try to load them.

const U32 AtlasFile::getTOCCount (  )  const [inline]

AtlasTOC* AtlasFile::getTOC ( U32  index  )  [inline]


Friends And Related Function Documentation

friend class AtlasTOC [friend]


Member Data Documentation

We actually manage the raw file IO with a subclass.

See also:
AtlasDeferredFile for more information.

List of the TOCs which are present in this file, in order.

Queue of notes that have had processing started on them by the IO thread.

Queue of notes that are awaiting processing by the IO thread.

Queue of notes that are done with IO and awaiting deserialization.

Queue of notes that are ready for processing by the main thread.

The deserialization thread.

This thread is responsible for performing the appropriate deserialization on raw data once it is read from disk, and queuing it for instatement.

See also:
deserializerThread

Control flag for the deserialization thread.

Track time of last sync() call.

The total amount of requests we allow to pile up in deserializer queue.

If the desializer has fewer pending requests than this and the load queue is empty, we allow new requests to enter the pipeline.