org.apache.nutch.ndfs
Class FSDataset

java.lang.Object
  org.apache.nutch.ndfs.FSDataset

All Implemented Interfaces:

FSConstants

public class FSDataset

extends Object

implements FSConstants

FSDataset manages a set of data blocks. Each block has a unique name and an extent on disk.

Author:

Mike Cafarella

Field Summary

Fields inherited from interface org.apache.nutch.ndfs.FSConstants

BLOCK_SIZE, BLOCKREPORT_INTERVAL, CHUNKED_ENCODING, COMPLETE_SUCCESS, DATANODE_STARTUP_PERIOD, EXPIRE_INTERVAL, HEARTBEAT_INTERVAL, LEASE_PERIOD, OBSOLETE_INTERVAL, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ABANDONBLOCK, OP_CLIENT_ABANDONBLOCK_ACK, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DATANODE_HINTS, OP_CLIENT_DATANODE_HINTS_ACK, OP_CLIENT_DATANODEREPORT, OP_CLIENT_DATANODEREPORT_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_EXISTS, OP_CLIENT_EXISTS_ACK, OP_CLIENT_ISDIR, OP_CLIENT_ISDIR_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_MKDIRS, OP_CLIENT_MKDIRS_ACK, OP_CLIENT_OBTAINLOCK, OP_CLIENT_OBTAINLOCK_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RAWSTATS, OP_CLIENT_RAWSTATS_ACK, OP_CLIENT_RELEASELOCK, OP_CLIENT_RELEASELOCK_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_RENEW_LEASE, OP_CLIENT_RENEW_LEASE_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_READSKIP_BLOCK, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, OPERATION_FAILED, RUNLENGTH_ENCODING, STILL_WAITING, SYSTEM_STARTUP_PERIOD, WRITE_COMPLETE

Constructor Summary

FSDataset(File dir)
An FSDataset has a directory where it loads its data files.

Method Summary

void	finalizeBlock(Block b) Complete the block write!
InputStream	getBlockData(Block b) Get a stream of data from the indicated block.
Block[]	getBlockReport() Return a table of block data
long	getCapacity() Return total capacity, used and unused
long	getLength(Block b) Find the block's on-disk length
long	getRemaining() Return how many bytes can still be stored in the FSDataset
void	invalidate(Block[] invalidBlks) We're informed that a block is no longer valid.
boolean	isValidBlock(Block b) Check whether the given block is a valid one.
boolean	startBlock(Block b) A Block b will be coming soon!
OutputStream	writeToBlock(Block b) Start writing to a block file

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSDataset

public FSDataset(File dir)
          throws IOException

An FSDataset has a directory where it loads its data files.

Method Detail

getCapacity

public long getCapacity()
                 throws IOException

Return total capacity, used and unused

Throws:

IOException

getRemaining

public long getRemaining()
                  throws IOException

Return how many bytes can still be stored in the FSDataset

Throws:

IOException

getLength

public long getLength(Block b)
               throws IOException

Find the block's on-disk length

Throws:

IOException

getBlockData

public InputStream getBlockData(Block b)
                         throws IOException

Get a stream of data from the indicated block.

Throws:

IOException

startBlock

public boolean startBlock(Block b)
                   throws IOException

A Block b will be coming soon!

Throws:

IOException

writeToBlock

public OutputStream writeToBlock(Block b)
                          throws IOException

Start writing to a block file

Throws:

IOException

finalizeBlock

public void finalizeBlock(Block b)
                   throws IOException

Complete the block write!

Throws:

IOException

getBlockReport

public Block[] getBlockReport()

Return a table of block data

isValidBlock

public boolean isValidBlock(Block b)

Check whether the given block is a valid one.

invalidate

public void invalidate(Block[] invalidBlks)
                throws IOException

We're informed that a block is no longer valid. We could lazily garbage-collect the block, but why bother? just get rid of it.

Throws:

IOException