org.apache.nutch.ndfs
Class NDFSClient

java.lang.Object
  org.apache.nutch.ndfs.NDFSClient

All Implemented Interfaces:

FSConstants

public class NDFSClient

extends Object

implements FSConstants

NDFSClient does what's necessary to connect to a Nutch Filesystem and perform basic file tasks.

Author:

Mike Cafarella, Tessa MacDuff

Field Summary

static Logger

LOG

Fields inherited from interface org.apache.nutch.ndfs.FSConstants

BLOCK_SIZE, BLOCKREPORT_INTERVAL, CHUNKED_ENCODING, COMPLETE_SUCCESS, DATANODE_STARTUP_PERIOD, EXPIRE_INTERVAL, HEARTBEAT_INTERVAL, LEASE_PERIOD, OBSOLETE_INTERVAL, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ABANDONBLOCK, OP_CLIENT_ABANDONBLOCK_ACK, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DATANODE_HINTS, OP_CLIENT_DATANODE_HINTS_ACK, OP_CLIENT_DATANODEREPORT, OP_CLIENT_DATANODEREPORT_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_EXISTS, OP_CLIENT_EXISTS_ACK, OP_CLIENT_ISDIR, OP_CLIENT_ISDIR_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_MKDIRS, OP_CLIENT_MKDIRS_ACK, OP_CLIENT_OBTAINLOCK, OP_CLIENT_OBTAINLOCK_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RAWSTATS, OP_CLIENT_RAWSTATS_ACK, OP_CLIENT_RELEASELOCK, OP_CLIENT_RELEASELOCK_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_RENEW_LEASE, OP_CLIENT_RENEW_LEASE_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_READSKIP_BLOCK, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, OPERATION_FAILED, RUNLENGTH_ENCODING, STILL_WAITING, SYSTEM_STARTUP_PERIOD, WRITE_COMPLETE

Constructor Summary

NDFSClient(InetSocketAddress namenode)

Method Summary

void	close()
NFSOutputStream	create(UTF8 src) Create an output stream that writes to all the right places.
NFSOutputStream	create(UTF8 src, boolean overwrite)
DatanodeInfo[]	datanodeReport()
boolean	delete(UTF8 src) Make a direct connection to namenode and manipulate structures there.
boolean	exists(UTF8 src)
boolean	isDirectory(UTF8 src)
NDFSFileInfo[]	listFiles(UTF8 src)
void	lock(UTF8 src, boolean exclusive)
boolean	mkdirs(UTF8 src)
NFSInputStream	open(UTF8 src) Create an input stream that obtains a nodelist from the namenode, and then reads from all the right places.
void	release(UTF8 src)
boolean	rename(UTF8 src, UTF8 dst) Make a direct connection to namenode and manipulate structures there.
long	totalRawCapacity()
long	totalRawUsed()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final Logger LOG

Constructor Detail

NDFSClient

public NDFSClient(InetSocketAddress namenode)

Method Detail

close

public void close()
           throws IOException

Throws:

IOException

open

public NFSInputStream open(UTF8 src)
                    throws IOException

Create an input stream that obtains a nodelist from the namenode, and then reads from all the right places. Creates inner subclass of InputStream that does the right out-of-band work.

Throws:

IOException

create

public NFSOutputStream create(UTF8 src)
                       throws IOException

Create an output stream that writes to all the right places. Basically creates instance of inner subclass of OutputStream that handles datanode/namenode negotiation.

Throws:

IOException

create

public NFSOutputStream create(UTF8 src,
                              boolean overwrite)
                       throws IOException

Throws:

IOException

rename

public boolean rename(UTF8 src,
                      UTF8 dst)
               throws IOException

Make a direct connection to namenode and manipulate structures there.

Throws:

IOException

delete

public boolean delete(UTF8 src)
               throws IOException

Make a direct connection to namenode and manipulate structures there.

Throws:

IOException

exists

public boolean exists(UTF8 src)
               throws IOException

Throws:

IOException

isDirectory

public boolean isDirectory(UTF8 src)
                    throws IOException

Throws:

IOException

listFiles

public NDFSFileInfo[] listFiles(UTF8 src)
                         throws IOException

Throws:

IOException

totalRawCapacity

public long totalRawCapacity()
                      throws IOException

Throws:

IOException

totalRawUsed

public long totalRawUsed()
                  throws IOException

Throws:

IOException

datanodeReport

public DatanodeInfo[] datanodeReport()
                              throws IOException

Throws:

IOException

mkdirs

public boolean mkdirs(UTF8 src)
               throws IOException

Throws:

IOException

lock

public void lock(UTF8 src,
                 boolean exclusive)
          throws IOException

Throws:

IOException

release

public void release(UTF8 src)
             throws IOException

Throws:

IOException