tachyon.hadoop

Class TFS

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

tachyon.hadoop.TFS

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable

public final class TFS
extends org.apache.hadoop.fs.FileSystem

An Apache Hadoop FileSystem interface implementation. Any program working with Hadoop HDFS can work with Tachyon transparently by using this class. However, it is not as efficient as using the Tachyon API in tachyon.client package.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields

Modifier and Type	Field and Description
static String	FIRST_COM_PATH
static String	RECOMPUTE_PATH

Fields inherited from class org.apache.hadoop.fs.FileSystem

FS_DEFAULT_NAME_KEY, statistics

Constructor Summary

Constructors

Constructor and Description
TFS()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path cPath, int bufferSize, org.apache.hadoop.util.Progressable progress)
void	close()
org.apache.hadoop.fs.FSDataOutputStream	create(org.apache.hadoop.fs.Path cPath, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
org.apache.hadoop.fs.FSDataOutputStream	createNonRecursive(org.apache.hadoop.fs.Path cPath, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated. API only for 0.20-append
boolean	delete(org.apache.hadoop.fs.Path path) Deprecated.
boolean	delete(org.apache.hadoop.fs.Path cPath, boolean recursive)
long	getDefaultBlockSize()
org.apache.hadoop.fs.BlockLocation[]	getFileBlockLocations(org.apache.hadoop.fs.FileStatus file, long start, long len)
org.apache.hadoop.fs.FileStatus	getFileStatus(org.apache.hadoop.fs.Path path) Return the status of a single file.
String	getScheme() Get the URI schema that maps to the FileSystem.
TachyonFS	getTachyonFS() Returns an object implementing the Tachyon-specific client API.
URI	getUri()
org.apache.hadoop.fs.Path	getWorkingDirectory()
void	initialize(URI uri, org.apache.hadoop.conf.Configuration conf) Initialize the class, have a lazy connection with Tachyon through mTFS.
protected boolean	isZookeeperMode() Determines if zookeeper should be used for the FileSystem.
org.apache.hadoop.fs.FileStatus[]	listStatus(org.apache.hadoop.fs.Path path)
boolean	mkdirs(org.apache.hadoop.fs.Path cPath, org.apache.hadoop.fs.permission.FsPermission permission)
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.Path cPath, int bufferSize)
boolean	rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
void	setWorkingDirectory(org.apache.hadoop.fs.Path path)

Methods inherited from class org.apache.hadoop.fs.FileSystem

addFileSystemForTesting, append, append, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, deleteOnExit, exists, get, get, get, getAllStatistics, getBlockSize, getCanonicalServiceName, getCanonicalUri, getContentSummary, getDefaultPort, getDefaultReplication, getDefaultUri, getDelegationToken, getFileChecksum, getHomeDirectory, getLength, getLocal, getName, getNamed, getReplication, getStatistics, getStatistics, getUsed, globStatus, globStatus, isDirectory, isFile, listStatus, listStatus, listStatus, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, open, printStatistics, processDeleteOnExit, setDefaultUri, setDefaultUri, setOwner, setPermission, setReplication, setTimes, setVerifyChecksum, startLocalOutput

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FIRST_COM_PATH

public static final String FIRST_COM_PATH

See Also:

Constant Field Values

RECOMPUTE_PATH

public static final String RECOMPUTE_PATH

See Also:

Constant Field Values

Constructor Detail

TFS

public TFS()

Method Detail

getScheme

public String getScheme()

Get the URI schema that maps to the FileSystem. This was introduced in Hadoop 2.x as a means to make loading new FileSystems simpler. This doesn't exist in Hadoop 1.x, so can not put

Returns:

schema hadoop should map to.

See Also:

FileSystem.createFileSystem(java.net.URI, org.apache.hadoop.conf.Configuration)

isZookeeperMode

protected boolean isZookeeperMode()

Determines if zookeeper should be used for the FileSystem. This method should only be used for initialize(java.net.URI, org.apache.hadoop.conf.Configuration).

Returns:

true if zookeeper should be used

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path cPath,
                                             int bufferSize,
                                             org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Specified by:

append in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

create

public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path cPath,
                                             org.apache.hadoop.fs.permission.FsPermission permission,
                                             boolean overwrite,
                                             int bufferSize,
                                             short replication,
                                             long blockSize,
                                             org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Specified by:

create in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

createNonRecursive

@Deprecated
public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path cPath,
                                                                    org.apache.hadoop.fs.permission.FsPermission permission,
                                                                    boolean overwrite,
                                                                    int bufferSize,
                                                                    short replication,
                                                                    long blockSize,
                                                                    org.apache.hadoop.util.Progressable progress)
                                                           throws IOException

Deprecated. API only for 0.20-append

TODO: We need to refactor this method after having a new internal API support (TACHYON-46).

Opens an FSDataOutputStream at the indicated Path with write-progress reporting. Same as create(), except fails if parent directory doesn't already exist.

Overrides:

createNonRecursive in class org.apache.hadoop.fs.FileSystem

Parameters:

cPath - the file name to open

overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.

bufferSize - the size of the buffer to be used.

replication - required block replication for the file.

blockSize -

progress -

Throws:

IOException

See Also:

FileSystem.setPermission(Path, FsPermission)

delete

@Deprecated
public boolean delete(org.apache.hadoop.fs.Path path)
               throws IOException

Deprecated.

Specified by:

delete in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

delete

public boolean delete(org.apache.hadoop.fs.Path cPath,
             boolean recursive)
               throws IOException

Specified by:

delete in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getDefaultBlockSize

public long getDefaultBlockSize()

Overrides:

getDefaultBlockSize in class org.apache.hadoop.fs.FileSystem

getFileBlockLocations

public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus file,
                                                         long start,
                                                         long len)
                                                           throws IOException

Overrides:

getFileBlockLocations in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path path)
                                              throws IOException

Return the status of a single file. If the file does not exist in Tachyon, query it from HDFS.

Specified by:

getFileStatus in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getTachyonFS

public TachyonFS getTachyonFS()

Returns an object implementing the Tachyon-specific client API.

Returns:

null if initialize() hasn't been called.

getUri

public URI getUri()

Specified by:

getUri in class org.apache.hadoop.fs.FileSystem

getWorkingDirectory

public org.apache.hadoop.fs.Path getWorkingDirectory()

Specified by:

getWorkingDirectory in class org.apache.hadoop.fs.FileSystem

initialize

public void initialize(URI uri,
              org.apache.hadoop.conf.Configuration conf)
                throws IOException

Initialize the class, have a lazy connection with Tachyon through mTFS.

Overrides:

initialize in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

listStatus

public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path path)
                                             throws IOException

Specified by:

listStatus in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

mkdirs

public boolean mkdirs(org.apache.hadoop.fs.Path cPath,
             org.apache.hadoop.fs.permission.FsPermission permission)
               throws IOException

Specified by:

mkdirs in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path cPath,
                                          int bufferSize)
                                            throws IOException

Specified by:

open in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

rename

public boolean rename(org.apache.hadoop.fs.Path src,
             org.apache.hadoop.fs.Path dst)
               throws IOException

Specified by:

rename in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

setWorkingDirectory

public void setWorkingDirectory(org.apache.hadoop.fs.Path path)

Specified by:

setWorkingDirectory in class org.apache.hadoop.fs.FileSystem