tachyon.master

Class EditLog

java.lang.Object

tachyon.master.EditLog

public final class EditLog
extends Object

Master operation journal.

Constructor Summary

Constructors

Constructor and Description
EditLog(String path, boolean inactive, long transactionId) Create a new EditLog

Method Summary

Methods

Modifier and Type	Method and Description
void	addBlock(int fileId, int blockIndex, long blockLength, long opTimeMs) Log an addBlock operation.
void	addCheckpoint(int fileId, long length, TachyonURI checkpointPath, long opTimeMs) Log an addCheckpoint operation.
void	close() Close the log.
void	completeFile(int fileId, long opTimeMs) Log a completeFile operation.
void	createDependency(List<Integer> parents, List<Integer> children, String commandPrefix, List<ByteBuffer> data, String comment, String framework, String frameworkVersion, DependencyType dependencyType, int depId, long creationTimeMs) Log a createDependency operation.
void	createFile(boolean recursive, TachyonURI path, boolean directory, long blockSizeByte, long creationTimeMs) Log a createFile operation.
void	createRawTable(int tableId, int columns, ByteBuffer metadata) Log a createRawTable operation.
void	delete(int fileId, boolean recursive, long opTimeMs) Log a delete operation.
void	deleteCompletedLogs(String path, int upTo) Delete the completed logs.
void	flush() Flush the log onto the storage.
Pair<Long,Long>	getTransactionIds() Get the current TransactionId and FlushedTransactionId
static long	load(MasterInfo info, String path, int currentLogFileNum) Load edit log.
static void	loadSingleLog(MasterInfo info, String path) Load one edit log.
static void	markUpToDate(String path) Make the edit log up-to-date, It will delete all editlogs since sBackUpLogStartNum.
void	rename(int fileId, TachyonURI dstPath, long opTimeMs) Log a rename operation.
void	rotateEditLog(String path) The edit log reaches the max log size and needs rotate.
void	setPinned(int fileId, boolean pinned, long opTimeMs) Log a setPinned operation.
void	updateRawTableMetadata(int tableId, ByteBuffer metadata) Log an updateRawTableMetadata operation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EditLog

public EditLog(String path,
       boolean inactive,
       long transactionId)
        throws IOException

Create a new EditLog

Parameters:

path - The path of the edit logs.

inactive - If a master is replaying an edit log, the current edit log is inactive.

transactionId - The beginning transactionId of the edit log

Throws:

IOException

Method Detail

load

public static long load(MasterInfo info,
        String path,
        int currentLogFileNum)
                 throws IOException

Load edit log.

Parameters:

info - The Master Info.

path - The path of the edit logs.

currentLogFileNum - The smallest completed log number that this master has not loaded

Returns:

The last transaction id.

Throws:

IOException

loadSingleLog

public static void loadSingleLog(MasterInfo info,
                 String path)
                          throws IOException

Load one edit log.

Parameters:

info - The Master Info

path - The path of the edit log

Throws:

IOException

markUpToDate

public static void markUpToDate(String path)

Make the edit log up-to-date, It will delete all editlogs since sBackUpLogStartNum.

Parameters:

path - The path of the edit logs

addBlock

public void addBlock(int fileId,
            int blockIndex,
            long blockLength,
            long opTimeMs)

Log an addBlock operation. Do nothing if the edit log is inactive.

Parameters:

fileId - The id of the file

blockIndex - The index of the block to be added

blockLength - The length of the block to be added

opTimeMs - The time of the addBlock operation, in milliseconds

addCheckpoint

public void addCheckpoint(int fileId,
                 long length,
                 TachyonURI checkpointPath,
                 long opTimeMs)

Log an addCheckpoint operation. Do nothing if the edit log is inactive.

Parameters:

fileId - The file to add the checkpoint

length - The length of the checkpoint

checkpointPath - The path of the checkpoint

opTimeMs - The time of the addCheckpoint operation, in milliseconds

close

public void close()

Close the log.

completeFile

public void completeFile(int fileId,
                long opTimeMs)

Log a completeFile operation. Do nothing if the edit log is inactive.

Parameters:

fileId - The id of the file

opTimeMs - The time of the completeFile operation, in milliseconds

createDependency

public void createDependency(List<Integer> parents,
                    List<Integer> children,
                    String commandPrefix,
                    List<ByteBuffer> data,
                    String comment,
                    String framework,
                    String frameworkVersion,
                    DependencyType dependencyType,
                    int depId,
                    long creationTimeMs)

Log a createDependency operation. The parameters are like creating a new Dependency. Do nothing if the edit log is inactive.

Parameters:

parents - The input files' id of the dependency

children - The output files' id of the dependency

commandPrefix - The prefix of the command used for recomputation

data - The list of the data used for recomputation

comment - The comment of the dependency

framework - The framework of the dependency, used for recomputation

frameworkVersion - The version of the framework

dependencyType - The type of the dependency, DependencyType.Wide or DependencyType.Narrow

depId - The id of the dependency

creationTimeMs - The create time of the dependency, in milliseconds

createFile

public void createFile(boolean recursive,
              TachyonURI path,
              boolean directory,
              long blockSizeByte,
              long creationTimeMs)

Log a createFile operation. Do nothing if the edit log is inactive.

Parameters:

recursive - If recursive is true and the filesystem tree is not filled in all the way to path yet, it fills in the missing components.

path - The path to create

directory - If true, creates an InodeFolder instead of an Inode

blockSizeByte - If it's a file, the block size for the Inode

creationTimeMs - The time the file was created

createRawTable

public void createRawTable(int tableId,
                  int columns,
                  ByteBuffer metadata)

Log a createRawTable operation. Do nothing if the edit log is inactive.

Parameters:

tableId - The id of the raw table

columns - The number of columns in the table

metadata - Additional metadata about the table

delete

public void delete(int fileId,
          boolean recursive,
          long opTimeMs)

Log a delete operation. Do nothing if the edit log is inactive.

Parameters:

fileId - the file to be deleted.

recursive - whether delete the file recursively or not.

opTimeMs - The time of the delete operation, in milliseconds

deleteCompletedLogs

public void deleteCompletedLogs(String path,
                       int upTo)

Delete the completed logs.

Parameters:

path - The path of the logs

upTo - The logs in the path from 0 to upTo-1 are completed and to be deleted

flush

public void flush()

Flush the log onto the storage.

getTransactionIds

public Pair<Long,Long> getTransactionIds()

Get the current TransactionId and FlushedTransactionId

Returns:

(TransactionId, FlushedTransactionId)

rename

public void rename(int fileId,
          TachyonURI dstPath,
          long opTimeMs)

Log a rename operation. Do nothing if the edit log is inactive.

Parameters:

fileId - The id of the file to rename

dstPath - The new path of the file

opTimeMs - The time of the rename operation, in milliseconds

rotateEditLog

public void rotateEditLog(String path)

The edit log reaches the max log size and needs rotate. Do nothing if the edit log is inactive.

Parameters:

path - The path of the edit log

setPinned

public void setPinned(int fileId,
             boolean pinned,
             long opTimeMs)

Log a setPinned operation. Do nothing if the edit log is inactive.

Parameters:

fileId - The id of the file

pinned - If true, the file is never evicted from memory

opTimeMs - The time of the setPinned operation, in milliseconds

updateRawTableMetadata

public void updateRawTableMetadata(int tableId,
                          ByteBuffer metadata)

Log an updateRawTableMetadata operation. Do nothing if the edit log is inactive.

Parameters:

tableId - The id of the raw table

metadata - The new metadata of the raw table