Package tachyon.worker

Worker process and utils for working with the worker remotely.

See: Description

Interface Summary

Interface	Description
DataServer	Defines how to interact with a server running the data protocol.

Class Summary

Class	Description
BlockHandler	Base class for handling block I/O.
BlockHandlerLocal	BlockHandler for files on LocalFS, such as RamDisk, SSD and HDD.
BlocksLocker	Handle local block locking.
SpaceCounter	The worker space counter, in charge of counting and granting spaces in a worker daemon.
TachyonWorker	Entry point for a worker daemon.
WorkerClient	The client talks to a worker server.
WorkerServiceHandler	WorkerServiceHandler handles all the RPC calls to the worker.
WorkerStorage	The structure to store a worker's information in worker node.

Enum Summary

Enum	Description
NetworkType	Enum over the different networking implementations.

Package tachyon.worker Description

Worker process and utils for working with the worker remotely. Main entry point for the worker is TachyonWorker.main(String[]) which gets started by the tachyon start scripts. The TachyonWorker class spins up the different RPC services (thrift, data) which are mostly wrappers around WorkerStorage.

Services

Thrift

The thrift service is mostly responsible for metadata operations (with a few operations that effect the worker's cached memory).

Checkpoint

The act of moving temporary data into accessible data on UnderFileSystem. This is triggered by WriteType.isThrough() operations. Implementation can be found at WorkerStorage.addCheckpoint(long, int)

Cache Block

Move's user generated blocks to the tachyon data directory. This operation expects that the caller is a local (to the node) caller, and that the input are under the user directories. Implementation can be found at WorkerStorage.cacheBlock(long, long)

Lock / Unlock

Tachyon supports cacheing blocks to local disk (client side). When this happens, a lock is given to the client to protect it from the remote block changing. Implementation can be found at WorkerStorage.lockBlock(long, long) and WorkerStorage.unlockBlock(long, long).

Data

This service is the main interaction between users and reading blocks. Currently this service only supports reading blocks (writing is to local disk). There are two different implementations of this layer: NettyDataServer and NIODataServer; netty is the default. To support both, a DataServer interface is used that just defines how to start/stop, and get port details; to start, object init is used. The current read protocol is defined in DataServerMessage. This has two different types: read DataServerMessage.createBlockRequestMessage(long, long, long) and write DataServerMessage.createBlockResponseMessage(boolean, long, long, long, ByteBuffer). Side note, the netty implementation does not use this class, but has defined two classes for the read and write case: BlockRequest, BlockResponse; theses classes are network compatible.