Next: The Current State of Up: Bacula Concepts and Overview Previous: New Features in 3.0.0 Contents Index
We plan to have plugins register to receive events that they normally would not receive, such as an event for each file examined for backup or restore. This feature is not yet implemented.
The second type of plugin, which is more useful and fully implemented in the current version is what we call a command plugin. As with all plugins, it gets notified of important events as noted above (details described below), but in addition, this kind of plugin can accept a command line, which is a:
Plugin = <command-string>
directive that is placed in the Include section of a FileSet and is very similar to the "File = " directive. When this Plugin directive is encountered by Bacula during backup, it passes the "command" part of the Plugin directive only to the plugin that is explicitly named in the first field of that command string. This allows that plugin to backup any file or files on the system that it wants. It can even create "virtual files" in the catalog that contain data to be restored but do not necessarily correspond to actual files on the filesystem.
The important features of the command plugin entry points are:
Bacula passes information to the plugin through this call and it gets back information that it needs to use the plugin. Later, Bacula will call particular functions that are defined by the loadPlugin interface.
When Bacula is finished with the plugin (when Bacula is going to exit), it will call the unloadPlugin entry point.
The two entry points are:
bRC loadPlugin(bInfo *lbinfo, bFuncs *lbfuncs, pInfo **pinfo, pFuncs **pfuncs) and bRC unloadPlugin()
both these external entry points to the shared object are defined as C entry points to avoid name mangling complications with C++. However, the shared object can actually be written in any language (preferrably C or C++) providing that it follows C language calling conventions.
The definitions for bRC and the arguments are src/filed/fd-plugins.h and so this header file needs to be included in your plugin. It along with src/lib/plugins.h define basically the whole plugin interface. Within this header file, it includes the following files:
#include <sys/types.h> #include "config.h" #include "bc_types.h" #include "lib/plugins.h" #include <sys/stat.h>
Aside from the bc_types.h and confit.h headers, the plugin definition uses the minimum code from Bacula. The bc_types.h file is required to ensure that the data type defintions in arguments correspond to the Bacula core code.
The return codes are defined as:
typedef enum {
bRC_OK = 0, /* OK */
bRC_Stop = 1, /* Stop calling other plugins */
bRC_Error = 2, /* Some kind of error */
bRC_More = 3, /* More files to backup */
} bRC;
At a future point in time, we hope to make the Bacula libbac.a into a shared object so that the plugin can use much more of Bacula's infrastructure, but for this first cut, we have tried to minimize the dependence on Bacula.
bRC loadPlugin(bInfo *lbinfo, bFuncs *lbfuncs, pInfo **pinfo, pFuncs **pfuncs)
and the arguments are:
typedef struct s_baculaInfo {
uint32_t size;
uint32_t version;
} bInfo;
The exact definition as of this writing is:
typedef struct s_baculaFuncs {
uint32_t size;
uint32_t version;
bRC (*registerBaculaEvents)(bpContext *ctx, ...);
bRC (*getBaculaValue)(bpContext *ctx, bVariable var, void *value);
bRC (*setBaculaValue)(bpContext *ctx, bVariable var, void *value);
bRC (*JobMessage)(bpContext *ctx, const char *file, int line,
int type, utime_t mtime, const char *fmt, ...);
bRC (*DebugMessage)(bpContext *ctx, const char *file, int line,
int level, const char *fmt, ...);
void *(*baculaMalloc)(bpContext *ctx, const char *file, int line,
size_t size);
void (*baculaFree)(bpContext *ctx, const char *file, int line, void *mem);
} bFuncs;
We will discuss these entry points and how to use them a bit later when describing the plugin code.
The exact definition as of this writing is:
typedef struct s_pluginInfo {
uint32_t size;
uint32_t version;
const char *plugin_magic;
const char *plugin_license;
const char *plugin_author;
const char *plugin_date;
const char *plugin_version;
const char *plugin_description;
} pInfo;
Where:
The pInfo structure must be defined in static memory because Bacula does not copy it and may refer to the values at any time while the plugin is loaded. All values must be supplied or the plugin will not run (not yet implemented). All text strings must be either ASCII or UTF-8 strings that are terminated with a zero byte.
The pFuncs structure must be defined in static memory because Bacula does not copy it and may refer to the values at any time while the plugin is loaded.
The exact definition as of this writing is:
typedef struct s_pluginFuncs {
uint32_t size;
uint32_t version;
bRC (*newPlugin)(bpContext *ctx);
bRC (*freePlugin)(bpContext *ctx);
bRC (*getPluginValue)(bpContext *ctx, pVariable var, void *value);
bRC (*setPluginValue)(bpContext *ctx, pVariable var, void *value);
bRC (*handlePluginEvent)(bpContext *ctx, bEvent *event, void *value);
bRC (*startBackupFile)(bpContext *ctx, struct save_pkt *sp);
bRC (*endBackupFile)(bpContext *ctx);
bRC (*startRestoreFile)(bpContext *ctx, const char *cmd);
bRC (*endRestoreFile)(bpContext *ctx);
bRC (*pluginIO)(bpContext *ctx, struct io_pkt *io);
bRC (*createFile)(bpContext *ctx, struct restore_pkt *rp);
bRC (*setFileAttributes)(bpContext *ctx, struct restore_pkt *rp);
bRC (*checkFile)(bpContext *ctx, char *fname);
} pFuncs;
The details of the entry points will be presented in separate sections below.
Where:
Sample code for loadPlugin:
bfuncs = lbfuncs; /* set Bacula funct pointers */ binfo = lbinfo; *pinfo = &pluginInfo; /* return pointer to our info */ *pfuncs = &pluginFuncs; /* return pointer to our functions */ return bRC_OK;
where pluginInfo and pluginFuncs are statically defined structures. See bpipe-fd.c for details.
The bpContext structure will be passed to the plugin, and during this call, if the plugin needs to have its private working storage that is associated with the particular instance of the plugin, it should create it from the heap (malloc the memory) and store a pointer to its private working storage in the pContext variable. Note: since Bacula is a multi-threaded program, you must not keep any variable data in your plugin unless it is truely meant to apply globally to the whole plugin. In addition, you must be aware that except the first and last call to the plugin (loadPlugin and unloadPlugin) all the other calls will be made by threads that correspond to a Bacula job. The bpContext that will be passed for each thread will remain the same throughout the Job thus you can keep your privat Job specific data in it (bContext).
typedef struct s_bpContext {
void *pContext; /* Plugin private context */
void *bContext; /* Bacula private context */
} bpContext;
This context pointer will be passed as the first argument to all the entry points that Bacula calls within the plugin. Needless to say, the plugin should not change the bContext variable, which is Bacula's private context pointer for this instance (Job) of this plugin.
typedef struct s_bEvent {
uint32_t eventType;
} bEvent;
which defines what event has been triggered, and for each event, Bacula will pass a pointer to a value associated with that event. If no value is associated with a particular event, Bacula will pass a NULL pointer, so the plugin must be careful to always check value pointer prior to dereferencing it.
The current list of events are:
typedef enum {
bEventJobStart = 1,
bEventJobEnd = 2,
bEventStartBackupJob = 3,
bEventEndBackupJob = 4,
bEventStartRestoreJob = 5,
bEventEndRestoreJob = 6,
bEventStartVerifyJob = 7,
bEventEndVerifyJob = 8,
bEventBackupCommand = 9,
bEventRestoreCommand = 10,
bEventLevel = 11,
bEventSince = 12,
} bEventType;
Most of the above are self-explanatory.
Note, if you intend to backup a file, this is an important first point to write code that copies the command string passed into your pContext area so that you will know that a backup is being performed and you will know the full contents of the "Plugin =" command (i.e. what to backup and what virtual filename the user wants to call it.
See the notes above concerning backup and the command string. This is the point at which Bacula passes you the original command string that was specified during the backup, so you will want to save it in your pContext area for later use when Bacula calls the plugin again.
During each of the above calls, the plugin receives either no specific value or only one value, which in some cases may not be sufficient. However, knowing the context of the event, the plugin can call back to the Bacula entry points it was passed during the loadPlugin call and get to a number of Bacula variables. (at the current time few Bacula variables are implemented, but it easily extended at a future time and as needs require).
struct save_pkt {
int32_t pkt_size; /* size of this packet */
char *fname; /* Full path and filename */
char *link; /* Link name if any */
struct stat statp; /* System stat() packet for file */
int32_t type; /* FT_xx for this file */
uint32_t flags; /* Bacula internal flags */
bool portable; /* set if data format is portable */
char *cmd; /* command */
int32_t pkt_end; /* end packet sentinel */
};
The second argument is a pointer to the save_pkt structure for the file to be backed up. The plugin is responsible for filling in all the fields of the save_pkt. If you are backing up a real file, then generally, the statp structure can be filled in by doing a stat system call on the file.
If you are backing up a database or something that is more complex, you might want to create a virtual file. That is a file that does not actually exist on the filesystem, but represents say an object that you are backing up. In that case, you need to ensure that the fname string that you pass back is unique so that it does not conflict with a real file on the system, and you need to artifically create values in the statp packet.
Example programs such as bpipe-fd.c show how to set these fields. You must take care not to store pointers the stack in the pointer fields such as fname and link, because when you return from your function, your stack entries will be destroyed. The solution in that case is to malloc() and return the pointer to it. In order to not have memory leaks, you should store a pointer to all memory allocated in your pContext structure so that in subsequent calls or at termination, you can release it back to the system.
Once the backup has begun, Bacula will call your plugin at the pluginIO entry point to "read" the data to be backed up. Please see the bpipe-fd.c plugin for how to do I/O.
Example of filling in the save_pkt as used in bpipe-fd.c:
struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext; time_t now = time(NULL); sp->fname = p_ctx->fname; sp->statp.st_mode = 0700 | S_IFREG; sp->statp.st_ctime = now; sp->statp.st_mtime = now; sp->statp.st_atime = now; sp->statp.st_size = -1; sp->statp.st_blksize = 4096; sp->statp.st_blocks = 1; p_ctx->backup = true; return bRC_OK;
Note: the filename to be created has already been created from the command string previously sent to the plugin and is in the plugin context (p_ctx->fname) and is a malloc()ed string. This example creates a regular file (S_IFREG), with various fields being created.
In general, the sequence of commands issued from Bacula to the plugin to do a backup while processing the "Plugin = " directive are:
The data in the restore_pkt is passed to the plugin and is based on the data that was originally given by the plugin during the backup and the current user restore settings (e.g. where, RegexWhere, replace). This allows the plugin to first create a file (if necessary) so that the data can be transmitted to it. The next call to the plugin will be a pluginIO command with a request to open the file write-only.
This call must return one of the following values:
enum {
CF_SKIP = 1, /* skip file (not newer or something) */
CF_ERROR, /* error creating file */
CF_EXTRACT, /* file created, data to extract */
CF_CREATED /* file created, no data to extract */
};
in the restore_pkt value create_status. For a normal file, unless there is an error, you must return CF_EXTRACT.
struct restore_pkt {
int32_t pkt_size; /* size of this packet */
int32_t stream; /* attribute stream id */
int32_t data_stream; /* id of data stream to follow */
int32_t type; /* file type FT */
int32_t file_index; /* file index */
int32_t LinkFI; /* file index to data if hard link */
uid_t uid; /* userid */
struct stat statp; /* decoded stat packet */
const char *attrEx; /* extended attributes if any */
const char *ofname; /* output filename */
const char *olname; /* output link name */
const char *where; /* where */
const char *RegexWhere; /* regex where */
int replace; /* replace flag */
int create_status; /* status from createFile() */
int32_t pkt_end; /* end packet sentinel */
};
Typical code to create a regular file would be the following:
struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext; time_t now = time(NULL); sp->fname = p_ctx->fname; /* set the full path/filename I want to create */ sp->type = FT_REG; sp->statp.st_mode = 0700 | S_IFREG; sp->statp.st_ctime = now; sp->statp.st_mtime = now; sp->statp.st_atime = now; sp->statp.st_size = -1; sp->statp.st_blksize = 4096; sp->statp.st_blocks = 1; return bRC_OK;
This will create a virtual file. If you are creating a file that actually exists, you will most likely want to fill the statp packet using the stat() system call.
Creating a directory is similar, but requires a few extra steps:
struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext; time_t now = time(NULL); sp->fname = p_ctx->fname; /* set the full path I want to create */ sp->link = xxx; where xxx is p_ctx->fname with a trailing forward slash sp->type = FT_DIREND sp->statp.st_mode = 0700 | S_IFDIR; sp->statp.st_ctime = now; sp->statp.st_mtime = now; sp->statp.st_atime = now; sp->statp.st_size = -1; sp->statp.st_blksize = 4096; sp->statp.st_blocks = 1; return bRC_OK;
The link field must be set with the full cononical path name, which always ends with a forward slash. If you do not terminate it with a forward slash, you will surely have problems later.
As with the example that creates a file, if you are backing up a real directory, you will want to do an stat() on the directory.
Note, if you want the directory permissions and times to be correctly restored, you must create the directory after all the file directories have been sent to Bacula. That allows the restore process to restore all the files in a directory using default directory options, then at the end, restore the directory permissions. If you do it the other way around, each time you restore a file, the OS will modify the time values for the directory entry.
See the definition of restre_pkt in the above section.
enum {
IO_OPEN = 1,
IO_READ = 2,
IO_WRITE = 3,
IO_CLOSE = 4,
IO_SEEK = 5
};
struct io_pkt {
int32_t pkt_size; /* Size of this packet */
int32_t func; /* Function code */
int32_t count; /* read/write count */
mode_t mode; /* permissions for created files */
int32_t flags; /* Open flags */
char *buf; /* read/write buffer */
const char *fname; /* open filename */
int32_t status; /* return status */
int32_t io_errno; /* errno code */
int32_t lerror; /* Win32 error code */
int32_t whence; /* lseek argument */
boffset_t offset; /* lseek argument */
bool win32; /* Win32 GetLastError returned */
int32_t pkt_end; /* end packet sentinel */
};
The particular Unix function being simulated is indicated by the func, which will have one of the IO_OPEN, IO_READ, ... codes listed above. The status code that would be returned from a Unix call is returned in status for IO_OPEN, IO_CLOSE, IO_READ, and IO_WRITE. The return value for IO_SEEK is returned in offset which in general is a 64 bit value.
When there is an error on Unix systems, you must always set io_error, and on a Win32 system, you must always set win32, and the returned value from the OS call GetLastError() in lerror.
For all except IO_SEEK, status is the return result. In general it is a positive integer unless there is an error in which case it is -1.
The following describes each call and what you get and what you should return:
Note: Bacula will call IO_SEEK only when writing a sparse file.
cd <bacula-source> ./configure <your-options> make ... cd src/plugins/fd make make test
After building Bacula and changing into the src/plugins/fd directory, the make command will build the bpipe-fd.so plugin, which is a very useful and working program.
The make test command will build the example-plugin-fd.so plugin and a binary named main, which is build from the source code located in src/filed/fd_plugins.c.
If you execute ./main, it will load and run the example-plugin-fd plugin simulating a small number of the calling sequences that Bacula uses in calling a real plugin. This allows you to do initial testing of your plugin prior to trying it with Bacula.
You can get a good idea of how to write your own plugin by first studying the example-plugin-fd, and actually running it. Then it can also be instructive to read the bpipe-fd.c code as it is a real plugin, which is still rather simple and small.
When actually writing your own plugin, you may use the example-plugin-fd.c code as a template for your code.