Administration Reference Guide

This is the install program:

Given arguments specify where to install the DB. They may be paths for directories in the local host, or host names, or host and directory names using the syntax host:dir.

Instead of arguments, a configuration file may be given using the -f flag. The configuration file has been described before in this document.

Flag -k may be used to specify the key to download the distribution.

Flag '-K' must be used when installing on AWS to supply the path (without file name extension) to the pem/pub key files.

By default, the distribution is retrieved from the leanXcale artifactory. However, it is possible to supply one or more times the -d flag giving it the name for a .tgz file, a directory, or a URL. The special argument leanxcale stands for the official repository for the distribution.

Distribution packages will be retrieved by looking at those distribution sources. By convention, the first source is usally the ./lxdist directory, used to keep the distribution when downloaded.

For example, this installs just what is downloaded at ./lxdist, without downloading anything else:

Once downloaded, files are not downloaded again. Flag ''-c'' cleans the ''./lxdist'' directory to force a download of everything. If a different directory is specified by the user, it is not cleaned, although packages not found will be still downloaded there. To force a download of particular packages, remove the packages desired from the ''./lxdist'' directory (or the directory specified in the arguments).

For example, this downloads a fresh copy of the distribution and installs it:

And this tries first the ../lxdist directory and then the standard leanxcale repository:

This is actually the default when no distribution source is specified. Also, if no directory source is specified as a first option, lxdist is used as a directory to download components (other than files found on the local host).

The database has users (different from UNIX users). The user lxadmin is the administrator for the whole system. During the install you will be asked to type a password for lxadmin. To specify the password without being asked, you can set the password in the LXPASS environment variable:

When the users file ./lxowner exists, that file is used as the file describing the lxadmin user and its secret, instead of creating one using the user supplied password for lxadmin.

Do not use a existing file unless you know what you are doing. The reason is that this file must define lxadmin as the first user and that such user must have access to all resources, as configured.

Flag -i makes lxinst ignore system limits, to install on places with reduced disk or memory and in a hurry.

By default, the install performs a large install, trying to use all the machine resources for the service. Flag -s selects a small instead, and flag -m selects a medium install.

The install size affects components and sizes not specified by the user in command line arguments or in the configuration file. In a small install, at most 4 kdvs are added per host, and component memory is limited to 1GiB. In a medium install, at most 8 kvds components are added, and component memory is left with default values.

Flag -u is used to update a previous install. Still WIP, do not use it for now.

Shell for running LeanXcale control programs. This simply fixes the environment for the installed host and runs the command given as an argument:

When flag -d is used, the current working directory is set to the install directory at the host before running the given command.

Most commands follow the same conventions regarding options and arguments. We describe them here for convenience:

Flag -l is used when executing the command locally. This is used by the lx command framework, and should not be used in general by the end user.

Flag -D enables verbose diagnostics

Arguments specify what to operate (e.g., what to start, stop, etc.) may be empty to rely on the defaults (whole DB) or may specify a particular host and/or component name:

This may be repeated to specify different hosts and/or components.

The special host names db, repl, and repl2 may be used and stand for hosts without the nodb attribute, hosts for the first replica, and hosts for the second replica (hosts that are a mirror of other ones).

Add the given file(s) to the lib directory of the installed hosts, to add or update a library or jar in the installation:

Create, list, and restore backups from a installed host:

This command copies disk contents to a backup directory or restores them. See lxbackup for backups/restores on external hosts.

By default, the backup is $LXDIR/dump at the host used to issue backup commands. The convention is to use the first host to keep backups.

This directory can be a mount point or a link to a remote directory to keep backups at a different machine.

Directories under dump/ are named using the date, with .1, .2, etc. appended if more than one backup is created on the same date.

For incremental backups, a final + is added to the directory name.

For example, create a full, cold, backup when leanXcale is not running:

The printed path is the path for the directory keeping the backup, as used when restoring it.

Using flag -z makes the tool gzip backup files, preserving modification times despite compression. For example:

Using flag -C both encrypts and compresses the backup files. The flag argument is the path to the file keeping the file used to encrypt/decrypt.

To create an incremental (cold) backup:

It is possible to backup just one or a few components, supplying arguments that address them like in other commands, as described before. In this case, it is advisable to perform always backups for the same set of components. This command does not know if the last full backup is for the whole database or just for a few components. The last full backup is considered the full backup for the whole thing.

In general, the system should not be running while doing a backup. To perform a hot backup, backup just the query engine files as an incremental dump:

Incremental backups can be mixed for different components, as they overwrite some of the files saved in the full backup.

Create a full backup for just kvds100:

Create an incremental backup adding just kvds200:

List the backups known:

Those with a + in their names are incremental backups. Those encrypted are reported as such.

List the backups known for any kvds:

Restore the backup with the given name:

Here, if the name refers to an incremental backup, the restore recovers also the files found in previous incremental backups and in the total backup made before them.

Also, target directories to restore are identified by the current configuration of the system. That is, if kvds100 changed its location to a different host, the restore process will restore its disk at the new location, not at the location used to create the backup.

Restore just the files for kvds components from the given backup:

To restore a backup, it is usually desirable to format the disk for the involved components before using backup to restore their disks. The restore process copies restored files back, but does not remove anything else found on the disk for the component. For example:

To remove a backup, it suffices to remove its directory. For example:

Here we used the flag “-d” for lx to change to $LXDIR before executing the remove command, which makes it easy to name the directory used for the dump.

An important note is that file modification times are preserved in backup files. They are used to learn if a file must be copied or not in an incremental backup. If a backup is relocated to a different place, and later copied back into the standard location, preserve file modification times in the process.

Using flag -z makes the tool gzip backup files, preserving modification times despite compression.

When restoring encrypted backups, flag -C must be used to supply the path to the file keeping the secret used to decrypt.

Create, list, and restore backups from an external host:

This command can be copied along with the installed configuration to an external host, to perform external backups/restores. The external host must have ssh access to the installed hosts.

See lx backup for backups/restores on installed hosts.

Using lxbackup is exactly like using lx backup with a few differences:

For example, create a full backup for just kvds100:

The lxbackup command can be found at the installed $LXDIR/bin directory on any installed host. If you are not sure regarding the $LXDIR value, use this command to find it:

Flag -d for lx makes it change to the $LXDIR directory before running the given command.

The detailed configuration file to be used is created at lxinst.conf when installing. The configuration can be retrieved also by the lx config command. For example:

Therefore, in the running example, we can setup an external host named orion to perform backups in this way:

And then just:

This was for a full (cold) system backup. To perform a hot backup create an incremental backup with the query engine files:

Using flag -z in calls to lxbackup makes the tool gzip backup files, preserving modification times despite compression.

Using flag -C both encrypts and compresses the backup files. The flag argument is the path to the file keeping the secret used to encrypt/decrypt.

If any backup is encrypted, -C must be used also to restore, as in

Inspect or update the configuration:

By default, lx config prints the whole configuration, or that for elements given as arguments.

The arguments follow the conventional syntax used by most commands, but knows property names also:

For example,

prints all configuration.

prints the configuration for hosts atlantis and mariner (assuming those are configured host names).

prints the configuration for kvms components found at atlantis and lxqe components found at mariner

prints the configuration for any kvms component and the kvds101 one.

prints the addr attribute for any kvms component.

prints the awsdisk property from the (global) configuration.

Use flag -o to save the configuration (or the parts selected) to the given file.

Use flag -d to remove the selected configuration entries. Do not remove hosts or components, and use this with caution.

Use flag -s to update the selected properties with new values. In this case, the argument for a property includes both its name and the new value (e.g., mem=50m)

For example:

Disable transactions.

For example,

disables transactions for just the entire system, until the they are enabled.

While transactions are disabled, lx status shows the system status as waiting (for transactions to become available).

Before using this command, stop the watcher if the system was starting with flag -w or lx watch is running.

Otherwise, because new JDBC connections are not accepted while transactions are disabled, the watcher might consider that the system is not in good health, and then restart the system.

Enable transactions.

For example,

enables transactions for just the qe100 query engine.

Format the whole DB or the indicated hosts or components:

For example, format the kvds101 disk:

Ask for help:

Prints the list of known commands with quick usage information, or detailed usage information about the given command.

Checks the license status or installs a new license:

For example, ask for the current status:

or install a new file lxlicense with the desired license:

List and inspect logs for installed components:

For example, list the logs for kvds at atlantis, but only those after the time 230518.0551 (yymmdd.hhmm, or a prefix of this, can be used):

Print the last log for kvds101:

Grep all the logs for lines with fatal:

Copy all (not just the last one) kvds101 logs to /tmp:

Print the transaction log contents:

The command prints the log contents for the given log directory (or file). With flag -v, it prints the logged kv message, otherwise it prints just the first line of each logged message.

With flag -h (or -v) it prints the header information too.

It is suggested not to use this on a running log, just in case the program modifies the logger (although that should not happen).

List the processes for the DB:

For example, list the processes at host blade123:

Or to see the port ustage status:

Report system status and debug information.

This program collects information from the system and builds an archive to be sent to support.

As printed by the command output, the resulting tar file should be sent to support.

The archive includes:

When kvms is still running, the archive includes also:

Run a command on the selected installed hosts, using the lx environment to run it:

Arguments specify hosts where to run (as usual in the rest of commands), perhaps none (to imply all DB hosts), and then the keyword cmd must be given, followed by the command and arguments to run on each host.

For example, discover where this thing is installed:

Or, as an exceptional measure, kill all processes running:

Or, kill just those at blade123:

Or kill -9 any kvds at blade123:

Dump process stacks

The command understands the conventional syntax to select hosts and/or components. For example, to dump the stack of kvms components:

With flag -v, local variables are printed too.

For java processes, both the native stack and the java stacks are printed.

Starts the DB for operation:

For example, to start the kvds components at host atlantis and leave the rest of the installation alone:

With flag -w, start will run the watch service. This pings the DB to make sure it can answer queries, and, when that is not the case, try to stop and restart the system. See the Section 1.22 section for details.

By default, start will remove files on the installed directories with names suggesting they are core dumps. Flag -k prevents this from happening and can be used to keep core dump files for debugging.

Prints the status for the system or waits for a given status:

For example, to learn the status:

Or, to wait until the status is running:

The status can be any of:

Halts the DB or stops individual hosts or components:

For example, halt the DB:

Stop just the kvds servers:

If the watch service is running, make sure to stop it before stopping individual components. Otherwise it might decide to wait for lxmeta to stop and then restart the system.

Without arguments, stop will first stop the watch service and there is no extra caution needed.

Under flag -w, stop will not stop the watch process.

Flag -k may be used to kill components. Do not use this unless you know what you are doing. For example, to kill the lxqe100

Print the installed version:

The program reports the leanXcale version name, along with detailed version information for distribution packages installed:

Watch out the system and restart it if needed:

This program is started by lx start when the flag -w is given to it. It is usually a bad idea to execute this command explicitly.

It waits until the DB is running, doing nothing until that point. From that point on, if the DB ceases to be running, the program will wait for lxmeta to stop, and, then try to stop and start the whole system, and finally exit.

When restarting the system, the flag -w is used, to start a new watcher for the new system.

The implications are that a failure to start will not restart the system more times, and that stopping individual components requires to stop this program first, or it might take actions on its own.

This is a command for local use only:

Use lx run to run it at any/all of the installed hosts.

Without any flag, locates the DB process pids looking at the files reporting them, and then kills them.

With flag -a, locates any process in the system by process name (all the ones started by the DB use bin/…​ as a name), and all java processes starting with LX, and kills them.

A TERM signal is sent, and then a KILL signal after a few seconds.

Flag -9 can be used to send only a KILL signal.

If process/component names are given (eg, kvds, or lxqe100), only those processes are killed.

For example, send a kill signal to any kvds at blade123:

Properties with upper-case names are exported as environment variables for the process(es) executing the component involved. This includes both global properties and host properties as well. For example, to enable debug flags DOM for the kv datastore:

These are other properties not described before.