Command-line Tools

lvectl

lvectl is the primary tool for LVE management. To use it, you have to have administrator access. lvectl is a part of lve-utils package.

lvectl syntax

Usage: lvectl command [veid] [options]

Commands

apply apply config settings to specified LVE
apply all apply config settings to all the LVEs
apply-many to apply LVE limits to multiple distinct LVEs (uids of users are read from stdin)
set set parameters for a LVE and/or create a LVE
set-user set parameters for a LVE and/or create a LVE using username instead of ID
list list loaded LVEs
list-user list loaded LVEs, display username instead of user id
limits show limits for loaded LVEs
delete delete LVE and set configuration for that LVE to defaults
delete-user delete LVE and set configuration for that user to defaults
destroy destroy LVE (configuration file remains unchanged)
destroy all destroy all LVE (configuration file remains unchanged)
destroy-many to destroy LVE limits to multiple distinct LVEs (uids of users are read from stdin)
package-set set LVE parameters for a package
package-list list LVE parameters for packages
package-delete delete LVE parameters for a package
paneluserslimits show current user's limits for control panel
limit limit PID into specified LVE. Parameters PID LVE_ID
release release PID from LVE. Parameters PID
set-binary add binary to be run inside LVE on execution
del-binary remove binary from being run inside LVE on execution
list-binaries list all binaries to be run inside LVE on execution
load-binaries load binaries (used on startup) from config file
reload-binaries re-load list of binaries from config file
help (-h) show this message
version (-v) version number
lve-version LVE version number
set-reseller create LVE container and set LVE parameters for a reseller
set-reseller-default set default limits for resellers users
remove-reseller delete LVE container and the record in the config, move LVE containers to the host container

Options

--cpu=N limit CPU usage; (deprecated. Use --speed)
--speed=N% limit CPU usage in percentage; 100% is one core
--speed=Nmhz\ghz limit CPU usage in mhz\ghz
--ncpu=N limit VCPU usage (deprecated)
--io=N define io limits (KB/s)
--nproc=N limit number of processes
--pmem=N limit physical memory usage for applications inside LVE
--iops=N limit io per second
--mem=N mem alias for vmem (deprecated)
--vmem=N limit virtual memory for applications inside LVE
--maxEntryProcs=N limit number of entry processes
--save save configuration settings (use with set) (deprecated)
--save-all-parameters save all parameters even if they match with defaults settings
--json returns result of command json formatted
--unlimited set all limits to unlimited
--save-username save username in the config file. This parameter is used in conjunction with set-user

Examples

  • Reset all LVEs settings based on configuration in /etc/container/ve.cfg:

    $ lvectl apply all
    
  • Set new default CPU & Physical memory limit:

    $ lvectl set default --speed=100% --pmem=256m
    
  • Reset all LVE's killing processes inside them:

    $ lvectl destroy all
    
  • Show list of LVEs and their limits:

    $ lvectl list
    

lveps

lveps tool shows information about running LVEs, processes and threads belonging to them, CPU/memory/IO usage consumed by LVEs and their individual processes/threads. LVE is only reported if it is considered active (at least one thread belongs to that LVE or was running during measurement in dynamic mode).

Usage:

lveps [-p] [-n] [-o <fmt1:width1,...>] [-d] [-c <time>] [-s <style>] [-t] [-h]

Options:

-p to print per-process/per-thread statistics
-n to print LVE ID instead of username
-o to use formatted output (fmt=id,ep,pid,tid,cpu,mem,io)
-d to show dynamic cpu usage instead of total cpu usage
-c to calculate average cpu usage for <time> seconds (used with -d)
-r to run under realtime priority for more accuracy (needs privileges)
-s to sort LVEs in output (cpu, process, thread, mem, io)
-t to run in the top-mode
-h to print this brief help message

Command like lveps -p will display processes running inside 'active' LVEs.

CPU The number of seconds LVE/process/thread has been running (each CPU /core is counted separately), or the average CPU load (100% is all CPU resources) if used with -d.
MEM The number of megabytes of resident memory in use by LVE/process/thread (shared memory is not included).
IO The number of kilobytes read and written in sum by LVE, or kb/sec if used with -d.
ID LVE ID or username.
EP The number of entry processes inside LVE.
COM Command name for this process.
PID PID of the process.
PNO The number of processes belonging to the LVE.
TID TID of the thread.
TNO The number of threads belonging to the LVE.
DO The number of disk operations belonging to the LVE from the time it was created.
DT Total amount of disk transfer in megabytes from LVE creation time.
IOPS The number of I/O operations per second

lvetop

lvetop utility allows to monitor LVE usage:

ID        EP PNO TNO SPEED MEM  IO   IPOS
testuser1 0  1   1   0%    7    0    0
testuser2 0  0   0   5%    0    3    0
testuser3 1  2   2   0%    102  2727 0
testuser4 0  1   1   0%    12   84   1
testuser5 0  2   2   1%    52   0    0

lvetop fields:

ID user name if LVE id matches user id in /etc/passwd, or LVE id
EP number of entry processes (concurrent scripts executed)
PNO number of processes within LVE
TNO number of threads within LVE
CPU CPU usage by LVE, relative to total CPU resources of the server
MEM Memory usage by LVE, in KB
I/O I/O usage
IOPS number of read/write operations per second

cldetect

Note

lve-utils 1.2-10+

cldetect is used to detect installed software, and adjust CloudLinux options accordingly.

Usage:

/usr/bin/cldetect [--options]
cldetect -h/-h/--help show this message
--detect-cp prints control panel and its version (CP_NAME,CP_VERSION)
--detect-cp-full prints control panel, version and panel specific data (CP_NAME,CP_VERSION,...). Specific data: for ISP Manager5 - Master/Node
--detect-cp-nameonly prints control panel name (CP_NAME)
--get-admin-email prints control panel admin email (CP_ADMIN_EMAIL)
--cxs-installed check if CXS is installed. Returns 0 if installed, 1 otherwise
--cpanel-suphp-enabled check if suPHP is enabled in cPanel.Returns 0 if enabled, 1 otherwise
--detect-litespeed check if LiteSpeed is installed. Returns 0 if installed, 1 otherwise
--detect-postgresql check if PostGreSQL is installed. Returns 0 if installed, 1 otherwise
--print-apache-gid prints current apache gid
--print-da-admin prints DirectAdmin admin user
--set-securelinks-gid changes /etc/sysctl.conf if apache gid != 48 (default)
--set-nagios do some adjustments to make nagios work correctly if it's installed. Called as a part of --setup-supergids
--setup-supergids do some adjustments to make special users/software (nagios, cPanel’s mailman) work correctly if it is installed to the system
--cl-setup check if CloudLinux is installing. Returns 0 if installing, 1 otherwise
--update-license updates license
--update-new-key updates license with new key
--check-license check license. Returns OK if license is not older than 3 days, error message otherwise
-q check license. Returns 0 if license is not older than 3 days, 1 otherwise
--no-valid-license-screen returns no valid license found screen
--license-out-of-date-email returns License out of Date Email.
--check-openvz returns enviroment id

clsupergid auto-configuration

Each time lve-utils package is installed or upgraded it does some automatic system re-configuration to make some software (like nagios) work correctly, if it’s installed, by calling cldetect --setup-supergids command.

Starting from lve-utils 3.0-21 a behaviour of cldetect --set-nagios (now, it’s a part of cldetect --setup-supergids) command slightly changed.

Old behavior New behavior
If fs.proc_super_gid is 0 (which means it’s not configured) or it’s set to some GID that doesn’t exist in the system. Command will set sysctl fs.proc_super_gid to point to Nagios GID. Command will create special clsupergid group, setup sysctl fs.proc_super_gid to point to it’s GID and add Nagios user to this group.

If fs.proc_super_gid was configured by an admin to some existing group, the command will just add Nagios user to this group.

cldiag

cldiag utility is included in |lve-utils package and is intended for:

  • server diagnostics performed by a server administrator for detecting the most common errors in the configuration or software operation;
  • the focused check of the servers for typical errors performed by the support engineers before proceeding to the detailed analysis of the customer tickets;
  • servers automatic check by cron with the following generation of the reports and email them to the server administrator.

In all cases, for the negative checker result, the exit code will be > 0 (at the moment it will be equal to the number of failed checkers).

All the checkers support additional --json option with the unified output in an appropriate format.

For example:

# cldiag --symlinksifowner --json

{"total_errors": 0, "Check fs.enforce_symlinksifowner is correctly enabled in /etc/sysctl.conf": {"res": "OK", "msg": "fs.enforce_symlinksifowner = 1"}}

Each checker returns a typical result with the name of the check performed (brief description), one of the possible statuses of its execution, and detailed information on its fail or success.

For example:

Check fs.enforce_symlinksifowner is correctly enabled in /etc/sysctl.conf:

OK: fs.enforce_symlinksifowner = 1

Check suexec has cagefs jail:

SKIPPED: SuEXEC is not enabled

Possible checkers results:

  1. OK - the check succeeded.
  2. FAILED - the check revealed a problem.
  3. SKIPPED - the check was skipped as it made no sense in the actual environment (e.g. wrong control panel) or it could not be executed for some reason (e.g. no users with enabled CageFS found). Such result doesn’t mean there is a problem and can be considered as a positive.
  4. INTERNAL_TEST_ERROR - the check has failed because of a problem in the checker itself. Please report such problems using https://cloudlinux.zendesk.com/agent/.

The utility includes several built-in checkers, and can also import and run checkers from other utilities.

Currently implemented checkers:

  1. --diag-cp

Checks control panel and its configuration (for DirectAdmin only).

Checking control panel availability, thereby detecting it with our code. Displaying control panel name and version. Also, for DirectAdmin, checking if CloudLinux support is enabled in its config.

Fails if /usr/local/directadmin/custombuild/options.conf does not contain cloudlinux=yes line (for DirectAdmin control panel).

  1. --symlinksifowner

Checks fs.enforce_symlinksifowner is correctly enabled in /etc/sysctl.conf.

Checking specified kernel setup described in this docs section for deprecated value and displaying its current value.

Fails if /proc/sys/fs/enforce_symlinksifowner contains value 2 (it is deprecated and can cause issues for the system operation).

  1. --check-suexec

Checks suexec has cagefs jail.

In case if CageFS is installed and SuEXEC is on, checking if CageFS is enabled for SuEXEC.

Fails if CageFS is not enabled for suexec binary.

  1. --check-suphp

Checks suphp has cagefs jail.

In case if CageFS is installed and SuPHP is on, checking if CageFS is enabled for SuPHP.

Fails if CageFS is not enabled for suphp binary.

  1. --check-usepam

Checks UsePAM in /etc/ssh/sshd_config.

Checking if /etc/ssh/sshd_config config file contains UsePAM yes line, which is required for pam_lve correct work with sshd.

Fails if /etc/ssh/sshd_config contains UsePAM no line.

  1. --check-symlinkowngid

Checks fs.symlinkown_gid.

First checking if user Apache is available in the system (on some panels users httpd or nobody with special GID are present instead of Apache, they are detected correctly as well). Then, if such user exists, checking that his GID equals to the one specified in sysctl or that this user belongs to this supplemental group. If these conditions are met, then the protection effect described in this docs section is applied to this user, and the appropriate message will be displayed.

Fails if Apache user is not in the group specified in /proc/sys/fs/symlinkown_gid.

  1. --check-cpanel-packages

Checks existence of all user's packages (cPanel only)

Reading PLAN= for all users from /var/cpanel/users and checking if all of them are present in /var/cpanel/packages and if not, then displaying them in pairs like user: plan. default and undefined packages are excluded from the check.

Fails if users from /var/cpanel/users/ directory have non-existing packages (packages do not exist in /var/cpanel/packages/ directory, except for undefined and default).

  1. --check-defaults-cfg

Checks /etc/cl.selector/default.cfg.

Checking that if this config exists, the default PHP version is not disabled in it. Also performing minimal syntax checks for PHP modules settings and displaying the incorrect.

Fails if there are some problems in /etc/cl.selector/default.cfg found.

Possible reasons for failure:

  • Default version is undefined, which means /etc/cl.selector/default.cfg file does not contain section [versions] with the defined default version.
  • Default PHP version is disabled.
  1. --check-cagefs

All checks for CageFS are described separately in this docs section and their start from cagefsctl utility is completely equivalent to the start from cldiag and is designed only for a better experience.

This checker includes a set of CageFS sub-checkers, failure of one (or more) of them causes general checker failure.

  1. --check-php-conf

Checks /etc/cl.selector/php.conf.

Checking the config syntax for acceptable blocks and directives.

Fails if /etc/cl.selector/php.conf has incorrect format.

  • File contains an invalid parameter (valid parameters: Directive, Default, Type, Comment, Range, Remark).

  • File contains an invalid setting for the parameter Type (valid settings for the Type parameter: value, list, bool)

  1. --check-phpselector

Checks compatibility for the PHP Selector

Detecting which PHP handler has been configured on the server and checking its compatibility with the CloudLinux PHP Selector according to this table and displaying the corresponding message with the link to the documentation in case of a problem detected. No checks are performed for EasyApache3.

Failure reasons:

  • The installed mod_ruid package is incompatible with the PHP Selector
  • mod_suexec is not installed
  • mod_suphp is not installed
  • Absence of the /usr/local/lsws/conf/httpd_config.xml file (only for LiteSpeed server)

Note

The following checkers are available in lve-utils >= 3.1.2

  1. --check-lve-limits

Checks the validity of LVE limits on the server.

See this page for detailed description.

  1. --check-rpmdb

Checks the RPM database integrity.

Check that rpm database is operable and utils using it (e.g. yum) can work properly.

To start all available checkers at once, the keys -a | --all are used. This does not include Check compatibility for PHP Selector, it must be started separately with --check-phpselector key.

Automatic problems notifications cPanel only

The utility generates HTML reports and emails them to the administrator. You can change email for notifications by adding the following line to the /etc/sysconfig/cloudlinux.

EMAIL=username@domain.zone

The automatic checks using cldiag utility by cron job is enabled on a server by default. You can disable it in the /etc/sysconfig/cloudlinux config file by adding ENABLE_CLDIAG=no option. When calling this utility automatically by cron, it checks all limits existing on the server and sends an administrator a report with limits check results.

Usage examples

  • Run only one checker
# cldiag --check-phpselector

Check compatibility for PHP Selector:

  OK: It looks ok [php.conf:cgi with suexec, suphp]

There are 0 errors found.
  • Run all available checkers
# cldiag -a

Check control panel and it's configuration(for DirectAdmin only):

  OK: Control Panel - cPanel; Version 79.9999;

Check fs.enforce_symlinksifowner is correctly enabled in sysctl conf:

  OK: fs.enforce_symlinksifowner = 1

Check suexec has cagefs jail:

  OK: binary has jail

Check suphp has cagefs jail:

  OK: binary has jail

There are 0 errors found.

Common problems troubleshooting

Reasons and recommendations on how to fix common cldiag checkers failures.

  • --diag-cp

    Checks control panel and its configuration (for DirectAdmin only).

    On servers with DirectAdmin control panel, CloudLinux support should be enabled in custombuild config. To do so, set yes for the cloudlinux parameter via /usr/local/directadmin/custombuild/build utility. Please read this article to see how to set values to parameters.

  • --symlinksifowner

    Checks fs.enforce_symlinksifowner is correctly enabled in /etc/sysctl.conf.

    To fix warning, change the value of /proc/sys/fs/enforce_symlinksifowner. See Symlink Owner Match Protection to know more about the fs.enforce_symlinksifowner parameter and configure it correctly.

    Fixing that warning makes the server more protected against symlink attacks and enables protection of PHP configs and other sensitive files.

  • --check-symlinkowngid

    Checks fs.symlinkown_gid.

    Symlink Owner Match Protection is not enabled for the Apache user. To enable it, see Symlink Owner Match Protection and set value of apache GID to the fs.symlinkown_gid parameter.

    Enabling Symlink Owner Match Protection provides protection for the Apacheuser and it may improve your server security.

  • --check-suexec

    Checks suexec has CageFS jail.

    Check that suexec package was installed from CloudLinux repository and if yes, try to reinstall it, probably suexec binary was broken.

    If you are running server without a control panel, you need to configure suexec by yourself. Please read this article to setup suexec on your server.

    Fix that warning to be sure that users run their sites inside CageFS and provide stable work of sites that are using Apache suexec module. This may improve server security.

  • --check-suphp

    Checks suphp has CageFS jail.

    Check that suphp package was installed from CloudLinux repo and if yes, try to reinstall it, probably suphp binary was broken.

    If you are running server without a control panel, you need to configure suphp by yourself. Please read this article | Installation and setup of a server using suPHP to setup suphp on your server.

    Fix that warning to be sure that users run their sites inside CageFS and provide stable work of sites that are using Apache suphp module. This may improve server security.

  • --check-usepam

    Checks UsePAM in /etc/ssh/sshd_config.

    To fix the issue, replace UsePAM no with Use PAM yes in the /etc/ssh/sshd_config file.

    Fix the warning to provide correct work of pam_lve module with sshd and CageFS ssh sessions, for details, please read this documentation about LVE PAM.

  • --check-defaults-cfg

    Checks /etc/cl.selector/default.cfg

    • In case of undefined default version error, go to LVE Manager | Selector Tab and set the default PHP version;
    • In case of default version disabled error, enable that version to fix the warning;

    /etc/cl.selector/defaults.cfg config file is used by the PHP Selector and stores its global options, so it is important to keep needed configurations and valid syntax for PHP modules settings to avoid PHP Selector misconfiguration.

  • --check-cagefs

    Depending on the error you get, resolution options may differ. See the full list of checks that --check-cagefs performs here. There you also can find possible failure reasons.

  • --check-phpselector

    Checks compatibility for the PHP Selector.

    Possible failures:

    • In case of installed mod_ruid2 package - remove it, PHP Selector is incompatible with that module.
    • In case of non-installed mod_suexec package - install it.
    • In case of non-installed mod_suphp package - install it.
    • In case of unsupported handler - see this table to set the compatible handler.

cloudlinux-config

cloudlinux-config utility shows/sets various parameters related to LVE Manager UI, MySQL Governor and faults notifications for LVE-Stats 2

Usage:

cloudlinux-config get [--json] [--for-reseller resname ]
cloudlinux-config set [--json] --data <str> [--for-reseller resname ]
cloudlinux-config set [--json] --reset-inodes-limit

Commands:

get Shows the values of all supported parameters
set Sets values for supported parameters

Options

--json Sets/returns values in json format
--data Sets values from the data string that follows
--for-reseller Sets/limits the output only to the data related to reseller resname
--reset-inodes-limit Resets inode limits (in case disk quota breaks)

JSON data structure

All options are enclosed inside options (Level 0) string. All options are enclosed in " ", while values come as is. Here's an example:

'{"options":{"L1_option1":value1, "L1_option2":value2}}'

Each Level1 option can have nested Level2 options specified using the same syntax, the same goes for Level2 and Level3 options respectively.

LVE-Stats 2 faults notification settings

Level1 Level2 Level3 Level4 Possible values Description
faultsNotification faultsToInclude concurrentConnections true/false Include concurrent connection value in notification emails
faultsNotification faultsToInclude cpu true/false Include CPU consumption value in notification emails
faultsNotification faultsToInclude io true/false Include IO value in notification emails
faultsNotification faultsToInclude iops true/false Include IOPS value in notification emails
faultsNotification faultsToInclude mem true/false Include RAM consumption value in notification emails
faultsNotification faultsToInclude nproc true/false Include number of processes value in notification emails
faultsNotification fminimumNumberOfFaultsToNotify admin N The minimum number of faults to notify for admin
faultsNotification fminimumNumberOfFaultsToNotify user N The minimum number of faults to notify for users
faultsNotification fminimumNumberOfFaultsToNotify reseller * N The minimum number of faults to notify for reseller
faultsNotification fminimumNumberOfFaultsToNotify customer * N The minimum number of faults to notify for reseller's customers
faultsNotification notify admin period N The period of faults notifications for admin
faultsNotification notify admin unitOfTime minutes/hours/days Time units of period of faults notifications for admin
faultsNotification notify user period N The period of faults notifications for users
faultsNotification notify user|<span class="notranslate">unitOfTime</span>|<span class="notranslate">minutes</span>/<span class="notranslate">hours</span>/<span class="notranslate">days` Time units of period of faults notifications for users
faultsNotification notify reseller * period N The period of faults notifications for resellers
faultsNotification notify reseller * unitOfTime minutes/hours/days Time units of period of faults notifications for resellers
faultsNotification notify customer * period N The period of faults notifications for reseller's customers
faultsNotification notify customer*|<span class="notranslate">unitOfTime</span>|<span class="notranslate">minutes</span>/<span class="notranslate">hours</span>/<span class="notranslate">days` Time units of period of faults notifications for reseller's customers
faultsNotification notifyAdmin true/false Send faults notifications to admin
faultsNotification notifyResellerCustomers true/false Send faults notification to reseller users
faultsNotification notifyResellerOnCustomers * true/false Send users' faults notifications to the respective reseller
faultsNotification notifyReseller * true/false Send faults notification to reseller
faultsNotification notifyCustomers true/false Send faults notifications to users
faultsNotification notifyResellers true/false Send faults notification to resellers

Note

Options marked with * are for reseller use only

MySQL Governor settings

Level1 Level2 Level3 Level4 Possible values Description
mySQLGovSettings errorLog level DEBUG/ERROR Sets error log level
mySQLGovSettings errorLog logPath /var/log/dbgovernor-error.log Sets error log destination file
mySQLGovSettings gatherDataForDetailedStats true/false Enables gathering data for detailed statistics
mySQLGovSettings logRestrictedUsersQueries true/false Enables logging of restricted user queries
mySQLGovSettings modeOfOperation off/single/abusers/all Sets the mode of operation
mySQLGovSettings restrictLog format SHORT/MEDIUM/LONG/VERYLONG Sets the format for restrict log
mySQLGovSettings restrictLog logPath /var/log/dbgovernor-restrict.log Sets the format for restrict log
mySQLGovSettings restrictType mode period/limit Sets the restriction mode
mySQLGovSettings restrictType unlimit period N Sets the restriction expiration period
mySQLGovSettings restrictType unlimit unitOfTime seconds/minutes/hours/days Sets the restriction expiration period units of time
mySQLGovSettings restrictedTimePeriods level1 period N Sets L1 restriction time period
mySQLGovSettings restrictedTimePeriods level1 unitOfTime seconds/minutes/hours/days Sets L1 restriction time period units of time
mySQLGovSettings restrictedTimePeriods level2 period N Sets L2 restriction time period
mySQLGovSettings restrictedTimePeriods level2 unitOfTime seconds/minutes/hours/days Sets L2 restriction time period units of time
mySQLGovSettings restrictedTimePeriods level3 period N Sets L3 restriction time period
mySQLGovSettings restrictedTimePeriods level3 unitOfTime seconds/minutes/hours/days Sets L3 restriction time period units of time
mySQLGovSettings restrictedTimePeriods level4 period N Sets L4 restriction time period
mySQLGovSettings restrictedTimePeriods level4 unitOfTime seconds/minutes/hours/days Sets L4 restriction time period units of time
mySQLGovSettings restrictedTimePeriods timeout period N Sets restriction time period timeout
mySQLGovSettings restrictedTimePeriods timeout unitOfTime seconds/minutes/hours/days Sets L1 restriction time period timeout units of time
mySQLGovSettings scriptPath /path/to/script Path to script to be triggered when account is restricted
mySQLGovSettings slowQueries kill true/false Enables killing of slow queries
mySQLGovSettings slowQueries logPath /path/to/sqkill.log Sets the path to slow query kill log file
mySQLGovSettings slowQueries timeout N Time to kill slow SELECT queries for account (seconds)
mySQLGovSettings userMaxConnections N Sets the maximum number of user connections to database

LVE Manager UI settings

Level1 Level2 Possible values Description
inodeLimits showUserInodesUsage true/false Show end user inode usage
uiSettings hideLVEUserStat true/false Hide LVE end user usage statistic
uiSettings hidePHPextensions true/false Hide PHP extension selection
uiSettings hidePythonApp true/false Hide Python App in web-interface
uiSettings hideRubyApp true/false Hide Ruby App in web-interface

Examples

  • display Python/Ruby Selector in User UI (cPanel)
$ cloudlinux-config set --json --data '{"options":{"uiSettings":{"hideRubyApp":false, "hidePythonApp":false}‌}}'