CloudLinux OS kernel

Hybrid Kernels

CloudLinux 6 Hybrid kernel

CloudLinux 6 Hybrid Kernel is CloudLinux 7 (3.10.0) kernel compiled for CloudLinux 6 OS. New 3.10 kernel features a set of performance and scalability improvements related to IO , networking and memory management, available in CloudLinux 7 OS . It also features improved CPU scheduler for better overall system throughput and latency.

Please find information on the main features of 3.10 kernel branch on the links:

https://kernelnewbies.org/Linux_3.10#head-e740f930dfd021616cc42e8abf21c79d0b07e217

https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.10.1

CloudLinux 7 Hybrid kernel

CloudLinux 7 Hybrid Kernel is essentially EL8-based (4.18) kernel compiled for CloudLinux OS 7.

You can find more information on 4.18 kernel branch using this link:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/8.0_release_notes/new-features#kernel

How to migrate from the normal to hybrid channel (CL6h):

Note

The system must be registered in CLN.

  1. Update rhn-client-tools from production

  2. Run normal-to-hybrid script.

  3. Reboot after script execution is completed.

yum update rhn-client-tools
normal-to-hybrid
reboot

How to migrate from the normal to hybrid channel (CL7h):

Note

The system must be registered in CLN.

  1. Update rhn-client-tools rhn-check rhn-setup from testing repository

  2. Run normal-to-hybrid script.

  3. Reboot after script execution is completed.

yum update rhn-client-tools rhn-check rhn-setup
normal-to-hybrid
reboot

How to migrate from hybrid to the normal channel (for both CL6h and CL7h):

Note

The system should be registered in CLN.

  1. Run hybrid-to-normal script.

  2. Reboot after script execution is completed.

hybrid-to-normal
reboot

Known limitations and issues of CloudLinux 6 Hybrid kernel

  1. We do not remove Hybrid kernel after migration from Hybrid to the normal channel, but we remove linux-firmware package which is needed to boot Hybrid kernel. This is because CloudLinux 6 does not allow to remove the package of currently running kernel. Proper removal procedure will be implemented, but for now, we should warn users not to boot Hybrid kernel if they have migrated to normal channel.

  2. Kernel module signature isn't checking for now, as 3.10 kernel is using x509 certificates to generate keys and CL6 cannot detect signatures created in such way. The solution will be implemented.

Known limitations and issues of CloudLinux 7 Hybrid kernel

Features that are absent in the current kernel build:

  • Per LVE traffic accounting

Note that Symlink Owner Match Protection is enabled by default in CL7 Hybrid kernel. To disable it, use sysctl utility:

sysctl -w fs.enforce_symlinksifowner=0

Find more details on symlink owner match protection.

CloudLinux provides comprehensive protection against symbolic link attacks popular in shared hosting environment.

The protection requires setting multiple kernel options to be enabled.

fs.enforce_symlinksifowner

To protect against symlink attack where attacker tricks Apache web server to read some other user PHP config files, or other sensitive file, enable:

fs.enforce_symlinksifowner=1

Setting this option will deny any process running under gid fs.symlinkown_gid to follow the symlink if owner of the link doesn’t match the owner of the target file.

Defaults:

fs.enforce_symlinksifowner = 1
fs.symlinkown_gid = 48
fs.enforce_symlinksifowner = 0 do not check symlink ownership
fs.enforce_symlinksifowner = 1 deny if symlink ownership doesn’t match target, and process gid matches _symlinkown_gid _

When fs.enforce_symlinksifowner set to 1, processes with GID 48 will not be able to follow symlinks if they are owned by user1 , but point to file owned user2 .

Please, note that fs.enforce_symlinksifowner = 2 is deprecated and can cause issues for the system operation.

fs.symlinkown_gid

On standard RPM Apache installation, Apache is usually running under GID 48. On cPanel servers, Apache is running under user nobody, GID 99.

To change GID of processes that cannot follow symlink , edit the file /etc/sysctl.conf , add the line:

fs.symlinkown_gid = XX
And execute:
$ sysctl -p

To disable symlink owner match protection feature, set fs.enforce_symlinksifowner = 0 in /etc/sysctl.conf , and execute

$ sysctl -p

WARNING

/proc/sys/fs/global_root_enable [CloudLinux 7 kernel only] [applicable for kernels 3.10.0-427.36.1.lve1.4.42+]

proc/sys/fs/global_root_enable flag enables following the symlink with root ownership. If global_root_enable=0 , then Symlink Owner Match Protection does not verify the symlink owned by root.

For example, in the path /proc/self/fd , self is a symlink , which leads to a process directory.  The symlink owner is root . When global_root_enable=0 , Symlink Owner Match Protection excludes this element from the verification. When global_root_enable=1 , the verification will be performed, which could block the access to fd and cause violation of web site performance.

It is recommended to set /proc/sys/fs/global_root_enable=0 by default. If needed, set /proc/sys/fs/global_root_enable=1 to increase the level of protection.

Note

Starting from lve-utils 3.0-21.2, fs.symlinkown_gid parameter values for httpd service user and fs.proc_super_gid for nagios service user is written to /etc/sysctl.d/90-cloudlinux.conf.

CageFS is extremely powerful at stopping most information disclosure attacks, where a hacker could read sensitive files like /etc/passwd .

Yet, CageFS does not work in each and every situation. For example, on cPanel servers, it is not enabled in WebDAV server, cPanel file manager and webmail, as well as some FTP servers don’t include proper change rooting.

This allows an attacker to create symlink or hardlink to a sensitive file like /etc/passwd and then use WebDAV , filemanager, or webmail to read the content of that file.

Starting with CL6 kernel 2.6.32-604.16.2.lve1.3.45, you can prevent such attacks by preventing user from creating symlinks and hardlinks to files that they don’t own.

This is done by set following kernel options to 1:

fs.protected_symlinks_create = 1
fs.protected_hardlinks_create = 1

WARNING

We do not recommend to use protected_symlinks option for cPanel users as it might break some of the cPanel functionality.

Note

Link Traversal Protection is disabled by default for the new CloudLinux OS installations/convertations.

fs.protected_symlinks_create = 0
fs.protected_hardlinks_create = 0
Then setup:
fs.protected_symlinks_allow_gid = id_of_group_linksafe
fs.protected_hardlinks_allow_gid = id_of_group_linksafe
This is for example needed by PHP Selector to work (new versions of Alt-PHP can already correctly configure those settings).

To manually adjust the settings, edit: /etc/sysctl.d/cloudlinux-linksafe.conf and execute:

sysctl -p /etc/sysctl.d/cloudlinux-linksafe.conf
or:
sysctl --system

Note

Starting from lvemanager 4.0-25.5, if there is no /etc/sysctl.d/cloudlinux-linksafe.conf config file, selectorctl for PHP with --setup-without-cagefs and --revert-to-cagefs keys writes fs.protected_symlinks_create and fs.protected_hardlinks_create parameters to /etc/sysctl.d/90-cloudlinux.conf.

File change API

General

General description

One of the main problems on a shared hosting system for file backup operations is to figure out which files have changed. Using INOTIFY on a 1T drive with a large number of small files and directories guarantees slow startup times, and a lot of context switching between kernel and userspace - generating additional load. On the other hand scanning disk for newly modified files is very IO intensive, and can kill the performance of the fastest disks.

CloudLinux approach

CloudLinux File Change API is a kernel level technology with the user space interface that buffers lists of modified files in the kernel and then off-loads that list to user space daemon.

After that - any software (with enough permissions) can get a list of files that has been modified for the last 24 hours.

The software is very simple to use and produces the list of modified files. As such we expect file backup software, including integrated cPanel backup system to integrate with this API soon.

Usage and integration

Userland utilities

/usr/bin/cloudlinux-backup-helper is a utility for getting the list of changed files.

It is supposed to be run by a super user only.

Command line parameters:

-t | --timestamp retrieve file names for files modified after specified timestamp
-u | --uid       retrieve file names for particular UID only
If no UID specified, are retrieved for all users. If no timestamp specified, all database events are shown.

Output format

protocol version (1 right now), timestamp (in seconds) - up to which time data was collected
UID:absolute path to file changed
UID:absolute path to file changed
…

Note

The timestamp in output is needed so you can clearly identify from which timestamp to get list of changed files next.

Examples:

[root@localhost ~]# cloudlinux-backup-helper -t 1495533489 -u <UID>
1,1495533925
1001:/home/user2/public_html/output.txt
1001:/home/user2/public_html/info.php
[root@localhost ~]# cloudlinux-backup-helper -t 1495533489
1,1495533925
1000:/home/user1/.bashrc
1001:/home/user2/public_html/output.txt
1001:/home/user2/public_html/info.php
1003:/home/user3/logs/data.log
**Getting changed files by end user**

/usr/bin/cloudlinux-backup-helper-uid is a SUID wrapper for the cloudlinux-backup-helper utility that enables an end user to get the list of files changed. It accepts timestamp argument only and retrieves data of the user who is running it only.

Examples:

[user@localhost ~]$ cloudlinux-backup-helper-uid               
1,1495530576
1000:/home/user/.bash_history 

[user@localhost ~]$ cloudlinux-backup-helper-uid -t 1495547922
1,1495548343
1000:/home/user/file1.txt
1000:/home/user/file2.txt
This command is available within CageFS.

Installation and configuration

cloudlinux-fchange-0.1-5

Requirements

CloudLinux OS 6 (requires Hybrid kernel) or 7 Kernel Version: 3.10.0-427.36.1.lve1.4.47

Installation and configuration

To install cloudlinux-fchange system run:

CloudLinux 7:

yum install cloudlinux-fchange --enablerepo=cloudlinux-updates-testing

CloudLinux 6 Hybrid:

yum install cloudlinux-fchange --enablerepo=cloudlinux-hybrid-testing
Configuration file can be found in /etc/sysconfig/cloudlinux-fchange

Database containing list of modified files is located at /var/lve/cloudlinux-fchange.db by default.

Starting and stopping

After successful installation the event collecting daemon starts automatically, providing all kernel-exposed data are in place.

To start daemon: CloudLinux 7:

systemctl start cloudlinux-file-change-collector

CloudLinux 6 Hybrid:

service cloudlinux-file-change-collector start
To stop daemon: _CloudLinux 7:_
systemctl stop cloudlinux-file-change-collector

CloudLinux 6 Hybrid:

service cloudlinux-file-change-collector stop

To uninstall cloudlinux-fchange run:

yum remove cloudlinux-fchange

Configuration details

Configuration resides in /etc/sysconfig/cloudlinux-fchange. The following is the default configuration (see comments):

# sqlite database file path. If commented out a default value is used
#database_path=/var/lve/cloudlinux-fchange.db

# If uncommented paths starting with 'include' one are processed only
# Pay attention this parameter is a regular string, not a regex
# To include more than one item just specify several lines to include:
# include=/one
# include=/two

# If uncommented exclude paths which contain 'exclude'
# Pay attention this parameter is a regular string, not a regex
# To exclude more than one item just specify several lines to exclude:
# exclude=var
# exclude=tmp

# Daemon polling interval in seconds
polling_interval=5 

# Time to keep entries in days. Does not clean if commented out or zero
time_to_keep=1

# User read-only mode minimal UID
# If file change collector stopped, all users with UID >= user_ro_mode_min_uid
# are restricted to write to their home directory. This prevents to miss
# a file change event.
# Value of -1 (default) allows to disable the feature
user_ro_mode_min_uid=-1

# Minimal UID of events to be processed.
# Events of users with UID less then specified are not handled.
# By default 500 (non-system users for redhat-based systems)
#minimal_event_uid=500
 
# SQLite shared lock prevents setting more restrictive locks. That is a
# process cannot write to a database table when a concurrent process reads
# from the table. As saving data to database is considered far more important
# than getting them (data could be reread a second later after all), database
# writer could try to terminate concurrent reading processes. Just set
# terminate rivals to 'yes' to turn this ability on.
# terminate_rivals=no 

# Events to be handled. Currently the following types of events are processed:
# 1. file creation
# 2. file deletion
# 3. directory creation
# 4. directory deletion
# 5. file content/metadata modification
# 6. file/directory attributes/ownership modification
# 7. hardlink creation
# 8. symlink creation
# 9. file/directory moving/renaming
# By default all events are processed. Keep in mind that events for a filepath
# are cached, i.e if a file was deleted and then a file with the same absolute
# name is created, only the deletion event is triggerred. Changing file
# modification timestamp with command 'touch' will trigger modification event
# as if a file content is modified.
# Currently supported options are:
# file_created, file_modified, file_deleted, dir_created, dir_deleted,
# owner_changed, attrib_changed, moved, hardlink_created, symlink_created, all
# Options that don't have 'file' or 'dir' prefix, applied to both files and
# directories. To set more than one options, separate them with commas,
# e.g. event_types=file_created,file_deleted,file_modified. Unknown options are
# ignored.
#
# event_types=all

Note

Please keep in mind, that current implementation implies that one process is writing to a database and another is reading from it. As reading sets shared lock to a database table, the writing process cannot write to the table until the lock is released. That’s why passing a timestamp to cloudlinux-backup-helpermatters: this way the number of records to be returned is substantially decreased, lowering the processing time and filtering out old records. Likewise, pay attention to narrowing the scope of events being recorded. Chances are that changing attributes, ownership, directory creation/deletion, symlink events are not relevant and there’s no need to keep them.

Low-level access

Note

Using this options is dangerous, and might cause problems with CloudLinux File Change API.

The kernel exposes the functionality to /proc/sys/fs/datacycle/ folder.

  1. enable - enable/disable the functionality. Write 1 to this file to enable, 0 to disable. If disabled, no events are coming to events file.

  2. events - the modified files log itself. Events in the format <EVENT_ID>:<EVENT_TYPE_ID>:<USER_ID>:<FILE_PATH> are constantly appending to the end of the file if datacycle enabled. File events are never duplicated: if we have file modification event, we would not get file deletion event if the file has been later deleted. This events buffer has limited capacity, therefore from time to time, the events log requires flushing.

  3. flush - a file for clearing events log. For flushing, the last event_id from the events file is written to this file. Right after this, events log is truncated to that event_id .

  4. user_ro_mode - forbidding users with UIDs equal or bigger that set in this file writing to their home directories. At the boot, the file has -1. When it’s written positive value, say 500, the system starts effectively preventing users from modifying their home dirs (on write attempt a user gets ‘read-only filesystem’ error). This feature is designed to prevent users from updating their home dirs when events are not handled.

  5. entries_in_buffer - just counter of log entries of events file.

  6. min_event_uid - this file has minimal UID of events to be handled. Events from users with smaller UID are not handled. By default 500 (non-system users in redhat-based systems).

Tuned-profiles-cloudlinux

The tuned-profiles-cloudlinux package brings a range of kernel under-the-hood tunings to address high LA, iowait issues what were detected earlier on particular users deploys. The package also encloses OOM adjustments to prioritize the elimination of overrun PHP, lsphp, Phusion Passenger workers processes over other processes (e.g. ssh, a cron job).

There are three profiles provided by CloudLinux:

# tuned-adm list | grep cloudlinux
- cloudlinux-default          - Default CloudLinux tuned profile
- cloudlinux-dummy            - Empty CloudLinux tuned profile
- cloudlinux-vz               - Empty CloudLinux tuned profile

cloudlinux-dummy and cloudlinux-vz are used for internal needs or when Virtuozzo/OpenVZ detected and actually do nothing.

cloudlinux-default is one to be used, it actually does the following:

  1. Switches CPU power consumption mode to the maximum. CPU operates at maximum performance at the maximum clock rate:
governor=performance
energy_perf_bias=performance

Note

If standard software CPU governors are used.

  1. Applies the following kernel options:
    vm.force_scan_thresh=100 - Improves kernel memory clean-up in case of big number of running LVE.

UBC parameters set the limits for the containers:

ubc.dirty_ratio=100 - Defines maximum RAM percentage for dirty memory pages. dirty_background_ratio=75 - Defines RAM percentage when to allow writing dirty pages on the disk.

  1. [CloudLinux 7 only] Detects used disk types and changes elevator to 'deadline' for HDD and to 'noop' for SSD in /sys/block/[blockname]/queue/scheduler .

Note

The script uses /sys/block/[blockname]/queue/rotational flag, some RAID controllers can not set it properly. For example, SSD used for RAID but rotational is set to 1 by RAID driver. As a workaround add the following to /etc/rc.d/rc.local to make it applied on boot:

echo "noop" > /sys/block/[blockname]/queue/scheduler  
echo "0" > /sys/block/[blockname]/queue/rotational

Where [blockname] is used device name, like sda/sdb .

And make it executable:

chmod +x /etc/rc.d/rc.local
  1. [CloudLinux 7 only] The profile sets I/O scheduler. For the normal discs the Deadline Scheduler is set to improve IO performance and decrease IO latency, for SSD - noop. When configuring scheduler I/O queue is changed and set to the value 1024 which improves overall I/O subsystem performance by caching IO requests in memory.

  2. Disables transparent HugePage .

  3. Provides adjustment group file for OOM-Killer to kill overrun php, lsphp and Phusion Passenger workers first.

To install:

yum install tuned-profiles-cloudlinux

To start using a profile:

tuned-adm profile cloudlinux-default

To stop using a profile:

tuned-adm off

Kernel config variables

Starting from lvemanager 4.0-25.5 , lve-utils 3.0-21.2 , and cagefs-6.1-26 , CloudLinux OS utilities can read/write kernel config variables from a custom config /etc/sysctl.d/90-cloudlinux.conf (earlier, the parameters were read/written only from sysctl.conf ).

CloudLinux OS utilities get parameter by using sysctl system utility. So for now, even if a config variable is not set in the sysctl.conf and in the /etc/sysctl.d config files, this variable will be read by sysctl utility directly from /proc/sys .

If some kernel variable was set in /etc/sysctl.d/90-cloudlinux.conf do

sysctl --system
to apply the parameters before reading and after writing.

Starting from cagefs-6.1-27, fs.proc_can_see_other_uid will be migrated (one time) from /etc/sysctl.conf into /etc/sysctl.d/90-cloudlinux.conf . If this variable is not set in either file, it will default to 0. It is strongly advised against setting this variable in 90-cloudlinux.conf . Define it in /etc/sysctl.conf or in some other config file with an index number greater than 90-cloudlinux.conf , e.g. /etc/sysctl.d/95-custom.conf.

Starting from lve-utils-3.0-23.7 fs.proc_super_gid and fs.symlinkown_gid will be migrated (one time) from /etc/sysctl.conf into /etc/sysctl.d/90-cloudlinux.conf .

For lve-utils versions from 3.0-21.2 to 3.0-23.7 the migration was performed the same way, but during every package install/update. Variables setting guidelines are the same as for CageFS (see above).

As requested by some our customers, we've implemented a new kernel setting to hide /proc/net/{tcp,udp,unix} files for additional security/isolation. You can hide them by runnign sysctl -w kernel.proc_disable_net=1 command, and by default, it's 0 (nothing hidden).

Virtualized /proc filesystem

You can prevent user from seeing processes of other users (via ps/top command) as well as special files in /proc file system by setting fs.proc_can_see_other_uid sysctl.

To do that, edit /etc/sysctl.conf

fs.proc_can_see_other_uid=0
fs.proc_super_gid=600
And do:
# sysctl -p
fs.proc_can_see_other_uid=0

If fs.proc_can_see_other_uid is set to 0, users will not be able to see special files. If it is set to 1 - user will see other processes IDs in /proc filesystem.

fs.proc_super_gid=XX

The fs.proc_super_gid sets group ID which will see system files in /proc, add any users to that group so they will see all files in /proc. Usually needed by some monitoring users like nagios or zabbix and cldetect utility can configure few most commonly used monitoring software automatically.

Virtualized /proc filesystem will only display following files (as well as directories for PIDs for the user) to unprivileged users:

/proc/cpuinfo
/proc/version
/proc/stat
/proc/uptime
/proc/loadavg
/proc/filesystems
/proc/stat
/proc/cmdline
/proc/meminfo
/proc/mounts
/proc/tcp
/proc/tcp6
/proc/udp
/proc/udp6
/proc/assocs
/proc/raw
/proc/raw6
/proc/unix
/proc/dev

Note

Starting from lve-utils 3.0-21.2, fs.proc_super_gid parameter in da_add_admin utility is written to /etc/sysctl.d/90-cloudlinux.conf.

Remounting procfs with "hidepid" option

In lve-utils-2.1-3.2 and later /proc can be remounted with hidepid=2 option to enable additional protection for procfs. This remount is performed in lve_namespaces service. This option is in sync with fs.proc_can_see_other_uid kernel parameter described above. When /etc/sysctl.conf does not contain fs.proc_can_see_other_uid setting, the protection is off (procfs is remounted with hidepid=0 option). In this case fs.proc_super_gid setting is ignored. Users are able to see full /proc including processes of other users on a server. This is a default behavior.

If /etc/sysctl.conf contains fs.proc_can_see_other_uid=1 setting, then /proc will be remounted with hidepid=0 option (disable hidepid protection for all users). If /etc/sysctl.conf contains fs.proc_can_see_other_uid=0 setting, then /proc will be remounted with hidepid=2 option (enable hidepid protection for all users). If /etc/sysctl.conf contains fs.proc_can_see_other_uid=0 and fs.proc_super_gid=$GID settings, then /proc will be remounted with hidepid=2, gid=$GID options (enable hidepid for all users except users in group with gid $GID).

To apply /etc/sysctl.conf changes, you should execute

service lve_namespaces restart
Or
/usr/share/cloudlinux/remount_proc.py

So, admin can prevent users from seeing processes of other users via fs.proc_can_see_other_uid and fs.proc_super_gid settings in /etc/sysctl.conf, like earlier.

Also, you can override this by specifying desired options for /proc in /etc/fstab.

To disable hidepid, add to /etc/fstab the following:

proc /proc proc defaults,hidepid=0,gid=0 0 0
Or you can specify desired hidepid and gid values explicitly:
proc /proc proc defaults,hidepid=2,gid=clsupergid 0 0
You should execute
mount -o remount /proc

to apply /etc/fstab changes.
Nevertheless, we recommend to manage procfs mount options via /etc/sysctl.conf as described above for backward compatibility.

Note

There is a known issue on CloudLinux 6 systems. User cannot see full /proc inside CageFS even when this user is in “super” group, that should see full /proc. This issue does not affect users with CageFS disabled. CloudLinux 7 is not affected.

Note

Starting from lve-utils 3.0-21.2, lve_namespaces service can read parameters from the /etc/sysctl.d/90-cloudlinux.conf.

Note

Even if fs.proc_can_see_other_uid and fs.proc_super_gid parameters are not set in config files but specified in /proc/sys, then when restarting lve_namespaces service the parameters from /proc/sys will be used. So, /proc will be remounted according to these parameters.

Ptrace block

Starting with kernel 3.10.0-427.18.s2.lve1.4.21 ( CloudLinux 7) and 2.6.32-673.26.1.lve1.4.17 ( CloudLinux 6) we re-implemented ptrace block to protect against ptrace family of vulnerabilities. It prevents end user from using any ptrace related functionality, including such commands as strace, lsof or gdb .

By default, CloudLinux doesn't prevent ptrace functionality.

Defaults:

kernel.user_ptrace = 1
kernel.user_ptrace_self = 1

The option kernel.user_ptrace disables PTRACE_ATTACH functionality, option kernel.user_ptrace_self disables PTRACE_TRACEME .

To disable all ptrace functionality change both sysctl options to 0, add this section to /etc/sysctl.conf :

## CL. Disable ptrace for users
kernel.user_ptrace = 0
kernel.user_ptrace_self = 0
##

Apply changes with:

$ sysctl -p

Different software could need different access to ptrace , you may need to change only one option to 0 to make them working. In this case, there will be only partial ptrace protection.

WARNING

ptrace protection is known to break PSA service for Plesk 11

Xen XVDA

2.6.32 kernels have different mode of naming Xen XVDA drives.

By adding xen_blkfront.sda_is_xvda=0 to kernel boot line in grub.conf you will make sure no naming translation is done, and the drives will be identified as xvde .

By default, this option is set to 1 in the kernel, and drives are detected as xvda . This is needed only for CloudLinux 6 and Hybrid kernels.

IO limits latency

[lve1.2.29+]

When customer reaches IO Limit, the processes that are waiting for IO will be placed to sleep to make sure they don't go over the limit. That could make some processes sleep for a very long time. By defining IO latency, you can make sure that no process sleeps due to IO limit for more then X milliseconds. By doing so, you will also let customers to burst through the limits, and use up more than they were limited too in some instances.

This option is OFF by default.

For CloudLinux 6 and CloudLinux 7 (since Hybrid kernel lve1.4.x.el5h):

To enable IO Limits latency and set it to 10 seconds:

# echo 10000 > /sys/module/kmodlve/parameters/latency
To disable latency:
# echo 2000000000 > /sys/module/kmodlve/parameters/latency

It is possible to set, for example, 1000 as a permanent value. To do so, create a file /etc/modprobe.d/kmodlve.conf with the following content:
options kmodlve latency=1000

For CloudLinux 5 (OBSOLETE):

To enable IO Limits latency and set it to 10 seconds:

# echo 10000 > /sys/module/iolimits/**parameters/latency
To disable latency:
# echo 2000000000 > /sys/module/iolimits/**parameters/latency

Reading LVE usage

CloudLinux kernel provides real time usage data in file.

All the statistics can be read from that file in real time. Depending on your kernel version you will get either Version 6 of the file, or version 4 of the file. You can detect the version by reading the first line of the file. It should look like:

6:LVE... for version 6
4:LVE... for version 4

First line presents headers for the data. Second line shows default limits for the server, with all other values being 0. The rest of the lines present limits & usage data on per LVE bases.

Version 6 (CL6 & hybrid kernels):

6:LVE        EP        lCPU        lIO        CPU        MEM        IO        lMEM        lEP        nCPU        fMEM        fEP        lMEMPHY        lCPUW        lNPROC        MEMPHY        fMEMPHY        NPROC        fNPROC
0        0        25        1024        0        0        0        262144        20        1        0        0        262144        100        0        0        0        00
300        0        25        1024        1862407        0        0        262144        20        1        0        0        262144        100        0        31        000
Label Description Value Supported versions
LVE LVE ID number
EP Number of entry processes number
lCPU CPU Limit % relative to total CPU power
lIO IO limits for CL6 KB/s for v6, from 1 to 100 for v4
CPU CPU usage since reboot in nanoseconds for v6, hertz for v4
MEM Virtual memory usage number of 4k pages
IO IO usage KB/s for v6, 0 for v4
lMEM Virtual memory limit number of 4k pages
lEP Entry Processes limit number
nCPU Number of cores limit number of cores
fMEM Virtual memory faults number of faults
fEP Entry Processes faults number of faults v6+
lMEMPHY Physical memory limit number v6+
lCPUW CPU weight (not used) from 1 to 100 v6+
lNPROC Number of processes limit number v6+
MEMPHY Physical memory usage number of 4k pages v6+
fMEMPHY Physical memory faults number of faults v6+
NPROC Number of processes number v6+
fNPROC Number of processes faults number of faults v6+
IOPS IO operations since reboot number v8+

Flashcache

Note

Available only for x86_64, CloudLinux 6 and Hybrid servers

Flashcache is a module originally written and released by Facebook (Mohan Srinivasan, Paul Saab and Vadim Tkachenko ) in April of 2010. It is a kernel module that allows Writethrough caching of a drive on another drive. This is most often used for caching a rotational drive on a smaller solid-state drive for performance reasons. This gives you the speed of an SSD and the size of a standard rotational drive for recently cached files. Facebook originally wrote the module to speed up database I/O , but it is easily extended to any I/O .

To install on CloudLinux 6 & Hybrid servers:

$ yum install flashcache

More info on flashcache : https://github.com/facebook/flashcache/

ArchLinux has a good page explaining how to use flashcache : https://wiki.archlinux.org/index.php/Flashcache

OOM killer for LVE processes

When LVE reaches its memory limit, the processes inside that LVE are killed by OOM Killer and appropriate message is written to /var/log/messages . When any LVE hits huge number of memory limits in short period of time, then OOM Killer could cause system overload. Starting from kernel 2.6.32-673.26.1.lve1.4.15 ( CloudLinux 6) and from kernel 3.10.0-427.18.2.lve1.4.14 ( CloudLinux 7) heavy OOM Killer could be disabled. If so - lightweight SIGKILL will be used instead.

By default OOM Killer is enabled, to disable it please run:

For CloudLinux 6 :

# echo 1 > /proc/sys/ubc/ubc_oom_disable

Also, add the following to /etc/sysctl.conf file to apply the same during boot:

ubc.ubc_oom_disable=1

For CloudLinux 7:

# echo 1 > /proc/sys/kernel/memcg_oom_disable

Also, add the following to /etc/sysctl.conf file to apply the same during boot:

kernel.memcg_oom_disable=1

File system quotas

In Ext4 file system, the process with enabled capability CAP_SYS_RESOURCE is not checked on the quota exceeding by default. It allows userland utilities selectorctl and cagefs to operate without fails even if a user exceeds a quota.

To disable quota checking in XFS file system set cap_res_quota_disable option to 1 using the following command:

# echo 1 > /proc/sys/fs/xfs/cap_res_quota_disable