CloudLinux has support for the following limits:
|Limits||Units||Default Value||Description||Supported Kernels / OS|
|SPEED||% of a core, or HZ||100%||CPU speed limit, relative to a single core, or specified in HZ (portable across CPU s)||all|
|CPU [deprecated]||% of CPU||25%||CPU Limit (smallest of CPU & NCPU is used)||all|
|NCPU [deprecated]||number of cores||1 CORE||Max number of cores (smallest of CPU & NCPU used)||all|
|PMEM||KB||1024MB||Physical memory limit (RSS field in ps/RES in top). Also includes shared memory and disk cache||CL5 hybrid kernel, CL5 lve1.x+ kernel, CL6 and CL7|
|VMEM||KB||0||Virtual memory limit (VSZ field in ps/VIRT in top)||all|
|IO||KB/sec||1024KB/sec||IO throughput - combines both read & write operations||CL7, CL6 lve1.1.9+ kernel|
|IOPS [lve1.3+]||Operations per second||1024||Restricts total number of read/write operations per second.||CL7 and CL6 kernel|
|NPROC||number||100||Max number of processes within LVE||CL6 and CL7 kernels|
|EP||number||20||Limit on entry processes. Usually represents max number of concurrent connections to apache dynamic scripts as well as SSH and cron jobs running simultaneously.||all|
It is always better to disable VMEM limits (set them to 0) in your system at all because they are deprecated in CloudLinux 6/7 system and are causing unexpected issues.
Bellow you can find recommendations for your typical shared hosting setup. The recommendations don't depend on the power of your server. They only depend on how "fast" you want your hosting accounts to be.
Typical Hosting Account
High End Hosting Account
LVE is a kernel level technology developed by the CloudLinux team. The technology has common roots with container based virtualization and uses cgroups in its latest incarnation. It is lightweight and transparent. The goal of LVE is to make sure that no single web site can bring down your web server.
Today, a single site can consume all CPU, IO, Memory resources or Apache processes - and bring the server to a halt. LVE prevents that. It is done via collaboration of Apache module, PAM module and kernel.
mod_hostinglimits is Apache module that:
- detects VirtualHost from which the request came;
- detects if it was meant for CGI or PHP script;
- puts Apache process used to serve that request into LVE for the user determined via SuexecUserGroup directive for that virtual host;
- lets Apache to serve the request;
- removes Apache process from user's LVE.
The kernel makes sure that all LVEs get fair share of the server's resources, and that no customer can use more then the limits set for that customer. Today we can limit CPU , Memory (virtual and physical), IO, number of processes as well as the number of entry processes (concurrent connections to apache).
Each LVE limits amount of entry processes (Apache processes entering into LVE) to prevent single site exhausting all Apache processes. If the limit is reached, then mod_hostinglimits will not be able to place Apache process into LVE, and will return error code 508. This way very heavy site would slow down and start returning 508 errors, without affecting other users.
- If the site is limited by CPU or IO, then the site will start responding slower.
- If the site is limited by memory or number of processes limits, then the user will receive 500 or 503 errors that server cannot execute the script.
Checking if LVE is installed
To use LVE you should have CloudLinux kernel installed, and LVE module loaded. You can check the kernel by running the following command:
$ uname -r
You should see something like 2.6.18-294.8.1.el5.lve0.8.60. The kernel should have lve in its name. To see if lve kernel module is loaded run:
$ lsmod|grep lve lve 46496 0
Starting from kernels lve1.4.x iolimits module is a part of kmod-lve and could not be used separately.
You can toggle LVE on/off by editing
Setting it to
yeswill enable LVE, setting it to
nowill disable LVE.
You can toggle IO limits by editing
You need to reboot the server, after you set this option to make the changes live.
Controlling LVE limits
The best way to control LVE limits is using LVE Manager in your favorite control panel. Alternatively, you can use command line tool
lvectl to control limits.
The limits are saved in
<?xml version="1.0" ?> <lveconfig> <defaults> <cpu limit="25"/> <ncpu limit="1"/> <io limit="1024"/> <mem limit="262144"/> <other maxentryprocs="200"/> <pmem limit="262144"/> <nproc limit="0"/> </defaults> <lve id="532"> <cpu limit="30"/> <ncpu limit="5"/> </lve> </lveconfig>
Sets CPU limit to 25%, IO limit to 1024KB/s, virtual memory limit to 1GB (memory limit is set as a number of 4096 bytes pages), physical memory limit to 1GB, CPU cores per LVE to 1, maximum entry processes to 200 and no limit for number of processes for all LVEs. It also sets the limit of 30% and number of processes limit to 5 for LVE with ID 532.
Checking LVE usage
One of the best way to monitor current usage is lvetop:
$ lvetop ID EP PNO TNO CPU MEM I/O test 1 2 2 2% 728 0
You can also check the content of
/proc/lve/list file that has all the data about LVE usage for all LVEs:
[root@localhost tests]$ cat /proc/lve/list 4:LVE EP lCPU lIO CPU MEM IO lMEM lEP nCPU fMEM fEP 0 0 75 25 0 0 0 262144 20 2 0 0 500 0 75 25 0 0 0 4294967 20 3 2 1 700 1 75 25 1403247 202 0 262144 20 2 0 0
Additionally, you can use tool lveps to see CPU usage, and processes within LVE.
CPU SPEED limit allows to set CPU limit in terms of % of a single core, or as a fixed number of Hz.
--speed=XX% would set performance relative to one core.
--speed=50%would mean 1/2 core.
--speed=100%would mean 1 core,
--speed=150%would mean 1.5 cores
--speed=XXmhz would automatically detect CPU speed of each core, and adjust the CPU scheduler to make sure user cannot go over that limit.
For example, on 1ghz CPU , setting of
--speed=2ghz would mean 2 cores, while on 4ghz CPU same setting would mean 1/2 of a core.
This should allow hosting companies to set same approximate performance level limits across different hardware using single setting.
Note. We strongly recommend setting CPU speed limits not less than 100%. As such limits cause CPU context switching which leads to increased
Memory is controlled using virtual (VMEM) and physical (PMEM) memory limits.
Virtual memory limit
Virtual memory limit corresponds to the amount of memory processes can allocate within LVE. You can see individual process virtual memory usage by monitoring VIRT column in top output for the process.
When process tries to allocate more memory, CloudLinux checks if the new total virtual memory used by all processes within LVE is more then a limit set. In such case CloudLinux will prevent memory from being allocated and increments fVMEM counter. In most cases, but not all of them - this causes process to fail. For CGI/PHP scripts it will usually cause 500 and 503 error.
It is recommended to disable VMEM limits (set them to 0) in your system at all because they are deprecated in CloudLinux 6 and 7 system and can cause unexpected issues.
Physical memory limit
Physical memory limit corresponds to the amount of memory actually used by end customer's processes. You can see individual process physical memory usage by monitoring RES column in top output for the process. Because similar processes (like PHP) share a lot of their memory, physical memory usage is often much lower then virtual memory usage.
Additionally physical memory includes shared memory used by the customer, as well as disk cache. In case of disk cache – if a user is starting to lack physical memory, the memory used for disk cache will be freed up, without causing any memory faults.
When LVE goes over physical memory limit, CloudLinux will first free up memory used for disk cache, and if that is not enough, it will kill some of the processes within that LVE, and increment fPMEM counter. This will usually cause web server to serve 500 and 503 errors. Physical memory limit is a much better way to limit memory for shared hosting.
Checking personal users disk cache (If lveinfo shows memory usage but there are no processes there)
If you see no processes under some user, but lve manager keeps telling it is using some memory, then most probably memory is taken by users disk cache. To check personal users disk cache (if lveinfo shows memory usage but not processes there):
On CloudLinux 7 and CloudLinux 6 Hybrid systems, the file is different:
Cached: 67300 kB
where XXX is user id, could be taken with:
IO limits restrict the data throughput for the customer. They are in KB/s. When limit is reached, the processes are throttled (put to sleep). This makes sure that processes within LVE cannot go over the limit,. Yet don't stop working, nor getting killed – they just work slower when the limit is reached.
IO limits are available with kernels el6.lve1.x and higher.
The IO limits will only affect DISK IO, and will have no effect on network. It also doesn't take into consideration any disk cache accesses. So, even if file is loaded from disk cache 1000 times – it will not be counted towards IO limits.
IOPS limits restrict the total number of read/write operations per second. When the limit is reached the read/write operations stop until current second expires.
Entry processes limit controls the number of entries into LVE. Each time a process 'enters' into LVE, we increment the counter. Each time process exits LVE, we decrement the counter. We don't count processes that are created inside LVE itself. It is also know as 'Apache concurrent connections' limit.
The process enter's into LVE when there is a new HTTP request for CGI/PHP.
This limit was created to prevent DoS attacks against web server. One of the fairly popular attacks is to tie up all the Apache connections by hitting some slow page on a server. Once all Apache slots are used up, no one else will be able to connect to the web server, causing it to appear to be down. The issue is worsened by CPU limits, as once site starts to get slow due to CPU limit – it will respond to requests slower and slower, causing more and more connections to be tied up.
To solve that, we have created entry processes (often called concurrent connections) limit. It will limit the number of concurrent connections to Apache, causing web server to serve error 508 page ( Resource Limit Reached), once there number of concurrent requests for the site goes above the limit.
Number of processes
NPROC controls the total number of processes and threads within LVE. Once the limit is reached, no new process can be created (until another one dies). When that happens NPROC counter is incremented. Apache might return 500 or 503 errors in such case.
Network traffic bandwidth control and accounting system
Requires kernel lve1.4.4.el6 or higher, or lve1.4.56.el7 or higher
Network traffic bandwidth control and accounting systems in CloudLinux 6 allows for each LVE container:
- Limiting outgoing network traffic bandwidth
- Accounting incoming and outgoing network traffic
The system supports IPv4 only protocol.
How to limit outgoing network traffic
All outgoing IP packets generated inside LVE container are marked with LVE identifier. Traffic control utility tc from iproute2 package uses this marker to set required bandwidth.
CloudLinux doesn’t limit the network traffic itself, it only marks IP packets with specific LVE id.
- We create class with HTB qdiscs and rate 10kbit :
tc qdisc add dev eth1 root handle 1: htb tc class add dev eth1 parent 1: classid 1:1 htb rate 10kbit
- All packets marked with LVE id will be processed by class 1:1 (rate 10kbit ).
tc filter add dev eth1 parent 1: handle 2121 fw flowid 1:1
- As an example we create class with HTB qdiscs and rate 100mbit and class 1:10 will be used by default:
tc qdisc add dev eth3 root handle 1: htb default 10 tc class add dev eth3 parent 1: classid 1:1 htb rate 100mbit
- For class 1:1 we create two branches with rate 5 mbit and 10 kbit accordingly, with classid 1:10 and 1:20.
tc class add dev eth3 parent 1:1 classid 1:10 htb rate 5mbit tc class add dev eth3 parent 1:1 classid 1:20 htb rate 10kbit
- All packets marked with LVE id=2121 are processed by 10 kbit class.
tc filter add dev eth3 protocol ip parent 1: prio 1 handle 2121 fw flowid 1:20
More info about
tc and its syntax can be found on the link http://tldp.org/HOWTO/Traffic-Control-HOWTO/index.html
Traffic accounting is performed for each LVE container. Network statistics is collected at
/proc/lve/list file. Network-related data found at fields:
lNETO- output traffic limit by volume, equals 0*
lNETI- input traffic limit by volume, equals 0*
NETO- current outgoing traffic value
NETI- current incoming traffic value
The data is also collected at
id is an LVE container identifier.
net_stat file contains 4 values in one row:
- Outgoing traffic limit by volume, equals 0*
- Incoming traffic limit by volume, equals 0*
- Current outgoing traffic value
- Current incoming traffic value
The current version of CloudLinux network control system doesn’t limit network traffic volume for a specific period of time (for example 3GB per day), it limits only network bandwidth.
Network limits are supported only for processes inside LVE. By default it does not limit static content, but only PHP/cgi scripts processed by Apache and processes launched over ssh etc.
Starting from lve-utils version 3.1-1, the validation of EP and NPROC limits is supported. If an administrator sets the NPROC limit less than (EP + 15), the following warning is shown:
error: You're trying to set invalid LVE limits. NPROC limit must be greater than EP + 15 limit, because number of processes and threads within LVE includes also Apache processes/threads, SSH sessions and etc, which enter into LVE.
Validation does not affect limits operation in any way. Even if invalid limits have been set, they will be applied for users/resellers.
Commands that support validation:
This command allows validation of an LVE ID which does not have a corresponding UID in the system. I.e., you can set limits for any LVE ID and they can be validated.
This command allows validation when setting limits using a user name instead of LVE ID.
This command supports limits validation both for inactive reseller and active one.
This command supports validation when setting default limits for a reseller.
This command supports limits validation both for packages existing in the system and nonexisting ones.
cloudlinux-limitscommands support all validation types described above, and support limits validation and exceptions lists as described below.
Exceptions list (validation is not supported)
- a) When EP limit for a package is greater than a custom NPROC limit for a user included in this package.
b) when NPROC limit for a package is less than a custom EP limit for a user included in this package.
- a) When default EP limit for a hoster is greater than a custom NPROC limit for a user/package which inherits the default limit.
b) When default NPROC limit for a hoster is less than a custom EP limit for a user/package which inherits the default limit.
- When using the following commands:
lvectl set-reseller --all
cloudlinux-limits --json enable-reseller-limits --all
Existing limits validation
The automatic validation using
cldiag utility by cron job is enabled on a server by default. You can disable it in the
/etc/sysconfig/cloudlinux config file using
ENABLE_CLDIAG option (Warning! This option disables all automatic checks using cldiag!) When calling this utility automatically by cron, it checks all limits existing on the server and send an administrator a report with limits check results. You can use the following command to validate existing limits:
The important difference between checking existing and setting limits is that even if validation fails when setting limits (see exceptions list above), checking existing limits will catch invalid limits in any case. I.e. even if a server administrator set invalid limits, validation of existing limits will catch invalid limit in any case.
Set NPROC limit greater than (EP + 15).
|Web Server / PHP||CPU||Virtual & Physical Memory||EP||NPROC||IO||CageFS||PHP Selector|
|Apache / suPHP||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Apache / FCGID||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Apache / CGI||Yes||Yes||Yes||Yes||Yes||Yes||Yes|
|Apache / PHP-FPM||Yes 3||Yes||Yes||Yes||Yes||Yes 3||No|
|Apache / mod_php||Yes||No||Yes||Yes||Yes||No||No|
|Apache / mod_ruid2||Yes||No||Yes||Yes||Yes||No||No|
|Apache / MPM ITK||Yes||No||Yes||Yes||Yes||Yes 1||No|
|NGINX / PHP-FPM||Yes 3||Yes||No||Yes||Yes||Yes||No|
- Requires patched version of MPM-ITK. CL httpd RPM has ITK worker with the patch. Patch is also available at: http://repo.cloudlinux.com/cloudlinux/sources/da/cl-apache-patches.tar.gz
- CloudLinux 7 and CloudLinux 6 kernels only.
- The DirectAdmin and CloudLinux PHP provide patched version. For other PHP distributions, please use patches available here: http://repo.cloudlinux.com/cloudlinux/sources/da/cl-apache-patches.tar.gz
Please note that mod_lsapi does not work when php-fpm is enabled because php-fpm is also a PHP Handler just as mod_lsapi.