# Control Panel Integration

• Introduction
• New API in a nutshell
• General
• Control panel API integration
• Control panel hooks integration
• Web UI integration
• PHP-based integration of WEB UI with the control panel
• How to integrate CageFS with a control panel
• Integrating CloudLinux OS PHP Selector
• Integration of Apache modules in control panels
• MySQL Governor
• CloudLinux OS LVE and CageFS patches
• Hardened PHP
• CloudLinux OS kernel set-up
• How to integrate X-Ray and AccelerateWP with a conrol panel

# Introduction

There are several possible ways of integration with CloudLinux OS:

• Complete integration using new API - exactly what is described in this document. This way a panel vendor will get all CloudLinux OS features (current and future) and maximum support. It’s recommended.
• Manually Ad-hoc - using low-level CLI utils. It is not recommended. It’s kind of “do it yourself way” - a control panel might use low-level utils like lvectl to set limits directly to a raw LVE by ID and control everything (including edge-cases) in its own code. There are many downsides of such approach e.g. only a small number of features can be implemented this way and any new changes in CloudLinux OS will require more and more updates to the control panel code, and can possibly even introduce bugs, etc. And although this way looks easier at first, it will become more and more difficult to maintain it over time.
• Old API for CloudLinux OS limits - described here. It’s deprecated. It still works but will not get any updates.

# New API in a nutshell

The goal of the new API is to shift all complexity for controlling CloudLinux OS components from a control panel to the CloudLinux OS web UI and utils. Most of the integration is done within a few steps:

1. A control panel vendor implements 7 simple scripts (any language) and specifies them in a special config file. All Cloudlinux OS components will look up and call them with some arguments to get all needed information from the control panel (e.g. users list, their hosting plans, domains, etc.)
2. Control panel should call hooks described below in a response to Admin/Users actions (e.g. changing end-user’s domain, creating new end-user, etc.). CloudLinux OS utils will do their stuff to reconfigure itself according to these events. There is no need to worry about what exactly they do because it will be related only to CloudLinux OS components and control panel will not be affected.
3. Configure the control panel and CageFS to work together by changing some configs.
4. Optionally embed CloudLinux OS web UI into the control panel. It is highly recommended because it does all you may need. You can use PHP scripts to embed SPA application to the specially prepared pages (with your menus) for LVE Manager and Selectors.

# General

• Example of the integration config

To integrate CloudLinux OS into a control panel, you should implement the following interfaces required by the CloudLinux OS utilities:

• CPAPI — a small scripts/files-type interface to get the required information from the control panel
• CP Hooks Integration — a set of the CloudLinux OS scripts/commands. Control panel calls this set in response to its internal events
• Web UI Integration(optional) — various configuration parameters and scripts to embed interface SPA application into the control panel native interface. Optional, if CLI is enough
• CageFS Integration — some changes may be required in the config files to work and integrate with CageFS as well as additional support from a control panel
• CloudLinux OS kernel set-up

Files related to integration (integration config file, CPAPI integration scripts and all their dependencies) as well as parent directories to place them (e.g. /opt/cpvendor/etc/) should be delivered/managed by the control panel vendor. They don't exist by default and should be created to activate integration mechanism. Most likely, they should be bundled with the control panel or installed with some CloudLinux OS support module (if CloudLinux OS support is not built-in).

Configuration related to integration should be placed in a valid INI file (further “integration config file”):

-rw-r--r-- root root /opt/cpvendor/etc/integration.ini

All users (and in CageFS too) should have read access to this file because some CPAPI methods have to work from end-user too. CloudLinux OS components only read this file and never write/create it.

The control panel vendor decides where to put CPAPI scripts (CloudLinux OS utils will read path to scripts from the integration file) but we recommend to put them into /opt/cpvendor/bin/ (should be created by the vendor first) because /opt/* is mounted to CageFS by default and the standard location will save some time during possible support incidents.

We recommend to deliver a set of scripts for CPAPI and integration file as a separate RPM package. This can greatly simplify support for vendors as they will be able to release any updates and compatibility fixes separately from the panel updates and use dependencies/conflicts management provided by a package manager.

Assumed that this package contains a list of paths to the integration scripts (or to the one script if it deals with all cpapi commands). You can specify default script arguments in the integration_scripts section, additional arguments like filter and --name will be automatically added after the defined ones.

# Versioning

We provide information about the current version of the API in the form of rpm package capability (Provides public_cp_vendors_api = VERSION) added to our packages. While improving integration API, we may add new features and scripts and change VERSION.

That means that you can add Conflicts: public_cp_vendors_api < VERSION to the spec of rpm package with your integration scripts and that will force yum to search and update our packages in order to support public_cp_vendors_api that your scripts require. It also means that you can protect your Control Panel from situations when your scripts and our API version are incompatible.

# Changelog

# Example of the integration config

[integration_scripts]

panel_info = /opt/cpvendor/bin/panel_info
db_info = /opt/cpvendor/bin/db_info
packages = /opt/cpvendor/bin/packages
users = /opt/cpvendor/bin/users
domains = /opt/cpvendor/bin/vendor_integration_script domains
resellers = /opt/cpvendor/bin/vendor_integration_script resellers
admins = /opt/cpvendor/bin/vendor_integration_script admins
php = /opt/cpvendor/bin/vendor_integration_script php



[lvemanager_config]
ui_user_info = /panel/path/ui_user_info.sh
base_path = /panel/path/lvemanager

run_service = 1
service_port = 9000
use_ssl = 1
ssl_cert = /path/to/domain_srv.crt
ssl_key = /path/to/domain_srv.key

# Control panel API integration

To integrate CPAPI, set the paths to the scripts in the integration file in the integration_scripts section (see example above).

You can use different scripts for different CPAPI methods or only one script and call it with different arguments. The script is any executable file written in any programming language, that returns a valid JSON file of the specified format in response to arguments it is expected to be called.

# Integration scripts requirements

• Legend

• By default, integration scripts should run only from UNIX user with the same credentials as LVE Manager is run, unless otherwise stated in the script description.

• Some scripts are run from both root and end-user. If the script is run from end-user it should return information related to this user only, filtering the irrelevant data for security/isolation purposes or return PermissionDenied if script is not intended to be run as this user at all.

• Scripts run from the end-user should also work from inside CageFS. You can find details on how to implement this here

# Legend

• Necessity always means it’s required nearly by all CloudLinux OS code and should be implemented no matter what features you are interested in. Other methods might not be implemented and this will affect only some CloudLinux OS features leaving others to work fine.
• All UNIX users means any Linux system account (from /etc/passwd) that is able to execute CloudLinux OS utilities directly or indirectly.

# Working with CPAPI & CageFS

Some integration scripts required to be called not only from root but from end-user. Developing such scripts, you should keep in mind that CageFS is an optional component, so anything that works in CageFS must also work even if CageFS is disabled for a particular user or not installed at all.

This means that you should do one of the following:

• all required information (code, all its dependencies, all called binaries, all configs/files/caches it reads) should be available for a user (even in CageFS)
• you should use CageFS's proxyexec mechanism to make your script run itself in a real system but with user privileges
• use both configured sudo and proxyexec if your script needs privilege escalation (e.g. to read some credentials or other files that shouldn't be disclosed to end-user). SUDO mechanism is needed for cases when CageFS is not installed or disabled for a user and proxyexec is needed when a user executes script in CageFS

When using SUDO you should create the wrapper that will call your script using SUDO command, like this:

[root]# cat /opt/cpvendor/bin/domains
#!/usr/bin/env bash
sudo opt/cpvendor/bin/domains_real

Of course, you must configure /etc/sudoers to make it work without a password prompt and with enough security.

***************************************************************************
*                                                                         *
*             !!!!  WARNING: YOU ARE INSIDE LVE !!!!                      *
*IF YOU RESTART ANY SERVICES STABILITY OF YOUR SYSTEM WILL BE COMPROMIZED *
*        CHANGE YOUR USER'S GROUP TO wheel to safely use SU/SUDO          *
*                             MORE INFO:                                  *
*      https://docs.cloudlinux.com/index.html?lve_pam_module.html          *
*                                                                         *
***************************************************************************

As an alternative for sudo you can use regular SUID scripts, but we strongly recommend using SUDO because of builtin logging of permissions escalation.

You can find more details on how to configure your scripts in CageFS properly here:

• Configuration. General information
• How to integrate CageFS with any control panel

# Expected scripts responses

• Internal errors
• Restricted access
• Error in arguments
• Nonexistent entities

Any integration script should return a valid UTF-8 encoded JSON.

There are two expected formats of integration script responses. In case of success, the metadata.result field is ok and the data field contains data according to the specification of the specific integration script. Return code is 0.

{
  "data": object or array,
  "metadata": {
    "result": "ok"
  }
}

In case of error, the output should be one of the following: Internal errors, Restricted access, Error in arguments, Nonexistent entities.

# Internal errors

When data is temporarily unavailable due to internal errors in the integration script (database unavailable, file access problems, etc), the output is as follows:

{
  "data": null,
  "metadata": {
    "result": "InternalAPIError",
    "message": "API is restarting, try again later"
  }
}

Data description

# Restricted access

When data is unavailable due to restricted access of a user that called the script, the output is as follows:

{
  "data": null,
  "metadata": {
    "result": "PermissionDenied",
    "message": "Only sudoers can view this section."
  }
}

Data description

# Error in arguments

When wrong arguments are passed to the utility, for example:

• unknown argument
• unknown fields in --fields

The output is as follows:

{
  "data": null,
  "metadata": {
    "result": "BadRequest",
    "message": "Unknown field ‘nosuchfield’"
  }
}

Data description

# Nonexistent entities

When during data filtering the target entity doesn't exist in the control panel, the output is as follows:

{
  "data": null,
  "metadata": {
    "result": "NotFound",
    "message": "No such user `unix_user_name`"
  }
}

Data description

# The list of the integration scripts

• panel_info
• db_info
• packages
• users
• domains
• resellers
• admins
• php

# panel_info

Returns the information about the control panel in the specified format.

Usage example

/scripts/panel_info

Output example

{
	"data": {
		"name": "SomeCoolWebPanel",
		"version": "1.0.1",
		"user_login_url": "https://{domain}:1111/",
		"supported_cl_features": {
			"php_selector": true,
			"ruby_selector": true,
			"python_selector": true,
			"nodejs_selector": false,
			"mod_lsapi": true,
			"mysql_governor": true,
			"cagefs": true,
			"reseller_limits": true,
			"xray": false,
			"accelerate_wp": false,
      "autotracing": true
		}
	},
	"metadata": {
		"result": "ok"
	}
}

Data description

# db_info

Returns the information about databases that are available to the control panel users and are managed by the control panel.

Integration script is optional, when there is no script, lve-stats won’t collect SQL snapshots.

Usage example

/scripts/db_info

Output

{
  "data": {
    "mysql": {
      "access": {
        "login": "root",
        "password": "Cphxo6z",
        "host": "localhost",
        "port": 3306
      },
      "mapping": {
        "unix_user_name": [
          "db_user_name1",
          "db_user_name2"
        ],
        "unix_user_name2": [
          "db_user_name3",
          "db_user_name4"
        ]
      }
    }
  },
  "metadata": {
    "result": "ok"
  }
}

Data description

# packages

The package is an abstraction that represents a group of users that have the same default limits.

Usage example

/scripts/packages [--owner=<owner>]

Arguments

Output example

{
  "data": [
    {
      "name": "package",
      "owner": "root"
    },
    {
      "name": "package",
      "owner": "admin"
    },
    {
      "name": "package",
      "owner": "reseller"
    }  
  ],
  "metadata": {
    "result": "ok"
  }
}

Data description

# users

Returns information about UNIX users, created and managed by the control panel. You will be able to manage the limits of these users via LVE Manager.

If a reseller user or administrator user has a corresponding UNIX-user in the system, this user should be included into this list.

Usage example

/scripts/users [--owner=<string> | (--package-name=<string> & --package-owner=<string>) | --username=<string> | --unix-id=<int>] [--fields=id,username,package,...]

Arguments

Output example

{
  "data": [
    {
      "id": 1000,
      "username": "ins5yo3",
      "owner": "root",
      "domain": "ins5yo3.com",
      "package": {
        "name": "package", 
        "owner": "root"
      },
      "email": "ins5yo3@ins5yo3.com",
      "locale_code": "EN_us"
    },
    {
      "id": 1001,
      "username": "ins5yo4",
      "owner": "root",
      "domain": "ins5yo4.com",
      "package": {
        "name": "package", 
        "owner": "root"
      },
      "email": "ins5yo4@ins5yo4.com",
      "locale_code": "EN_us"
    }  
  ],
  "metadata": {
    "result": "ok"
  }
}

Example to output specific data fields

/scripts/users --fields=id,email

Output

{
  "data": [
    {
      "id": 1000,
      "email": "ins5yo3@ins5yo3.com"
    },
    {
      "id": 1001,
      "email": "ins5yo3@ins5yo3.com"
    }  
  ],
  "metadata": {
    "result": "ok"
  }
}

Data description

# domains

Returns key-value object, where a key is a domain (or subdomain) and a value is a key-value object contains the owner name (UNIX users), the path to the site root specified in the HTTP server config, and the domain status (main or alternative).

X-Ray support is available since API v1.2 (see versioning). In order to enable X-Ray support, the value for each domain should include php configuration. Full X-Ray integration documentation can be found here AccelerateWP support is available since API v1.4 (see versioning). In order to enable AccelerateWP support, the php_version_id fields should be included in php configuration. php_version_id should match unique identifier retured by php script and handler must be specified, one of the following: fpm, cgi, lsapi.

Usage example

/scripts/domains ([--owner=<unix_user>] | [--name=<name>] | [--with-php])

Arguments

Output example

{
  "data": {
    "domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/",
      "is_main": true
    },
    "subdomain.domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/subdomain/",
      "is_main": false  
    }
  },
  "metadata": {
    "result": "ok"
  }
}

Output example with optional flag --with-php (X-Ray and Accelerate WP support only)

{
  "data": {
    "domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/",
      "is_main": true,
      "php": {
        "php_version_id": "alt-php56",
        "version": "56",
        "ini_path": "/opt/alt/php56/link/conf",
        "is_native": true,
        "handler": "lsapi"
       }
    },
    "subdomain.domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/subdomain/",
      "is_main": false,
      "php": {
        "php_version_id": "alt-php72",
        "version": "72",
        "ini_path": "/opt/alt/php72/link/conf",
        "fpm": "alt-php72-fpm",
        "handler": "fpm"
      }
    }
  },
  "metadata": {
    "result": "ok"
  }
}

Data description

# resellers

This script returns the information about resellers. A reseller is an entity that can own users in the control panel. Resellers do not obligatory have UNIX account with the same name. It can exist only as an account in the control panel.

Usage example

/scripts/resellers ([--id=<id>] | [--name=<name>])

Arguments

Output example

{
  "data": [
    {
      "name": "reseller",
      "locale_code": "EN_us",
      "email": "reseller@domain.zone",
      "id": 10001   
    }
  ],
  "metadata": {
    "result": "ok"
  }
}

Data description

# admins

Gives information about panel’s administrators, output information about all panel’s administrators who:

• could be (or actually are) the owners of the users, listed in users()
• could be (or actually are) the owners of the packages, listed in packages()
• has UNIX users with the rights to run LVE Manager UI

Usage example

/scripts/admins ([--name=<string>] | [--is-main=<true|false>])

Arguments

Output example

{
  "data": [
    {
      "name": "admin1",
      "unix_user": "admin",
      "locale_code": "EN_us",
      "email": "admin1@domain.zone",
      "is_main": true
    },
	{
      "name": "admin2",
      "unix_user": "admin",
      "locale_code": "Ru_ru",
      "email": "admin2@domain.zone",
      "is_main": false
    },
  ],
  "metadata": {
    "result": "ok"
  }
}

Data description

# php

Gives information about panel’s supported php versions.

Usage example

/scripts/php

Output example

{
  "data": [
    {
      "identifier":  "alt-php74",
      "version": "7.4",
      "modules_dir":  "/opt/alt/php74/usr/lib64/modules",
      "dir": "/opt/alt/php74/",
      "bin":  "/opt/alt/php74/usr/bin/php",
      "ini": "/opt/alt/php74/link/conf/default.ini"
    },
    {
      "identifier":  "ea-php74",
      "version": "7.4",
      "modules_dir":  "/opt/cpanel/ea-php74/usr/lib64/modules",
      "dir": "/opt/cpanel/ea-php74/",
      "bin":  "/opt/cpanel/ea-php74/usr/bin/php",
      "ini": "/opt/cpanel/ea-php74/etc/php.ini"
    }
  ],
  "metadata": {
    "result": "ok"
  }
}

Data description

# Control panel hooks integration

For proper CloudLinux OS operation, you should implement a hooks mechanism. Hooks are scripts that control panel invokes with the exact arguments at a specific time. The control panel is responsible for executing these scripts as a system root user.

# Managing administrators

After creating a new administrator, the control panel should call the following command:

/usr/share/cloudlinux/hooks/post_modify_admin.py create --username admin

After removing the administrator, the following command should be called by the control panel:

/usr/share/cloudlinux/hooks/post_modify_admin.py delete --name admin

# Managing users

To manage user limits properly, CloudLinux OS utilities need information about the following control panel events.

After creating a new user, the following script should be called:

/usr/share/cloudlinux/hooks/post_modify_user.py create --username user --owner owner

After renaming a user (when a user name or a home directory was changed), the following script should be called:

/usr/share/cloudlinux/hooks/post_modify_user.py modify -u user --new-username newuser

After moving the user to the new owner, the following command should be run:

/usr/share/cloudlinux/hooks/post_modify_user.py modify -u user --new-owner new_owner

Before removing a user, the following command should be run:

/usr/share/cloudlinux/hooks/pre_modify_user.py delete --username user

After removing the user, you should call the following command:

/usr/share/cloudlinux/hooks/post_modify_user.py delete --username user

After restoring the user fron backup, you should call the following command:

/usr/share/cloudlinux/hooks/post_modify_user.py restore --username user

# Managing domains

We expect that all domain-related data is stored inside the user's home directory. If your control panel stores domain-related data outside the user's home directory and that data can potentially change, please contact us so we can add support for this to our integration mechanism.

After renaming a domain (or any equivalent domain removal operation with transfer of all data to another domain), the following command should be run:

/usr/share/cloudlinux/hooks/post_modify_domain.py modify -u user --domain old_domain --new-domain new_domain [--include-subdomains]

# Managing packages (hosting plans)

To manage packages limits properly, CloudLinux OS utilities need information about the following control panel events.

After renaming a package, the following command should be run:

/usr/share/cloudlinux/hooks/post_modify_package.py rename --name package_name --new-name new_package_name

# Web UI integration

UI integration can be configured in lvemanager_config section of the /opt/cpvendor/etc/integration.ini:

Example:

[lvemanager_config]
# Required
ui_user_info = /panel/path/ui_user_info.sh

# Can be optional or required depend on vendor implementation
base_path = /panel/path/lvemanager
run_service = 1
service_port = 9000
use_ssl = 1
ssl_cert = /path/to/domain_srv.crt
ssl_key = /path/to/domain_srv.key

# ui_user_info script

• Known issues for GUI unification

This script should return information about the current user opening Web UI. It is only used in UI part of the LVE Manager and utilities.

• Input value (first positional argument) — the current authentication token, passed on plugin open (for example, open.php?token=hash)
• Expected data format

{
    "userName": "user1",
    "userId": 1000,
    "userType": "user",
    "baseUri": "/user2/lvemanager/",
    "assetsUri": "/userdata/assets/lvemanager",
    "lang": "en",
    "userDomain": "current-user-domain.com"
}

• userName is used in queries to Selectors and Resource Usage plugin.
• userId - system user ID if exist.
• userType - can have the following values: “user”, “reseller”, “admin”
• lang - used to identify a current locale in the user interface (‘en’, ‘ru’ - two-character language code)
• userDomain - a current user domain. It can be the default domain or any selected user domain. Used to display in Selector. For example, in DirectAdmin the control panel opens only if a domain is selected — the selected domain is set. In cPanel, a domain is not selected — the default domain is set. If a domain is absent — leave empty.
• baseUri - URI where LVE Manager files are available on the webserver
• assetsUri - URI where assets files for LVE Manager are available on the webserver

The following configuration file parameters are used to determine the location of the plugin UI part.

• base_path - directory where file assets are copied for accessibility from the control panel. This is optional if the default directories /usr/share/l.v.e-manager/commons and /usr/share/l.v.e-manager/panelless-version/lvemanager are accessible from the control panel and correctly configured in the web server. The command yum update lvemanager performs file copying or replacement.
• run_service - parameter that activates the LVE Manager web server. Set it to 1 during the installation or updating of LVE Manager to run the web server automatically.
• service_port - port used for running a web server for access LVE Manager without the control panel (required if run_service = 1)
• use_ssl - optional parameter that mandates the use of HTTPS for the LVE Manager web server when set to 1.
• ssl_cert - conditional parameter that becomes essential when use_ssl is activated. Identifies the path to the SSL certificate file for the domain.
• ssl_key - conditional parameter that becomes mandatory when use_ssl is enabled. Specifies the path to the SSL certificate key file for the domain.

# Known issues for GUI unification

# PHP-based integration of WEB UI with the control panel

The first use case:

require_once('lvemanager/widget.php');

This script defines the type of user and displays icons to open plugins included in LVE Manager. You can customize your styles by copying the file content and setting your control panel styles. In this case, all LVE Manager plugins will be opened in a new window.

The second use case:

Embedding the specific plugin into the control panel page.

<?php
require_once('lvemanager/LveManager.php');
$manager = new LveManager(‘<pluginName>’);
$manager->setCSRFToken();
?>
<html><body>
<?php
echo $manager->insertPanel();
?>
</body></html>

Pass the plugin name to the class constructor to initialize the plugin (user name will be defined). At the time of the insertPanel method invocation, the SPA application with the specified Selector will be embedded.

vendor_php (optional for PHP integration only) - a full path to the vendor.php file. This option allows the vendor to implement specific logic before initializing the plugin (for example drop permission).

Example:

vendor_php = /opt/cpvendor/etc/vendor.php

Queries to the backend are created separately in the points (PHP files) which are located in the LVE Manager catalog.

# Implementing Logic for Plugin Visibility Control

Vendors can implement logic to control the visibility of end-user plugins within their control panel. The current UI settings are stored in the read-only config file located at /usr/share/l.v.e-manager/lvemanager-config.json. Note: This file is read-only and cannot be modified by vendors. Vendors should read this file and implement logic in their own systems to show or hide icons based on the settings in this config file.

{
    "ui_config": {
        "inodeLimits": {
            "showUserInodesUsage": false
        },
        "uiSettings": {
            "hideRubyApp": true,
            "hideLVEUserStat": false,
            "hidePythonApp": false,
            "hideNodeJsApp": false,
            "hidePHPextensions": false,
            "hideDomainsTab": false,
            "hidePhpApp": false,
            "hideXrayApp": true,
            "hideAccelerateWPApp": false
        }
    }
}

The following options are available under ui_config.uiSettings:

• hideLVEUserStat - control the visibility of the “Resource Usage” plugin for end-users (hide if value is true, show if value is false).
• hidePhpApp - control the visibility of the “PHP Selector” plugin for end-users (hide if value is true, show if value is false).
• hidePythonApp - control the visibility of the “Python Selector” plugin for end-users (hide if value is true, show if value is false).
• hideNodeJsApp - control the visibility of the “Nodejs Selector” plugin for end-users (hide if value is true, show if value is false).
• hideXrayApp - control the visibility of the “X-Ray” plugin for end-users (hide if value is true, show if value is false).
• hideAccelerateWPApp - control the visibility of the “AccelerateWP” plugin for end-users (hide if value is true, show if value is false).

Brand assets located in /usr/share/l.v.e-manager/commons/brand-assets. In the images folder present icons which vendor can use while embeds plugin into control panel. Icons for the following plugins are available:

• CloudLinux Manager
• Resource Usage
• Python/NodeJS, PHP Selectors
• X-Ray
• AccelerateWP

# UI with CageFS enabled

LVE Manager scripts run outside of CageFS (on production systems). When calling API, LVE Manager scripts enter CageFS if needed (Node.js Selector, Python Selector). PHP Selector API does not work if the script is started inside CageFS and returns an error.

As for Node.js Selector and Python Selector, they are supposed to be contained inside CageFS because end-users can start scripts from UI (npm run, pip install). That is the reason why they should run inside LVE.

PHP Selector run outside of CageFS. When PHP Selector is called, it is not limited by LVE which allows an end-user whose web site is reaching its limits to use PHP Selector UI without problems.

# How to integrate CageFS with a control panel

CageFS documentation can be found here: CageFS

# CageFS MIN_UID

CageFS has UID_MIN setting. Users with UIDs < UID_MIN will not be limited by CageFS. This setting is configured based on UID_MIN setting from /etc/login.defs file by default. So, typically UID_MIN is 500 on CloudLinux OS 6 and 1000 on CloudLinux OS 7.

You can obtain current UID_MIN value by executing the following command:

cagefsctl --get-min-uid

You can set UID_MIN by executing the following command:

cagefsctl --set-min-uid <value>

For example, to set UID_MIN to 10000, you should execute the following command:

cagefsctl --set-min-uid 10000

# PAM configuration for CageFS

CageFS is enabled for su, ssh and cron services by default. PAM Configuration

Also you can enable CageFS for any service that uses PAM. LVE PAM Module

# Integrating CageFS with Apache

You should apply CloudLinux OS patches to integrate CageFS with Apache. Details can be found here: Integration of Apache modules in Control Panels

Note that it may be required to execute cagefsctl --force-update after Apache rebuild in order to update CageFS.

# Running commands inside CageFS

You may want to execute commands inside CageFS for security reasons. To do so, you can execute a command inside CageFS in the following way:

/bin/su -s /bin/bash -c “$COMMAND” - $USER
/sbin/cagefs_enter_user $USER "$COMMAND"

The commands above require root privileges. You can use the following command when running as a user:

/bin/cagefs_enter "$COMMAND"

# CageFS - temporary LVEs with lvectl options

Starting from lve-wrappers 0.7.5-1, cagefs_enter_user accepts a subset of lvectl limit parameters as input. Using any of them will result in a creation of a temporary LVE with a unique ID. That LVE will have the provided limits set and will be destroyed after the given command is run.

The following parameters are accepted: --speed, --pmem, --vmem, --io, --iops, --nproc, --maxEntryProcs.

cagefs_enter_user --pmem=512M --speed=70% <user> <command>

Refer to documentation on limits for more details on the provided parameters.

# Updating CageFS skeleton

Updating CageFS skeleton is required after update of the system RPM packages. It may be also required after update of hosting control panel itself.

To update CageFS skeleton, execute the following command:

cagefsctl --force-update

CageFS RPM package installs a daily cron job that updates CageFS skeleton

/etc/cron.d/cp-cagefs-cron

You can set update period for CageFS skeleton in days using

cagefsctl --set-update-period <days>

# Setting up directories for PHP sessions

Each user in CageFS has its own /tmp directory that is not visible for other users. PHP scripts save sessions in /tmp directory by default (when session.save_path directive is empty). However, you can set up a different location for PHP sessions specific to each PHP versions using CageFS per-user mounts like it is done for alt-php. Simply add a line like below to /etc/cagefs/cagefs.mp

@/opt/alt/php54/var/lib/php/session,700

and then execute

cagefsctl --remount-all

to apply changes. After that, session files will be stored in /opt/alt/php54/var/lib/php/session inside user’s CageFS. In real file systems these files can be found inside .cagefs directory in user’s home directory. For above example, given that user’s home directory is /home/user, the files will reside in /home/user/.cagefs/opt/alt/php54/var/lib/php/session directory. TMP directory for the user will be located in /home/user/.cagefs/tmp. Temporary files including php sessions in /tmp directories in CageFS are cleaned using tmpwatch. You can change the period of removing old temporary files or set up additional directories for cleaning according to the documentation: TMP directories.

Knowing the location where PHP sessions are stored (described above), you can also implement any custom script for cleaning PHP sessions. Remember to drop permissions (switch to the appropriate user) when removing the files (for example using sudo).

# Creating new user account

When user or admin account has been created, you should execute an appropriate hook script. More about CloudLinux OS hooks subsystem: Control Panel Hooks Integration

rm -rf /var/cagefs/`/usr/sbin/cagefsctl --getprefix $username`/$username/etc/cl.selector
rm -rf /var/cagefs/`/usr/sbin/cagefsctl --getprefix $username`/$username/etc/cl.php.d
/usr/sbin/cagefsctl --force-update-etc "$username"

# Modifying user accounts

You should execute appropriate hook script when an account has been modified: Control Panel Hooks Integration

# Removing user accounts

CageFS has userdel hook script /usr/bin/userdel.cagefs, that is configured in /etc/login.defs file:

USERDEL_CMD /usr/bin/userdel.cagefs

This script performs all necessary actions when a user is being removed. Anyway, you should execute appropriate hook script when removing an account: Control Panel Hooks Integration

# Excluding users from CageFS

If you need to exclude some system users from CageFS that are specific to your control panel, you can do this by creating a file (any name would work) inside /etc/cagefs/exclude folder, and list users that you would like to exclude from CageFS in that file (each user in separate line). Then execute

cagefsctl --user-status USER

to apply changes and check that the command shows Disabled: Excluding users

# How to add a file or a directory to CageFS

• Copy files and directories to CageFS
• Mount an entire directory with needed files to CageFS

There are two major ways to add files and directories to CageFS:

1. copy files and directories to CageFS
2. mount an entire directory with needed files to CageFS

# Copy files and directories to CageFS

In order to copy files and directories to CageFS, you can manually create a custom config file with *.cfg extension in /etc/cagefs/conf.d directory.

Also, you can add files and directories that belong to rpm package to CageFS as follows:

• execute cagefsctl --addrpm PACKAGE command
• execute cagefsctl --force-update command to apply changes, i.e. to copy files and directories to the /usr/share/cagefs-skeleton directory.

There is a daily cron job that copies new and removes unneeded files and directories, i.e. updates cagefs-skeleton.

You should execute cagefsctl --force-update after each change of system configuration or after installing new software. Otherwise, CageFS will only be updated every 24 hours by a daily cron job.

The benefits of this approach:

• users in CageFS cannot modify files in the real file system (for example, in case of wrong permissions), because files in cagefs-skeleton are the copies of files from the real file system (and not mounted to CageFS)
• you can copy specific safe files from some directory from the real file system to CageFS, and not copy all files and not mount the entire directory.

You can find more info here.

# Mount an entire directory with needed files to CageFS

The second way to add files and directories to CageFS is to mount the entire directory containing the files to CageFS.

This approach should be used for frequently updated files (like UNIX sockets) or for directories with a large number of files or with files that occupy much disk space.

The benefits of this approach:

• no delay between update of the file in the real file system and update of the file in CageFS; when file or directory is changed in the real file system they will be changed in CageFS immediately
• data is mounted but not copied to CageFS, so this minimizes IO load while updating cagefs-skeleton

Please make sure that the mounted directory does not contain sensitive data.

You can mount a directory from the real system to CageFS via /etc/cagefs/cagefs.mp config file.

To apply changes of mount points in /etc/cagefs/cagefs.mp file you should execute the following command:

cagefsctl --remount-all

You can find more info here.

# Users' home directory

CageFS mounts users' home directories to CageFS automatically. Usually, there is no need to configure anything. However, if your control panel uses a custom path to users' home directories (for example /home/$USER/data instead of /home/$USER), then it may be necessary to configure Base Home Directory setting: Base Home Directory

The modes of mounting users' home directories into CageFS are described here: Mounting user’s home directory inside CageFS

# Excluding files from CageFS

CageFS has a default set of files and directories that are visible to users inside CageFS. If you wish to exclude some of these files or directories from CageFS, you can do this as described below: Excluding files

# Executing commands outside CageFS via proxyexec

SUID programs cannot run inside CageFS due to “nosuid” mounts. But you can give users in CageFS the ability to execute some specific commands outside CageFS via proxyexec.

Typically, these commands are suid programs or other programs that cannot run inside CageFS (due to complex dependencies, for example). Examples of such programs are passwd, exim or sendmail commands.

• Regular (not suid) commands will be executed as user inside the user’s LVE, but outside of CageFS.
• SUID commands will be executed in the user’s LVE outside of CageFS with the effective UID set to the owner of the file being executed.

You can find more details here.

# Filtering options for commands executed by proxyexec

It is possible to disable specific dangerous options of programs executed via proxyexec. More details: Filtering options for commands executed by proxyexec.

# Integrating CloudLinux OS PHP Selector

Complete documentation for CloudLinux OS PHP Selector can be found here: PHP Selector

If your control panel has some PHP version selector installed already, we suggest you not to enable CloudLinux OS PHP Selector for your users, so they will not be confused by two multiple PHP Selectors. However, if you want to use CloudLinux OS PHP Selector anyway, you can refer to the following article to configure loading of extensions for alt-php: PHP extensions

For proper operation of CloudLinux OS PHP Selector, you should specify location of native PHP binaries of your control panel in /etc/cl.selector/native.conf. Then execute

cagefsctl --setup-cl-selector

to apply changes. More details: Native PHP Configuration

You can configure CloudLinux OS PHP Selector to allow your customers to select any custom PHP version specific to your control panel. Here's how to do this: Roll your own PHP

Also, you can compile and add your own PHP extensions to CloudLinux OS PHP Selector: Compiling your own extensions

# Integration of Apache modules in control panels

We provide control panels with the ability to build their own Apache packages with modules such as alt-mod-passenger, mod_lsapi, mod_hostinglimits. This is due to the fact that control panels often have their own customly-built Apache. At the same time, the modules must be compiled with the version of Apache and APR libraries that control panels use. Therefore, if your control panel uses Apache that is provided by CloudLinux OS, you can use native packages with modules. The following table represents the RPM package compatibility.

RPMs of Apache provided by CloudLinux OS, if you are using:

If you use custom Apache, follow this documentation on how to build a package for your Apache.

# alt-mod-passenger

1. Download alt-mod-passenger sources

yumdownloader --source alt-mod-passenger --enablerepo=*source

2. Rebuild the package with your Apache using rpmbuild

rpmbuild --rebuild alt-mod-passenger-ver.cloudlinux.src.rpm

3. Remove dependencies from httpd, apr, apr-util packages (-devel packages as well) and set your Apache packages instead, in the alt-mod-passenger spec file. Also, set the path for the directory where your Apache configs are located with __apache_conf_dir in the spec file.

4. Rebuild the package again, this time you should get to the compilation stage. During compilation, you may encounter such errors:

* Checking for Apache 2 development headers...
     Found: no
* Checking for Apache Portable Runtime (APR) development headers...
     Found: no
* Checking for Apache Portable Runtime Utility (APU) development headers...
     Found: no

In this case, you should specify the paths of your custom Apache in alt-mod-passenger-5.3.7/utils/passenger file. You need to add paths to binaries such as httpd, apu-1-config, and apxs. For example the following lines should be added before the passenger-install-apache2-module call:

export APU_CONFIG="${PATH_IN_YOUR_SYSTEM}/bin/apu-1-config"
export HTTPD="${PATH_IN_YOUR_SYSTEM}/sbin/httpd"
export APXS2="${PATH_IN_YOUR_SYSTEM}/bin/apxs"

5. Rebuild the package again, if everything is set correctly, there shouldn't be any problems.

6. Install the module, check that it is successfully loaded into Apache.

# mod_hostinglimits

1. Download mod_hostinglimits sources

yumdownloader --source mod_hostinglimits --enablerepo=*source

2. Rebuild the package with your Apache using rpmbuild

rpmbuild --rebuild mod_hostinglimits-ver.el7.cloudlinux.1.src.rpm

3. Remove dependencies such as httpd, (-devel packages as well) and set your Apache packages instead, in the mod_hostinglimits spec file. Also, check the paths that lead to the modules or conf.d directories and change them if they are different on your system.

4. Rebuild the package again, this time you should get to the compilation stage. During compilation, you may encounter the following errors:

-- Not Found Apache Bin Directory: APACHE2_2_HTTPD_BIN-NOTFOUND, APACHE2_2_HTTPD_MODULES-NOTFOUND
-- Can't find Apache2.2: APACHE2_2_HTTPD_INCLUDE_DIR-NOTFOUND

In this case, you should specify the paths of your custom Apache in mod_hostinglimits-1.0/cmake/FindApacheForBuild.cmake file. According to macros above, add the paths to the directory where httpd bin is located, to the dir where apache modules are located and to the dir where include files are located. It may look the following way:

FIND_PATH (APACHE2_2_HTTPD_INCLUDE_DIR 
   NAMES 
      httpd.h
   PATHS
      ${APACHE2_2_INCLUDES_DIR}
      /usr/local/include/httpd
      /usr/include/httpd
      /usr/include/apache
      /usr/local/apache/include/       # added line
)

5. Rebuild the package again, if you set everything correctly, there shouldn't be any problems.

6. Install the module, check that it is successfully loaded into Apache.

# mod_lsapi

1. Install the packages needed to build mod_lsapi from the source. Some packages may be missing on some platforms, this is not critical

yum install -y rpm-build
yum install -y liblsapi liblsapi-devel
yum install -y autoconf cmake apr-devel apr-util-devel
yum install -y python-lxml pytest python-mock

On CloudLinux OS 7 and above you also should install criu related packages

yum install -y criu-lve python-criu-lve criu-lve-devel crit-lve

On CloudLinux OS 8 and above you also should install next packages

yum install -y alt-python37-mock alt-python37-pytest httpd-devel

2. Download mod_lsapi sources

yumdownloader --source mod_lsapi --enablerepo=*source

3. Rebuild the package with your Apache using rpmbuild

rpmbuild --rebuild mod_lsapi-ver.el7.cloudlinux.src.rpm

4. Remove building dependency from httpd-devel in the mod_lsapi.spec spec file located in the ~/rpmbuild/SPECS/ directory.

5. Change the paths to copy mod_lsapi.so and mod_lsapi.conf in the %install section of the mod_lsapi.spec file. For example, if Apache root is /usr/local/apache, then modules directory and config files directory will be /usr/local/apache/modules and /usr/local/apache/conf.d respectively. So for the config file you should substitute the following line in the %install section:

install -D -m 644 conf/mod_lsapi.conf $RPM_BUILD_ROOT%{g_path}/confs/lsapi.conf

with the following line:

install -D -m 644 conf/mod_lsapi.conf $RPM_BUILD_ROOT/usr/local/apache/conf/lsapi.conf

And, also, for the module you should substitute the following line in the %install section:

install -D -m 755 build/lib/mod_lsapi.so $RPM_BUILD_ROOT%{g_path}standard/mod_lsapi.so 

with the following line:

install -D -m 755 build/lib/mod_lsapi.so $RPM_BUILD_ROOT/usr/local/apache/modules/mod_lsapi.so

Also you should add mentions of both module and config files in the %files section of mod_lsapi.spec. For example, for /usr/local/apache Apache root the following two lines should be added:

%config /usr/local/apache/conf.d/lsapi.conf
/usr/local/apache/modules/mod_lsapi.conf

6. Rebuild the package again, this time you should get to the compilation stage.

Use this command to rebuild a package

During compilation, you may encounter some errors:

-- Not Found Apache Bin Directory: APACHE2_2_HTTPD_BIN-NOTFOUND, APACHE2_2_HTTPD_MODULES-NOTFOUND
-- Can't find Apache2.2: APACHE2_2_HTTPD_INCLUDE_DIR-NOTFOUND

In this case, you should specify the paths of your custom Apache in the mod_lsapi-1.1/cmake/FindApacheForBuild.cmake file. According to macros above, add the paths to the directory where httpd binary is located, to the dir where apache modules are located and to the dir where include files are located. It may look a such way:

FIND_PATH (APACHE2_2_HTTPD_INCLUDE_DIR 
   NAMES 
      httpd.h
   PATHS
      ${APACHE2_2_INCLUDES_DIR}
      /usr/local/include/httpd
      /usr/include/httpd
      /usr/include/apache
      /usr/local/apache/include/       # added line
)

Starting from mod_lsapi-1.1-57, you can use macros for custom paths to the Apache/APR includes/binaries.

A custom Apache location can be defined via the CUSTOM_APACHE_ROOT variable. This implies the following structure under the ${CUSTOM_APACHE_ROOT}:

If the real structure of Apache root differs from the implied one, it's possible to define a custom location for every single component.

7. To customize the switch_mod_lsapi script add into mod_lsapi.spec custom script and its configuration file. Configuration file name should be config.ini file. Script file name can be arbitrary, because its name should be mentioned in config.ini file. Both of them should be copied into /usr/share/lve/modlscapi/custom in the %install section of the mod_lsapi.spec file. For example, if script file name is custom.sh, you should add the following lines into %install section of mod_lsapi.spec file:

install -D -m 644 config.ini $RPM_BUILD_ROOT%{g_path}/custom/config.ini
install -D -m 755 custom.sh $RPM_BUILD_ROOT%{g_path}/custom/custom.sh

Also you should add mentions of both files in the %files section of mod_lsapi.spec:

/usr/share/lve/modlscapi/custom/config.ini 
/usr/share/lve/modlscapi/custom/custom.sh

The requirements to the config.ini file and script file are described in the following section

8. Rebuild the package again, if you set everything correctly, there shouldn't be any problems.

Use this command to rebuild a package

9. Install the module, check that it is successfully loaded into Apache.

# How to integrate switch_mod_lsapi script with custom panels

To be able to use switch_mod_lsapi you have to do the steps mentioned below. We assume that if the /usr/share/lve/modlscapi/custom/config.ini file exists, then it is a Custom Panel.

1. Create config.ini file /usr/share/lve/mod_lsapi/custom/ directory. As an example we have created a config.ini.example file. All directives that are shown in example file needed in GLOBAL section:

[GLOBAL]
VERSION = 1.0.0
APACHECTL_BIN_LOCATION = /usr/sbin/apachectl
DOC_URL = http://docs.cloudlinux.com/index.html?apache_mod_lsapi.html
EXECUTABLE_BIN = custom.sh
PANEL_NAME=CustomPanel

2. Create executable script in /usr/share/lve/mod_lsapi/custom/executable.sh. Name it as you want and specify in the config.ini file. Set EXECUTABLE_BIN=executable.sh in the GLOBAL section of the ini file. Script must be located in the /usr/share/lve/mod_lsapi/custom/ directory.

3. Custom script options correspond to the options of the switch_mod_lsapi. For more information use the this document.

# What you need to write in your custom executable.sh file.

You have to implement options in your script. Switch_mod_lsapi will call executable with corresponding arguments. Script must return exit code 0 as success or some integer in case of fail.

Supported options are:

switch_mod_lsapi --setup
/usr/share/lve/modlscapi/custom/executable.sh --setup

switch_mod_lsapi --uninstall
/usr/share/lve/modlscapi/custom/executable.sh --uninstall

switch_mod_lsapi --enable-domain test.com
/usr/share/lve/modlscapi/custom/executable.sh --enable-domain test.com

Script executable.sh executed with two arguments --disable-domain and domain name:

switch_mod_lsapi --disable-domain test.com
/usr/share/lve/modlscapi/custom/executable.sh --disable-domain test.com

Script executable.sh executed with --enable-global argument:

switch_mod_lsapi --enable-global
/usr/share/lve/modlscapi/custom/executable.sh --enable-global

Script executable.sh executed with --disable-global argument:

switch_mod_lsapi --disable-global
/usr/share/lve/modlscapi/custom/executable.sh --disable-global

Сustom script executed with --verbose and --force arguments if switch_mod_lsapi are called with.

Not supported options are:

--setup-light
--build-native-lsphp
--build-native-lsphp-cust
--check-php
--stat

mod_lsapi uses one predefined lsphp handler application/x-httpd-lsphp, which relates to /usr/local/bin/lsphp binary. All other lsphp handlers should be added by custom script into /etc/container/php.handler file.

An example of /etc/container/php.handler file

application/x-lsphp52 /opt/alt/php52/usr/bin/lsphp
application/x-lsphp53 /opt/alt/php53/usr/bin/lsphp
application/x-lsphp54 /opt/alt/php54/usr/bin/lsphp
application/x-lsphp55 /opt/alt/php55/usr/bin/lsphp
application/x-lsphp56 /opt/alt/php56/usr/bin/lsphp
application/x-lsphp70 /opt/alt/php70/usr/bin/lsphp
application/x-lsphp71 /opt/alt/php71/usr/bin/lsphp
application/x-lsphp72 /opt/alt/php72/usr/bin/lsphp

application/x-httpd-php81-lsphp /usr/local/apps/php81/bin/lsphp

An example of executable.sh script:

#!/bin/bash
 
CMD=$1
 
if [ "$CMD" == "--setup" ]; then
    	echo "application/x-httpd-php81-lsphp /usr/local/apps/php81/bin/lsphp"     >>/etc/container/php.handler
fi 
 
if [ "$CMD" == "--uninstall" ]; then
    sed -i "#application/x-httpd-php81-lsphp#d" /etc/container/php.handler
fi
 
if [ "$CMD" == "--enable-domain" ]; then
    http_root=`/opt/ExamplePanel/domain2root $2`
    if [ -e $http_root/.htaccess ]; then
        sed -i '/lsphp/d' $http_root/.htaccess
    fi
fi 
 
if [ "$CMD" == "--disable-domain" ]; then
    http_root=`/opt/ExamplePanel/domain2root $2`
    if [ -e $http_root/.htaccess ]; then
        sed -i '/lsphp/d' $http_root/.htaccess
    fi
fi 
 
# if command not supported
if [ "$CMD" == "--enable-global" ]; then
    echo "LSAPI enable global command is not supported by ExamplePanel"; exit 1
fi 

# MySQL Governor

:::Warning MySQL Governor is not available in CloudLinux OS Admin edition. :::

1. Install MySQL Governor

yum install governor-mysql

2. MySQL Governor supports only cl-MariaDB** or cl-MySQL** packages. Follow the documentation to figure out all supported versions.

/usr/share/lve/dbgovernor/mysqlgovernor.py --mysql-version=mysqlXX

3. Backup your databases.

4. Run the cl-MySQL/cl-MariaDB installation.

/usr/share/lve/dbgovernor/mysqlgovernor.py --install

5. After installation, check that the database server is working properly. If you have any problems, contact support

6. Configure user mapping to the database. The mapping format is described in the following section. The control panel should automatically generate such mapping and write it to /etc/container/dbuser-map. Usually, it is enough to write a hook when adding, deleting or renaming a database for a user. The control panel should implement such a mechanism for MySQL Governor to operate properly. MySQL Governor automatically applies changes from the dbuser-map file every five minutes.

7. MySQL Governor configuration can be found in the following section.

8. MySQL Governor CLI tools description can be found in the following section

9. Having configured the mapping use dbtop to see the current user load on the database (you'd need to make some database queries).

dbtop

10. If the load appears in the dbtop output, then you have successfully configured MySQL Governor.

# CloudLinux OS LVE and CageFS patches

If you use custom Apache, you need to apply patches so that the processes launched by the Apache are working with LVE and CageFS properly. Cloudlinux OS provides patches for the following packages:

• suphp
• suexec
• php-fpm
• apr library
• Apache mpm itk

First of all, download patches using the following link https://repo.cloudlinux.com/cloudlinux/sources/da/cl-apache-patches.tar.gz

To apply the patch, follow these steps.

1. Untar archive which you were going to patch and go to the created directory with sources.

2. Transfer the patch to the directory with sources.

3. Use the patch utility inside the directory with sources

4. Recompile your project.

Here's an example with apr library:

# ls 
apr-1.4.8.tar.bz2  apr-2.4-httpd.2.patch
# tar xvjf apr-1.4.8.tar.bz2
# cp apr-2.4-httpd.2.patch ./apr-1.4.8
# cd ./apr-1.4.8
# patch -p3 < apr-2.4-httpd.2.patch 
patching file include/apr_thread_proc.h
Hunk #1 succeeded at 204 (offset -2 lines).
patching file threadproc/unix/proc.c

Patch applied Successfully, recompile apr. 

If you build custom RPM packages, it might make sense to apply patches using spec file.

In this case, add to the Source section (example for apr):

Source0: apr-1.4.8.tar.bz2
Patch100: apr-2.4-httpd.2.patch

Then apply this patch in the %setup section:

%prep
%setup -q
%patch100 -p3 -b .lve

If no errors appear during the build, the patch has been successfully applied.

Instead of patching the apr library, you can build your custom Apache with one of the libapr-devel packages provided by CloudLinux OS. Every version of CloudLinux OS provides two versions of libapr - the system library as a package named apr and the alternative library as a package named alt-apr. Their versions are listed in the following table:

|-|-|-| |CloudLinux OS version |apr version |alt-apr version | |6 |1.3.9 |1.7.4 | |7 |1.4.8 |1.7.4 | |8 |1.6.3 |1.7.4 | |9 |1.7.0 |1.7.4 |

If your custom Apache is already linked with a non-patched apr library with the major version equal to one provided by apr or alt-apr package, you can just change the library symlink. For example, if your Apache binary /usr/local/apps/apache2/bin/httpd is linked with /usr/local/apps/apache2/lib/libapr-1.so.0 library which, in turn, is a link to libapr library with version 1.7.4, you can install the alt-apr package and change the symlink - it should link to the patched library provided by alt-apr package instead of the original one.

[root@cloudlinux ~]# ldd /usr/local/apps/apache2/bin/httpd | grep libapr-1.so.0
        libapr-1.so.0 => /usr/local/apps/apache2/lib/libapr-1.so.0 (0x00007f191e35c000)
[root@cloudlinux ~]#
[root@cloudlinux ~]# ls -l /usr/local/apps/apache2/lib/libapr-1.so.0
lrwxrwxrwx 1 root root 17 Feb 10  2023 /usr/local/apps/apache2/lib/libapr-1.so.0 -> libapr-1.so.0.7.4
[root@cloudlinux ~]#
[root@cloudlinux ~]# rpm -ql alt-apr | grep libapr-1.so.0.7.4
/opt/alt/alt-apr17/lib64/libapr-1.so.0.7.4
[root@cloudlinux ~]#
[root@cloudlinux ~]# ln -sf /opt/alt/alt-apr17/lib64/libapr-1.so.0.7.4 libapr-1.so.0

# Hardened PHP

All versions of alt-php should work properly in any control panel.

To install all PHP versions, run:

yum groupinstall alt-php

To install a particular version of PHP, run:

yum groupinstall alt-php{some_version}

For details, see PHP Selector Installation and Update

# CloudLinux OS kernel set-up

# /proc filesystem access control

• Integrating control panel with CloudLinux OS

CloudLinux OS kernel allows the server administrator to control the user access to the /proc filesystem with the special parameters. Unprivileged system users will not be able to see the processes of other system users, they can only see their processes. See details here.

If the fs.proc_can_see_other_uid parameter is not defined in sysctl config during the CageFS package installation, it is set to 0: fs.proc_can_see_other_uid=0.

You can also set this parameter in the /etc/sysctl.conf file and then run the sysctl -p command.

hidepid parameter is set based on the fs.proc_can_see_other_uid parameter value automatically during the lve_namespaces service starting (i.e. during lve-utils package installation and server booting).

# Integrating control panel with CloudLinux OS

Run the following command to add administrators into proc_super_gid group (if the panel administrator is not a root user or there are several administrators):

/usr/share/cloudlinux/hooks/post_modify_admin.py create --username <admin_name>

This command also adds a specified administrator to the clsudoers group, so the LVE Manager admin's plugin works properly.

This command is a hook that the control panel should call after adding the admin user to the system.

If the admins list script exists when installing lve-utils package then the existing administrators will be processed automatically. It the script does not exist, you should run the above mentioned command for each administrator.

# Symlink protection

• How to enable symlink protection

CloudLinux OS kernels have symlink attacks protection. When this protection is enabled, the system does not allow users to create symlinks (and hard links) to non-existent/not their own files.

Here you can find all protection mechanism parameters.

See also: the special module to protect web-server.

You can run cldiag utility with the --check-symlinkowngid parameter to check whether your web-server protection mechanisms are correctly set-up.

cldiag --symlinksifowner
Check fs.enforce_symlinksifowner is correctly enabled in sysctl conf:
	OK: fs.enforce_symlinksifowner = 1

There are 0 errors found.  
............ 

cldiag --check-symlinkowngid
Check fs.symlinkown_gid:
	OK: Web-server user is protected by Symlink Owner Match Protection

There are 0 errors found.

• Symlink traversal protection is disabled by default.
• Symlink owner match protection is enabled by default.

# How to enable symlink protection

Set the symlinkown_gid parameter to the value of the GID group of the web-server process.

Before enabling symlink protection, ensure it does not break the control panel functionality.

For example, if an unprivileged user is allowed to create symlinks or hard links to files or directories he doesn’t own then when enabling symlink traversal protection he will not be able to do so.

To allow an unprivileged user to create such symlink or hard link, a file/directory which symlink is referred to should have linksafe group.

# File system quotas

Some of CloudLinux OS utilities (cagefsctl, selectorctl, cloudlinux-selector, etc) writes files to the user’s home directory on behalf of this user.

So they are subject to the disk quotas if any are set for the user. The utilities can override these quotas to avoid failures. See File system quotas for kernel parameters controlling these permissions. This parameter is used only for XFS file system.

# How to integrate X-Ray and AccelerateWP with a control panel

In order to integrate X-Ray or AccelerateWP into your panel, you should follow two simple steps.

1. Extend the output of your existing domains integration script as follows:

1.1. Implement an optional flag --with-php, upon which all domains with their PHP configuration will be returned. Thus, if your integration config file /opt/cpvendor/etc/integration.ini has

domains = /opt/cpvendor/bin/vendor_integration_script domains

in [integration_scripts] section, X-Ray and AccelerateWP would call this script as

/opt/cpvendor/bin/vendor_integration_script domains --with-php

1.2. In order to enable X-Ray or AccelerateWP support, the script output should return a representation of all domains on the server: a key-value object, where a key is a domain (or subdomain) and a value is a key-value object contains the owner name (UNIX users) and PHP interpreter configuration. If you are extending the existing domains integration script, you just include the PHP configuration.

PHP interpreter configuration for each domain is to be placed in the nested key-value object of the following format:

“php”: {
   “version”: str
   “ini_path”: str
   “is_native”: bool
   “fpm”: str
}

Output example

{
  "data": {
    "domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/",
      "is_main": true,
      "php": {
        "version": "56",
        "ini_path": "/opt/alt/php56/link/conf",
        "is_native": true
       }
    },
    "subdomain.domain.com": {
      "owner": "username",
      "document_root": "/home/username/public_html/subdomain/",
      "is_main": false,
      "php": {
        "version": "72",
        "ini_path": "/opt/alt/php72/link/conf",
        "fpm": "alt-php72-fpm"
      }
    }
  },
  "metadata": {
    "result": "ok"
  }
}

Data description

Key	Required	Description
version	True	The version of php that is selected by the end user or administrator in the panel for this domain. Should be presented in the format XY where XY is the exact version of PHP. List of supported versions is presented below.
ini_path	True	Path where PHP expects additional .ini files to scan, usually compiled path into binary PHP.Example where to get this value:
is_native	False	If your panel has integrated CloudLinux OS PHP Selector, then there is the list of native PHP binaries in the native.conf (see details here). The is_native value must be set to true if the binary of the above version of PHP is specified in the native.conf, false or omitted otherwise. This way the X-Ray will determine if CloudLinux OS PHP Selector is enabled for a given domain
fpm	False	The name of the FPM service that uses specified domain, optional value. To make the X-Ray work on domains using FPM, you need to restart FPM service. In X-Ray added automatic restart of the FPM service if the tracing task is created for the domain that uses the FPM therefore we need this field, otherwise it can be omitted
handler	*True	The name of used handler. Can be one of the follwing: fpm, cgi, fcgi, lsapi. Use lsapi in case if you use litespeed web server.

• handler is only required for Accelerate WP integration.

2. After finishing the X-Ray integration script, enable X-Ray support through panel_info integration script: add xray component into supported_cl_features object.

Output example

{
	"data": {
		"name": "SomeCoolWebPanel",
		"version": "1.0.1",
		"user_login_url": "https://{domain}:1111/",
		"supported_cl_features": {
			"php_selector": true,
			"ruby_selector": true,
			"python_selector": true,
			"nodejs_selector": false,
			"mod_lsapi": true,
			"mysql_governor": true,
			"cagefs": true,
			"reseller_limits": true,
			"xray": true
		}
	},
	"metadata": {
		"result": "ok"
	}
}