Documentation
Submit Request Try Free
Documentation: CloudLinux OS+ (BETA)

CloudLinux OS+ (BETA)

X-Ray beta

Description

Note

Please note that X-Ray is a new experimental tool. It is in beta testing now. If you’d like to be a beta tester, please fill out this form. You will be able to use the X-Ray tool after receiving the CLN manager approval.

Warning!

X-Ray beta can be started only in NON-RESELLER CLN accounts.

Warning!

X-Ray is available for cPanel, Plesk, and DirectAdmin. Non-panel installations is not planned.

X-Ray is a tool developed for website performance monitoring and performance issues detection.

X-Ray can gather and visualize information about top N slowest system functions, external requests, software modules and database queries of the client’s website.

First release of X-Ray is offered for cPanel administrators and support to find the cause of website performance issues.

X-Ray can monitor websites that were developed on cPanel hosts and use PHP (see PHP version list) or WordPress.

Installation

  1. Make sure you are an approved X-Ray beta-tester (only non-reseller accounts apply)

  2. Make sure you have installed LVE Manager version 6.2 or later. You can install it with the following command:

    # yum install lvemanager --enablerepo=cloudlinux-updates-testing

  3. Run the following command:

    # rhn_check

    If the rhn_check command is not found, run the following command:

    # yum install rhn-check rhn-setup --enablerepo=cloudlinux-updates-testing

  4. Then install the alt-php-xray package

    • Via user interface

      • Go to the X-Ray tab.
      • Click Install to start installation.

    • Via SSH by running the following command:

    # yum install lvemanager alt-php-xray --enablerepo=cloudlinux-updates-testing

  5. After installation, use the Start tracing button to create your first tracing task for a slow site.

How to manage X-Ray

Create a new tracing task

  1. Go to the X-Ray tab
  2. Click the Start tracing button to create a new task
  3. In the opened popup specify a website URL to trace
  4. Click the Run button
  5. Tracing will run in the default mode. In the default mode X-Ray traces the first 20 requests for a specified URL

  • URL should be a valid URL of the domain which exists on the current hosting server. The URL field supports wildcard matching. To learn more about wildcard matching, click How to use special characters.

  • Advanced settings allow you to set an IP address and tracing options: by time or by number of queries.

Advanced settings

  • Client’s IP: it is an IPv4 address of a machine to trace. For example, if you have a production website that processes requests from different IP addresses and you do not want to add these requests to the tracing task. So, you can set a specific IP address and X-Ray will analyze requests only from this specific IP address. Record for
  • Time period: how much time X-Ray collect the requests (2 days max)
  • Requests: the amount of requests that X-Ray will collect

After creating, the task appears in the list of tracing tasks.

View tracing tasks list

Tracing status

A tracing task can have the following statuses:

  • Running – tracing is in progress
  • Stopped – tracing was stopped by administrator
  • On hold – the same URL already exists in the lists. Task processing will not start automatically. Administrator should start it manually.
  • Completed – period of time is finished or number of requests is reached.

Stop tracing task

Click to stop the tracing task.

The tracing task status will be changed to Stopped. Data will not be collected anymore but you can see already collected information or continue tracing later by clicking .

Delete tracing task

Click to delete the tracing task.

Warning!

When you have deleted a tracing task, all collected data will be unavailable.

View collected requests for tracing task

Warning!

Collected requests are available in the UI for two weeks.

Click to open a list of collected requests.

Tracing tasks

The slowest request is highlighted.

  • Collected requests displays how many requests were collected according to tasks requirements.
  • Pending requests displays how many of collected requests are not visible in the table yet.

X-Ray collects the following data for each request:

  • Top issues – the slowest items of a request
  • Software modules/plugins by execution time (only for WordPress plugins)
  • Database queries by execution time
  • External requests by execution time
  • Other system functions by execution time

Software modules/plugins

The Software modules/plugins section displays the following data:

  • Software type – a type a module/plugin. For now, X-Ray can analyze only WordPress software
  • Software module – a name of the WordPress plugin
  • Duration – plugin execution time
  • Duration (%) – plugin execution time as a percentage of the total duration of the request

Database queries

The Database queries section displays the following data:

  • Query – the executed SQL-query
  • File – the file and the line of the executed query and backtrace
  • Software module – a WordPress plugin name from which the request was completed. If the request does not belong to any of the WordPress plugin, the name of the function that executed the given request is displayed
  • Calls – the number of identical SQL queries
  • Duration – execution time as a percentage of the total duration of a request and the function processing time (in brackets)

External requests

The External requests section displays the following data:

  • URL – the URL of the executed request
  • File – the file and the line of the executed request and backtrace
  • Duration – execution time as a percentage of the total duration of a request and the function processing time (in brackets)

System functions

The System functions section displays the following data:

  • Function – the executed function
  • File – the file and the line of the executed request
  • Duration – execution time as a percentage of the total duration of a request and the function processing time (in brackets)

X-Ray client

X-Ray client is a PHP extension named xray.so. It analyzes the processing time of the entire request and its parts and then sends the data to the X-Ray agent.

List of supported PHP versions

The list of currently supported PHP versions:
ALT PHP: EA PHP: Plesk PHP
  • alt-php54
  • alt-php55
  • alt-php56
  • alt-php70
  • alt-php71
  • alt-php72
  • alt-php73
  • alt-php74
  • ea-php54
  • ea-php55
  • ea-php56
  • ea-php70
  • ea-php71
  • ea-php72
  • ea-php73
  • ea-php74
  • php54
  • php55
  • php56
  • php70
  • php71
  • php72
  • php73
  • php74

Functions that X-Ray client can hook

Database queries

  • Functions from the MySQL extension:
    • mysql_query
    • mysql_db_query
    • mysql_unbuffered_query
  • Functions from the MySQLi extension:
    • mysqli_query
    • mysqli::query
    • mysqli_multi_query
    • mysqli::multi_query
    • mysqli_real_query
    • mysqli::real_query
  • Functions from the PDO extension:
    • PDO::exec
    • PDO::query
    • PDOStatement::execute

External requests

System PHP functions

It may be any PHP system function which can be related to a PHP engine or other PHP extension, for example fopen() or json_encode(). A list of these functions can be found here.

Configuration Options

xray.enabled

Syntax: xray.enabled=On/Off

Default: On

Changeable: PHP_INI_SYSTEM

Description: Enable or disable X-Ray extension from php.ini

xray.database_queries

Syntax: xray.database_queries=[number]

Default: 20

Changeable: PHP_INI_SYSTEM

Description: The number of the slowest SQL queries which will be sent to the X-Ray agent. The min value is 0 and the max value is 100. If the variable value is more, the default value will be used.

xray.external_requests

Syntax: xray.external_requests=[number]

Default: 20

Changeable: PHP_INI_SYSTEM

Description: The number of the slowest external requests (the curl_exec function) which will be sent to the X-Ray agent. The min value is 0 and the max value is 100. If the variable value is more, the default value will be used.

xray.system_functions

Syntax: xray.system_functions=[number]

Default: 20

Changeable: PHP_INI_SYSTEM

Description: The number of the slowest system functions which will be sent to the X-Ray agent. The min value is 0 and the max value is 100. If the variable value is more, the default value will be used.

xray.backtrace_depth

Syntax: xray.backtrace_depth=[number]

Default: 10

Changeable: PHP_INI_SYSTEM

Description: The backtrace depth to the main() function which will be sent to the X-Ray agent. The min value is 0 and the max value is 20. If the variable value is more, the default value will be used.

xray.processor

Syntax: xray.processor=[processor_name]

Default: xray

Changeable: PHP_INI_SYSTEM

Description: Tells the X-Ray client which processor to use. The new processors may be added in the future. The default processor is xray which means to send data to the X-Ray agent.

xray.tasks

Syntax: xray.tasks=host:uri:ip:id

Default: no value

Changeable: PHP_INI_SYSTEM

Description: The current tracing tasks for the given PHP request. This directive is added automatically by the X-Ray manager when creating a task. It is not allowed to edit manually, as X-Ray may stop working.

xray.to_file

Syntax: xray.to_file=On/Off

Default: Off

Changeable: PHP_INI_SYSTEM

Description: Only for debug purposes. Writes to a file data which is sent to the processor.

xray.debug

Syntax: xray.debug=On/Off

Default: Off

Changeable: PHP_INI_SYSTEM

Description: Only for debug purposes. Enables debug output during request processing. In the On mode can slow down the domain.

xray.debug_file

Syntax: xray.debug_file=[path_to_file]

Default: /tmp/xray-debug.log

Changeable: PHP_INI_SYSTEM

Description: Only for debug purposes. Specifies a file for logging debug information.

X-Ray agent

This is a service that receives data from the X-Ray client and sends it to the remote storage.

Managing X-Ray service

The X-Ray agent is managed by the service utility.

  • To start the X-Ray agent, run the following command:

    # service xray-agent start

  • To stop the X-Ray agent, run the following command:

    # service xray-agent stop

  • To restart the X-Ray agent, run the following command:

    # service xray-agent restart

FAQ

Does X-Ray affect website performance?

X-Ray affects website performance. Our tests show 5-10 % overhead from a website loading time with X-Ray tracing enabled. X-Ray allows you to find website performance issues and should not be enabled permanently. If your website is very slow, you can enable X-Ray to find the cause and then disable it.

What should I do if I see the warning "Task is duplicated by URL"?

This warning means that you already have a task to trace this URL in the list of your tracing tasks. If you see this warning, a new task can be created only with the On hold status and you will be able to run it only when the previous task with the same URL will be completed.

Note that the URL field supports wildcard matching and you can have a case when X-Ray is tracing the URL=domain.com/* and you are trying to create a new task with URL=domain.com/xray.php. In this case, you will see that warning because the * URLs array includes xray.php.

I started a tracing task and made requests to URL but did not see any results in the UI. What should I do?

  1. Check that xray extension is enabled for the domain. To do so, go to the phpinfo() page and make a request. In the phpinfo output try to find the following section:

If you cannot see that section, try to restart PHP processes for that user (the simplest way is to restart Apache) and check that you can see the xray extension.

  1. If you can see the xray extension in the phpinfo, check that X-Ray agent service is running with the service xray-agent status command. If it is not running, start it with the service xray-agent start command.

  2. X-Ray may not send data if a site uses a caching plugin, as the caching plugin is outputting HTML, thus there are no PHP scripts to examine. We encountered such issues with sites that use LSCache and WP Super Cache plugins. Check that your site does not use caching plugins. If so, disable it while tracing a site to get information from X-Ray.

  3. If you set a client’s IP when creating the tracing task, check that your requests come to the server with this IP via phpinfo (since there may be NAT between your local machine and the server).

  4. If, after checking the previous items, the issue persists, contact our support team.

What to do if X-Ray is not found in the phpinfo() page?

If you managed to create a tracing task, this means that the xray.ini file was created in a system. Therefore, there may be two reasons why it did not appear in the phpinfo page of the domain.

  1. PHP process wasn't reloaded after adding the xray.ini. To solve this, you should restart the Apache or fpm service for the domain on which the tracing was started. At the moment, this is done automatically by the X-Ray manager after creating the task.

  2. Your domain uses a PHP version different from the one which was detected by the X-Ray manager. To solve this, check the scan dir for additional ini files for your domain.

    Then check the ini_location that was passed to the X-Ray manager by running the following command:

    # cat /usr/share/alt-php-xray/manager.log | grep ini_location

    Find your tracing task in the output and check that the xray.ini exists in this directory, also check that the ini path is the same in the phpinfo page output and in the ini_location directive for your tracing task. If they are the same, you should reload your PHP. If they are different that means that the X-Ray manager could not correctly determine the PHP version your domain uses. In this case, contact our support team at https://cloudlinux.zendesk.com/hc/requests/new.

I use LiteSpeed, X-Ray is enabled and it is shown in the phpinfo() page but does not collect data when sending requests to a site. What to do?

Check for the CacheLookup on option in the htaccess file for your domain. If the option is there, LiteSpeed processes requests bypassing the PHP X-Ray extension. In this case, to get tracing information, you should remove the CacheLookup on option.

What is the proper format for the URL?

All of the examples below are correct:

  • http://domain.com
  • http://domain.com/
  • https://domain.com
  • https://domain.com/

You can use any of them with a prefix www. and it is also correct.

Centralized Monitoring beta

Description

Centralized Monitoring is a tool that allows hosting administrators to monitor load for all their servers and users.

Centralized Monitoring allows you to:

  • View system metrics for all clients’ end servers
  • View the LVE statistics per user for all clients’ end servers

Installation

Note

Make sure that cm.cloudlinux.com is available on your end server.

Warning!

Centralized Monitoring beta can be started only in NON-RESELLER CLN accounts.

Note

Skip the first and second steps and start from the third step if you are already an X-Ray beta tester.

  1. Send a request to become the Centralized Monitoring beta tester here: https://cln.cloudlinux.com/console/dashboard/products
  2. Wait for the approval from the manager.
  3. Register CloudLinux+ servers or use the existing servers.
  4. Log in to the https://cm.cloudlinux.com/ using CLN credentials (if you are already logged in via CLN, authorization via CM is not necessary, it uses SSO).
  5. You can find the list of servers in the Centralized Monitoring UI: https://cm.cloudlinux.com/#/servers or you can find the list of servers in your CLN account: https://cln.cloudlinux.com/console/cloudlinux/centralized-monitoring. Servers will have the N/A status.
  6. Update/install the rhn-client-tools package version 2.0.2-31.cl7 for CloudLinux 7, version 1.1.15-3.el6 for CloudLinux 6, version 2.8.16-14.module_el8.1.0+6074+9dc6073e.cloudlinux.2 for CloudLinux 8, and rhn-check and rhn-setup packages:
yum update/install rhn-client-tools rhn-check rhn-setup
rhn_check
  1. Update/install the lve-utils package version 4.2.11-1 or higher:
yum update/install lve-utils --enablerepo=cloudlinux-updates-testing

  1. Beta

    Set up your server to send statistics. Run this command

    /usr/share/cloudlinux/cl_plus/manage_clplus enable

    to install the cl-end-server-tools package and start service collecting and sending statistics to the central database. Then, check that the cl-end-server-tools package is installed successfully:

    rpm -q cl-end-server-tools

    Production (not available yet)

    Within an hour the cl-end-server-tools package will be installed on your server and the collecting and sending statistics daemon will be turned on.

  2. Check the status of service by running this command:

service cl_plus_sender status
  1. Check that all collectors are initiated:
cat /var/log/clplus_sender.log
  1. Wait some minutes and check the server statistics in the Centralized Monitoring UI | servers list: https://cm.cloudlinux.com/#/servers for those servers where the cl_plus_sender service works.
  2. List of users https://cm.cloudlinux.com/#/users contains users from all servers where the cl_plus_sender service works and have had any load during the last 30 days.

Centralized Monitoring user interface

You can access Centralized Monitoring in your CLN account. Click C-Monitoring in the left menu.

Servers

This page contains the list of all clients’ end servers. The server appears in the list after finishing Installation. By default, there is a descending sort by CPU usage.

The following values are available for each server:

  • Load Avg 15m – average system load for the last 15 min
  • CPU Usage – CPU usage for the last 15 min (the number of cores can be found in the hint)
  • Memory Usage – free available memory, the second value is the total memory for the last 15 min
  • IO read/write – disk read bytes/disk written bytes for the last 15 min

Note

The values are calculated using a 15 min time period but the metric state is updated automatically every minute by default or you can choose from one of the predefined periods.

  • Idle state – there were no statistics for the server for the last minute.
  • N/A state – there were no statistics for the server for the last 30 days. This can happen if a new server was added but statistics sending was not configured.

There is no pagination on the All servers page and all columns can be sorted by absolute value. Use the search tool to operate with the data.

Servers details

To get the detailed statistics for the server via charts, click a desired server line in the table. All charts are auto-refreshed and there is an ability to select the period for metrics data to be updated for the chart.

Charts

Disk space usage

Open file descriptor/Context switches

System load 1m , 5m, 15m

CPU usage (total, system, user, iowait, steal)

Network traffic usage

Disk space usage

Memory usage (total, used, available)

Time spent doing I/Os

Disk IOps Completed

Disk read/write data

Disk read/write time

Apache connections (number)/Number of requests per minute/Max connections

Note

In the current version, we collect these metrics for the cPanel end servers only. We are planning to add other panels support soon.

Note

In the current version, we collect these metrics only for Apache (NOT for LiteSpeed, Nginx, etc.). The charts will be empty for LiteSpeed, Nginx, etc..

MySQL queries

The most loaded server users for the last minute

We calculate the user’s load by LVE statistics that we collect on the end server. The idle state for the user means that the LVE statistics were not collected for the last minute for some reason.

In each cell there are current usage/limit values for the basic LVE limits:

  • CPU Usage
  • Entry Processes
  • Physical Memory Usage
  • IOPS
  • IO Usage
  • Number of Processes
  • MySQL CPU
  • MySQL IO

In the hint, there is a number of faults for each limit. The values in the columns are underlined (it is red if load-to-limit ratio >=90% and it is yellow if load-to-limit ratio >= 50%). For the current implementation, the only sort by the load-to-limit ratio is available. By default, there is a descending sort by the CPU usage column.

When sorting by a column, the lines with the load-to-limit ratio >=90% for this column will have the red background color, and lines with the load-to-limit ratio >=50% for this column will have the yellow background color.

Note

The users with unlimited resources (∞) will be at the bottom of the table.

Users

This page contains all users for the all server of the client and their LVE statistics for the last minute. You can select the number of users on this page and search by user’s data.

The description of this page is the same as The most loaded server users for the last minute of the top 5 loaded users.

FAQ

How can I disable collecting and sending statistics on a client’s server?

Run this command:

/usr/share/cloudlinux/cl_plus/manage_clplus disable

Where can I view all my servers load?

You can find all your servers load in your CM personal account here: https://cm.cloudlinux.com/#/servers or in your CLN personal account here: https://cln.cloudlinux.com/console/cloudlinux/centralized-monitoring.

Where can I view all my users load?

You can find all your users load in your CM personal account here: https://cm.cloudlinux.com/#/users or in your CLN personal account here: https://cln.cloudlinux.com/console/cloudlinux/centralized-monitoring

Where can I view a server load details for a period?

Click the desired server in the server list in the UI.

Where can I view a user load details for the period?

Click the desired user in the user list in the UI.

How long are statistics stored in the central database?

30 days.

How can I change the charts period on the details page?

Choose the desired period in the upper right corner or select it directly on the chart.

I don’t understand how to read the user load chart.

The user load chart contains three lines:

  • limit
  • current load
  • count of faults

Limit and current load are drawing regarding the left vertical axis, the count of faults is drawing regarding the right vertical axis. You can focus on a particular line by clicking a required legend.

Do I need to submit a request to be a Centralized Monitoring beta tester if I'm already an X-Ray beta tester?

No, you don't need, just follow the Centralized Monitoring instruction starting from the third step.

Troubleshooting

I can't see a server statistics

  1. Check that your server is registered by key or by IP license of the CloudLinux+ account, i.e., it should be seen in the list of servers in your CLN account here: https://cln.cloudlinux.com/console/auth/login
  2. Check that the following required packages are installed on the end server:
  • cl-end-server-tools >= 1.0.1-1
  • cl-node-exporter >= 1.1.0-1
  • rhn-client-tools
    • CloudLinux 6 >= 1.1.15-3.el6.cloudlinux.26
    • CloudLinux 7 >= 2.0.2-31.el7.clouldinux
    • CloudLinux 8 >= 2.8.16-14.module_el8.1.0+6074+9dc6073e.cloudlinux.2
  • lve-stats >= 3.0.6-1
  • lve-utils >= 4.2.11-1
  • alt-python27-cllib >= 2.1.8-1
  1. Check that service collecting and sending statistics is running:
service cl_plus_sender status
  1. Check that log of the cl_plus_sender service doesn't contain errors:
/var/log/clplus_sender.log

Where can I view the events log on the client's server?

You can view the events log on the client's server here:

/var/log/clplus_sender.log

Can I get monitoring metrics from LiteSpeed, Nginx or other (Not Apache) web server?

No, you can not. We will announce in our blog when we implement this.

Edit this page
We are Cloudlinux
2020. CloudLinux Inc
How to
Getting started
Report an error in documentation
Blog