Doris Manager Unified Deployment and Multi-Cluster Management

With business expansion and the diversification of product forms, there is a need to deploy multiple sets of offline and real-time data warehouse clusters. While the previous Ansible Playbook method for deploying Doris FE/BE was basically sufficient, it fell short for subsequent Doris version upgrades, scaling, visual management, and onboarding junior operations personnel. Therefore, this document was written to facilitate quick future reference.

1. Overview

1.1 Introduction to Doris Manager

Doris Manager is a graphical operation and maintenance tool launched by the Apache Doris community for unified deployment, management, and monitoring of Doris clusters. It simplifies the entire process from operating system initialization and component deployment to daily operation and maintenance management, making it particularly suitable for rapidly setting up and maintaining large-scale Doris clusters in production environments.

From the current perspective, the official open-source repository is no longer maintained. As SelectDB is a commercial company, for its own development, it has evolved Doris Manager into a free, closed-source operation and maintenance tool, though it can still be downloaded and used.

1.2 Main Features

• Cluster Deployment: Automated deployment of FE, BE, Broker, and other components.
• Node Management: Unified management of all nodes, supporting scaling out and scaling in.
• Configuration Management: Centralized configuration management and parameter adjustment.
• Monitoring and Alerting: Cluster status monitoring and alert notifications.
• Log Viewing: Centralized log viewing and analysis.

2. Prerequisites

2.1 Environment Planning

Before deployment, please confirm the following information:

2.2 Software Preparation

Official download address: https://selectdb.com/download/enterprise#manager

Download to the company software repository in advance (needs to be downloaded from the selectdb official website):

Repository Address: https://repo.test.com/bigdata/

Download Commands:

1
2
3
4
5
6
7

# Create download directory
mkdir -p /opt/downloads/doris_manager /opt/downloads/doris
cd /opt/downloads/doris_manager

# Download installation packages (from company repository)
wget https://repo.test.com/bigdata/doris-manager-24.1.5-x64-bin.tar.gz
wget https://repo.test.com/bigdata/apache-doris-2.1.8.1-bin-x64.tar.gz

2.3 System Requirements

3. System Initialization and Optimization

Before deploying Doris Manager and the Doris cluster, system initialization configuration is required for all nodes.

3.1 Data Disk Initialization

Important: Production environments must use an independent data disk; do not share it with the system disk.

Manual initialization (single node)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# 1. View data disk devices
lsblk
# Assume the data disk is /dev/vdb

# 2. Create ext4 file system (optimized parameters)
# -m 1: Reduce reserved space from 5% to 1% to increase available space
# -T largefile: Optimize for large file storage
# -L: Set volume label for easy identification
mkfs.ext4 -m 1 -T largefile -L doris-data /dev/vdb

# 3. Create mount point directory
mkdir -p /data

# 4. Mount data disk
mount /dev/vdb /data

# 5. Get disk UUID
blkid /dev/vdb
# Output example: /dev/vdb: UUID="e86ca90c-ca31-460d-b450-1fe914ebae70" TYPE="ext4"

# 6. Configure automatic mount on boot (replace with actual UUID)
echo "UUID=e86ca90c-ca31-460d-b450-1fe914ebae70 /data ext4 defaults,noatime 0 2" >> /etc/fstab
# noatime: Do not update access time to improve performance (especially for big data read/write)
# 0: Do not backup
# 2: Check file system on startup

# 7. Create Doris data directory
mkdir -p /data/doris

# 8. Verify mount
df -h | grep /data

3.2 Disable Swap Partition

Reason: When Doris is under high memory pressure, swap swaps memory pages to disk, causing GC pauses, query lag, or even node pseudo-death, severely affecting performance.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 1. Temporarily turn off swap
sudo swapoff -a

# 2. Permanently disable swap (comment out the swap line in /etc/fstab)
sudo sed -i '/swap/s/^/#/' /etc/fstab

# 3. Verify swap is turned off
free -h
# Swap should show 0B

swapon --show
# Should have no output, indicating swap is fully closed

3.3 Optimize Kernel Parameters

3.3.1 Increase Virtual Memory Areas (VMA)

Reason: Doris maps a large number of files to memory, requiring sufficient virtual memory areas.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Add kernel parameters
cat >> /etc/sysctl.conf << EOF
vm.max_map_count = 2000000
EOF

# Apply configuration
sysctl -p

# Verify
sysctl vm.max_map_count
# Should output: vm.max_map_count = 2000000

3.3.2 Optimize Network Parameters (Optional)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Optimize TCP connection parameters for high concurrency scenarios
cat >> /etc/sysctl.conf << EOF
# TCP optimization parameters
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.core.somaxconn = 1024
net.ipv4.tcp_max_syn_backlog = 2048
EOF

sysctl -p

3.4 Increase File Handle Limits

Reason: Doris BE opens a large number of files (data shards, indexes, logs, etc.), so the file descriptor limit must be increased.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# 1. Modify limits configuration
sudo tee -a /etc/security/limits.conf <<EOF
* soft nofile 65536
* hard nofile 65536
EOF

# 2. Create independent configuration in limits.d directory
sudo tee -a /etc/security/limits.d/20-nproc.conf <<EOF
* soft nofile 65536
* hard nofile 65536
EOF

# 3. Verify configuration
ulimit -n
# Should output: 65536

3.5 Apply All Configurations and Reboot

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Apply all kernel parameters
sysctl -p

# Reboot nodes for all configurations to take effect
reboot

# Verification after reboot
# 1. Check swap: free -h (Swap should be 0)
# 2. Check file handles: ulimit -n (should be 65536)
# 3. Check VMA: sysctl vm.max_map_count (should be 2000000)
# 4. Check data disk: df -h | grep /data

In actual deployment, you can customize virtual machine hosts or cloud host images in advance to include basic initialization operations.

4. Deploy Doris Manager

4.1 Extraction and Installation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# 1. Extract installation package
cd /opt/downloads/doris_manager
tar zxvf doris-manager-24.1.5-x64-bin.tar.gz

# 2. Create deployment directory
mkdir -p /opt/doris/manager

# 3. Move files to deployment directory
mv ./doris-manager-24.1.5-x64-bin/* /opt/doris/manager/

# 4. Prepare Doris installation package resources
# Manager needs local source files for subsequent cluster deployment
mkdir -p /opt/downloads/doris
cp /opt/downloads/doris_manager/apache-doris-2.1.8.1-bin-x64.tar.gz /opt/downloads/doris/
cp /opt/downloads/doris_manager/selectdb-doris-2.0.3-b4-bin-x64.tar.gz /opt/downloads/doris/

4.2 Start Doris Manager

1
2
3
4
5
6

# Start Doris Manager WebServer
cd /opt/doris/manager/
./webserver/bin/start.sh

# View logs (optional)
tail -f webserver/log/webserver.log

Startup Verification: Service listens on port 8004

1
2
3
4
5
6
7

# Check if port is listening normally
ss -an | grep 8004
# Expected output:
# tcp    LISTEN    0      100    *:8004    *:*

# Or use netstat
netstat -tlnp | grep 8004

4.3 Configure Auto-start on Boot (Recommended)

Use systemd to manage the Doris Manager service, ensuring automatic startup after node restarts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

# 1. Create systemd service file
cat > /etc/systemd/system/doris-manager.service <<'EOF'
[Unit]
Description=Doris Manager WebServer Service
Documentation=https://doris.apache.org/
After=network.target

[Service]
Type=forking
WorkingDirectory=/opt/doris/manager
ExecStart=/opt/doris/manager/webserver/bin/start.sh
ExecStop=/opt/doris/manager/webserver/bin/stop.sh
Restart=on-failure
RestartSec=10
User=root
Group=root
TimeoutStartSec=120
TimeoutStopSec=120
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF

# 2. Reload systemd configuration
sudo systemctl daemon-reload

# 3. Enable auto-start on boot
sudo systemctl enable doris-manager

# 4. Check service status
sudo systemctl status doris-manager

4.4 Access Manager UI

Access the Doris Manager management interface via a browser:

http://{manager-ip}:8004

First Access Steps:

1. Enter the initialization user page to create the first Manager admin user.
2. The Manager admin account is independent of cluster accounts and is only used for Manager permission control.
3. Log in with that account after successful creation.

5. Configure Nginx Reverse Proxy

In production environments, it is recommended to configure domain names and HTTPS access via Nginx.

5.1 Nginx Configuration File

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

server {
  listen 80;
  listen 443 ssl;
  charset utf-8;
  server_name ug-test-doris-manager.test.com;
  index  index.html index.htm index.jsp index.php;

  # SSL configuration (optional)
  # if ($scheme = http) {
  #   rewrite ^(.*)$ https://$host$1 redirect;
  # }

  include rule.d/ssl.conf;
  include rule.d/ssl_test.conf;

  client_max_body_size 5000m;

  location  / {
    include rule.d/proxy_header.conf;
    include rule.d/proxy_connection_small.conf;
    proxy_pass http://10.156.179.58:8004;
  }

  access_log  /var/log/nginx/ug-test-doris-manager.test.com.access.log main;
  error_log   /var/log/nginx/ug-test-doris-manager.test.com.error.log;
}

5.2 Restart Nginx

1
2
3
4
5
6
7
8

# Check configuration file syntax
nginx -t

# Restart Nginx
systemctl restart nginx

# Verify
curl -I http://ug-test-doris-manager.test.com

Now you can access Doris Manager via the domain name:

https://ug-test-doris-manager.test.com

6. Initialize Manager Configuration

6.1 Create Admin Account

1. Access UI Interface: Access Doris Manager via the configured domain or IP.

   

2. Create Admin Account:

   • Set username and password.
   • Service Configuration: Since external network policies may not be open, it is recommended to turn off email alerts first.
   • Click to start the doris-manager service.

   

3. Enter Management Console: After successful installation, enter the main interface.

   

6.2 Upload Doris Installation Packages

Upload the Doris installation package on the Manager interface:

• apache-doris-2.1.8.1-bin-x64.tar.gz

Or use the command line to place in the specified directory:

1
2
3

# Ensure the path is accessible by Manager
mkdir -p /opt/downloads/doris
cp apache-doris-2.1.8.1-bin-x64.tar.gz /opt/downloads/doris/

7. Deploy Doris Agent

Agent is the proxy program of Doris Manager on each node, responsible for executing deployment and management tasks.

7.1 Get Installation Command

On the Manager interface:

1. Go to “Cluster Management” -> “Agent Management”.

2. Click the “Add Agent” button.

3. Get the installation command (usually two lines).

   

Example Command:

1

wget http://10.156.179.58:8004/api/download/deploy.sh -O deploy_agent.sh && chmod +x deploy_agent.sh && ./deploy_agent.sh

Note: Replace the IP address with the actual Manager service IP.

7.2 Execute Installation on Target Nodes

Log in to each target node (FE, BE nodes) and execute the above Agent installation command:

1
2
3
4

# Execute on every Doris node
wget http://{manager-ip}:8004/api/download/deploy.sh -O deploy_agent.sh
chmod +x deploy_agent.sh
./deploy_agent.sh

7.3 Configure Agent Auto-start on Boot

Create a systemd service file to manage Agent:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# 1. Create systemd service file
cat > /etc/systemd/system/doris-manager-agent.service <<'EOF'
[Unit]
Description=Doris Manager Agent Service
Documentation=https://doris.apache.org/
After=network.target

[Service]
Type=forking
# Type=forking is because the start.sh script starts the process in the background and exits immediately

WorkingDirectory=/root/manager-agent

ExecStart=/root/manager-agent/bin/start.sh
ExecStop=/root/manager-agent/bin/stop.sh

Restart=on-failure
RestartSec=10

User=root
Group=root

TimeoutStartSec=120
TimeoutStopSec=120

StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF

# 2. Reload systemd configuration
sudo systemctl daemon-reload

# 3. Enable auto-start on boot
sudo systemctl enable doris-manager-agent

# 4. Start service
sudo systemctl start doris-manager-agent

# 5. Check service status
sudo systemctl status doris-manager-agent

# 6. View real-time logs (if startup fails)
sudo journalctl -u doris-manager-agent -f

7.4 Manually Create Data Directory

Manually create the data storage directory on all nodes:

1

mkdir -p /data/doris

7.5 Verify Agent in Manager Interface

On the “Agent Management” page of the Manager interface, you should see the installed Agent nodes with a status of “Online”.

8. Create Doris Cluster

8.1 Start Creating Cluster

1. Enter Cluster Management: Click “Cluster Management” on the Manager main interface.

2. Click “New Cluster”: Start creating a new Doris cluster.

   

3. Fill in Cluster Configuration:

   • Cluster Name: e.g., prod-cluster
   • Cluster Version: Select apache-doris-2.1.8.1
   • Select Installation Package: Choose from the uploaded packages.

   

8.2 Add FE Nodes

1. Add FE Nodes:

   • Select FE nodes from the list of nodes with Agent installed.
   • It is recommended to have at least 3 FE nodes to form a high-availability cluster.
   • Select FE role (Follower or Observer).

2. Configure FE Parameters:

   • Ports: http_port=8030, query_port=9030, edit_log_port=9010
   • Memory Allocation: Configure according to the node memory size.

8.3 Add BE Nodes

1. Add BE Nodes:

   • Select BE nodes from the list of nodes with Agent installed.
   • Select an appropriate number of BE nodes according to business requirements.

2. Configure BE Parameters:

   • Ports: be_port=9060, webserver_port=8040
   • Data Directory: /data/doris
   • Memory Allocation: It is recommended to reserve system memory and allocate 70-80% of the remainder to BE.

8.4 Environment Detection

Before starting deployment, Manager will automatically perform environment detection:

• System parameter checks (file handles, VMA, swap, etc.)
• Disk space checks
• Network connectivity checks
• Port occupation checks

Wait for all detection items to pass, then click “Next” to continue deployment.

If there are failed detection items, please fix them according to the prompts and re-detect.

9. Cluster Deployment and Configuration

9.1 System Initialization Configuration

Before deploying the cluster, system parameter initialization configuration is required:

Execute the following system parameter adjustments on all nodes:

Modify System Limits (Limits):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Modify file handle limits
sudo tee -a /etc/security/limits.conf <<EOF
* soft nofile 65536
* hard nofile 65536
EOF

sudo tee -a /etc/security/limits.d/20-nproc.conf <<EOF
* soft nofile 65536
* hard nofile 65536
EOF

Apply Configuration and Reboot:

1
2
3
4
5

# Apply kernel parameters
sysctl -p

# Reboot node
reboot

Start Agent After Reboot: After the node restart is complete, you must manually start the Agent service.

1

systemctl start doris-manager-agent

9.2 Deploy Cluster

1. Cluster Settings:

   • Table Name Case Sensitivity: It is recommended to select “Insensitive” (for easier application compatibility).
   • Confirm all configurations are correct.

2. Start Deployment: Click the “Deploy Cluster” button.

   

3. Monitor Deployment Progress:

   • Manager will automatically deploy FE and BE components on each node.
   • You can view real-time deployment logs on the interface.
   • The deployment process usually takes 5-10 minutes.

4. Deployment Complete:

   • All component statuses show as “Running”.
   • Enter the cluster details page to view the cluster overview.

   

9.2 Configure Timezone (Important)

Business Scenario: To meet the business needs of different countries, the cluster timezone needs to be adjusted according to the deployment region.

Timezone Mapping Table:

9.2.1 Modify FE Timezone

On the Manager interface:

1. Go to “Cluster Management” -> Select Cluster -> “Configuration Management”.
2. Find the FE configuration file fe.conf.
3. Add or modify the following parameter:

1
2

# fe/conf/fe.conf
time_zone = Africa/Abidjan

4. Save the configuration and restart the FE node.

9.2.2 Modify BE Timezone

On the Manager interface:

1. Find the BE configuration file be.conf in “Configuration Management”.
2. Add or modify the following parameter:

1
2

# be/conf/be.conf
time_zone = Africa/Abidjan

3. Save the configuration and restart the BE node.

9.2.3 Verify Timezone Configuration

Method 1: Verify via MySQL Client

1
2
3
4
5
6
7
8

# Connect to FE
mysql -h {fe_host} -P9030 -u root -p

# View timezone settings
SHOW VARIABLES LIKE '%time_zone%';

# View current time
SELECT NOW();

Expected Output:

+---------------+--------+
| Variable_name | Value  |
+---------------+--------+
| time_zone     | AAI    |  # Abbreviation for Africa/Abidjan
+---------------+--------+

Method 2: Verify via Logs

1
2
3
4
5

# View FE logs
grep -i timezone /opt/doris/fe/log/fe.log

# View BE logs
grep -i timezone /opt/doris/be/log/be.INFO

10. Cluster Verification and Testing

10.1 Check Cluster Status

View in the Manager interface:

• FE Node Status: Should be “ALIVE”
• BE Node Status: Should be “ALIVE”
• Cluster Health Status: Should be “Healthy”

10.2 Verify via Command Line

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# 1. Connect to FE
mysql -h {fe_host} -P9030 -u root -p

# 2. View FE status
SHOW FRONTENDS;

# 3. View BE status
SHOW BACKENDS;

# 4. Create test database
CREATE DATABASE test_db;

# 5. Create test table
USE test_db;
CREATE TABLE test_table (
  id INT,
  name VARCHAR(100)
) DUPLICATE KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 10;

# 6. Insert test data
INSERT INTO test_table VALUES (1, 'test'), (2, 'doris');

# 7. Query test data
SELECT * FROM test_table;

10.3 Verify Timezone

1
2
3
4
5

-- Query current time to confirm timezone is correct
SELECT NOW(), CURRENT_TIMESTAMP();

-- View timezone variables
SHOW VARIABLES LIKE '%time_zone%';

11. Troubleshooting

11.1 Agent Cannot Connect to Manager

Symptom: Manager interface shows Agent as offline.

Troubleshooting Steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# 1. Check if Agent process is running
ps aux | grep agent

# 2. Check Agent logs
tail -f /root/manager-agent/log/agent.log

# 3. Check network connectivity
curl -I http://{manager-ip}:8004

# 4. Check firewall and ports
telnet {manager-ip} 8004

# 5. Restart Agent
systemctl restart doris-manager-agent

11.2 FE Node Fails to Start

Common Causes:

1. Port occupied
2. Metadata directory permission issues
3. JAVA_HOME not configured

Troubleshooting Steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 1. View FE logs
tail -f /opt/doris/fe/log/fe.log

# 2. Check port occupation
netstat -tlnp | grep -E '9030|9010|8030'

# 3. Check directory permissions
ls -la /data/doris/

# 4. Check Java environment
java -version
echo $JAVA_HOME

11.3 BE Node Fails to Start

Common Causes:

1. Data disk not mounted
2. Memory configuration too large
3. System parameters not effective

Troubleshooting Steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# 1. View BE logs
tail -f /opt/doris/be/log/be.INFO

# 2. Check data disk mount
df -h | grep /data

# 3. Check memory configuration
free -h
grep -i mem /opt/doris/be/conf/be.conf

# 4. Check system parameters
ulimit -n
sysctl vm.max_map_count

11.4 Timezone Configuration Not Taking Effect

Troubleshooting Steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 1. Confirm configuration file has been modified
grep time_zone /opt/doris/fe/conf/fe.conf
grep time_zone /opt/doris/be/conf/be.conf

# 2. Restart FE and BE
# Restart via Manager interface or manually

# 3. View startup logs
grep -i timezone /opt/doris/fe/log/fe.log | tail -20

# 4. Verify via MySQL client
mysql -h {fe_host} -P9030 -u root -e "SHOW VARIABLES LIKE '%time_zone%';"

12. Operations Management

12.1 Cluster Scaling Out

Add BE Nodes:

1. Deploy Agent on the new node (refer to Chapter 7).
2. In Manager interface, select Cluster -> “Node Management” -> “Add Node”.
3. Select the BE nodes to add.
4. Confirm to automatically deploy and start BE.

Add FE Nodes:

1. Deploy Agent on the new node.
2. Add FE Follower node in Manager interface.
3. Wait for FE to complete data synchronization and join the cluster.

12.2 Cluster Scaling In

Delete BE Nodes:

1. Select the BE node to delete in Manager interface.
2. Click the “Delete” button.
3. The system will automatically perform data migration (takes a certain time).
4. The node will automatically go offline after data migration is complete.

Delete FE Nodes:

• Deleting FE Follower nodes is not recommended in production environments.
• If necessary, please ensure the cluster has at least 3 FE nodes first.

12.3 Configuration Modification

You can easily modify cluster configurations in the Manager interface:

1. Go to “Cluster Management” -> Select Cluster -> “Configuration Management”.
2. Select the configuration file to modify (fe.conf or be.conf).
3. Save after modifying parameters.
4. Click “Apply Configuration” to restart the corresponding components.

Common Configuration Adjustments:

12.4 Monitoring and Alerting

Cluster Monitoring Metrics:

• CPU usage
• Memory usage
• Disk space usage
• Query QPS
• Query response time

Alert Configuration: Configure alert rules in the Manager interface:

1. Go to “Cluster Management” -> “Alert Management”.
2. Create alert rules (e.g., disk usage > 80%).
3. Configure alert receiving methods (Email, DingTalk, WeCom).

13. Appendix

13.1 Port Mapping Table

13.2 Directory Structure

/opt/doris/
├── manager/                    # Doris Manager
│   ├── webserver/
│   │   ├── bin/
│   │   └── log/
│   └── ...
├── fe/                         # Frontend
│   ├── bin/
│   ├── conf/                   # Configuration files
│   │   ├── fe.conf
│   │   └── fe_custom.conf
│   ├── log/                    # Log directory
│   │   ├── fe.log
│   │   └── fe.warn.log
│   └── ...
├── be/                         # Backend
│   ├── bin/
│   ├── conf/                   # Configuration files
│   │   ├── be.conf
│   │   └── be_custom.conf
│   ├── log/                    # Log directory
│   │   ├── be.INFO
│   │   └── be.out
│   └── ...
└── ...

/root/manager-agent/            # Agent working directory
├── bin/
├── conf/
└── log/

/data/doris/                    # Data storage directory
├── fe/                         # FE data
│   ├── meta/
│   └── ...
└── be/                         # BE data
    ├── storage/
    └── ...

13.3 Reference Documents

• Apache Doris Official Documentation
• Doris Manager Deployment Guide

At this point, the deployment of Doris Manager and the setup of the Doris cluster are complete!