ClickHouse Directory Layout and Configuration

This article is used to explain the directory layout, configuration entry and recommended maintenance methods of clickhouse_ansible.

1. Top-level directory

Typical path:

/usr/local/dbbot/clickhouse_ansible

Key directory:

  • downloads/: ClickHouse and ZooKeeper installation packages.
  • inventory/: Host list for deployment, backup, recovery, NFS server and other scenarios.
  • playbooks/: Entry Playbook, pre-task and variable files.
  • roles/: System initialization, coordination component deployment, ClickHouse deployment and other roles.
  • artifacts/manifests/: Backup and restore running products.

2. Directory structure example

clickhouse_ansible/
├── artifacts/
│   └── manifests/
├── downloads/
├── inventory/
│   ├── hosts.deploy.ini
│   ├── hosts.dr_deploy.ini
│   ├── hosts.backup.ini
│   ├── hosts.dr_backup.ini
│   ├── hosts.restore.ini
│   ├── hosts.nfs_server.ini
│   └── hosts.single.ini
├── playbooks/
│   ├── deploy_cluster.yml
│   ├── deploy_single.yml
│   ├── setup_nfs_server.yml
│   ├── setup_nfs_client_mount_rc_local.yml
│   ├── prepare_backup_disk.yml
│   ├── backup_cluster.yml
│   ├── restore_cluster.yml
│   ├── uninstall_cluster.yml
│   ├── pre_tasks/
│   ├── tasks/
│   └── vars/
└── roles/
    ├── init_system/
    ├── deploy_zookeeper/
    └── deploy_clickhouse/

3. Inventory file responsibilities

  • hosts.deploy.ini: Deployment-specific, requires dbbot_inventory_purpose=deploy.
  • hosts.dr_deploy.ini: dedicated for disaster recovery cluster deployment, requires dbbot_inventory_purpose=deploy.
  • hosts.backup.ini: Backup-specific, requires dbbot_inventory_purpose=backup.
  • hosts.dr_backup.ini: dedicated for NFS mounting and backup disk preparation of disaster recovery targets, requires dbbot_inventory_purpose=backup.
  • hosts.restore.ini: Restore-specific, requires dbbot_inventory_purpose=restore.
  • hosts.nfs_server.ini: NFS server-specific, requires dbbot_inventory_purpose=nfs_server.
  • hosts.single.ini: dedicated for single-node validation.

The purpose of this split is to reduce the probability of misoperation and avoid accidentally hitting backup or recovery commands to the deployment inventory.

4. Variable file responsibilities

4.1 playbooks/vars/common_config.yml

Used to maintain common parameters, such as:

  • clickhouse_version
  • clickhouse_user / clickhouse_group
  • clickhouse_uid / clickhouse_gid
  • clickhouse_data_dir_base
  • clickhouse_default_password
  • fcs_allow_dbbot_default_passwd
  • clickhouse_enable_ssl
  • use_clickhouse_keeper
  • clickhouse_install_debug_package

Description:

  • The public default password for clickhouse_default_password is Dbbot_default@8888.
  • Defaults to fcs_allow_dbbot_default_passwd: false, so if this public default password is still used, the related playbook will fail directly in the pre_tasks stage.

4.2 playbooks/vars/cluster_config.yml

Used to maintain cluster-level parameters, such as:

  • clickhouse_cluster_name
  • clickhouse_tcp_port_base
  • clickhouse_http_port_base
  • clickhouse_interserver_http_port_base

4.3 playbooks/vars/backup_config.yml

For backup policy and manifest output, for example:

  • backup_mode
  • backup_storage_disk
  • backup_mount_dir
  • backup_checkpoint_mode
  • backup_require_replicated_tables

4.4 playbooks/vars/restore_config.yml

Used to restore behavioral control, such as:

  • restore_batch_id
  • restore_to_all_replicas
  • restore_allow_non_empty_tables
  • restore_enable_two_phase_mv_compat

4.5 playbooks/vars/standalone_config.yml

Used in standalone validation mode, for example:

  • clickhouse_topology_mode
  • clickhouse_single_host_required

4.6 playbooks/vars/uninstall_config.yml

Used to control uninstall behavior, for example:

  • uninstall_purge_clickhouse_config
  • uninstall_purge_clickhouse_data
  • uninstall_remove_backup_mount
  • uninstall_purge_zookeeper

5. Execution logic overview

The entry Playbook generally follows the following sequence:

  1. pre_tasks: parameter validation, topology analysis, environment check, public default password interception.
  2. roles: Execute system initialization, coordination component deployment, ClickHouse deployment or backup and recovery logic.
  3. artifacts/manifests: Run metadata for backup and recovery process output.

6. Practical suggestions

  • Maintain separate inventories for deployment, backup, and recovery.
  • Maintain separate hosts.nfs_server.ini for NFS server.
  • Unify clickhouse_uid / clickhouse_gid in multiple environments to reduce cross-cluster permission restoration issues.
  • Prioritize the use of fixed version numbers and fixed batch numbers for production environments to facilitate auditing and rollback.
  • The debug package is not installed by default; enable clickhouse_install_debug_package=true only when failure analysis is clearly needed.