Home

Welcome to the ceph-14.2.9-complete-install-guide wiki!

Architecture and assumptions:

• I have tried many methods to install ceph. Although installation and configuration is the easiest part in ceph administration but there's always some glitches and headaches. Last time the problem was that I've decided to install Nautilus and ceph-deploy could not have been instructed to install this version and was insisting on installing Mimic which was too old for me. If you are OK with Mimic then I think ceph-deploy is the best option for you. But I love this version of howto of mine :D
• I'm going to configure 3 nodes, each having two network card for public and private network and each having 1 hdd for OS and 4 HDD as OSD
• I'm going to use this cluster as a test bed to check if my application is OK to work with an S3 API or not, so this installation is not good for production use. And I'm going to install and config rgw (radosgw).
• I'm going to install only one monitor to keep is as simple as possible.
• Good monitoring is needed, so prometheus and grafana is going to be installed and configured. Graphs and data are far better than you can imagine.
• All 3 nodes are VMware vCenter VMs and they should be located on the same host, you have to use affinity rules to stick them to the same host so that you won't face high latency issues. So all 3 nodes will communicate switch-free and through hypervisor localhost.
• All disks are going to be provisioned as VMDK from the same VMFS datastore backed with RAID10 LUN from a SAN. I know that separated JBOD for each OSD is the best practice but I cannot go for that option in this scenario.
• Limited user that I'm going to work with is cphus
• Domain name I've defined is ceph.clc and node hostnames are like: node#.ceph.cls. You can replace them with your own domain.
• IPs for public NICs are provided by IaaS, but I've chosen 10.20.30.0/24 for private network, you can replace them and you can go with my choice.
• Resources assigned to each machine is arbitrary, you can change them. But remember if you add or remove hdds you would need to change yml files as well.
• I've cated my final config. There's a lot more options you can tweak and mess with, but this config worked just fine for me.
• I'm using my first node as admin node and ansible provisioner, you can deploy a separated node for it or use another node if you want.
• Understanding this document need basic knowledge of linux and ceph architecture, so don't jump in before knowing what you are doing!
• I've used ceph official docs and mainly this doc here.

Proceed to:

1. Prepare a VM TMP

2. Ceph Installation