Introduction
This is part of the Distributed Storage series. We assume you have a Kubernetes cluster running as described in Kubernetes With Microk8s. Using the same virtual machines, we have deployed a Ceph cluster and enabled different types of storage: file, object, and block (default). This ensures that we have covered the prerequisites.
Rook is an open source cloud-native storage orchestrator, providing the platform, framework, and support for Ceph storage to natively integrate with cloud-native environments. The storage architecture is well documented. We focus on getting it up and running.
Enable Rook
First, we enable rook on the k8s cluster
manas@manas-s01:~$ sudo microk8s enable rook-ceph
Infer repository core for addon rook-ceph
Add Rook Helm repository <https://charts.rook.io/release>
"rook-release" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "rook-release" chart repository
Update Complete. ⎈Happy Helming!⎈
Install Rook version v1.11.9
NAME: rook-ceph
LAST DEPLOYED: Fri Jun 13 16:01:21 2025
NAMESPACE: rook-ceph
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Rook Operator has been installed. Check its status by running:
kubectl --namespace rook-ceph get pods -l "app=rook-ceph-operator"
...
Check Rook Status
manas@manas-s01:~$ microk8s kubectl --namespace rook-ceph get pods -l "app=rook-ceph-operator"
NAME READY STATUS RESTARTS AGE
rook-ceph-operator-684bbd569f-82dv9 1/1 Running 0 30m
Connect Ceph and k8s cluster
manas@manas-s01:~$ sudo microk8s connect-external-ceph
[sudo] password for manas:
Looking for MicroCeph on the host
Detected existing MicroCeph installation
Attempting to connect to Ceph cluster
Successfully connected to e43d58a8-deb0-43d9-b7ef-d6159f114c02 (192.168.148.134:0/1580887393)
Creating pool microk8s-rbd0 in Ceph cluster
Configuring pool microk8s-rbd0 for RBD
Successfully configured pool microk8s-rbd0 for RBD
Creating namespace rook-ceph-external
namespace/rook-ceph-external created
Configuring Ceph CSI secrets
Successfully configured Ceph CSI secrets
Importing Ceph CSI secrets into MicroK8s
secret/rook-ceph-mon created
configmap/rook-ceph-mon-endpoints created
secret/rook-csi-rbd-node created
secret/rook-csi-rbd-provisioner created
storageclass.storage.k8s.io/ceph-rbd created
Importing external Ceph cluster
W0613 16:32:46.334927 129114 warnings.go:70] unknown field "spec.upgradeOSDRequiresHealthyPGs"
NAME: rook-ceph-external
LAST DEPLOYED: Fri Jun 13 16:32:45 2025
NAMESPACE: rook-ceph-external
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Ceph Cluster has been installed. Check its status by running:
kubectl --namespace rook-ceph-external get cephcluster
Visit <https://rook.io/docs/rook/latest/CRDs/Cluster/ceph-cluster-crd/> for more information about the Ceph CRD.
Important Notes:
- You can only deploy a single cluster per namespace
- If you wish to delete this cluster and start fresh, you will also have to wipe the OSD disks using `sfdisk`
=================================================
Successfully imported external Ceph cluster. You can now use the following storageclass
to provision PersistentVolumes using Ceph CSI:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ceph-rbd rook-ceph.rbd.csi.ceph.com Delete Immediate true 2s
Check Ceph cluster status
manas@manas-s01:~$ microk8s kubectl --namespace rook-ceph-external get cephcluster
NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID
rook-ceph-external /var/lib/rook 3 67s true
manas@manas-s01:~$ microk8s kubectl --namespace rook-ceph-external get cephcluster
NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID
rook-ceph-external /var/lib/rook 3 6m1s Connected Cluster connected successfully HEALTH_OK true e43d58a8-deb0-43d9-b7ef-d6159f114c02
ma
Wait until all k8s resources are in Running status:
manas@manas-s01:~$ microk8s kubectl get all --namespace rook-ceph
NAME READY STATUS RESTARTS AGE
pod/csi-cephfsplugin-chz2w 2/2 Running 0 5m7s
pod/csi-cephfsplugin-n2wxd 2/2 Running 0 5m7s
pod/csi-cephfsplugin-provisioner-7bd8fb7c64-fbmw6 5/5 Running 0 5m7s
pod/csi-cephfsplugin-provisioner-7bd8fb7c64-tbqlp 5/5 Running 0 5m7s
pod/csi-cephfsplugin-zpltk 2/2 Running 0 5m7s
pod/csi-rbdplugin-7dgxz 2/2 Running 0 5m7s
pod/csi-rbdplugin-8x5x7 2/2 Running 0 5m7s
pod/csi-rbdplugin-provisioner-5f7d95b6fb-s4znf 5/5 Running 0 5m7s
pod/csi-rbdplugin-provisioner-5f7d95b6fb-wjk7q 5/5 Running 0 5m7s
pod/csi-rbdplugin-zccc7 2/2 Running 0 5m7s
pod/rook-ceph-operator-684bbd569f-82dv9 1/1 Running 0 39m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/csi-cephfsplugin 3 3 3 3 3 <none> 5m7s
daemonset.apps/csi-rbdplugin 3 3 3 3 3 <none> 5m8s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/csi-cephfsplugin-provisioner 2/2 2 2 5m7s
deployment.apps/csi-rbdplugin-provisioner 2/2 2 2 5m7s
deployment.apps/rook-ceph-operator 1/1 1 1 39m
NAME DESIRED CURRENT READY AGE
replicaset.apps/csi-cephfsplugin-provisioner-7bd8fb7c64 2 2 2 5m7s
replicaset.apps/csi-rbdplugin-provisioner-5f7d95b6fb 2 2 2 5m7s
replicaset.apps/rook-ceph-operator-684bbd569f 1 1 1 39m
manas@manas-s01:~$
Meanwhile, If you want to explore the possible API Resources, you the following command:
manas@manas-s01:~$ microk8s kubectl api-resources --namespace rook-ceph-external | grep ceph
cephblockpoolradosnamespaces ceph.rook.io/v1 true CephBlockPoolRadosNamespace
cephblockpools ceph.rook.io/v1 true CephBlockPool
cephbucketnotifications ceph.rook.io/v1 true CephBucketNotification
cephbuckettopics ceph.rook.io/v1 true CephBucketTopic
cephclients ceph.rook.io/v1 true CephClient
cephclusters ceph.rook.io/v1 true CephCluster
cephfilesystemmirrors ceph.rook.io/v1 true CephFilesystemMirror
cephfilesystems ceph.rook.io/v1 true CephFilesystem
cephfilesystemsubvolumegroups ceph.rook.io/v1 true CephFilesystemSubVolumeGroup
cephnfses nfs ceph.rook.io/v1 true CephNFS
cephobjectrealms ceph.rook.io/v1 true CephObjectRealm
cephobjectstores ceph.rook.io/v1 true CephObjectStore
cephobjectstoreusers rcou,objectuser ceph.rook.io/v1 true CephObjectStoreUser
cephobjectzonegroups ceph.rook.io/v1 true CephObjectZoneGroup
cephobjectzones ceph.rook.io/v1 true CephObjectZone
cephrbdmirrors ceph.rook.io/v1 true CephRBDMirror
To see Block storage status, you can use cephblockpools. Similarly, cephobjectstores is for Object and cephnfses is for NFS.
Refer to examples from https://github.com/rook/rook/tree/master/deploy/examples
Create Block Storage
$ microk8s kubectl create -f storageClass.yaml
cephblockpool.ceph.rook.io/replicapool created
storageclass.storage.k8s.io/rook-ceph-block created
$ microk8s kubectl -n rook-ceph get cephblockpools
NAME PHASE
replicapool Progressing
$ microk8s kubectl create -f mysql.yaml
service/wordpress-mysql created
persistentvolumeclaim/mysql-pv-claim created
deployment.apps/wordpress-mysql created
Create Object Stores
This requires rgw that we have already enabled
$ kubectl create -f object.yaml
$ microk8s kubectl -n rook-ceph get cephobjectstores
NAME PHASE
my-store Progressing
Create Share File Systems
$ kubectl create -f file.yaml
$ microk8s kubectl get cephfilesystems -n rook-ceph
NAME ACTIVEMDS AGE PHASE
myfs 1 5m38s Progressing
That’s it! We have completed deploying a cloud native storage solution, that too, locally.
