Cannot use DatastoreClusters With PVC Successfully

[jinglejengel]

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug

What happened:

I'm having issues using datastore clusters for dynamic PVCs on vSphere for our kubernetes clusters based on this documentation: https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/storageclass.html

We have a few datastore clusters that I’d like to allocate individual Storage Classes for (one is SSD, one is HHD). I’m unable to get this to work, but it does work when I use a specific datastore under the datastore cluster.

It’s worth noting we’re running vCenter 5.5 currently.

I notice on govc I can see the datastore cluster under datastores/ but specifying it to kubernetes unfortunately yields a ‘Failed to provision volume with StorageClass "test-cluster": datastore 'DatastoreCluster/IT-EntApp-HHD-NonSSD' not found’

I also tried feeding it the following values with no luck:

'datastore/IT-EntApp-HHD-NonSSD'
'IT-EntApp-HHD-NonSSD'
'/MyCompany/datastore/IT-EntApp-HHD-NonSSD'

It's also worth noting that using a datastore directly does work (e.g. one under the datastore cluster)

What you expected to happen: To be able to use a DatastoreCluster to use for dynamic PVC

How to reproduce it (as minimally and precisely as possible):

Here's my StorageClass that I'm trying to get working:

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: test-cluster
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: thin
  datastore: "DatastoreCluster/IT-EntApp-HHD-NonSSD"

And here's the corresponding PVC:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvcsc002
  annotations:
    volume.beta.kubernetes.io/storage-class: test-cluster
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Anything else we need to know?: N/A

Environment:

• Kubernetes version (use kubectl version): 1.8.10
• Cloud provider or hardware configuration: vSphere
• OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

• Kernel (e.g. uname -a): Linux $HOST 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[BaluDontu]

Hi @Joeskyyy,
Currently vSphere cloud Provider(VCP) doesn't have support for using datastore clusters directly in the storage class definition for provisioning persistent volumes. The user has to specify the datastore name in the storage class definition.

The following 2 storage class definitions works:

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: test-cluster
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: thin
  datastore: "DatastoreCluster/vsanDatastore"

Here, "DatastoreCluster/vsanDatastore" specifies vsanDatastore is the datastore on DatastoreCluster.

Alternatively the following storage class definition also works

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: test-cluster
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: thin
  datastore: "vsanDatastore"

Here, the user directly specifies the datastore name instead of the full datastore path as explained above.

We have an open issue for this request - #203. Please track this issue for any updates.