Skip to content

HyperShift API Reference

Packages:

hypershift.openshift.io/v1alpha1

Package v1alpha1 contains the HyperShift API.

The HyperShift API enables creating and managing lightweight, flexible, heterogeneous OpenShift clusters at scale.

HyperShift clusters are deployed in a topology which isolates the “control plane” (e.g. etcd, the API server, controller manager, etc.) from the “data plane” (e.g. worker nodes and their kubelets, and the infrastructure on which they run). This enables “hosted control plane as a service” use cases.

HostedCluster

HostedCluster is the primary representation of a HyperShift cluster and encapsulates the control plane and common data plane configuration. Creating a HostedCluster results in a fully functional OpenShift control plane with no attached nodes. To support workloads (e.g. pods), a HostedCluster may have one or more associated NodePool resources.

Field Description
apiVersion string hypershift.openshift.io/v1alpha1
kind string HostedCluster
metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.
spec HostedClusterSpec

Spec is the desired behavior of the HostedCluster.



release Release

Release specifies the desired OCP release payload for the hosted cluster.

Updating this field will trigger a rollout of the control plane. The behavior of the rollout will be driven by the ControllerAvailabilityPolicy and InfrastructureAvailabilityPolicy.

clusterID string (Optional)

ClusterID uniquely identifies this cluster. This is expected to be an RFC4122 UUID value (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx in hexadecimal values). As with a Kubernetes metadata.uid, this ID uniquely identifies this cluster in space and time. This value identifies the cluster in metrics pushed to telemetry and metrics produced by the control plane operators. If a value is not specified, an ID is generated. After initial creation, the value is immutable.

infraID string (Optional)

InfraID is a globally unique identifier for the cluster. This identifier will be used to associate various cloud resources with the HostedCluster and its associated NodePools.

platform PlatformSpec

Platform specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies the availability policy applied to critical control plane components. The default value is SingleReplica.

Value must be one of: "HighlyAvailable", "SingleReplica"

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies the availability policy applied to infrastructure services which run on cluster nodes. The default value is HighlyAvailable.

Value must be one of: "HighlyAvailable", "SingleReplica"

dns DNSSpec

DNS specifies DNS configuration for the cluster.

networking ClusterNetworking

Networking specifies network configuration for the cluster.

autoscaling ClusterAutoscaling (Optional)

Autoscaling specifies auto-scaling behavior that applies to all NodePools associated with the control plane.

etcd EtcdSpec

Etcd specifies configuration for the control plane etcd cluster. The default ManagementType is Managed. Once set, the ManagementType cannot be changed.

services []ServicePublishingStrategyMapping

Services specifies how individual control plane services are published from the hosting cluster of the control plane.

If a given service is not present in this list, it will be exposed publicly by default.

pullSecret Kubernetes core/v1.LocalObjectReference

PullSecret references a pull secret to be injected into the container runtime of all cluster nodes. The secret must have a key named “.dockerconfigjson” whose value is the pull secret JSON.

sshKey Kubernetes core/v1.LocalObjectReference

SSHKey references an SSH key to be injected into all cluster node sshd servers. The secret must have a single key “id_rsa.pub” whose value is the public part of an SSH key.

issuerURL string (Optional)

IssuerURL is an OIDC issuer URL which is used as the issuer in all ServiceAccount tokens generated by the control plane API server. The default value is kubernetes.default.svc, which only works for in-cluster validation.

serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster. When specifying a service account signing key, a IssuerURL must also be specified.

configuration ClusterConfiguration (Optional)

Configuration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has mTLS the kubeconfig will contain client keys. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

This field is currently only supported on the IBMCloud platform.

imageContentSources []ImageContentSource (Optional)

ImageContentSources specifies image mirrors that can be used by cluster nodes to pull content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle is a reference to a ConfigMap containing a PEM-encoded X.509 certificate bundle that will be added to the hosted controlplane and nodes

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption specifies a Kubernetes secret encryption strategy for the control plane.

fips bool (Optional)

FIPS indicates whether this cluster’s nodes will be running in FIPS mode. If set to true, the control plane’s ignition server will be configured to expect that nodes joining the cluster will be FIPS-enabled.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

Value must be one of: "guest", "management"

status HostedClusterStatus

Status is the latest observed status of the HostedCluster.

NodePool

NodePool is a scalable set of worker nodes attached to a HostedCluster. NodePool machine architectures are uniform within a given pool, and are independent of the control plane’s underlying machine architecture.

Field Description
apiVersion string hypershift.openshift.io/v1alpha1
kind string NodePool
metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.
spec NodePoolSpec

Spec is the desired behavior of the NodePool.



clusterName string

ClusterName is the name of the HostedCluster this NodePool belongs to.

TODO(dan): Should this be a LocalObjectReference?

release Release

Release specifies the OCP release used for the NodePool. This informs the ignition configuration for machines, as well as other platform specific machine properties (e.g. an AMI on the AWS platform).

platform NodePoolPlatform

Platform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

nodeCount int32 (Optional)

Deprecated: Use Replicas instead. NodeCount will be dropped in the next api release.

replicas int32 (Optional)

Replicas is the desired number of nodes the pool should maintain. If unset, the default value is 0.

management NodePoolManagement

Management specifies behavior for managing nodes in the pool, such as upgrade strategies and auto-repair behaviors.

autoScaling NodePoolAutoScaling (Optional)

Autoscaling specifies auto-scaling behavior for the NodePool.

config []Kubernetes core/v1.LocalObjectReference

Config is a list of references to ConfigMaps containing serialized MachineConfig resources to be injected into the ignition configurations of nodes in the NodePool. The MachineConfig API schema is defined here:

https://github.com/openshift/machine-config-operator/blob/master/pkg/apis/machineconfiguration.openshift.io/v1/types.go#L172

Each ConfigMap must have a single key named “config” whose value is the JSON or YAML of a serialized MachineConfig.

nodeDrainTimeout Kubernetes meta/v1.Duration (Optional)

NodeDrainTimeout is the total amount of time that the controller will spend on draining a node. The default value is 0, meaning that the node can be drained without any time limitations. NOTE: NodeDrainTimeout is different from kubectl drain --timeout TODO (alberto): Today changing this field will trigger a recreate rolling update, which kind of defeats the purpose of the change. In future we plan to propagate this field in-place. https://github.com/kubernetes-sigs/cluster-api/issues/5880

status NodePoolStatus

Status is the latest observed status of the NodePool.

AESCBCSpec

(Appears on: SecretEncryptionSpec)

AESCBCSpec defines metadata about the AESCBC secret encryption strategy

Field Description
activeKey Kubernetes core/v1.LocalObjectReference

ActiveKey defines the active key used to encrypt new secrets

backupKey Kubernetes core/v1.LocalObjectReference (Optional)

BackupKey defines the old key during the rotation process so previously created secrets can continue to be decrypted until they are all re-encrypted with the active key.

APIServerNetworking

(Appears on: ClusterNetworking)

APIServerNetworking specifies how the APIServer is exposed inside a cluster node.

Field Description
advertiseAddress string

AdvertiseAddress is the address that nodes will use to talk to the API server. This is an address associated with the loopback adapter of each node. If not specified, 172.20.0.1 is used.

port int32

Port is the port at which the APIServer is exposed inside a node. Other pods using host networking cannot listen on this port. If not specified, 6443 is used.

AWSCloudProviderConfig

(Appears on: AWSPlatformSpec)

AWSCloudProviderConfig specifies AWS networking configuration.

Field Description
subnet AWSResourceReference (Optional)

Subnet is the subnet to use for control plane cloud resources.

zone string (Optional)

Zone is the availability zone where control plane cloud resources are created.

vpc string

VPC is the VPC to use for control plane cloud resources.

AWSEndpointAccessType

(Appears on: AWSPlatformSpec)

AWSEndpointAccessType specifies the publishing scope of cluster endpoints.

Value Description

"Private"

Private endpoint access allows only private API server access and private node communication with the control plane.

"Public"

Public endpoint access allows public API server access and public node communication with the control plane.

"PublicAndPrivate"

PublicAndPrivate endpoint access allows public API server access and private node communication with the control plane.

AWSKMSAuthSpec

(Appears on: AWSKMSSpec)

AWSKMSAuthSpec defines metadata about the management of credentials used to interact with AWS KMS

Field Description
credentials Kubernetes core/v1.LocalObjectReference

Credentials contains the name of the secret that holds the aws credentials that can be used to make the necessary KMS calls. It should at key AWSCredentialsFileSecretKey contain the aws credentials file that can be used to configure AWS SDKs

AWSKMSKeyEntry

(Appears on: AWSKMSSpec)

AWSKMSKeyEntry defines metadata to locate the encryption key in AWS

Field Description
arn string

ARN is the Amazon Resource Name for the encryption key

AWSKMSSpec

(Appears on: KMSSpec)

AWSKMSSpec defines metadata about the configuration of the AWS KMS Secret Encryption provider

Field Description
region string

Region contains the AWS region

activeKey AWSKMSKeyEntry

ActiveKey defines the active key used to encrypt new secrets

backupKey AWSKMSKeyEntry (Optional)

BackupKey defines the old key during the rotation process so previously created secrets can continue to be decrypted until they are all re-encrypted with the active key.

auth AWSKMSAuthSpec

Auth defines metadata about the management of credentials used to interact with AWS KMS

AWSNodePoolPlatform

(Appears on: NodePoolPlatform)

AWSNodePoolPlatform specifies the configuration of a NodePool when operating on AWS.

Field Description
instanceType string

InstanceType is an ec2 instance type for node instances (e.g. m5.large).

instanceProfile string

InstanceProfile is the AWS EC2 instance profile, which is a container for an IAM role that the EC2 instance uses.

subnet AWSResourceReference (Optional)

Subnet is the subnet to use for node instances.

ami string (Optional)

AMI is the image id to use for node instances. If unspecified, the default is chosen based on the NodePool release payload image.

securityGroups []AWSResourceReference (Optional)

SecurityGroups is an optional set of security groups to associate with node instances.

rootVolume Volume (Optional)

RootVolume specifies configuration for the root volume of node instances.

resourceTags []AWSResourceTag (Optional)

ResourceTags is an optional list of additional tags to apply to AWS node instances.

These will be merged with HostedCluster scoped tags, and HostedCluster tags take precedence in case of conflicts.

See https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html for information on tagging AWS resources. AWS supports a maximum of 50 tags per resource. OpenShift reserves 25 tags for its use, leaving 25 tags available for the user.

AWSPlatformSpec

(Appears on: PlatformSpec)

AWSPlatformSpec specifies configuration for clusters running on Amazon Web Services.

Field Description
region string

Region is the AWS region in which the cluster resides. This configures the OCP control plane cloud integrations, and is used by NodePool to resolve the correct boot AMI for a given release.

cloudProviderConfig AWSCloudProviderConfig (Optional)

CloudProviderConfig specifies AWS networking configuration for the control plane.

TODO(dan): should this be named AWSNetworkConfig?

serviceEndpoints []AWSServiceEndpoint (Optional)

ServiceEndpoints specifies optional custom endpoints which will override the default service endpoint of specific AWS Services.

There must be only one ServiceEndpoint for a given service name.

roles []AWSRoleCredentials

Roles must contain exactly 4 entries representing the locators for roles supporting the following OCP services:

  • openshift-ingress-operator/cloud-credentials
  • openshift-image-registry/installer-cloud-credentials
  • openshift-cluster-csi-drivers/ebs-cloud-credentials
  • cloud-network-config-controller/cloud-credentials

Each role has unique permission requirements whose documentation is TBD.

TODO(dan): revisit this field; it’s really 3 required fields with specific content requirements

kubeCloudControllerCreds Kubernetes core/v1.LocalObjectReference

KubeCloudControllerCreds is a reference to a secret containing cloud credentials with permissions matching the cloud controller policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file.

TODO(dan): document the “cloud controller policy”

nodePoolManagementCreds Kubernetes core/v1.LocalObjectReference

NodePoolManagementCreds is a reference to a secret containing cloud credentials with permissions matching the node pool management policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file.

TODO(dan): document the “node pool management policy”

controlPlaneOperatorCreds Kubernetes core/v1.LocalObjectReference

ControlPlaneOperatorCreds is a reference to a secret containing cloud credentials with permissions matching the control-plane-operator policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file.

TODO(dan): document the “control plane operator policy”

resourceTags []AWSResourceTag (Optional)

ResourceTags is a list of additional tags to apply to AWS resources created for the cluster. See https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html for information on tagging AWS resources. AWS supports a maximum of 50 tags per resource. OpenShift reserves 25 tags for its use, leaving 25 tags available for the user.

endpointAccess AWSEndpointAccessType (Optional)

EndpointAccess specifies the publishing scope of cluster endpoints. The default is Public.

Value must be one of: "Private", "Public", "PublicAndPrivate"

AWSResourceReference

(Appears on: AWSCloudProviderConfig, AWSNodePoolPlatform)

AWSResourceReference is a reference to a specific AWS resource by ID, ARN, or filters. Only one of ID, ARN or Filters may be specified. Specifying more than one will result in a validation error.

Field Description
id string (Optional)

ID of resource

arn string (Optional)

ARN of resource

filters []Filter (Optional)

Filters is a set of key/value pairs used to identify a resource They are applied according to the rules defined by the AWS API: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Filtering.html

AWSResourceTag

(Appears on: AWSNodePoolPlatform, AWSPlatformSpec)

AWSResourceTag is a tag to apply to AWS resources created for the cluster.

Field Description
key string

Key is the key of the tag.

value string

Value is the value of the tag.

Some AWS service do not support empty values. Since tags are added to resources in many services, the length of the tag value must meet the requirements of all services.

AWSRoleCredentials

(Appears on: AWSPlatformSpec)

Field Description
arn string
namespace string
name string

AWSServiceEndpoint

(Appears on: AWSPlatformSpec)

AWSServiceEndpoint stores the configuration for services to override existing defaults of AWS Services.

Field Description
name string

Name is the name of the AWS service. This must be provided and cannot be empty.

url string

URL is fully qualified URI with scheme https, that overrides the default generated endpoint for a client. This must be provided and cannot be empty.

AgentNodePoolPlatform

(Appears on: NodePoolPlatform)

AgentNodePoolPlatform specifies the configuration of a NodePool when operating on the Agent platform.

Field Description
agentLabelSelector Kubernetes meta/v1.LabelSelector (Optional)

AgentLabelSelector contains labels that must be set on an Agent in order to be selected for a Machine.

AgentPlatformSpec

(Appears on: PlatformSpec)

AgentPlatformSpec specifies configuration for agent-based installations.

Field Description
agentNamespace string

AgentNamespace is the namespace where to search for Agents for this cluster

AvailabilityPolicy

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

AvailabilityPolicy specifies a high level availability policy for components.

Value Description

"HighlyAvailable"

HighlyAvailable means components should be resilient to problems across fault boundaries as defined by the component to which the policy is attached. This usually means running critical workloads with 3 replicas and with little or no toleration of disruption of the component.

"SingleReplica"

SingleReplica means components are not expected to be resilient to problems across most fault boundaries associated with high availability. This usually means running critical workloads with just 1 replica and with toleration of full disruption of the component.

AzureNodePoolPlatform

(Appears on: NodePoolPlatform)

Field Description
vmsize string
imageID string (Optional)

ImageID is the id of the image to boot from. If unset, the default image at the location below will be used: subscription/$subscriptionID/resourceGroups/$resourceGroupName/providers/Microsoft.Compute/images/rhcos.x86_64.vhd

diskSizeGB int32 (Optional)
availabilityZone string (Optional)

AvailabilityZone of the nodepool. Must not be specified for clusters in a location that does not support AvailabilityZone.

AzurePlatformSpec

(Appears on: PlatformSpec)

Field Description
credentials Kubernetes core/v1.LocalObjectReference
location string
resourceGroup string
vnetName string
vnetID string
subnetName string
subscriptionID string
machineIdentityID string
securityGroupName string

ClusterAutoscaling

(Appears on: HostedClusterSpec)

ClusterAutoscaling specifies auto-scaling behavior that applies to all NodePools associated with a control plane.

Field Description
maxNodesTotal int32

MaxNodesTotal is the maximum allowable number of nodes across all NodePools for a HostedCluster. The autoscaler will not grow the cluster beyond this number.

maxPodGracePeriod int32

MaxPodGracePeriod is the maximum seconds to wait for graceful pod termination before scaling down a NodePool. The default is 600 seconds.

maxNodeProvisionTime string

MaxNodeProvisionTime is the maximum time to wait for node provisioning before considering the provisioning to be unsuccessful, expressed as a Go duration string. The default is 15 minutes.

podPriorityThreshold int32 (Optional)

PodPriorityThreshold enables users to schedule “best-effort” pods, which shouldn’t trigger autoscaler actions, but only run when there are spare resources available. The default is -10.

See the following for more details: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-does-cluster-autoscaler-work-with-pod-priority-and-preemption

ClusterConfiguration

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ClusterConfiguration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

The API for individual configuration items is at: https://docs.openshift.com/container-platform/4.7/rest_api/config_apis/config-apis-index.html

Field Description
secretRefs []Kubernetes core/v1.LocalObjectReference (Optional)

SecretRefs holds references to any secrets referenced by configuration entries. Entries can reference the secrets using local object references.

configMapRefs []Kubernetes core/v1.LocalObjectReference (Optional)

ConfigMapRefs holds references to any configmaps referenced by configuration entries. Entries can reference the configmaps using local object references.

items []k8s.io/apimachinery/pkg/runtime.RawExtension (Optional)

Items embeds the serialized configuration resources.

ClusterNetworking

(Appears on: HostedClusterSpec)

ClusterNetworking specifies network configuration for a cluster.

Field Description
serviceCIDR string

ServiceCIDR is…

TODO(dan): document it

podCIDR string

PodCIDR is…

TODO(dan): document it

machineCIDR string

MachineCIDR is…

TODO(dan): document it

networkType NetworkType

NetworkType specifies the SDN provider used for cluster networking.

Value must be one of: "Calico", "OVNKubernetes", "OpenShiftSDN", "Other"

apiServer APIServerNetworking

APIServer contains advanced network settings for the API server that affect how the APIServer is exposed inside a cluster node.

ClusterVersionStatus

(Appears on: HostedClusterStatus)

ClusterVersionStatus reports the status of the cluster versioning, including any upgrades that are in progress. The current field will be set to whichever version the cluster is reconciling to, and the conditions array will report whether the update succeeded, is in progress, or is failing.

Field Description
desired Release

desired is the version that the cluster is reconciling towards. If the cluster is not yet fully initialized desired will be set with the information available, which may be an image or a tag.

history []github.com/openshift/api/config/v1.UpdateHistory (Optional)

history contains a list of the most recent versions applied to the cluster. This value may be empty during cluster startup, and then will be updated when a new update is being applied. The newest update is first in the list and it is ordered by recency. Updates in the history have state Completed if the rollout completed - if an update was failing or halfway applied the state will be Partial. Only a limited amount of update history is preserved.

observedGeneration int64

observedGeneration reports which version of the spec is being synced. If this value is not equal to metadata.generation, then the desired and conditions fields may represent a previous version.

ConditionType

Value Description

"EndpointAvailable"

AWSEndpointServiceAvailable indicates whether the AWS Endpoint has been created in the guest VPC

"EndpointServiceAvailable"

AWSEndpointServiceAvailable indicates whether the AWS Endpoint Service has been created for the specified NLB in the management VPC

"ClusterVersionFailing"

"ClusterVersionSucceeding"

ClusterVersionSucceeding indicates the current status of the desired release version of the HostedCluster as indicated by the Failing condition in the underlying cluster’s ClusterVersion.

"EtcdAvailable"

"Available"

HostedClusterAvailable indicates whether the HostedCluster has a healthy control plane.

"Available"

"IgnitionEndpointAvailable"

IgnitionEndpointAvailable indicates whether the ignition server for the HostedCluster is available to handle ignition requests.

"InfrastructureReady"

"KubeAPIServerAvailable"

"OIDCConfigurationInvalid"

OIDCConfigurationInvalid indicates if an AWS cluster’s OIDC condition is detected as invalid.

"ReconciliationPaused"

ReconciliationPaused indicates if reconciliation of the hostedcluster is paused.

"SupportedHostedCluster"

SupportedHostedCluster indicates whether a HostedCluster is supported by the current configuration of the hypershift-operator. e.g. If HostedCluster requests endpointAcess Private but the hypershift-operator is running on a management cluster outside AWS or is not configured with AWS credentials, the HostedCluster is not supported.

"UnmanagedEtcdAvailable"

UnmanagedEtcdAvailable indicates whether a user-managed etcd cluster is healthy.

"ValidConfiguration"

ValidHostedClusterConfiguration indicates (if status is true) that the ClusterConfiguration specified for the HostedCluster is valid.

"ValidHostedControlPlaneConfiguration"

DNSSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

DNSSpec specifies the DNS configuration in the cluster.

Field Description
baseDomain string

BaseDomain is the base domain of the cluster.

publicZoneID string (Optional)

PublicZoneID is the Hosted Zone ID where all the DNS records that are publicly accessible to the internet exist.

privateZoneID string (Optional)

PrivateZoneID is the Hosted Zone ID where all the DNS records that are only available internally to the cluster exist.

EtcdManagementType

(Appears on: EtcdSpec)

EtcdManagementType is a enum specifying the strategy for managing the cluster’s etcd instance

Value Description

"Managed"

Managed means HyperShift should provision and operator the etcd cluster automatically.

"Unmanaged"

Unmanaged means HyperShift will not provision or manage the etcd cluster, and the user is responsible for doing so.

EtcdSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

EtcdSpec specifies configuration for a control plane etcd cluster.

Field Description
managementType EtcdManagementType

ManagementType defines how the etcd cluster is managed.

Value must be one of: "Managed", "Unmanaged"

managed ManagedEtcdSpec (Optional)

Managed specifies the behavior of an etcd cluster managed by HyperShift.

unmanaged UnmanagedEtcdSpec (Optional)

Unmanaged specifies configuration which enables the control plane to integrate with an eternally managed etcd cluster.

EtcdTLSConfig

(Appears on: UnmanagedEtcdSpec)

EtcdTLSConfig specifies TLS configuration for HTTPS etcd client endpoints.

Field Description
clientSecret Kubernetes core/v1.LocalObjectReference

ClientSecret refers to a secret for client mTLS authentication with the etcd cluster. It may have the following key/value pairs:

etcd-client-ca.crt: Certificate Authority value
etcd-client.crt: Client certificate value
etcd-client.key: Client certificate key value

Filter

(Appears on: AWSResourceReference)

Filter is a filter used to identify an AWS resource

Field Description
name string

Name of the filter. Filter names are case-sensitive.

values []string

Values includes one or more filter values. Filter values are case-sensitive.

HostedClusterSpec

(Appears on: HostedCluster)

HostedClusterSpec is the desired behavior of a HostedCluster.

Field Description
release Release

Release specifies the desired OCP release payload for the hosted cluster.

Updating this field will trigger a rollout of the control plane. The behavior of the rollout will be driven by the ControllerAvailabilityPolicy and InfrastructureAvailabilityPolicy.

clusterID string (Optional)

ClusterID uniquely identifies this cluster. This is expected to be an RFC4122 UUID value (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx in hexadecimal values). As with a Kubernetes metadata.uid, this ID uniquely identifies this cluster in space and time. This value identifies the cluster in metrics pushed to telemetry and metrics produced by the control plane operators. If a value is not specified, an ID is generated. After initial creation, the value is immutable.

infraID string (Optional)

InfraID is a globally unique identifier for the cluster. This identifier will be used to associate various cloud resources with the HostedCluster and its associated NodePools.

platform PlatformSpec

Platform specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies the availability policy applied to critical control plane components. The default value is SingleReplica.

Value must be one of: "HighlyAvailable", "SingleReplica"

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies the availability policy applied to infrastructure services which run on cluster nodes. The default value is HighlyAvailable.

Value must be one of: "HighlyAvailable", "SingleReplica"

dns DNSSpec

DNS specifies DNS configuration for the cluster.

networking ClusterNetworking

Networking specifies network configuration for the cluster.

autoscaling ClusterAutoscaling (Optional)

Autoscaling specifies auto-scaling behavior that applies to all NodePools associated with the control plane.

etcd EtcdSpec

Etcd specifies configuration for the control plane etcd cluster. The default ManagementType is Managed. Once set, the ManagementType cannot be changed.

services []ServicePublishingStrategyMapping

Services specifies how individual control plane services are published from the hosting cluster of the control plane.

If a given service is not present in this list, it will be exposed publicly by default.

pullSecret Kubernetes core/v1.LocalObjectReference

PullSecret references a pull secret to be injected into the container runtime of all cluster nodes. The secret must have a key named “.dockerconfigjson” whose value is the pull secret JSON.

sshKey Kubernetes core/v1.LocalObjectReference

SSHKey references an SSH key to be injected into all cluster node sshd servers. The secret must have a single key “id_rsa.pub” whose value is the public part of an SSH key.

issuerURL string (Optional)

IssuerURL is an OIDC issuer URL which is used as the issuer in all ServiceAccount tokens generated by the control plane API server. The default value is kubernetes.default.svc, which only works for in-cluster validation.

serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster. When specifying a service account signing key, a IssuerURL must also be specified.

configuration ClusterConfiguration (Optional)

Configuration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has mTLS the kubeconfig will contain client keys. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

This field is currently only supported on the IBMCloud platform.

imageContentSources []ImageContentSource (Optional)

ImageContentSources specifies image mirrors that can be used by cluster nodes to pull content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle is a reference to a ConfigMap containing a PEM-encoded X.509 certificate bundle that will be added to the hosted controlplane and nodes

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption specifies a Kubernetes secret encryption strategy for the control plane.

fips bool (Optional)

FIPS indicates whether this cluster’s nodes will be running in FIPS mode. If set to true, the control plane’s ignition server will be configured to expect that nodes joining the cluster will be FIPS-enabled.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

Value must be one of: "guest", "management"

HostedClusterStatus

(Appears on: HostedCluster)

HostedClusterStatus is the latest observed status of a HostedCluster.

Field Description
version ClusterVersionStatus (Optional)

Version is the status of the release version applied to the HostedCluster.

kubeconfig Kubernetes core/v1.LocalObjectReference (Optional)

KubeConfig is a reference to the secret containing the default kubeconfig for the cluster.

kubeadminPassword Kubernetes core/v1.LocalObjectReference (Optional)

KubeadminPassword is a reference to the secret that contains the initial kubeadmin user password for the guest cluster.

ignitionEndpoint string (Optional)

IgnitionEndpoint is the endpoint injected in the ign config userdata. It exposes the config for instances to become kubernetes nodes.

oauthCallbackURLTemplate string

OAuthCallbackURLTemplate contains a template for the URL to use as a callback for identity providers. The [identity-provider-name] placeholder must be replaced with the name of an identity provider defined on the HostedCluster. This is populated after the infrastructure is ready.

conditions []Kubernetes meta/v1.Condition

Conditions represents the latest available observations of a control plane’s current state.

HostedControlPlaneSpec

HostedControlPlaneSpec defines the desired state of HostedControlPlane

Field Description
releaseImage string
pullSecret Kubernetes core/v1.LocalObjectReference
issuerURL string
serviceCIDR string
podCIDR string
machineCIDR string
networkType NetworkType

NetworkType specifies the SDN provider used for cluster networking.

Value must be one of: "Calico", "OVNKubernetes", "OpenShiftSDN", "Other"

sshKey Kubernetes core/v1.LocalObjectReference
clusterID string (Optional)

ClusterID is the unique id that identifies the cluster externally. Making it optional here allows us to keep compatibility with previous versions of the control-plane-operator that have no knowledge of this field.

infraID string
platform PlatformSpec
dns DNSSpec
serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster.

apiPort int32 (Optional)

APIPort is the port at which the APIServer listens inside a worker

apiAdvertiseAddress string (Optional)

APIAdvertiseAddress is the address at which the APIServer listens inside a worker.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies whether to run control plane controllers in HA mode Defaults to SingleReplica when not set

Value must be one of: "HighlyAvailable", "SingleReplica"

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies whether to run infrastructure services that run on the guest cluster nodes in HA mode Defaults to HighlyAvailable when not set

Value must be one of: "HighlyAvailable", "SingleReplica"

fips bool (Optional)

FIPS specifies if the nodes for the cluster will be running in FIPS mode

kubeconfig KubeconfigSecretRef (Optional)

KubeConfig specifies the name and key for the kubeconfig secret

services []ServicePublishingStrategyMapping

Services defines metadata about how control plane services are published in the management cluster.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has MTLS the kubeconfig will contain client keys. This is currently only supported in IBM Cloud. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

etcd EtcdSpec

Etcd contains metadata about the etcd cluster the hypershift managed Openshift control plane components use to store data.

configuration ClusterConfiguration

Configuration embeds resources that correspond to the openshift configuration API: https://docs.openshift.com/container-platform/4.7/rest_api/config_apis/config-apis-index.html

imageContentSources []ImageContentSource (Optional)

ImageContentSources lists sources/repositories for the release-image content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle references a ConfigMap containing a PEM-encoded X.509 certificate bundle

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption contains metadata about the kubernetes secret encryption strategy being used for the cluster when applicable.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

Value must be one of: "guest", "management"

HostedControlPlaneStatus

HostedControlPlaneStatus defines the observed state of HostedControlPlane

Field Description
ready bool

Ready denotes that the HostedControlPlane API Server is ready to receive requests This satisfies CAPI contract https://github.com/kubernetes-sigs/cluster-api/blob/cd3a694deac89d5ebeb888307deaa61487207aa0/controllers/cluster_controller_phases.go#L226-L230

initialized bool

Initialized denotes whether or not the control plane has provided a kubeadm-config. Once this condition is marked true, its value is never changed. See the Ready condition for an indication of the current readiness of the cluster’s control plane. This satisfies CAPI contract https://github.com/kubernetes-sigs/cluster-api/blob/cd3a694deac89d5ebeb888307deaa61487207aa0/controllers/cluster_controller_phases.go#L238-L252

externalManagedControlPlane bool

ExternalManagedControlPlane indicates to cluster-api that the control plane is managed by an external service. https://github.com/kubernetes-sigs/cluster-api/blob/65e5385bffd71bf4aad3cf34a537f11b217c7fab/controllers/machine_controller.go#L468

controlPlaneEndpoint APIEndpoint

ControlPlaneEndpoint contains the endpoint information by which external clients can access the control plane. This is populated after the infrastructure is ready.

oauthCallbackURLTemplate string

OAuthCallbackURLTemplate contains a template for the URL to use as a callback for identity providers. The [identity-provider-name] placeholder must be replaced with the name of an identity provider defined on the HostedCluster. This is populated after the infrastructure is ready.

version string

Version is the semantic version of the release applied by the hosted control plane operator

releaseImage string

ReleaseImage is the release image applied to the hosted control plane.

lastReleaseImageTransitionTime Kubernetes meta/v1.Time

lastReleaseImageTransitionTime is the time of the last update to the current releaseImage property.

kubeConfig KubeconfigSecretRef

KubeConfig is a reference to the secret containing the default kubeconfig for this control plane.

kubeadminPassword Kubernetes core/v1.LocalObjectReference (Optional)

KubeadminPassword is a reference to the secret containing the initial kubeadmin password for the guest cluster.

conditions []Kubernetes meta/v1.Condition

Condition contains details for one aspect of the current state of the HostedControlPlane. Current condition types are: “Available”

IBMCloudKMSAuthSpec

(Appears on: IBMCloudKMSSpec)

IBMCloudKMSAuthSpec defines metadata for how authentication is done with IBM Cloud KMS

Field Description
type IBMCloudKMSAuthType

Type defines the IBM Cloud KMS authentication strategy

Value must be one of: "Managed", "Unmanaged"

unmanaged IBMCloudKMSUnmanagedAuthSpec (Optional)

Unmanaged defines the auth metadata the customer provides to interact with IBM Cloud KMS

managed IBMCloudKMSManagedAuthSpec (Optional)

Managed defines metadata around the service to service authentication strategy for the IBM Cloud KMS system (all provider managed).

IBMCloudKMSAuthType

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSAuthType defines the IBM Cloud KMS authentication strategy

Value Description

"Managed"

IBMCloudKMSManagedAuth defines the KMS authentication strategy where the IKS/ROKS platform uses service to service auth to call IBM Cloud KMS APIs (no customer credentials requried)

"Unmanaged"

IBMCloudKMSUnmanagedAuth defines the KMS authentication strategy where a customer supplies IBM Cloud authentication to interact with IBM Cloud KMS APIs

IBMCloudKMSKeyEntry

(Appears on: IBMCloudKMSSpec)

IBMCloudKMSKeyEntry defines metadata for an IBM Cloud KMS encryption key

Field Description
crkID string

CRKID is the customer rook key id

instanceID string

InstanceID is the id for the key protect instance

correlationID string

CorrelationID is an identifier used to track all api call usage from hypershift

url string

URL is the url to call key protect apis over

keyVersion int

KeyVersion is a unique number associated with the key. The number increments whenever a new key is enabled for data encryption.

IBMCloudKMSManagedAuthSpec

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSManagedAuthSpec defines metadata around the service to service authentication strategy for the IBM Cloud KMS system (all provider managed).

IBMCloudKMSSpec

(Appears on: KMSSpec)

IBMCloudKMSSpec defines metadata for the IBM Cloud KMS encryption strategy

Field Description
region string

Region is the IBM Cloud region

auth IBMCloudKMSAuthSpec

Auth defines metadata for how authentication is done with IBM Cloud KMS

keyList []IBMCloudKMSKeyEntry

KeyList defines the list of keys used for data encryption

IBMCloudKMSUnmanagedAuthSpec

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSUnmanagedAuthSpec defines the auth metadata the customer provides to interact with IBM Cloud KMS

Field Description
credentials Kubernetes core/v1.LocalObjectReference

Credentials should reference a secret with a key field of IBMCloudIAMAPIKeySecretKey that contains a apikey to call IBM Cloud KMS APIs

IBMCloudPlatformSpec

(Appears on: NodePoolPlatform, PlatformSpec)

IBMCloudPlatformSpec defines IBMCloud specific settings for components

Field Description
providerType github.com/openshift/api/config/v1.IBMCloudProviderType

ProviderType is a specific supported infrastructure provider within IBM Cloud.

ImageContentSource

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ImageContentSource specifies image mirrors that can be used by cluster nodes to pull content. For cluster workloads, if a container image registry host of the pullspec matches Source then one of the Mirrors are substituted as hosts in the pullspec and tried in order to fetch the image.

Field Description
source string

Source is the repository that users refer to, e.g. in image pull specifications.

mirrors []string (Optional)

Mirrors are one or more repositories that may also contain the same images.

InPlaceUpgrade

(Appears on: NodePoolManagement)

InPlaceUpgrade specifies an upgrade strategy which upgrades nodes in-place without any new nodes being created or any old nodes being deleted.

KMSProvider

(Appears on: KMSSpec)

KMSProvider defines the supported KMS providers

Value Description

"AWS"

"IBMCloud"

KMSSpec

(Appears on: SecretEncryptionSpec)

KMSSpec defines metadata about the kms secret encryption strategy

Field Description
provider KMSProvider

Provider defines the KMS provider

Value must be one of: "AWS", "IBMCloud"

ibmcloud IBMCloudKMSSpec (Optional)

IBMCloud defines metadata for the IBM Cloud KMS encryption strategy

aws AWSKMSSpec (Optional)

AWS defines metadata about the configuration of the AWS KMS Secret Encryption provider

KubevirtCompute

(Appears on: KubevirtNodePoolPlatform)

KubevirtCompute contains values associated with the virtual compute hardware requested for the VM.

Field Description
memory k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Memory represents how much guest memory the VM should have

cores uint32 (Optional)

Cores represents how many cores the guest VM should have

KubevirtDiskImage

(Appears on: KubevirtRootVolume)

KubevirtDiskImage contains values representing where the rhcos image is located

Field Description
containerDiskImage string (Optional)

ContainerDiskImage is a string representing the container image that holds the root disk

KubevirtNodePoolPlatform

(Appears on: NodePoolPlatform)

KubevirtNodePoolPlatform specifies the configuration of a NodePool when operating on KubeVirt platform.

Field Description
rootVolume KubevirtRootVolume

RootVolume represents values associated with the VM volume that will host rhcos

compute KubevirtCompute (Optional)

Compute contains values representing the virtual hardware requested for the VM

KubevirtPersistentVolume

(Appears on: KubevirtVolume)

KubevirtPersistentVolume containes the values involved with provisioning persistent storage for a KubeVirt VM.

Field Description
size k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Size is the size of the persistent storage volume

storageClass string (Optional)

StorageClass is the storageClass used for the underlying PVC that hosts the volume

KubevirtRootVolume

(Appears on: KubevirtNodePoolPlatform)

KubevirtRootVolume represents the volume that the rhcos disk will be stored and run from.

Field Description
diskImage KubevirtDiskImage (Optional)

Image represents what rhcos image to use for the node pool

KubevirtVolume KubevirtVolume

(Members of KubevirtVolume are embedded into this type.)

KubevirtVolume represents of type of storage to run the image on

KubevirtVolume

(Appears on: KubevirtRootVolume)

KubevirtVolume represents what kind of storage to use for a KubeVirt VM volume

Field Description
type KubevirtVolumeType (Optional)

Type represents the type of storage to associate with the kubevirt VMs.

Value must be one of: "Persistent"

persistent KubevirtPersistentVolume (Optional)

Persistent volume type means the VM’s storage is backed by a PVC VMs that use persistent volumes can survive disruption events like restart and eviction This is the default type used when no storage type is defined.

KubevirtVolumeType

(Appears on: KubevirtVolume)

KubevirtVolumeType is a specific supported KubeVirt volumes

Value Description

"Persistent"

KubevirtVolumeTypePersistent represents persistent volume for kubevirt VMs

LoadBalancerPublishingStrategy

(Appears on: ServicePublishingStrategy)

LoadBalancerPublishingStrategy specifies setting used to expose a service as a LoadBalancer.

Field Description
hostname string (Optional)

Hostname is the name of the DNS record that will be created pointing to the LoadBalancer.

ManagedEtcdSpec

(Appears on: EtcdSpec)

ManagedEtcdSpec specifies the behavior of an etcd cluster managed by HyperShift.

Field Description
storage ManagedEtcdStorageSpec

Storage specifies how etcd data is persisted.

ManagedEtcdStorageSpec

(Appears on: ManagedEtcdSpec)

ManagedEtcdStorageSpec describes the storage configuration for etcd data.

Field Description
type ManagedEtcdStorageType

Type is the kind of persistent storage implementation to use for etcd.

Value must be one of: "PersistentVolume"

persistentVolume PersistentVolumeEtcdStorageSpec (Optional)

PersistentVolume is the configuration for PersistentVolume etcd storage. With this implementation, a PersistentVolume will be allocated for every etcd member (either 1 or 3 depending on the HostedCluster control plane availability configuration).

ManagedEtcdStorageType

(Appears on: ManagedEtcdStorageSpec)

ManagedEtcdStorageType is a storage type for an etcd cluster.

Value Description

"PersistentVolume"

PersistentVolumeEtcdStorage uses PersistentVolumes for etcd storage.

NetworkType

(Appears on: ClusterNetworking, HostedControlPlaneSpec)

NetworkType specifies the SDN provider used for cluster networking.

Value Description

"Calico"

Calico specifies Calico as the SDN provider

"OVNKubernetes"

OVNKubernetes specifies OVN as the SDN provider

"OpenShiftSDN"

OpenShiftSDN specifies OpenshiftSDN as the SDN provider

"Other"

Other specifies an undefined SDN provider

NodePoolAutoScaling

(Appears on: NodePoolSpec)

NodePoolAutoScaling specifies auto-scaling behavior for a NodePool.

Field Description
min int32

Min is the minimum number of nodes to maintain in the pool. Must be >= 1.

max int32

Max is the maximum number of nodes allowed in the pool. Must be >= 1.

NodePoolCondition

(Appears on: NodePoolStatus)

We define our own condition type since metav1.Condition has validation for Reason that might be broken by what we bubble up from CAPI. NodePoolCondition defines an observation of NodePool resource operational state.

Field Description
type string

Type of condition in CamelCase or in foo.example.com/CamelCase. Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important.

status Kubernetes core/v1.ConditionStatus

Status of the condition, one of True, False, Unknown.

severity string (Optional)

Severity provides an explicit classification of Reason code, so the users or machines can immediately understand the current situation and act accordingly. The Severity field MUST be set only when Status=False.

lastTransitionTime Kubernetes meta/v1.Time

Last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.

reason string (Optional)

The reason for the condition’s last transition in CamelCase. The specific API may choose whether or not this field is considered a guaranteed API. This field may not be empty.

message string (Optional)

A human readable message indicating details about the transition. This field may be empty.

observedGeneration int64

NodePoolManagement

(Appears on: NodePoolSpec)

NodePoolManagement specifies behavior for managing nodes in a NodePool, such as upgrade strategies and auto-repair behaviors.

Field Description
upgradeType UpgradeType

UpgradeType specifies the type of strategy for handling upgrades.

Value must be one of: "InPlace", "Replace"

replace ReplaceUpgrade

Replace is the configuration for rolling upgrades.

inPlace InPlaceUpgrade

InPlace is the configuration for in-place upgrades.

autoRepair bool (Optional)

AutoRepair specifies whether health checks should be enabled for machines in the NodePool. The default is false.

NodePoolPlatform

(Appears on: NodePoolSpec)

NodePoolPlatform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

Field Description
type PlatformType

Type specifies the platform name.

Value must be one of: "AWS", "Agent", "Azure", "IBMCloud", "KubeVirt", "None", "PowerVS"

aws AWSNodePoolPlatform (Optional)

AWS specifies the configuration used when operating on AWS.

ibmcloud IBMCloudPlatformSpec

IBMCloud defines IBMCloud specific settings for components

kubevirt KubevirtNodePoolPlatform (Optional)

Kubevirt specifies the configuration used when operating on KubeVirt platform.

agent AgentNodePoolPlatform (Optional)

Agent specifies the configuration used when using Agent platform.

azure AzureNodePoolPlatform
powervs PowerVSNodePoolPlatform (Optional)

PowerVS specifies the configuration used when using IBMCloud PowerVS platform.

NodePoolSpec

(Appears on: NodePool)

NodePoolSpec is the desired behavior of a NodePool.

Field Description
clusterName string

ClusterName is the name of the HostedCluster this NodePool belongs to.

TODO(dan): Should this be a LocalObjectReference?

release Release

Release specifies the OCP release used for the NodePool. This informs the ignition configuration for machines, as well as other platform specific machine properties (e.g. an AMI on the AWS platform).

platform NodePoolPlatform

Platform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

nodeCount int32 (Optional)

Deprecated: Use Replicas instead. NodeCount will be dropped in the next api release.

replicas int32 (Optional)

Replicas is the desired number of nodes the pool should maintain. If unset, the default value is 0.

management NodePoolManagement

Management specifies behavior for managing nodes in the pool, such as upgrade strategies and auto-repair behaviors.

autoScaling NodePoolAutoScaling (Optional)

Autoscaling specifies auto-scaling behavior for the NodePool.

config []Kubernetes core/v1.LocalObjectReference

Config is a list of references to ConfigMaps containing serialized MachineConfig resources to be injected into the ignition configurations of nodes in the NodePool. The MachineConfig API schema is defined here:

https://github.com/openshift/machine-config-operator/blob/master/pkg/apis/machineconfiguration.openshift.io/v1/types.go#L172

Each ConfigMap must have a single key named “config” whose value is the JSON or YAML of a serialized MachineConfig.

nodeDrainTimeout Kubernetes meta/v1.Duration (Optional)

NodeDrainTimeout is the total amount of time that the controller will spend on draining a node. The default value is 0, meaning that the node can be drained without any time limitations. NOTE: NodeDrainTimeout is different from kubectl drain --timeout TODO (alberto): Today changing this field will trigger a recreate rolling update, which kind of defeats the purpose of the change. In future we plan to propagate this field in-place. https://github.com/kubernetes-sigs/cluster-api/issues/5880

NodePoolStatus

(Appears on: NodePool)

NodePoolStatus is the latest observed status of a NodePool.

Field Description
replicas int32 (Optional)

Replicas is the latest observed number of nodes in the pool.

version string

Version is the semantic version of the latest applied release specified by the NodePool.

conditions []NodePoolCondition

Conditions represents the latest available observations of the node pool’s current state.

NodePortPublishingStrategy

(Appears on: ServicePublishingStrategy)

NodePortPublishingStrategy specifies a NodePort used to expose a service.

Field Description
address string

Address is the host/ip that the NodePort service is exposed over.

port int32

Port is the port of the NodePort service. If <=0, the port is dynamically assigned when the service is created.

OLMCatalogPlacement

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

OLMCatalogPlacement is an enum specifying the placement of OLM catalog components.

Value Description

"guest"

GuestOLMCatalogPlacement indicates OLM catalog components will be placed in the guest cluster.

"management"

ManagementOLMCatalogPlacement indicates OLM catalog components will be placed in the management cluster.

PersistentVolumeEtcdStorageSpec

(Appears on: ManagedEtcdStorageSpec)

PersistentVolumeEtcdStorageSpec is the configuration for PersistentVolume etcd storage.

Field Description
storageClassName string (Optional)

StorageClassName is the StorageClass of the data volume for each etcd member.

See https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1.

size k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Size is the minimum size of the data volume for each etcd member.

PlatformSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

PlatformSpec specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

Field Description
type PlatformType

Type is the type of infrastructure provider for the cluster.

Value must be one of: "AWS", "Agent", "Azure", "IBMCloud", "KubeVirt", "None", "PowerVS"

aws AWSPlatformSpec (Optional)

AWS specifies configuration for clusters running on Amazon Web Services.

agent AgentPlatformSpec (Optional)

Agent specifies configuration for agent-based installations.

ibmcloud IBMCloudPlatformSpec

IBMCloud defines IBMCloud specific settings for components

azure AzurePlatformSpec

Azure defines azure specific settings

powervs PowerVSPlatformSpec (Optional)

PowerVS specifies configuration for clusters running on IBMCloud Power VS Service. This field is immutable. Once set, It can’t be changed.

PlatformType

(Appears on: NodePoolPlatform, PlatformSpec)

PlatformType is a specific supported infrastructure provider.

Value Description

"AWS"

AWSPlatform represents Amazon Web Services infrastructure.

"Agent"

AgentPlatform represents user supplied insfrastructure booted with agents.

"Azure"

AzurePlatform represents Azure infrastructure.

"IBMCloud"

IBMCloudPlatform represents IBM Cloud infrastructure.

"KubeVirt"

KubevirtPlatform represents Kubevirt infrastructure.

"None"

NonePlatform represents user supplied (e.g. bare metal) infrastructure.

"PowerVS"

PowerVSPlatform represents PowerVS infrastructure.

PowerVSNodePoolPlatform

(Appears on: NodePoolPlatform)

PowerVSNodePoolPlatform specifies the configuration of a NodePool when operating on IBMCloud PowerVS platform.

Field Description
systemType string (Optional)

SystemType is the System type used to host the instance. systemType determines the number of cores and memory that is available. Few of the supported SystemTypes are s922,e880,e980. e880 systemType available only in Dallas Datacenters. e980 systemType available in Datacenters except Dallas and Washington. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The current default is s922 which is generally available.

processorType string (Optional)

ProcessorType is the VM instance processor type. It must be set to one of the following values: Dedicated, Capped or Shared.

Dedicated: resources are allocated for a specific client, The hypervisor makes a 1:1 binding of a partition’s processor to a physical processor core. Shared: Shared among other clients. Capped: Shared, but resources do not expand beyond those that are requested, the amount of CPU time is Capped to the value specified for the entitlement.

if the processorType is selected as Dedicated, then Processors value cannot be fractional. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The current default is Shared.

processors k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

Processors is the number of virtual processors in a virtual machine. when the processorType is selected as Dedicated the processors value cannot be fractional. maximum value for the Processors depends on the selected SystemType. when SystemType is set to e880 or e980 maximum Processors value is 143. when SystemType is set to s922 maximum Processors value is 15. minimum value for Processors depends on the selected ProcessorType. when ProcessorType is set as Shared or Capped, The minimum processors is 0.5. when ProcessorType is set as Dedicated, The minimum processors is 1. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The default is set based on the selected ProcessorType. when ProcessorType selected as Dedicated, the default is set to 1. when ProcessorType selected as Shared or Capped, the default is set to 0.5.

memoryGiB int32 (Optional)

MemoryGiB is the size of a virtual machine’s memory, in GiB. maximum value for the MemoryGiB depends on the selected SystemType. when SystemType is set to e880 maximum MemoryGiB value is 7463 GiB. when SystemType is set to e980 maximum MemoryGiB value is 15307 GiB. when SystemType is set to s922 maximum MemoryGiB value is 942 GiB. The minimum memory is 32 GiB.

When omitted, this means the user has no opinion and the platform is left to choose a reasonable default. The current default is 32.

image PowerVSResourceReference (Optional)

Image used for deploying the nodes. If unspecified, the default is chosen based on the NodePool release payload image.

storageType string (Optional)

StorageType for the image and nodes, this will be ignored if Image is specified. The storage tiers in PowerVS are based on I/O operations per second (IOPS). It means that the performance of your storage volumes is limited to the maximum number of IOPS based on volume size and storage tier. Although, the exact numbers might change over time, the Tier 3 storage is currently set to 3 IOPS/GB, and the Tier 1 storage is currently set to 10 IOPS/GB.

The default is tier1

imageDeletePolicy string (Optional)

ImageDeletePolicy is policy for the image deletion.

delete: delete the image from the infrastructure. retain: delete the image from the openshift but retain in the infrastructure.

The default is delete

PowerVSPlatformSpec

(Appears on: PlatformSpec)

PowerVSPlatformSpec defines IBMCloud PowerVS specific settings for components

Field Description
resourceGroup string

ResourceGroup is the IBMCloud Resource Group in which the cluster resides. This field is immutable. Once set, It can’t be changed.

region string

Region is the IBMCloud region in which the cluster resides. This configures the OCP control plane cloud integrations, and is used by NodePool to resolve the correct boot image for a given release. This field is immutable. Once set, It can’t be changed.

zone string

Zone is the availability zone where control plane cloud resources are created. This field is immutable. Once set, It can’t be changed.

subnet PowerVSResourceReference

Subnet is the subnet to use for control plane cloud resources. This field is immutable. Once set, It can’t be changed.

serviceInstanceID string

ServiceInstance is the reference to the Power VS service on which the server instance(VM) will be created. Power VS service is a container for all Power VS instances at a specific geographic region. serviceInstance can be created via IBM Cloud catalog or CLI. ServiceInstanceID is the unique identifier that can be obtained from IBM Cloud UI or IBM Cloud cli.

More detail about Power VS service instance. https://cloud.ibm.com/docs/power-iaas?topic=power-iaas-creating-power-virtual-server

This field is immutable. Once set, It can’t be changed.

vpc PowerVSVPC

VPC specifies IBM Cloud PowerVS Load Balancing configuration for the control plane. This field is immutable. Once set, It can’t be changed.

kubeCloudControllerCreds Kubernetes core/v1.LocalObjectReference

KubeCloudControllerCreds is a reference to a secret containing cloud credentials with permissions matching the cloud controller policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file. This field is immutable. Once set, It can’t be changed.

TODO(dan): document the “cloud controller policy”

nodePoolManagementCreds Kubernetes core/v1.LocalObjectReference

NodePoolManagementCreds is a reference to a secret containing cloud credentials with permissions matching the node pool management policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file. This field is immutable. Once set, It can’t be changed.

TODO(dan): document the “node pool management policy”

controlPlaneOperatorCreds Kubernetes core/v1.LocalObjectReference

ControlPlaneOperatorCreds is a reference to a secret containing cloud credentials with permissions matching the control-plane-operator policy. The secret should have exactly one key, credentials, whose value is an AWS credentials file. This field is immutable. Once set, It can’t be changed.

TODO(dan): document the “control plane operator policy”

PowerVSResourceReference

(Appears on: PowerVSNodePoolPlatform, PowerVSPlatformSpec)

PowerVSResourceReference is a reference to a specific IBMCloud PowerVS resource by ID, or Name. Only one of ID, or Name may be specified. Specifying more than one will result in a validation error.

Field Description
id string (Optional)

ID of resource

name string (Optional)

Name of resource

PowerVSVPC

(Appears on: PowerVSPlatformSpec)

PowerVSVPC specifies IBM Cloud PowerVS LoadBalancer configuration for the control plane.

Field Description
name string

Name for VPC to used for all the service load balancer. This field is immutable. Once set, It can’t be changed.

region string

Region is the IBMCloud region in which VPC gets created, this VPC used for all the ingress traffic into the OCP cluster. This field is immutable. Once set, It can’t be changed.

zone string (Optional)

Zone is the availability zone where load balancer cloud resources are created. This field is immutable. Once set, It can’t be changed.

subnet string (Optional)

Subnet is the subnet to use for load balancer. This field is immutable. Once set, It can’t be changed.

PublishingStrategyType

(Appears on: ServicePublishingStrategy)

PublishingStrategyType defines publishing strategies for services.

Release

(Appears on: ClusterVersionStatus, HostedClusterSpec, NodePoolSpec)

Release represents the metadata for an OCP release payload image.

Field Description
image string

Image is the image pullspec of an OCP release payload image.

ReplaceUpgrade

(Appears on: NodePoolManagement)

ReplaceUpgrade specifies upgrade behavior that replaces existing nodes according to a given strategy.

Field Description
strategy UpgradeStrategy

Strategy is the node replacement strategy for nodes in the pool.

Value must be one of: "OnDelete", "RollingUpdate"

rollingUpdate RollingUpdate

RollingUpdate specifies a rolling update strategy which upgrades nodes by creating new nodes and deleting the old ones.

RollingUpdate

(Appears on: ReplaceUpgrade)

RollingUpdate specifies a rolling update strategy which upgrades nodes by creating new nodes and deleting the old ones.

Field Description
maxUnavailable k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

MaxUnavailable is the maximum number of nodes that can be unavailable during the update.

Value can be an absolute number (ex: 5) or a percentage of desired nodes (ex: 10%).

Absolute number is calculated from percentage by rounding down.

This can not be 0 if MaxSurge is 0.

Defaults to 0.

Example: when this is set to 30%, old nodes can be deleted down to 70% of desired nodes immediately when the rolling update starts. Once new nodes are ready, more old nodes be deleted, followed by provisioning new nodes, ensuring that the total number of nodes available at all times during the update is at least 70% of desired nodes.

maxSurge k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

MaxSurge is the maximum number of nodes that can be provisioned above the desired number of nodes.

Value can be an absolute number (ex: 5) or a percentage of desired nodes (ex: 10%).

Absolute number is calculated from percentage by rounding up.

This can not be 0 if MaxUnavailable is 0.

Defaults to 1.

Example: when this is set to 30%, new nodes can be provisioned immediately when the rolling update starts, such that the total number of old and new nodes do not exceed 130% of desired nodes. Once old nodes have been deleted, new nodes can be provisioned, ensuring that total number of nodes running at any time during the update is at most 130% of desired nodes.

RoutePublishingStrategy

(Appears on: ServicePublishingStrategy)

RoutePublishingStrategy specifies options for exposing a service as a Route.

Field Description
hostname string (Optional)

Hostname is the name of the DNS record that will be created pointing to the Route.

SecretEncryptionSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

SecretEncryptionSpec contains metadata about the kubernetes secret encryption strategy being used for the cluster when applicable.

Field Description
type SecretEncryptionType

Type defines the type of kube secret encryption being used

Value must be one of: "aescbc", "kms"

kms KMSSpec (Optional)

KMS defines metadata about the kms secret encryption strategy

aescbc AESCBCSpec (Optional)

AESCBC defines metadata about the AESCBC secret encryption strategy

SecretEncryptionType

(Appears on: SecretEncryptionSpec)

SecretEncryptionType defines the type of kube secret encryption being used.

Value Description

"aescbc"

AESCBC uses AES-CBC with PKCS#7 padding to do secret encryption

"kms"

KMS integrates with a cloud provider’s key management service to do secret encryption

ServicePublishingStrategy

(Appears on: ServicePublishingStrategyMapping)

ServicePublishingStrategy specfies how to publish a ServiceType.

Field Description
type PublishingStrategyType

Type is the publishing strategy used for the service.

nodePort NodePortPublishingStrategy

NodePort configures exposing a service using a NodePort.

loadBalancer LoadBalancerPublishingStrategy

LoadBalancer configures exposing a service using a LoadBalancer.

route RoutePublishingStrategy

Route configures exposing a service using a Route.

ServicePublishingStrategyMapping

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ServicePublishingStrategyMapping specifies how individual control plane services are published from the hosting cluster of a control plane.

Field Description
service ServiceType

Service identifies the type of service being published.

servicePublishingStrategy ServicePublishingStrategy

ServicePublishingStrategy specifies how to publish Service.

ServiceType

(Appears on: ServicePublishingStrategyMapping)

ServiceType defines what control plane services can be exposed from the management control plane.

UnmanagedEtcdSpec

(Appears on: EtcdSpec)

UnmanagedEtcdSpec specifies configuration which enables the control plane to integrate with an eternally managed etcd cluster.

Field Description
endpoint string

Endpoint is the full etcd cluster client endpoint URL. For example:

https://etcd-client:2379

If the URL uses an HTTPS scheme, the TLS field is required.

tls EtcdTLSConfig

TLS specifies TLS configuration for HTTPS etcd client endpoints.

UpgradeStrategy

(Appears on: ReplaceUpgrade)

UpgradeStrategy is a specific strategy for upgrading nodes in a NodePool.

Value Description

"OnDelete"

UpgradeStrategyOnDelete replaces old nodes when the deletion of the associated node instances are completed.

"RollingUpdate"

UpgradeStrategyRollingUpdate means use a rolling update for nodes.

UpgradeType

(Appears on: NodePoolManagement)

UpgradeType is a type of high-level upgrade behavior nodes in a NodePool.

Value Description

"InPlace"

UpgradeTypeInPlace is a strategy which replaces nodes in-place with no additional node capacity requirements.

"Replace"

UpgradeTypeReplace is a strategy which replaces nodes using surge node capacity.

Volume

(Appears on: AWSNodePoolPlatform)

Volume specifies the configuration options for node instance storage devices.

Field Description
size int64

Size specifies size (in Gi) of the storage device.

Must be greater than the image snapshot size or 8 (whichever is greater).

type string

Type is the type of the volume.

iops int64 (Optional)

IOPS is the number of IOPS requested for the disk. This is only valid for type io1.