In OpenShift Container Platform 4.17, you can use the Agent-based Installer to install a cluster on Oracle® Cloud Infrastructure (OCI), so that you can run cluster workloads on infrastructure that supports dedicated, hybrid, public, and multiple cloud environments.
Installing a cluster on OCI is supported for virtual machines (VMs) and bare-metal machines.
You can install an OpenShift Container Platform cluster on Oracle® Cloud Infrastructure (OCI) by using the Agent-based Installer. Red Hat and Oracle test, validate, and support running OCI workloads in an OpenShift Container Platform cluster.
The Agent-based Installer provides the ease of use of the Assisted Installation service, but with the capability to install a cluster in either a connected or disconnected environment.
The following diagrams show workflows for connected and disconnected environments:
OCI provides services that can meet your regulatory compliance, performance, and cost-effectiveness needs. OCI supports 64-bit x86
instances and 64-bit ARM
instances. Additionally, OCI provides an OCVS service where you can move VMware workloads to OCI with minimal application re-architecture.
Consider selecting a nonvolatile memory express (NVMe) drive or a solid-state drive (SSD) for your boot disk, because these drives offer low latency and high throughput capabilities for your boot disk. |
By running your OpenShift Container Platform cluster on OCI, you can access the following capabilities:
Compute flexible shapes, where you can customize the number of Oracle® CPUs (OCPUs) and memory resources for your VM. With access to this capability, a cluster’s workload can perform operations in a resource-balanced environment. You can find all RHEL-certified OCI shapes by going to the Oracle page on the Red Hat Ecosystem Catalog portal.
Block Volume storage, where you can configure scaling and auto-tuning settings for your storage volume, so that the Block Volume service automatically adjusts the performance level to optimize performance.
Oracle® Cloud VMware Solution (OCVS), where you can deploy a cluster in a public-cloud environment that operates on a VMware® vSphere software-defined data center (SDDC). You continue to retain full-administrative control over your VMware vSphere environment, but you can use OCI services to improve your applications on flexible, scalable, and secure infrastructure.
To ensure the best performance conditions for your cluster workloads that operate on OCI and on the OCVS service, ensure volume performance units (VPUs) for your block volume is sized for your workloads. The following list provides some guidance in selecting the VPUs needed for specific performance needs:
Consider reserving additional VPUs to provide sufficient capacity for updates and scaling activities. For more information about VPUs, see Volume Performance Units (Oracle documentation). |
The following workflow describes a high-level outline for the process of installing an OpenShift Container Platform cluster on OCI using the Agent-based Installer:
Create OCI resources and services (Oracle).
Disconnected environments: Prepare a web server that is accessible by OCI instances (Red Hat).
Prepare configuration files for the Agent-based Installer (Red Hat).
Generate the agent ISO image (Red Hat).
Disconnected environments: Upload the rootfs image to the web server (Red Hat).
Configure your firewall for OpenShift Container Platform (Red Hat).
Upload the agent ISO image to a storage bucket (Oracle).
Create a custom image from the uploaded agent ISO image (Oracle).
Create compute instances on OCI (Oracle).
Verify that your cluster runs on OCI (Oracle).
You must create an OCI environment on your virtual machine (VM) or bare-metal shape. By creating this environment, you can install OpenShift Container Platform and deploy a cluster on an infrastructure that supports a wide range of cloud options and strong security policies. Having prior knowledge of OCI components can help you with understanding the concept of OCI resources and how you can configure them to meet your organizational needs.
The Agent-based Installer method for installing an OpenShift Container Platform cluster on OCI requires that you manually create OCI resources and services.
To ensure compatibility with OpenShift Container Platform, you must set
The |
You configured an OCI account to host the OpenShift Container Platform cluster. See Prerequisites (Oracle documentation).
Create the required OCI resources and services. See OCI Resources Needed for Using the Agent-based Installer (Oracle documentation).
You must create the install-config.yaml
and the agent-config.yaml
configuration files so that you can use the Agent-based Installer to generate a bootable ISO image. The Agent-based installation comprises a bootable ISO that has the Assisted discovery agent and the Assisted Service. Both of these components are required to perform the cluster installation, but the latter component runs on only one of the hosts.
At a later stage, you must follow the steps in the Oracle documentation for uploading your generated agent ISO image to Oracle’s default Object Storage bucket, which is the initial step for integrating your OpenShift Container Platform cluster on Oracle® Cloud Infrastructure (OCI).
You can also use the Agent-based Installer to generate or accept Zero Touch Provisioning (ZTP) custom resources. |
You reviewed details about the OpenShift Container Platform installation and update processes.
You read the documentation on selecting a cluster installation method and preparing the method for users.
You have read the "Preparing to install with the Agent-based Installer" documentation.
You downloaded the Agent-Based Installer and the command-line interface (CLI) from the Red Hat Hybrid Cloud Console.
If you are installing in a disconnected environment, you have prepared a mirror registry in your environment and mirrored release images to the registry.
Check that your
Example output for a shared registry binary
|
You have logged in to the OpenShift Container Platform with administrator privileges.
Create an installation directory to store configuration files in by running the following command:
$ mkdir ~/<directory_name>
Configure the install-config.yaml
configuration file to meet the needs of your organization and save the file in the directory you created.
install-config.yaml
file that sets an external platform# install-config.yaml
apiVersion: v1
baseDomain: <base_domain> (1)
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
network type: OVNKubernetes
machineNetwork:
- cidr: <ip_address_from_cidr> (2)
serviceNetwork:
- 172.30.0.0/16
compute:
- architecture: amd64 (3)
hyperthreading: Enabled
name: worker
replicas: 0
controlPlane:
architecture: amd64 (3)
hyperthreading: Enabled
name: master
replicas: 3
platform:
external:
platformName: oci (4)
cloudControllerManager: External
sshKey: <public_ssh_key> (5)
pullSecret: '<pull_secret>' (6)
# ...
1 | The base domain of your cloud provider. |
2 | The IP address from the virtual cloud network (VCN) that the CIDR allocates to resources and components that operate on your network. |
3 | Depending on your infrastructure, you can select either arm64 or amd64 . |
4 | Set OCI as the external platform, so that OpenShift Container Platform can integrate with OCI. |
5 | Specify your SSH public key. |
6 | The pull secret that you need for authenticate purposes when downloading container images for OpenShift Container Platform components and services, such as Quay.io. See Install OpenShift Container Platform 4 from the Red Hat Hybrid Cloud Console. |
Create a directory on your local system named openshift
. This must be a subdirectory of the installation directory.
Do not move the |
If you used a stack to provision OCI infrastructure resources: Copy and paste the dynamic_custom_manifest
output of the OCI stack into a file titled manifest.yaml
and save the file in the openshift
directory.
If you did not use a stack to provision OCI infrastructure resources: Download and prepare custom manifests to create an Agent ISO image:
Go to Configuration Files (Oracle documentation) and follow the link to the custom manifests directory on GitHub.
Copy the contents of the condensed-manifest.yml
file and save it locally to a file in the openshift
directory.
In the condensed-manifest.yml
file, update the sections marked with TODO
to specify the compartment Oracle® Cloud Identifier (OCID), VCN OCID, subnet OCID from the load balancer, and the security lists OCID.
Configure the agent-config.yaml
configuration file to meet your organization’s requirements.
agent-config.yaml
file for an IPv4 network.apiVersion: v1beta1
metadata:
name: <cluster_name> (1)
namespace: <cluster_namespace> (2)
rendezvousIP: <ip_address_from_CIDR> (3)
bootArtifactsBaseURL: <server_URL> (4)
# ...
1 | The cluster name that you specified in your DNS record. |
2 | The namespace of your cluster on OpenShift Container Platform. |
3 | If you use IPv4 as the network IP address format, ensure that you set the rendezvousIP parameter to an IPv4 address that the VCN’s Classless Inter-Domain Routing (CIDR) method allocates on your network. Also ensure that at least one instance from the pool of instances that you booted with the ISO matches the IP address value you set for the rendezvousIP parameter. |
4 | The URL of the server where you want to upload the rootfs image. This parameter is required only for disconnected environments. |
Generate a minimal ISO image, which excludes the rootfs image, by entering the following command in your installation directory:
$ ./openshift-install agent create image --log-level debug
The command also completes the following actions:
Creates a subdirectory, ./<installation_directory>/auth directory:
, and places kubeadmin-password
and kubeconfig
files in the subdirectory.
Creates a rendezvousIP
file based on the IP address that you specified in the agent-config.yaml
configuration file.
Optional: Any modifications you made to agent-config.yaml
and install-config.yaml
configuration files get imported to the Zero Touch Provisioning (ZTP) custom resources.
The Agent-based Installer uses Red Hat Enterprise Linux CoreOS (RHCOS). The rootfs image, which is mentioned in a later step, is required for booting, recovering, and repairing your operating system. |
Disconnected environments only: Upload the rootfs image to a web server.
Go to the ./<installation_directory>/boot-artifacts
directory that was generated when you created the minimal ISO image.
Use your preferred web server, such as any Hypertext Transfer Protocol daemon (httpd
), to upload the rootfs image to the location specified in the bootArtifactsBaseURL
parameter of the agent-config.yaml
file.
For example, if the bootArtifactsBaseURL
parameter states http://192.168.122.20
, you would upload the generated rootfs image to this location so that the Agent-based installer can access the image from http://192.168.122.20/agent.x86_64-rootfs.img
. After the Agent-based installer boots the minimal ISO for the external platform, the Agent-based Installer downloads the rootfs image from the http://192.168.122.20/agent.x86_64-rootfs.img
location into the system memory.
The Agent-based Installer also adds the value of the |
Consider that the full ISO image, which is in excess of |
Before you install OpenShift Container Platform, you must configure your firewall to grant access to the sites that OpenShift Container Platform requires. When using a firewall, make additional configurations to the firewall so that OpenShift Container Platform can access the sites that it requires to function.
For a disconnected environment, you must mirror content from both Red Hat and Oracle. This environment requires that you create firewall rules to expose your firewall to specific ports and registries.
If your environment has a dedicated load balancer in front of your OpenShift Container Platform cluster, review the allowlists between your firewall and load balancer to prevent unwanted network restrictions to your cluster. |
Set the following registry URLs for your firewall’s allowlist:
URL | Port | Function |
---|---|---|
|
443 |
Provides core container images |
|
443 |
Hosts a signature store that a container client requires for verifying images pulled from |
|
443 |
Hosts all the container images that are stored on the Red Hat Ecosystem Catalog, including core container images. |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
Provides core container images |
|
443 |
The |
You can use the wildcards *.quay.io
and *.openshiftapps.com
instead of cdn.quay.io
and cdn0[1-6].quay.io
in your allowlist.
You can use the wildcard *.access.redhat.com
to simplify the configuration and ensure that all subdomains, including registry.access.redhat.com
, are allowed.
When you add a site, such as quay.io
, to your allowlist, do not add a wildcard entry, such as *.quay.io
, to your denylist. In most cases, image registries use a content delivery network (CDN) to serve images. If a firewall blocks access, image downloads are denied when the initial download request redirects to a hostname such as cdn01.quay.io
.
Set your firewall’s allowlist to include any site that provides resources for a language or framework that your builds require.
If you do not disable Telemetry, you must grant access to the following URLs to access Red Hat Insights:
URL | Port | Function |
---|---|---|
|
443 |
Required for Telemetry |
|
443 |
Required for Telemetry |
|
443 |
Required for Telemetry |
|
443 |
Required for Telemetry and for |
Set your firewall’s allowlist to include the following registry URLs:
URL | Port | Function |
---|---|---|
|
443 |
Required both for your cluster token and to check if updates are available for the cluster. |
|
443 |
Required to download Red Hat Enterprise Linux CoreOS (RHCOS) images. |
Set your firewall’s allowlist to include the following external URLs. Each repository URL hosts OCI containers. Consider mirroring images to as few repositories as possible to reduce any performance issues.
URL | Port | Function |
---|---|---|
|
port |
A Kubernetes registry that hosts container images for a community-based image registry. This image registry is hosted on a custom Google Container Registry (GCR) domain. |
|
port |
A GitHub image registry where you can store and manage Open Container Initiative images. Requires an access token to publish, install, and delete private, internal, and public packages. |
|
443 |
A source of release image signatures, although the Cluster Version Operator needs only a single functioning source. |
|
port |
Replaces the |
To run a cluster on Oracle® Cloud Infrastructure (OCI), you must upload the generated agent ISO image to the default Object Storage bucket on OCI. Additionally, you must create a compute instance from the supplied base image, so that your OpenShift Container Platform and OCI can communicate with each other for the purposes of running the cluster on OCI.
OCI supports the following OpenShift Container Platform cluster topologies:
|
You generated an agent ISO image. See the "Creating configuration files for installing a cluster on OCI" section.
Upload the agent ISO image to Oracle’s default Object Storage bucket and import the agent ISO image as a custom image to this bucket. Ensure you that you configure the custom image to boot in Unified Extensible Firmware Interface (UEFI) mode. For more information, see Creating the OpenShift Container Platform ISO Image (Oracle documentation).
Create a compute instance from the supplied base image for your cluster topology. See Creating the OpenShift Container Platform cluster on OCI (Oracle documentation).
Before you create the compute instance, check that you have enough memory and disk resources for your cluster. Additionally, ensure that at least one compute instance has the same IP address as the address stated under |
Verify that your cluster was installed and is running effectively on Oracle® Cloud Infrastructure (OCI).
You created all the required OCI resources and services. See the "Creating OCI infrastructure resources and services" section.
You created install-config.yaml
and agent-config.yaml
configuration files. See the "Creating configuration files for installing a cluster on OCI" section.
You uploaded the agent ISO image to Oracle’s default Object Storage bucket, and you created a compute instance on OCI. For more information, see "Running a cluster on OCI".
After you deploy the compute instance on a self-managed node in your OpenShift Container Platform cluster, you can monitor the cluster’s status by choosing one of the following options:
From the OpenShift Container Platform CLI, enter the following command:
$ ./openshift-install agent wait-for install-complete --log-level debug
Check the status of the rendezvous
host node that runs the bootstrap node. After the host reboots, the host forms part of the cluster.
Use the kubeconfig
API to check the status of various OpenShift Container Platform components. For the KUBECONFIG
environment variable, set the relative path of the cluster’s kubeconfig
configuration file:
$ export KUBECONFIG=~/auth/kubeconfig
Check the status of each of the cluster’s self-managed nodes. CCM applies a label to each node to designate the node as running in a cluster on OCI.
$ oc get nodes -A
NAME STATUS ROLES AGE VERSION
main-0.private.agenttest.oraclevcn.com Ready control-plane, master 7m v1.27.4+6eeca63
main-1.private.agenttest.oraclevcn.com Ready control-plane, master 15m v1.27.4+d7fa83f
main-2.private.agenttest.oraclevcn.com Ready control-plane, master 15m v1.27.4+d7fa83f
Check the status of each of the cluster’s Operators, with the CCM Operator status being a good indicator that your cluster is running.
$ oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.17.0-0 True False False 6m18s
baremetal 4.17.0-0 True False False 2m42s
network 4.17.0-0 True True False 5m58s Progressing: …
…