1. Introducing Red Hat CodeReady Containers

1.1. About CodeReady Containers

Red Hat CodeReady Containers brings a minimal OpenShift 4 cluster to your local computer. This cluster provides a minimal environment for development and testing purposes. CodeReady Containers is mainly targeted at running on developers' desktops. For other use cases, such as headless or multi-developer setups, use the full OpenShift installer.

Refer to the OpenShift documentation for a full introduction to OpenShift.

CodeReady Containers includes the crc command-line interface (CLI) to interact with the CodeReady Containers virtual machine running the OpenShift cluster.

1.2. Differences from a production OpenShift installation

Red Hat CodeReady Containers is a regular OpenShift installation with the following notable differences:

  • The CodeReady Containers OpenShift cluster is ephemeral and is not intended for production use.

  • It uses a single node which behaves as both a master and worker node.

  • It disables the machine-config and monitoring Operators by default.

    • These disabled Operators cause the corresponding parts of the web console to be non-functional.

    • For the same reason, there is no upgrade path to newer OpenShift versions.

  • The OpenShift instance runs in a virtual machine. This may cause other differences, particularly with external networking.

2. Installation

2.1. Minimum system requirements

CodeReady Containers requires the following minimum hardware and operating system requirements.

2.1.1. Hardware requirements

CodeReady Containers requires the following system resources:

  • 4 virtual CPUs (vCPUs)

  • 8 GB of memory

  • 35 GB of storage space

Note

The OpenShift cluster requires these minimum resources to run in the CodeReady Containers virtual machine. Some workloads may require more resources.

2.1.2. Operating system requirements

CodeReady Containers requires the following minimum version of a supported operating system:

Microsoft Windows
  • On Microsoft Windows, CodeReady Containers requires the Windows 10 Pro Fall Creators Update (version 1709). CodeReady Containers does not work on earlier versions or editions of Microsoft Windows.

macOS
  • On macOS, CodeReady Containers requires macOS 10.12 Sierra or newer. CodeReady Containers does not work on earlier versions of macOS.

Linux
  • On Linux, CodeReady Containers is only supported on Red Hat Enterprise Linux/CentOS 7.5 or newer (including 8.x versions) and on the latest two stable Fedora releases.

  • When using Red Hat Enterprise Linux, the machine running CodeReady Containers must be registered with the Red Hat Customer Portal.

  • Ubuntu 18.04 LTS or newer and Debian 10 or newer are not officially supported and may require manual set up of the host machine.

  • See Required software packages to install the required packages for your Linux distribution.

2.2. Required software packages

CodeReady Containers requires the libvirt and NetworkManager packages. Consult the following table to find the command used to install these packages for your Linux distribution:

Table 1. Package installation commands by distribution
Linux Distribution Installation command

Fedora

sudo dnf install NetworkManager

Red Hat Enterprise Linux/CentOS

su -c 'yum install NetworkManager'

Debian/Ubuntu

sudo apt install qemu-kvm libvirt-daemon libvirt-daemon-system network-manager

2.3. Installing CodeReady Containers

Prerequisites
Procedure
  1. Download the latest release of CodeReady Containers for your platform and extract the contents of the archive to a location in your PATH.

2.4. Upgrading CodeReady Containers

Newer versions of the CodeReady Containers binary require manual set up to prevent potential incompatibilities with earlier versions.

Procedure
  • Download the latest release of CodeReady Containers.

  • Delete the existing CodeReady Containers virtual machine:

    $ crc delete
    Warning

    The crc delete command results in the loss of data stored in the CodeReady Containers virtual machine. Save any desired information stored in the virtual machine before running this command.

  • Replace the earlier crc binary with the binary of the latest release. Verify that the new crc binary is in use by checking its version:

    $ crc version
  • Set up the new CodeReady Containers release:

    $ crc setup
  • Start the new CodeReady Containers virtual machine:

    $ crc start

3. Using CodeReady Containers

3.1. Setting up CodeReady Containers

The crc setup command performs operations to set up the environment of your host machine for the CodeReady Containers virtual machine.

This procedure will create the ~/.crc directory if it does not already exist.

Prerequisites
  • Your user account has permission to use the sudo command.

Note
  • Do not run the crc binary as root (or Administrator). Always run the crc binary with your user account.

  • If you are setting up a new version, capture any changes made to the virtual machine before setting up a new CodeReady Containers release.

Procedure
  1. Set up your host machine for CodeReady Containers:

    $ crc setup

3.2. Starting the virtual machine

The crc start command starts the CodeReady Containers virtual machine and OpenShift cluster.

Prerequisites
Procedure
  • Start the CodeReady Containers virtual machine:

    $ crc start
  • When prompted, supply your user pull secret.

Note
  • The cluster takes a minimum of four minutes to start the necessary containers and Operators before serving a request.

  • If you see errors during crc start, check the Troubleshooting Red Hat CodeReady Containers section for potential solutions.

3.3. Accessing the OpenShift cluster

Access the OpenShift cluster running in the CodeReady Containers virtual machine via the OpenShift web console or client binary (oc).

3.3.1. Accessing the OpenShift web console

Prerequisites
Procedure

To access the OpenShift web console, follow these steps:

  1. Run crc console. This will open your web browser and direct it to the web console.

  2. Log in to the OpenShift web console as the developer user with the password printed in the output of the crc start command.

    Note
    • You can also view the password for the developer and kubeadmin users by running crc console --credentials.

    • You can initially access the cluster through either the kubeadmin or developer user. Use the developer user for creating projects or OpenShift applications and for application deployment. Only use the kubeadmin user for administrative tasks such as creating new users, setting roles, and so on.

See Troubleshooting Red Hat CodeReady Containers if you cannot access the CodeReady Containers OpenShift cluster.

Additional resources

3.3.2. Accessing the OpenShift cluster with oc

Prerequisites
Procedure

To access the OpenShift cluster via the oc command, follow these steps:

  1. Run the crc oc-env command to print the command needed to add the cached oc binary to your PATH:

    $ crc oc-env
  2. Run the printed command.

  3. Log in as the developer user:

    $ oc login -u developer https://api.crc.testing:6443
    Note

    The crc start command prints the password for the developer user. You can also view it by running the crc console --credentials command.

  4. You can now use oc to interact with your OpenShift cluster. For example, to verify that the OpenShift cluster Operators are available:

    $ oc get co
    Note
    • CodeReady Containers disables the machine-config and monitoring Operators by default.

See Troubleshooting Red Hat CodeReady Containers if you cannot access the CodeReady Containers OpenShift cluster.

Additional resources
  • The OpenShift documentation covers the creation of projects and applications.

  • You can also use OpenShift Do (odo) to create OpenShift projects and applications from the command line.

3.4. Stopping the virtual machine

The crc stop command stops the running CodeReady Containers virtual machine and OpenShift cluster. The stopping process will take a few minutes while the cluster shuts down.

Procedure
  • Stop the CodeReady Containers virtual machine and OpenShift cluster:

    $ crc stop

3.5. Deleting the virtual machine

The crc delete command deletes an existing CodeReady Containers virtual machine.

Procedure
  • Delete the CodeReady Containers virtual machine:

    $ crc delete
    Warning

    The crc delete command results in the loss of data stored in the CodeReady Containers virtual machine. Save any desired information stored in the virtual machine before running this command.

4. Configuring CodeReady Containers

4.1. About CodeReady Containers configuration

Use the crc config command to configure both the crc binary and the CodeReady Containers virtual machine. The crc config command requires a subcommand to act on the configuration. The available subcommands are get, set, unset, and view. The get, set, and unset subcommands operate on named configurable properties. Run the crc config --help command to list the available properties.

You can also use the crc config command to configure the behavior of the startup checks for the crc start and crc setup commands. By default, startup checks report an error and stop execution when their conditions are not met. Set the value of a property starting with skip-check or warn-check to true to skip the check or report a warning rather than an error, respectively.

4.2. Viewing CodeReady Containers configuration

The CodeReady Containers binary provides commands to view configurable properties and the current CodeReady Containers configuration.

Procedure
  • To view the available configurable properties:

    $ crc config --help
  • To view the values for a configurable property:

    $ crc config get <property>
  • To view the complete current configuration:

    $ crc config view
    Note

    The crc config view command does not return any information if the configuration consists of default values.

4.3. Configuring the virtual machine

Use the cpus and memory properties to configure the default number of vCPUs and amount of memory available to the CodeReady Containers virtual machine, respectively.

Important

You cannot change the configuration of an existing CodeReady Containers virtual machine. To enable configuration changes, you must delete the existing virtual machine and create a new one.

To create the new virtual machine, first delete the existing CodeReady Containers virtual machine, then start a new virtual machine with the configuration changes:

$ crc delete
$ crc start
Warning

The crc delete command results in the loss of data stored in the CodeReady Containers virtual machine. Save any desired information stored in the virtual machine before running this command.

Procedure
  • To increase the number of vCPUs available to the virtual machine:

    $ crc config set cpus <number>

    The default value for the cpus property is 4. The number of vCPUs to assign must be greater than or equal to the default.

  • To increase the memory available to the virtual machine:

    $ crc config set memory <number-in-mib>
    Note

    Values for available memory are set in mebibytes (MiB). One gibibyte (GiB) of memory is equal to 1024 MiB.

    The default value for the memory property is 8192. The amount of memory to assign must be greater than or equal to the default.

5. Networking

5.1. DNS configuration details

5.1.1. General DNS setup

The OpenShift cluster managed by CodeReady Containers uses 2 DNS domain names, crc.testing and apps-crc.testing. The crc.testing domain is for core OpenShift services. The apps-crc.testing domain is for accessing OpenShift applications deployed on the cluster.

For example, the OpenShift API server will be exposed as api.crc.testing while the OpenShift console is accessed through console-openshift-console.apps-crc.testing. These DNS domains are served by a dnsmasq DNS container running inside the CodeReady Containers virtual machine.

Running crc setup will adjust your system DNS configuration so that it can resolve these domains. Additional checks are done to verify DNS is properly configured when running crc start.

5.1.2. Linux

On Linux, CodeReady Containers expects the following DNS configuration:

  • CodeReady Containers expects NetworkManager to manage networking.

  • NetworkManager uses dnsmasq via the /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf configuration file.

  • The configuration file for this dnsmasq instance is /etc/NetworkManager/dnsmasq.d/crc.conf:

    server=/crc.testing/192.168.130.11
    server=/apps-crc.testing/192.168.130.11
    • The NetworkManager dnsmasq instance forwards requests for the crc.testing and apps-crc.testing domains to the 192.168.130.11 DNS server.

5.1.3. macOS

On macOS, CodeReady Containers expects the following DNS configuration:

  • CodeReady Containers creates a /etc/resolver/testing file which instructs macOS to forward all DNS requests for the testing domain to the CodeReady Containers virtual machine.

  • CodeReady Containers also adds an api.crc.testing entry to /etc/hosts pointing at the VM IP address. The oc binary requires this entry. See OpenShift issue #23266 for more information.

5.2. Starting CodeReady Containers behind a proxy

Prerequisites
  • To use an existing oc binary on your host machine, export the .testing domain as part of the no_proxy environment variable.

  • The embedded oc binary does not require manual settings. For more information on using the embedded oc binary, see Accessing the OpenShift cluster with oc.

Procedure
  • To start CodeReady Containers behind a proxy:

    1. Define a proxy using the http_proxy and https_proxy environment variables or using the crc config set command as follows:

      $ crc config set http-proxy http://example.proxy.com:<port>
      $ crc config set https-proxy http://example.proxy.com:<port>
      $ crc config set no-proxy <comma-separated-no-proxy-entries>

      The crc binary will be able to use the defined proxy once set via environment variables or CodeReady Containers configuration.

Note
  • Proxy-related values set in the configuration for CodeReady Containers have priority over values set via environment variables.

  • SOCKS proxies are not supported by OpenShift Container Platform.

6. Administrative tasks

6.1. Starting Monitoring, Alerting, and Telemetry

To make sure CodeReady Containers can run on a typical laptop, some resource-heavy services get disabled by default. One of these is Prometheus and the related monitoring, alerting, and telemetry functionality. Telemetry functionality is responsible for listing your cluster in the Red Hat OpenShift Cluster Manager.

Prerequisites
  • A running CodeReady Containers virtual machine and a working oc command. For more information, see Accessing the OpenShift cluster with oc.

  • You must log in via oc as the kubeadmin user.

  • You must assign additional memory to the CodeReady Containers virtual machine. At least 12 GiB of memory (a value of 12288) is recommended. For more information, see Configuring the virtual machine.

Procedure
  1. List unmanaged Operators and note the numeric index for cluster-monitoring-operator:

    • On Linux or macOS:

      $ oc get clusterversion version -ojsonpath='{range .spec.overrides[*]}{.name}{"\n"}{end}' | nl -v 0
  • On Microsoft Windows using PowerShell:

    PS> oc get clusterversion version -ojsonpath='{range .spec.overrides[*]}{.name}{"\n"}{end}' | % {$nl++;"`t$($nl-1) `t $_"};$nl=0
    1. Start monitoring, alerting, and telemetry services using the identified numeric index:

      $ oc patch clusterversion/version --type='json' -p '[{"op":"remove", "path":"/spec/overrides/<unmanaged-operator-index>"}]' -oyaml

7. Troubleshooting Red Hat CodeReady Containers

Note

The goal of Red Hat CodeReady Containers is to deliver an OpenShift environment for development and testing purposes. Issues occurring during installation or usage of specific OpenShift applications are outside of the scope of CodeReady Containers. Report such issues to the relevant project. For example, OpenShift tracks issues on GitHub.

7.1. Basic troubleshooting

Resolve most issues by stopping the CodeReady Containers virtual machine, deleting it, and starting a new instance.

Prerequisites
Procedure

To troubleshoot CodeReady Containers, perform the following steps:

  1. Stop the CodeReady Containers virtual machine:

    $ crc stop
  2. Delete the CodeReady Containers virtual machine:

    $ crc delete
  3. Start the CodeReady Containers virtual machine:

    $ crc start
    Note

    The cluster takes a minimum of four minutes to start the necessary containers and Operators before serving a request.

If your issue is not resolved by this procedure, perform the following steps:

  1. Search open issues for the issue that you are encountering.

  2. If no existing issue addresses the encountered issue, create an issue and attach the ~/.crc/crc.log file to the created issue. The ~/.crc/crc.log file has detailed debugging and troubleshooting information which can help diagnose the problem that you are experiencing.

7.2. Getting shell access to the OpenShift cluster

Direct access to the OpenShift cluster is not needed for regular use and is strongly discouraged. To access the cluster for troubleshooting or debugging purposes, follow this procedure.

Prerequisites
Procedure
  1. Run oc get nodes. The output will be similar to this:

    $ oc get nodes
    NAME                 STATUS   ROLES           AGE    VERSION
    crc-shdl4-master-0   Ready    master,worker   7d7h   v1.14.6+7e13ab9a7
  2. Run oc debug nodes/<node> where <node> is the name of the node printed in the previous step.

7.3. Troubleshooting expired certificates

Before Red Hat CodeReady Containers 1.2.0, the system bundle in each released crc binary expired 30 days after the release. This expiration was due to certificates embedded in the OpenShift cluster. As a result, using an older crc binary or system bundle results in an expired certificates error.

Starting from CodeReady Containers 1.2.0, the embedded certificates can be automatically renewed by crc. The crc start command triggers the certificate renewal process when needed. Certificate renewal can add up to five minutes to the start time of the cluster.

Procedure

With CodeReady Containers releases older than 1.2.0, to resolve expired certificate errors:

  1. Download the latest CodeReady Containers release and place the crc binary in your $PATH.

  2. Remove the cluster with certificate errors using the crc delete command:

    $ crc delete
    Warning

    The crc delete command results in the loss of data stored in the CodeReady Containers virtual machine. Save any desired information stored in the virtual machine before running this command.

  3. Set up the new release:

    $ crc setup
  4. Start the new virtual machine:

    $ crc start

7.4. Troubleshooting bundle version mismatch

Created CodeReady Containers virtual machines contain bundle information and instance data. Bundle information and instance data is not updated when setting up a new CodeReady Containers release. This information is not updated due to customization in the earlier instance data. This will lead to errors when running the crc start command:

$ crc start
...
FATA Bundle 'crc_hyperkit_4.2.8.crcbundle' was requested, but the existing VM is using
'crc_hyperkit_4.2.2.crcbundle'
Procedure
  1. Issue the crc delete command before attempting to start the instance:

    $ crc delete
    Warning

    The crc delete command results in the loss of data stored in the CodeReady Containers virtual machine. Save any desired information stored in the virtual machine before running this command.