Integration

CRC Housekeeping

CRC provides the ability to develop Openshift applications locally, without relying on a heavyweight infrastructure or a cloud environment. But developers will soon realise that multiple deployments will leave traces, and these will turn the system unstable. In other words, you might get  the infamous error message: Diskpreassure. You could reinstall CRC; it is in fact very simple. But nobody wants to recreate all those secrets and config maps. Fortunately, you have a second option: CRC Housekeeping.

Before we start let us see how we got into this situation. First connect via ssh to your system:

ssh  -i ${HOME}/.crc/machines/crc/id_rsa  core@api.crc.testing

and the run a ‘df -h’ to check the filesystem.

Disk space on the Openshift imageSysroot is the drive where the image registry is located and most of the artifacts are stored. Our main goal is to try to free up some space from “/sysroot” by using the tools provided by crc.

1-Pruning the deployments

Our first task is to recover space used by deployments that are no longer being used:

oc adm prune deployments --orphans

Without the ‘–confirm’ flag, the execution will be a dry run. In other words, it will be a test to see if it works and nothing will be really deleted. If you are happy with the test result, add the ‘–confirm’ flag to  prune the components definitely.

oc adm prune deployments --orphans –-confirm

2- Pruning the builds

Very similar als the one above, but this time you recover the space used by building the components. It will also execute a dry run if you don’t use the confirm flag.

oc adm prune builds --orphans –-confirm

3- Pruning the images

This is the one that will recover the most space because it is similar to the docker purge command. As with the ones above, it will execute as a dry run if you omit the confirm flag:

oc adm prune images --prune-registry

Once you are ready, execute it this way to actually delete the images:

oc adm prune images --prune-registry --confirm

But wait ….

error: failed to ping registry image-registry.openshift-image-registry.svc:5000: 
[Get https://image-registry.openshift-image-registry.svc:5000/: 
dial tcp: lookup image-registry.openshift-image-registry.svc on 192.168.178.1:53: no such host, 
Get http://image-registry.openshift-image-registry.svc:5000/: 
dial tcp: lookup image-registry.openshift-image-registry.svc on 192.168.178.1:53: no such host]

It seems that when we actually want to prune the registry it cannot connect to it. But fortunately there is a solution! Two things need to be done, first export the registry SSL certificate for our connection. Second, expose the registry as a route so that we can access it.

## Change to the Image registry Project in Openshift
oc project openshift-image-registry
## Get one of the secrets that contain the certificate
oc get secrets | grep pruner-token
##Export the certificate (change pruner-token-xxxx with the actual name you got above)
oc get secret pruner-token-xxxx -o yaml| awk '/service.ca/ {print $2}'|base64 -d > image-pruner.crt
##Patch the operator to expose the route
oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge
##Get the route url
oc get route

Now you can run the prune images again. But this time add also two new flags: registry-url and certificate-authority.

oc adm prune images --prune-registry --confirm --registry-url default-route-openshift-image-registry.apps-crc.testing --certificate-authority=image-pruner.crt

And voila, if you connect to your system via ssh again, you can see that some space was recovered.

Conclusion

The steps above explain how to do some Housekeeping in CRC in order to claim back some disk space. There are some more complex technics to resize the disk, like the ones in this thread: https://github.com/code-ready/crc/issues/127

Also, CRC is constantly evolving and it may implement a way to define the disk size before installing it, like minishift did. But in the meantime, this commands may save you a headache.