This readme contains guidance for users relating to the deployment and use of Kraan.
helm repo add kraan https://fidelity.github.io/kraan
helm install kraan kraan/kraan-controller --namespace gotk-system
This chart deploys a Kraan and the Gitops Toolkit components it uses on a Kubernetes cluster using the Helm package manager.
To install the chart with the release name kraan in namespace gotk-system:
kubectl create namespace gotk-system
helm install kraan kraan/kraan-controller --namespace gotk-system
To uninstall the kraan deployment:
helm uninstall kraan
The command removes all the Kubernetes components associated with the chart and deletes the release.
The following table lists the configurable parameters of the Prometheus chart and their default values.
| Parameter | Description | Default |
|---|---|---|
global.extraPodAnnotations |
list of annotations to add to kraan-controller, source-controller and helm-controller, see samples/local.yaml for example. |
{} |
global.extraLabels |
list of labels to add to kraan-controller, source-controller and helm-controller |
{} |
global.prometheusAnnotations |
Annotations to be applied to controllers to allow prometheus server scrapping. | prometheus.io/scrape: "true"prometheus.io/port: "8080" |
global.env.httpsProxy |
HTTPS proxy url to use. | |
global.env.noProxy |
no proxy CIDRs. | 10.0.0.0/8,172.0.0.0/8 |
kraan.crd.enabled |
install/update Kraan CRDs | true |
kraan.rbac.enabled |
install/update Kraan RBAC rules | true |
kraan.netpolicy.enabled |
install/update Kraan network policies | true |
kraan.kraanController.enabled |
install/update Kraan Controller | true |
kraan.kraanController.name |
name of Kraan Controller | kraan-controller |
kraan.kraanController.extraPodAnnotations |
list of annotations to add to kraan-controller |
{} |
kraan.kraanController.extraLabels |
list of labels to add to kraan-controller |
{} |
kraan.kraanController.prometheus.enabled |
apply prometheus annotations to kraan-controller |
true |
kraan.kraanController.imagePullSecrets.name |
name of Kraan Controller’s imagePullSecrets |
|
kraan.kraanController.image.repository |
Kraan Controller’s repository | kraan |
kraan.kraanController.image.name |
Kraan Controller’s image name | kraan-controller |
kraan.kraanController.image.tag |
Kraan Controller’s image tag | .chart.AppVersion |
kraan.kraanController.image.imagePullPolicy |
Kraan Controller’s image pull policy | InNotPresent |
kraan.kraanController.args.logLevel |
Kraan Controller’s log level, 0 for info, 1 for debug, 2 or greater for trace levels | 0 |
kraan.kraanController.args.syncPeriod |
The period between reprocessing of all AddonsLayers | 1m |
kraan.kraanController.devmode |
set to true when running a development image to allow writes to container filesystem | false |
kraan.kraanController.resources |
resource settings for kraan-controller |
limits:cpu: 1000mmemory: 1Girequests:cpu: 500mmemory: 128Mi |
kraan.kraanController.tolerations |
tolerations for kraan-controller |
{} |
kraan.kraanController.nodeSelector |
nodeSelector settings for kraan-controller |
{} |
kraan.kraanController.affinity |
affinity settings for kraan-controller |
{} |
gotk.rbac.enabled |
install/update Kraan RBAC rules | true |
gotk.netpolicy.enabled |
install/update Kraan network policies | true |
gotk.sourceController.crd.enabled |
install/update Source Controller CRDs | true |
gotk.sourceController.enabled |
install/update Source Controller | true |
gotk.sourceController.name |
name of Source Controller | source-controller |
gotk.sourceController.extraPodAnnotations |
list of annotations to add to source-controller |
{} |
gotk.sourceController.extraLabels |
list of labels to add to source-controller |
{} |
gotk.sourceController.prometheus.enabled |
apply prometheus annotations to source-controller |
true |
gotk.sourceController.imagePullSecrets.name |
name of Source Controller’s imagePullSecrets |
|
gotk.sourceController.image.repository |
Source Controller’s repository | ghcr.io/fluxcd |
gotk.sourceController.image.name |
Source Controller’s image name | source-controller |
gotk.sourceController.image.tag |
Source Controller’s image tag | v0.7.1 |
gotk.sourceController.image.imagePullPolicy |
image pull policy | InNotPresent |
gotk.sourceController.resources |
resource settings for source-controller |
limits:cpu: 1000mmemory: 1Girequests:cpu: 500mmemory: 128Mi |
gotk.sourceController.tolerations |
tolerations for source-controller |
{} |
gotk.sourceController.nodeSelector |
nodeSelector settings for source-controller |
{} |
gotk.sourceController.affinity |
affinity settings for source-controller |
{} |
gotk.helmController.crd.enabled |
install/update Helm Controller CRDs | true |
gotk.helmController.enabled |
install/update Helm Controller | true |
gotk.helmController.name |
name of Helm Controller | helm-controller |
gotk.helmController.extraPodAnnotations |
list of annotations to add to helm-controller |
{} |
gotk.helmController.extraLabels |
list of labels to add to helm-controller |
{} |
gotk.helmController.prometheus.enabled |
apply prometheus annotations to helm-controller |
true |
gotk.helmController.imagePullSecrets.name |
name of Helm Controller’s imagePullSecrets |
|
gotk.helmController.image.repository |
Helm Controller’s repository | ghcr.io/fluxcd |
gotk.helmController.image.name |
Source Controller’s image name | helm-controller |
gotk.helmController.image.tag |
Helm Controller’s image tag | v0.6.1 |
gotk.helmController.image.imagePullPolicy |
image pull policy | InNotPresent |
gotk.helmController.resources |
resource settings for helm-controller |
limits:cpu: 1000mmemory: 1Girequests:cpu: 500mmemory: 128Mi |
gotk.helmController.tolerations |
tolerations for helm-controller |
{} |
gotk.helmController.nodeSelector |
nodeSelector settings for helm-controller |
{} |
gotk.helmController.affinity |
affinity settings for helm-controller |
{} |
Specify each parameter using the --set key=value[,key=value] argument to helm install or use the --values to specify a comma seperated list of values files. The samples directory contains some example values files.
The Kraan-Controller is designed to deploy HelmReleases to a Kubernetes cluster. Using addonslayers.kraan.io custom resources it retrieves the definitions of one or more HelmReleases from a git repository and applies them to the cluster then waits until they are all successfully deployed before marking the AddonsLayer as Deployed. By creating multiple AddonsLayers custom resources and setting the dependencies between them the user can sequence the deployment of multiple layers of addons.
If a HelmRelease is removed from one AddonLayer’s git repository source path and added to another layer’s git repositiory source path then Kraan will detect this and not allow the existing HelmRelease on the cluster to be ‘adpoted’ by the layer it is now defined in rather than being deleted and readded. However to achieve this behaviour both AddonLayers need to be updated at the same time.
The ‘prune’ processing for the layer it has been removed from will mark the HelmReelease as ‘orphaned’ and wait for a period set by that layer’s interval setting, which defauts to one minute, before uninstalling it. This allows an opportunity for another layer to identify that the HelmRelease in its git repository source path is present on the cluster but currently owned by another layer. In this case if the HelmRelease has been updated by the other layer with an ‘orphaned’ label this layer will change the owner and apply it to the cluster, which will cause the Helm Controller to upgrade it if any changes have been made or do nothing if the HelmRelease definition matches the current state on the cluster.
If a HelmRelease is defined in multiple layers at the same time the first layer to process it will acquire ownership and the other layer(s) will fail to apply. This will be clearly reported in the AddonLayer status. It is possible that this error could occur briefly in the sceanrio where a HelmRelease is moved from one layer to another if the layer it is now defined in processes before the layer that it has been removed from, but this situation will be resolved when the layer it has been removed from processes and adds an ‘orphaned’ label.
An AddonsLayer references a gitrepository.source.toolkit.fluxcd.io custom resource which it uses to retrieve data from a git repository using the Source-Controller. The source element of the AddonsLayer custom resource references a GitRepository custom resource the. The path element under source is the path relative to the top directory of the git repository referenced by the GitRepository custom resource where the HelmReleases comprising this AddonsLayer are defined.
An AddonsLayer can also optionally include a prereqs element containing the minimum version of the Kubernetes API required by the AddonsLayer. If specified, the AddonsLayer will not be applied until the cluster API version is greater than or equal to the specified version. The Kraan-Controller will regularly check the Cluster API version.
The interval field is used to specify the period to wait before reprocessing an AddonsLayer. Note that all AddonsLayers are reprocessed periodically. The period between reprocessing of all AddonsLayers defaults to one minute but can set using the syncPeriod value, see Configuration section above.
The interval field is also used to define the period to wait for another layer to adopt a HelmRelease that has been removed from a layer. See Pruning section below for more details.
The timeout field is used to set the period to wait for HelmReleases to be deployed before setting the AddonsLayer’s status to failed.
The hold setting can be used to prevent processing of the AddonsLayer. Set to true to enable this feature.
The version field defines the version of the AddonsLayer. This can be used to define a new version of the AddonsLayer. Changing the version affects other AddonsLayers that are dependent on this layer. If you change the version of an AddonsLayer you need to update the version in dependsOn field in the dependent layer to make that layer dependent on the new version of this layer.
Modifying the version field can be used to force redeployment of HelmReleases that have not changed. This feature can be activated by adding an annotation to the HelmRelease definition. Setting kraan.updateVersion: "true" will cause the Kraan-Controller to add a value to that HelmRelease with key kraanVersion and value of the AddonsLayer’s version. This means that if the version has changed since the last time the AddonsLayer was processed the HelmRelease will be redeployed. This feature enables the user configure an integration test HelmRelease for an AddonsLayer which will be run on AddonsLayer version change even if it has not changed. By using the HelmRelease dependsOn feature you can ensure this HelmRelease is not deployed until all other HelmReleases in the layer are deployed, see testdata/addons/bootstrap for an example.
The version field does not need to be updated, simply commiting a change to the git repository branch referenced by the GitRepository custom resource that is referenced in the AddonsLayer’s source element or editing that GitRepository to reference a different tag, commit or even a different git repository will cause Kraan to reprocess the AddonsLayer.
apiVersion: kraan.io/v1alpha1
kind: AddonsLayer
metadata:
name: bootstrap
spec:
version: 0.1.01
hold: false
interval: 1m
source:
name: addons-config
namespace: gotk-system
path: ./testdata/addons/bootstrap
prereqs:
k8sVersion: "v1.16"
---
apiVersion: kraan.io/v1alpha1
kind: AddonsLayer
metadata:
name: base
spec:
version: 0.1.01
interval: 1m
source:
name: addons-config
namespace: gotk-system
path: ./testdata/addons/base
prereqs:
k8sVersion: "v1.16"
dependsOn:
- bootstrap@0.1.01
The Kraan-Controller monitors any helmrelease.helm.toolkit.fluxcd.io resources owned by a AddonsLayer and reprocesses the AddonsLayer when it detects changes to the HelmReleases it owns. This means that if a HelmRelease is deleted or amended using kubectl or other cluster management tools, Kraan-Controller will redeploy it using the definition in the git repository references by the AddonsLayer source field. To prevent this, set the hold field in the AddonsLayer to true.
When processing an AddonsLayer the Kraan-Controller first prunes any HelmReleases owned by the AddonsLayer that are not defined in the git repository. This means that removing a HelmRelease definition from a git repository will cause it to be uninstalled from the cluster.
Kraan-Controller performs the prune processing on all AddonLayers without reference to layer dependencies. It then proceeds with deploying addons, waiting for any layers it depends on to be deployed first. Before pruning HelmReleases the Kraan-Controller waits for a configurable period of time to see if the HelmRelease has been moved to another layer.
The interval field of the layer a HelmRelease has been removed from is used to define the period to wait for another layer to adopt a HelmRelease that has been removed from that layer. When a HelmRelease is removed from a layer an orphaned label containing the current date and time is added to the HelmRelease. The pruning process will be deferred for the interval period before proceeding with the uninstallation of the HelmRelease. This is designed to allow time for another layer to adopt the HelmRelease if it is now defined in its repository source. The HelmRelease is adopted by removing the orphaned label and changing the owner.
The Kraan Controller generates Kubernetes Events for AddonsLayer custom resources. It also includes the status of the HelmReleases managed by an AddonsLayer in the custom resource status.
kubectl describe al base
Name: base
Namespace:
Labels: <none>
Annotations: ...
API Version: kraan.io/v1alpha1
Kind: AddonsLayer
Metadata:
Creation Timestamp: 2020-11-30T10:31:11Z
Finalizers:
finalizers.kraan.io
Generation: 4
Managed Fields: ...
Spec:
Interval: 1m
Prereqs:
Depends On:
bootstrap@0.1.03
k8sVersion: v1.16
Source:
Kind: gitrepositories.source.toolkit.fluxcd.io
Name: addons-config
Path: ./testdata/addons/base
Timeout: 30s
Version: 0.1.03
Status:
Conditions:
Last Transition Time: 2020-12-01T10:28:16Z
Message: AddonsLayer failed, HelmRelease: base/microservice-1, not ready
Reason: Failed
Status: True
Type: Failed
Observed Generation: 4
Resources:
Kind: helmreleases.helm.toolkit.fluxcd.io
Last Transition Time: 2020-12-01T10:27:57Z
Name: microservice-1
Namespace: base
Status: TestFailed
Revision: master/6a226b05a5aa0a775c2147d5b8b3b14d1adfa094
State: Failed
Version: 0.1.03
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
....
Normal Deployed 47m kraan-controller AddonsLayer version 0.1.03 is Deployed, All HelmReleases deployed
Normal ApplyPending 37s kraan-controller Waiting for layer: bootstrap, to apply source revision: main/5bfb0..... Layer: bootstrap, current state: Applying, deployed revision: master/6a226...
Normal Applying 31s kraan-controller AddonsLayer is being applied
Normal Failed 1s (x2 over 23s) kraan-controller AddonsLayer failed, HelmRelease: base/microservice-1, not ready
A ‘HelmRelease not deployed’ log message will be emmitted if a HelmRelease fails to deploy. This message includes the layer name as well as HelmRelease namespace and name, e.g.
{
"level": "info",
"ts": "2020-12-01T10:28:22.700Z",
"logger": "kraan.controller.reconciler",
"msg": "HelmRelease not deployed",
"function": "controllers.(*AddonsLayerReconciler).setHelmReleaseFailed",
"source": "addons_controller.go",
"line": 417,
"layer": "base",
"name": "base/microservice-1"
}
The Kraan-Controller also generates metrics for Deployed and Failed conditions.