New to Stash? Please start here.
This tutorial will show you how to use Stash to backup a Kubernetes deployment. At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube. Now, install Stash in your cluster following the steps here.
In this tutorial, we are going to backup the /source/data folder of a busybox pod into a local backend. First deploy the following busybox Deployment in your cluster. Here we are using a git repository as source volume for demonstration purpose.
$ kubectl apply -f ./docs/examples/tutorial/busybox.yaml
deployment "stash-demo" created
apiVersion: apps/v1beta1
kind: Deployment
metadata:
labels:
app: stash-demo
name: stash-demo
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
app: stash-demo
name: busybox
spec:
containers:
- command:
- sleep
- "3600"
image: busybox
imagePullPolicy: IfNotPresent
name: busybox
volumeMounts:
- mountPath: /source/data
name: source-data
restartPolicy: Always
volumes:
- gitRepo:
repository: https://github.com/appscode/stash-data.git
name: source-data
Run the following command to confirm that busybox pods are running.
$ kubectl get pods -l app=stash-demo
NAME READY STATUS RESTARTS AGE
stash-demo-788ffcf9c6-6t6lj 1/1 Running 0 12s
Now, create a Secret that contains the key RESTIC_PASSWORD. This will be used as the password for your restic repository.
$ kubectl create secret generic stash-demo --from-literal=RESTIC_PASSWORD=changeit
secret "stash-demo" created
You can check that the secret was created like this:
$ kubectl get secret stash-demo -o yaml
apiVersion: v1
data:
RESTIC_PASSWORD: Y2hhbmdlaXQ=
kind: Secret
metadata:
creationTimestamp: 2017-12-04T05:24:22Z
name: stash-demo
namespace: default
resourceVersion: "22328"
selfLink: /api/v1/namespaces/default/secrets/stash-demo
uid: 62aa8ef8-d8b3-11e7-be92-0800277f19c0
type: Opaque
Now, create a Restic CRD with selectors matching the labels of the busybox Deployment.
$ kubectl apply -f ./docs/examples/tutorial/restic.yaml
restic "stash-demo" created
apiVersion: stash.appscode.com/v1alpha1
kind: Restic
metadata:
name: stash-demo
namespace: default
spec:
selector:
matchLabels:
app: stash-demo
fileGroups:
- path: /source/data
retentionPolicyName: 'keep-last-5'
backend:
local:
mountPath: /safe/data
hostPath:
path: /data/stash-test/restic-repo
storageSecretName: stash-demo
schedule: '@every 1m'
volumeMounts:
- mountPath: /source/data
name: source-data
retentionPolicies:
- name: 'keep-last-5'
keepLast: 5
prune: true
Here,
spec.selector is used to select workloads upon which this Restic configuration will be applied. Restic always selects workloads in the same Kubernetes namespace. In this tutorial, labels of busybox Deployment match this Restic's selectors. If multiple Restic objects are matched to a given workload, Stash operator will error out and avoid adding sidecar container.
spec.retentionPolicies defines an array of retention policies, which can be used in fileGroups using retentionPolicyName.
spec.fileGroups indicates an array of local paths that will be backed up using restic. For each path, users can also specify the retention policy for old snapshots using retentionPolicyName, which must be defined in spec.retentionPolicies. Here, we are backing up the /source/data folder and only keeping the last 5 snapshots.
spec.backend.local indicates that restic will store the snapshots in a local path /safe/data. For the purpose of this tutorial, we are using an hostPath to store the snapshots. But any Kubernetes volume that can be mounted locally can be used as a backend (example, NFS, Ceph, etc). Stash can also store snapshots in cloud storage solutions like S3, GCS, Azure, etc. To use a remote backend, you need to configure the storage secret to include your cloud provider credentials and set one of spec.backend.(s3|gcs|azure|swift|b2). Please visit here for more detailed examples.
spec.backend.storageSecretName points to the Kubernetes secret created earlier in this tutorial. Restic always points to secrets in its own namespace. This secret is used to pass restic repository password and other cloud provider secrets to restic binary.
spec.schedule is a cron expression that indicates that file groups will be backed up every 1 minute.
spec.volumeMounts refers to volumes to be mounted in stash sidecar to get access to fileGroup path /source/data.
Stash operator watches for Restic objects using Kubernetes api. Stash operator will notice that the busybox Deployment matches the selector for stash-demo Restic object. So, it will add a sidecar container named stash to busybox Deployment and restart the running busybox pods. Since a local backend is used in stash-demo Restic, sidecar container will mount the corresponding persistent volume.
$ kubectl get pods -l app=stash-demo
NAME READY STATUS RESTARTS AGE
stash-demo-788ffcf9c6-6t6lj 0/1 Terminating 0 3m
stash-demo-79554ff97b-wsdx2 2/2 Running 0 49s
$ kubectl get deployment stash-demo -o yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
restic.appscode.com/last-applied-configuration: |
{"kind":"Restic","apiVersion":"stash.appscode.com/v1alpha1","metadata":{"name":"stash-demo","namespace":"default","selfLink":"/apis/stash.appscode.com/v1alpha1/namespaces/default/restics/stash-demo","uid":"d8768901-d8b9-11e7-be92-0800277f19c0","resourceVersion":"27379","creationTimestamp":"2017-12-04T06:10:37Z"},"spec":{"selector":{"matchLabels":{"app":"stash-demo"}},"fileGroups":[{"path":"/source/data","retentionPolicyName":"keep-last-5"}],"backend":{"storageSecretName":"stash-demo","local":{"volumeSource":{"hostPath":{"path":"/data/stash-test/restic-repo"}},"path":"/safe/data"}},"schedule":"@every 1m","volumeMounts":[{"name":"source-data","mountPath":"/source/data"}],"resources":{},"retentionPolicies":[{"name":"keep-last-5","keepLast":5,"prune":true}]},"status":{}}
restic.appscode.com/tag: canary
creationTimestamp: 2017-12-04T06:08:55Z
generation: 2
labels:
app: stash-demo
name: stash-demo
namespace: default
resourceVersion: "27401"
selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/stash-demo
uid: 9c2bf209-d8b9-11e7-be92-0800277f19c0
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 2
selector:
matchLabels:
app: stash-demo
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: stash-demo
name: busybox
spec:
containers:
- command:
- sleep
- "3600"
image: busybox
imagePullPolicy: IfNotPresent
name: busybox
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /source/data
name: source-data
- args:
- backup
- --restic-name=stash-demo
- --workload-kind=Deployment
- --workload-name=stash-demo
- --run-via-cron=true
- --v=3
env:
- name: NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
image: appscode/stash:0.7.0
imagePullPolicy: IfNotPresent
name: stash
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /tmp
name: stash-scratchdir
- mountPath: /etc/stash
name: stash-podinfo
- mountPath: /source/data
name: source-data
readOnly: true
- mountPath: /safe/data
name: stash-local
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- gitRepo:
repository: https://github.com/appscode/stash-data.git
name: source-data
- emptyDir: {}
name: stash-scratchdir
- downwardAPI:
defaultMode: 420
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.labels
path: labels
name: stash-podinfo
- hostPath:
path: /data/stash-test/restic-repo
type: ""
name: stash-local
status:
conditions:
- lastTransitionTime: 2017-12-04T06:10:37Z
lastUpdateTime: 2017-12-04T06:10:37Z
message: Deployment does not have minimum availability.
reason: MinimumReplicasUnavailable
status: "False"
type: Available
- lastTransitionTime: 2017-12-04T06:08:55Z
lastUpdateTime: 2017-12-04T06:10:37Z
message: ReplicaSet "stash-demo-79554ff97b" is progressing.
reason: ReplicaSetUpdated
status: "True"
type: Progressing
observedGeneration: 2
replicas: 2
unavailableReplicas: 2
updatedReplicas: 1
Now, wait until the deployment is in running state. Once the pod with sidecar is in running state, it will create a Repository CRD for this deployment with name deployment.stash-demo. Check the Repository CRD object has been created successfully by,
$ kubectl get repository -l workload-name=stash-demo
NAME AGE
deployment.stash-demo 3m
Restic will take backup of the volume periodically with a 1-minute interval. You can verify that backup is taking successfully by,
$ kubectl get snapshots -l repository=deployment.stash-demo
NAME AGE
deployment.stash-demo-c1014ca6 10s
Here, deployment.stash-demo-c1014ca6 represents the name of the successful backup Snapshot taken by Stash in deployment.stash-demo repository.
You can also check the status.backupCount of deployment.stash-demo Repository CRD to see number of successful backup taken in this repository.
$ kubectl get repository deployment.stash-demo -o yaml
apiVersion: stash.appscode.com/v1alpha1
kind: Repository
metadata:
clusterName: ""
creationTimestamp: 2018-03-29T08:28:10Z
generation: 0
labels:
restic: stash-demo
workload-kind: Deployment
workload-name: stash-demo
name: deployment.stash-demo
namespace: default
resourceVersion: "999"
selfLink: /apis/stash.appscode.com/v1alpha1/namespaces/default/repositories/deployment.stash-demo
uid: 1d814fdb-332b-11e8-94e6-08002792cb23
spec:
backend:
local:
hostPath:
path: /data/stash-test/restic-repo
mountPath: /safe/data
storageSecretName: local-secret
backupPath: deployment/stash-demo
status:
backupCount: 1
firstBackupTime: 2018-03-29T08:29:10Z
lastBackupDuration: 2.105757874s
lastBackupTime: 2018-03-29T08:29:10Z
To stop Restic from taking backup, you can do following things:
Set spec.paused: true in Restic yaml and then apply the update. This means:
$ kubectl patch restic stash-demo --type="merge" --patch='{"spec": {"paused": true}}'
restic "stash-demo" patched
apiVersion: stash.appscode.com/v1alpha1
kind: Restic
metadata:
name: stash-demo
namespace: default
spec:
selector:
matchLabels:
app: stash-demo
fileGroups:
- path: /source/data
retentionPolicyName: 'keep-last-5'
backend:
local:
mountPath: /safe/data
hostPath:
path: /data/stash-test/restic-repo
storageSecretName: stash-demo
schedule: '@every 1m'
paused: true
volumeMounts:
- mountPath: /source/data
name: source-data
retentionPolicies:
- name: 'keep-last-5'
keepLast: 5
prune: true
$ kubectl delete restic stash-demo
restic "stash-demo" deleted
You can resume Restic to backup by setting spec.paused: false in Restic yaml and applying the update or you can patch Restic using,
$ kubectl patch restic stash-demo --type="merge" --patch='{"spec": {"paused": false}}'
To cleanup the Kubernetes resources created by this tutorial, run:
$ kubectl delete deployment stash-demo
$ kubectl delete secret stash-demo
$ kubectl delete restic stash-demo
$ kubectl delete repository deployment.stash-demo
If you would like to uninstall Stash operator, please follow the steps here.