From 7b5c58e3017ff41f1d4f4bb0af1abad9d8b7bb06 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Wed, 26 May 2021 01:18:48 -0400 Subject: [PATCH 1/9] Polish up the tutorial document Signed-off-by: jesus m. rodriguez --- docs/tutorial.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 3249df6..cff6a4f 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -1,27 +1,25 @@ ---- -title: Quarkus Operator Tutorial -linkTitle: Tutorial -weight: 30 -description: An in-depth walkthough of building and running a Quarkus-based operator. ---- +# Java Operator Tutorial +### An in-depth walkthrough of building and running a Java-based operator. ## Prerequisites -- Java through the [installation guide](https://java.com/en/download/help/download_options.html). +- [Operator SDK](https://sdk.operatorframework.io/docs/installation/) v1.8.0 or newer +- [Java](https://java.com/en/download/help/download_options.html) 11 +- [Maven 3.6.3](https://maven.apache.org/install.html) or newer - User authorized with `cluster-admin` permissions. -- Maven installation [installation guide](https://maven.apache.org/install.html) ## Overview We will create a sample project to let you know how it works and this sample will: - Create a Memcached Deployment if it doesn't exist -- Ensure that the Deployment size is the same as specified by the Memcached Custom Resource spec -- Update the Memcached Custom Resource status using the status writer with the names of the Custom Resource's pods +- Ensure that the Deployment size is the same as specified by the Memcached Custom Resource (CR) spec +- Update the Memcached CR status using the status writer with the names of the CR's pods ## Create a new project -Use the CLI to create a new memcached-quarkus-operator project: +Use the [Operator SDK](https://sdk.operatorframework.io/docs/installation/) CLI to create a +new memcached-quarkus-operator project: ```sh mkdir memcached-quarkus-operator @@ -447,4 +445,4 @@ Delete One of the Pod forcefully then Memcached operator will create new automat `kubectl delete pod pod-name` -In the end, change the size from the memcached-sample.yaml file and apply it to the cluster. After these steps, an operator will make sure that the cluster has an updated number of pods in it. \ No newline at end of file +In the end, change the size from the memcached-sample.yaml file and apply it to the cluster. After these steps, an operator will make sure that the cluster has an updated number of pods in it. From 80fdd3f9afc925cbce50e405d4401f0ca6f304cd Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Wed, 26 May 2021 01:51:12 -0400 Subject: [PATCH 2/9] Add more color commentary to the API section --- docs/tutorial.md | 61 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 14 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index cff6a4f..a7b382a 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -35,9 +35,19 @@ operator-sdk init --plugins quarkus --domain example.com --project-name memcache ### MemcachedQuarkusOperator -The main program of the operator `MemcachedQuarkusOperator.java` initializes and runs the operator. The operator uses java-operator-sdk which is similar to the Go lang version of controller-runtime. +The quarkus plugin will scaffold out several files during the `init` phase. One +of these files is the operator's main program, `MemcachedQuarkusOperator.java`. +This file initializes and runs the operator. The operator uses java-operator-sdk, +which is similar to +[controller-runtime](https://github.com/kubernetes-sigs/controller-runtime), to +make operator development easier. -The code below will initialize and define the informers/watches for your operator. +The important part of the `MemcachedQuarkusOperator.java` is the `run` method +which will start the operator and initializes the informers and watches for your +operator. + +Here is an example of the `run` method that will typically be scaffolded out by +this plugin: ``` @Override @@ -51,16 +61,21 @@ The code below will initialize and define the informers/watches for your operato ## Create a new API and Controller -Create a new Custom Resource Definition (CRD) API with group `cache` version `v1` and Kind `Memcached`. The plugin, still in its alpha state, will output debug messages which are normal. +An operator isn't much good without an API to work with. Create a new Custom +Resource Definition (CRD) API with group `cache`, version `v1`, and Kind +`Memcached`. -`create api` command will scaffold the `MemcachedController`, `MemcachedSpec`, `MemcachedStatus` and `Memcached`. +Use the `create api` command to scaffold the `MemcachedController`, +`MemcachedSpec`, `MemcachedStatus` and `Memcached`. These files represent the +API. The plugin may show some debug statements which is normal as it is still in +the alpha state. ```console $ operator-sdk create api --plugins quarkus --group cache --version v1 --kind Memcached -... ``` -After the `create api` command the file structure will be shown as below. +After running the `create api` command the file structure will change to match the +one shown as below. ``` $ tree @@ -93,10 +108,13 @@ The `java-operator-plugins` project uses the APIs from [java-operator-sdk](https #### `MemcachedSpec` -Initially, the scaffolded Spec file will be empty. The operator developer needs to add attributes to this file according to his need. For the Memcached example, we added the size field as shown below example. +Initially, the scaffolded Spec file will be empty. The operator developer needs +to add attributes to this file according to their needs. For the `Memcached` +example, we will add the size field as shown in the example below. + +The `MemcachedSpec` class defines the desired state of `Memcached`. ``` -// MemcachedSpec defines the desired state of Memcached public class MemcachedSpec { // Add Spec information here @@ -113,17 +131,25 @@ public class MemcachedSpec { } ``` +As you can see, we added the `size` attribute along with corresponding getter +and setter. + #### `MemcachedStatus` -Similar to the Spec file `MemcachedStatus` file got scaffolded as part of the `create api` command. The user has to modify a Status file. For the Memcached example, we too list of nodes as shown below. -The nodes field is a list of string values and it contains the name of the Memcached pods. +Similar to the Spec file above, the `MemcachedStatus` file was scaffolded as +part of the `create api` command. The user will need to modify a Status file +in order to add any desired attributes. For this `Memcached` example, we will +add a list of nodes as shown below. + +The nodes field is a list of string values and it contains the name of +the Memcached pods. The `MemcachedStatus` defines the observed state +of `Memcached`. ``` import java.util.ArrayList; import java.util.List; -// MemcachedStatus defines the observed state of Memcached public class MemcachedStatus { // Add Status information here @@ -143,14 +169,19 @@ public class MemcachedStatus { } ``` -**Note** The Node field is just to illustrate an example of a Status field. In real cases, it would be recommended to use [Conditions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties). +**Note** The Node field is just to illustrate an example of a Status field. In +real use cases, it is recommended that you use +[Conditions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties). #### `Memcached` -`Memcached` file scaffolded via `create api` command. It extends the properties of `MemcachedSpec` and `MemcachedStatus`. +Now that we have Spec and Status classes, let's look at the `Memcached` class. +This file was also scaffolded via `create api` command. Notice it extends both +`MemcachedSpec` and `MemcachedStatus`. + +The `Memcached` is the Schema for the Memcacheds API. ``` -// Memcached is the Schema for the memcacheds API @Version("v1alpha1") @Group("cache.example.com") @@ -158,6 +189,8 @@ public class Memcached extends CustomResource implements Namespaced {} ``` +You have now created the necessary classes for the API. + ### Apply Custom Resource and CRD's using below command #### CRD - From a182942a6c4601481671267dc83e18031208da83 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Wed, 26 May 2021 11:49:52 -0400 Subject: [PATCH 3/9] Add color to the createOrUpdateResource method --- docs/tutorial.md | 194 ++++++++++++++++++++++++++++++----------------- 1 file changed, 123 insertions(+), 71 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index a7b382a..8b0c20d 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -193,11 +193,15 @@ You have now created the necessary classes for the API. ### Apply Custom Resource and CRD's using below command -#### CRD - +There are a couple of ways to create the CRD. You can either create the file +manually. OR let the quarkus extensions defined in pom.xml use the annotations +on your Spec/Status classes to create the crd files for you. -Create a file with the name `crd.yaml`. CRD enables users to add their own/custom objects to the Kubernetes cluster. +#### Manually create `crd.yaml` -This file will contain the below content. +Create a file with the name `crd.yaml`. A CRD enables users to add their +own/custom objects to the Kubernetes cluster. Below you will find an example +`Memcached` CRD. ``` apiVersion: apiextensions.k8s.io/v1 @@ -218,7 +222,7 @@ spec: - name: v1 schema: openAPIV3Schema: - ... + type: object served: true storage: true subresources: @@ -231,15 +235,15 @@ status: storedVersions: [] ``` +### TODO: Creating the CRD seems out of place Create the CRD: `kubectl apply -f crd.yaml` +######### -### Create Memcached Custom Resource - memcached-sample.yaml +### Create sample Memcached Custom Resource -Create the sample Memcached Custom Resource manifest at k8s/samples/memcached-sample.yaml and define the spec as the following. - -This file will contain the below content. +Let's create the sample Memcached Custom Resource manifest at `memcached-sample.yaml` and define the spec as the following. ``` apiVersion: cache.example.com/v1 @@ -251,35 +255,51 @@ spec: size: 1 ``` +### TODO out of place Create the Custom Resource: `kubectl apply -f memcached-sample.yaml` +################3 ## Implement the Controller -Add the below-mentioned code snippet in `MemcachedController.java` file. Initially, this file will contain the empty methods `createOrUpdateResource` and `deleteResource`. Please add the below code in respective methods as part of controller logic. Also, add the `createMemcachedDeployment` method that will create the Deployment for your operator. +By now we have the API defined in `Memcached.java`, `MemcachedSpec.java`, +`MemcachedStatus.java`. We also have the CRD and the sample Custom Resource. +This isn't enough, we still need a controller to reconcile these items. + +The `create api` command will have scaffolded a skeleton `MemcachedController.java`. +This controller implements the `ResourceController` interface from the +`java-operator-sdk`. This interface has some important and useful methods. -**Note**: Next two subsections explain the two methods `createOrUpdateResource` and `deleteResource`. These two methods get called whenever some update/create/delete event occurs in the cluster. +Initially the `MemcachedController.java` will contain the empty stubs for +`createOrUpdateResource` and `deleteResource`. In this section we will fill in +the controller logic in these methods. We will also add a +`createMemcachedDeployment` method that will create the Deployment for our +operator. -These methods are already scaffolded as part of the `create api` command. In this memcached example, we will need to watch the deployment so we can react to size changes. We will accomplish this in the steps below. +The `createOrUpdateResource` and `deleteResource` get called whenever some +update/create/delete event occurs in the cluster. This will allow us to react to +changes to the Deployment. ### createOrUpdateResource -First, let's get the Deployment. In the MemachedController.java, find the createOrUpdateResource method and add the following code below the `// TODO: fill in logic` comment. It should look like: +In this section we will focus on implementing the `createOrUpdateResource` +method. In the `MemcachedController.java` you will see a `// TODO: fill in logic` +comment. At this line we will first add code to get the Deployment. ``` - Deployment deployment = - client - .apps() - .deployments() - .inNamespace(resource.getMetadata().getNamespace()) - .withName(resource.getMetadata().getName()) - .get(); -``` + Deployment deployment = client.apps() + .deployments() + .inNamespace(resource.getMetadata().getNamespace()) + .withName(resource.getMetadata().getName()) + .get(); -If the deployment is `null` that means we need to create the deployment for it. In the `MemachedController.java`, find the `createOrUpdateResource` method and add the following code. +``` -Below code will verify that Deployment within the cluster got created or not. If deployment is null then it will create deployment. `createMemcachedDeployment(resource)` creates the Deployment and then it will be applied by using `client.apps().deployments().create(newDeployment);` code. `createMemcachedDeployment(resource)` method explained in the next part. +Once we get the `deployment`, we have a couple of decisions to make. If it is +`null` it does not exist which means we need to create the deployment. In the +`MemachedController.java`, in the `createOrUpdateResource` method just below the +get deployment code we added above, add the following: ``` if (deployment == null) { @@ -289,17 +309,33 @@ Below code will verify that Deployment within the cluster got created or not. If } ``` +In the above code, we are checking to see if the deployment exists, if not we +will create it by calling the yet to be defined `createMemcachedDeployment` +method. + + +############################## +Below code will verify that Deployment within the cluster got created or not. If deployment is null then it will create deployment. `createMemcachedDeployment(resource)` creates the Deployment and then it will be applied by using `client.apps().deployments().create(newDeployment);` code. `createMemcachedDeployment(resource)` method explained in the next part. + +############################## + Once we create the deployment, we need to decide whether we have to reconcile it or not. -If there is no need of reconciliation then return `UpdateControl.noUpdate()` else we need to return `UpdateControl.updateStatusSubResource(resource)` +If there is no need of reconciliation then return `UpdateControl.noUpdate()` +else we need to return `UpdateControl.updateStatusSubResource(resource)` -After the Creation of the Deployment, get the current and required replicas by using the below code. +After getting the Deployment, we get the current and required replicas. Add the +following lines below the `if (deployment == null)` block in your +`MemcachedController.java` file. ``` int currentReplicas = deployment.getSpec().getReplicas(); int requiredReplicas = resource.getSpec().getSize(); ``` -If currentReplicas does not match with requiredReplicas then we need to update the `Deployment`. This process will be done by using the below code. +Once we get the replicas, we need to determine if they are different so that we +can reconcile. If `currentReplicas` does not match the `requiredReplicas` then +we need to update the `Deployment`. Add the following comparison block to your +controller. ``` if (currentReplicas != requiredReplicas) { @@ -309,22 +345,28 @@ If currentReplicas does not match with requiredReplicas then we need to update t } ``` -Then, let's get the list of pods and pod names. In the `MemachedController.java`, find the `createOrUpdateResource` method and add the following code. It should look like: +The above sections will cover reconciling any `size` changes to the Spec. In the +next section, we will look at handling the changes to the `nodes` list from the +Status. + +Let's get the list of pods and their names. In the `MemcachedController.java`, +add the following code below the `if (currentReplicas != requiredReplicas) {` +block. ``` - List pods = - client - .pods() - .inNamespace(resource.getMetadata().getNamespace()) - .withLabels(labelsForMemcached(resource)) - .list() - .getItems(); + List pods = client.pods() + .inNamespace(resource.getMetadata().getNamespace()) + .withLabels(labelsForMemcached(resource)) + .list() + .getItems(); List podNames = - pods.stream().map(p -> p.getMetadata().getName()).collect(Collectors.toList()); + pods.stream().map(p -> p.getMetadata().getName()).collect(Collectors.toList()); ``` -Now, check whether resources get created or not. Then, it verifies podnames with the Memcached resources. If there is a mismatch in either of these conditions then we need to do a reconciliation. +Now that we have the pods and names. What do we do next? Well, we check whether resources +were created. Then, it verifies podnames with the Memcached resources. If there is a +mismatch in either of these conditions then we need to do a reconciliation. ``` if (resource.getStatus() == null @@ -335,24 +377,19 @@ Now, check whether resources get created or not. Then, it verifies podnames with } ``` -The complete `createOrUpdateResource` function will look like below in `MemachedController.java`. +That's it we have completed the `createOrUpdateResource` method. The method +should now look like the following: ``` @Override public UpdateControl createOrUpdateResource( Memcached resource, Context context) { // TODO: fill in logic - System.out.println("Create or Update Control"); - Deployment deployment = - client - .apps() - .deployments() - .inNamespace(resource.getMetadata().getNamespace()) - .withName(resource.getMetadata().getName()) - .get(); - - System.out.println(deployment); - + Deployment deployment = client.apps() + .deployments() + .inNamespace(resource.getMetadata().getNamespace()) + .withName(resource.getMetadata().getName()) + .get(); if (deployment == null) { Deployment newDeployment = createMemcachedDeployment(resource); @@ -362,37 +399,62 @@ The complete `createOrUpdateResource` function will look like below in `Memached int currentReplicas = deployment.getSpec().getReplicas(); int requiredReplicas = resource.getSpec().getSize(); + if (currentReplicas != requiredReplicas) { deployment.getSpec().setReplicas(requiredReplicas); client.apps().deployments().createOrReplace(deployment); return UpdateControl.noUpdate(); } - List pods = - client - .pods() - .inNamespace(resource.getMetadata().getNamespace()) - .withLabels(labelsForMemcached(resource)) - .list() - .getItems(); + List pods = client.pods() + .inNamespace(resource.getMetadata().getNamespace()) + .withLabels(labelsForMemcached(resource)) + .list() + .getItems(); List podNames = - pods.stream().map(p -> p.getMetadata().getName()).collect(Collectors.toList()); + pods.stream().map(p -> p.getMetadata().getName()).collect(Collectors.toList()); + if (resource.getStatus() == null - || !CollectionUtils.isEqualCollection(podNames, resource.getStatus().getNodes())) { - if (resource.getStatus() == null) resource.setStatus(new MemcachedStatus()); - resource.getStatus().setNodes(podNames); - return UpdateControl.updateStatusSubResource(resource); + || !CollectionUtils.isEqualCollection(podNames, resource.getStatus().getNodes())) { + if (resource.getStatus() == null) resource.setStatus(new MemcachedStatus()); + resource.getStatus().setNodes(podNames); + return UpdateControl.updateStatusSubResource(resource); } return UpdateControl.noUpdate(); } ``` +Let's recap what we did. + +* Get the Deployment, if the deployment does not exist, we create it. +* If the deployment already exists, we get the current replicas and the desired + replicas. +* We compare the replicas, if they do not match, we replace the deployment with + the expected values. +* Next we look at the node list from the pods. If they do not match, we update + and reconcile. + +What's left? If you recall, in the if the deployment is `null`, we call +`createMemcachedDeployment(resource)`. This method still needs to get created. +In the next section, we will walk you through creating this helper method. + ### createMemcachedDeployment -Create `createMemcachedDeployment` method and add the below code snippet. This method simply creates the Deployment. +Creating Kubernetes objects via APIs can be quite verbose which is why putting +them in helper methods can make the code more readable. The +`MemcachedController.java` needs to create a Deployment if it does not exist. In +the `createOrUpdateResource` we make a call to a helper, +`createMemcachedDeployment`. + +Let's create the `createMemcachedDeployment` method. The following code will use +the [`fabric8`](https://fabric8.io/) `DeploymentBuilder` class. Notice the +Deployment specifies the `memcached` image for the pod. + +Below your `deleteResource(Memcached resource, Context context) {` +block in the `MemcachedController.java`, add the following method. ``` private Deployment createMemcachedDeployment(Memcached m) { @@ -441,18 +503,8 @@ Create `createMemcachedDeployment` method and add the below code snippet. This m ### deleteResource The `deleteResource` method is an implemented method. The deletion part will be taken care of by the Java Operator SDK library, that's why it is empty. -The code snippet for `deleteResource` is as shown below. -``` - @Override - public DeleteControl deleteResource(Memcached resource, Context context) { - // nothing to do here... - // framework takes care of deleting the resource object - // k8s takes care of deleting deployment and pods because of ownerreference set - System.out.println("Delete Control"); - return DeleteControl.DEFAULT_DELETE; - } -``` +We have not implemented the `MemcachedController.java`. ## Run the Operator Locally From 885ea6f733cc056bcd603f0106182d2abdaefe32 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 08:44:14 -0400 Subject: [PATCH 4/9] Cleanup run commands --- docs/tutorial.md | 234 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 209 insertions(+), 25 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 8b0c20d..4c0cb34 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -7,6 +7,7 @@ - [Java](https://java.com/en/download/help/download_options.html) 11 - [Maven 3.6.3](https://maven.apache.org/install.html) or newer - User authorized with `cluster-admin` permissions. +- [GNU Make](https://www.gnu.org/software/make/) ## Overview @@ -183,7 +184,7 @@ The `Memcached` is the Schema for the Memcacheds API. ``` -@Version("v1alpha1") +@Version("v1") @Group("cache.example.com") public class Memcached extends CustomResource implements Namespaced {} @@ -194,7 +195,7 @@ You have now created the necessary classes for the API. ### Apply Custom Resource and CRD's using below command There are a couple of ways to create the CRD. You can either create the file -manually. OR let the quarkus extensions defined in pom.xml use the annotations +manually. Or let the quarkus extensions defined in `pom.xml` use the annotations on your Spec/Status classes to create the crd files for you. #### Manually create `crd.yaml` @@ -222,6 +223,26 @@ spec: - name: v1 schema: openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + size: + format: int32 + type: integer + type: object + status: + properties: + nodes: + items: + type: string + type: array + type: object type: object served: true storage: true @@ -235,11 +256,10 @@ status: storedVersions: [] ``` -### TODO: Creating the CRD seems out of place -Create the CRD: +#### Via Quarkus extension -`kubectl apply -f crd.yaml` -######### +TODO: there is currently an issue with the CRD generation that the schema +validation is not properly generated. ### Create sample Memcached Custom Resource @@ -275,7 +295,7 @@ Initially the `MemcachedController.java` will contain the empty stubs for `createOrUpdateResource` and `deleteResource`. In this section we will fill in the controller logic in these methods. We will also add a `createMemcachedDeployment` method that will create the Deployment for our -operator. +operator and a `labelsForMemcached` method that returns the labels. The `createOrUpdateResource` and `deleteResource` get called whenever some update/create/delete event occurs in the cluster. This will allow us to react to @@ -313,12 +333,6 @@ In the above code, we are checking to see if the deployment exists, if not we will create it by calling the yet to be defined `createMemcachedDeployment` method. - -############################## -Below code will verify that Deployment within the cluster got created or not. If deployment is null then it will create deployment. `createMemcachedDeployment(resource)` creates the Deployment and then it will be applied by using `client.apps().deployments().create(newDeployment);` code. `createMemcachedDeployment(resource)` method explained in the next part. - -############################## - Once we create the deployment, we need to decide whether we have to reconcile it or not. If there is no need of reconciliation then return `UpdateControl.noUpdate()` else we need to return `UpdateControl.updateStatusSubResource(resource)` @@ -439,7 +453,27 @@ Let's recap what we did. What's left? If you recall, in the if the deployment is `null`, we call `createMemcachedDeployment(resource)`. This method still needs to get created. -In the next section, we will walk you through creating this helper method. +As well as the `labelsForMemcached` utility method. + +Let's create the utility method first. + +### labelsForMemcached + +A simple utility method to return a map of the labels we want to attach to some +of the resources. Below the `deleteResource` method add the following +helper: + +``` + private Map labelsForMemcached(Memcached m) { + Map labels = new HashMap<>(); + labels.put("app", "memcached"); + labels.put("memcached_cr", m.getMetadata().getName()); + return labels; + } +``` + +In the next section, we will walk you through creating the +`createMemcachedDeployment` utility method. ### createMemcachedDeployment @@ -453,8 +487,8 @@ Let's create the `createMemcachedDeployment` method. The following code will use the [`fabric8`](https://fabric8.io/) `DeploymentBuilder` class. Notice the Deployment specifies the `memcached` image for the pod. -Below your `deleteResource(Memcached resource, Context context) {` -block in the `MemcachedController.java`, add the following method. +Below your `labelsForMemcached(Memcached m)` block in the +`MemcachedController.java`, add the following method. ``` private Deployment createMemcachedDeployment(Memcached m) { @@ -465,7 +499,7 @@ block in the `MemcachedController.java`, add the following method. .withNamespace(m.getMetadata().getNamespace()) .withOwnerReferences( new OwnerReferenceBuilder() - .withApiVersion("v1alpha1") + .withApiVersion("v1") .withKind("Memcached") .withName(m.getMetadata().getName()) .withUid(m.getMetadata().getUid()) @@ -500,15 +534,147 @@ block in the `MemcachedController.java`, add the following method. } ``` +Now we have a `createOrUpdateResource` method. It calls +`createMemcachedDeployment` which we have implemented above. In the next section +we will discuss the deletion of the resource. + ### deleteResource -The `deleteResource` method is an implemented method. The deletion part will be taken care of by the Java Operator SDK library, that's why it is empty. +One of the benefits of the `java-operator-sdk` library is that it handles the +deletion portion for you. The scaffolded `deleteResource` is already implmented +for you. + +We have now implemented the `MemcachedController.java`. + +## Run the Operator + +You can run the operator in a couple of ways. You can run it locally where the +operator runs on your development machine and talks to the cluster. Or it can +build images of your operator and run it directly in the cluster. + +### Running the operator in the cluster + +The following steps will show how to run your operator in the cluster. + +1. Build and push your operator's image: + +``` +make docker-build docker-push IMG=quay.io/YOURUSER/memcached-quarkus-operator:0.0.1 +``` + +This will build the docker image +`quay.io/YOURUSER/memcached-quarkus-operator:0.0.1` and push it to the registry. + +**Note** You can use any docker registry you want i.e. `docker.io`, `quay.io`, +etc. + +You can verify it is in your docker registry: + +``` +$ docker images | grep memcached +quay.io/YOURUSER/memcached-quarkus-operator 0.0.1 c84d2616bc1b 29 seconds ago 236MB +``` + +2. Install the CRD + +Next we will install the CRD into the `default` namespace + +``` +make install +customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created +``` + +``` +$ kubectl apply -f /tmp/crd.yaml +customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created +``` + +3. Create rbac.yaml file -We have not implemented the `MemcachedController.java`. +The RBAC generated in the `kubernetes.yml` only has [view +permissions](https://quarkus.io/guides/deploying-to-kubernetes#using-the-kubernetes-client) +which is not enough to run the operator. For this example, we will simply grant +cluster-admin to the `memcached-quarkus-operator-operator` service account. -## Run the Operator Locally +Create a file called `rbac.yaml` with the following contents: -### Run locally outside the cluster +``` +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: memcached-operator-admin +subjects: +- kind: ServiceAccount + name: memcached-quarkus-operator-operator + namespace: default +roleRef: + kind: ClusterRole + name: cluster-admin + apiGroup: "" +``` + +Do not apply this yet. We will do that in a later step. + +4. Deploy the operator + +``` +make deploy +``` + +5. Grant `cluster-admin` to service account + +Let's grant the `memcached-quarkus-operator-operator` service account +the right privileges. + +``` +kubectl apply -f rbac.yaml +``` + +``` +$ kubectl get all -n default +NAME READY STATUS RESTARTS AGE +pod/memcached-quarkus-operator-operator-7db86ccf58-k4mlm 0/1 Running 0 18s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/kubernetes ClusterIP 10.96.0.1 443/TCP 6m18s +service/memcached-quarkus-operator-operator ClusterIP 10.96.244.231 8080/TCP 18s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/memcached-quarkus-operator-operator 0/1 1 0 18s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/memcached-quarkus-operator-operator-7db86ccf58 1 1 0 18s +``` + +6. Apply the memcached-sample to see the operator create the memcached-sample + pod. + +``` +$ k apply -f memcached-sample.yaml +memcached.cache.example.com/memcached-sample created +``` + +``` +$ k get all +NAME READY STATUS RESTARTS AGE +pod/memcached-quarkus-operator-operator-7b766f4896-kxnzt 1/1 Running 1 79s +pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 18s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/kubernetes ClusterIP 10.96.0.1 443/TCP 8h +service/memcached-quarkus-operator-operator ClusterIP 10.96.236.214 8080/TCP 79s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/memcached-quarkus-operator-operator 1/1 1 1 80s +deployment.apps/memcached-sample 1/1 1 1 19s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/memcached-quarkus-operator-operator-7b766f4896 1 1 1 80s +replicaset.apps/memcached-sample-6c765df685 1 1 1 19s +``` + + +### Running locally outside the cluster The following steps will show how to run your operator locally. @@ -516,17 +682,35 @@ Compile your operator with the below command `mvn clean install` -It will create a `.jar` file for your operator in `target/quarkus-app`. Now, run the `jar` file using the below command. +You should see a nice `BUILD SUCCESS` method like the one below: + +``` +[INFO] ------------------------------------------------------------------------ +[INFO] BUILD SUCCESS +[INFO] ------------------------------------------------------------------------ +[INFO] Total time: 11.193 s +[INFO] Finished at: 2021-05-26T12:16:54-04:00 +[INFO] ------------------------------------------------------------------------ + +``` + +After running this command, notice there is now a `target` directory. That +directory may be a bit overwhelming, but the key thing to know for running locally +is the `target/memcached-quarkus-operator-0.0.1.jar` file created for your +operator and the `quarkus-run.jar` in `target/quarkus-app` directory. + +Now, run the `jar` file using the below command. -`java -jar quarkus-run.jar` +`java -jar target/quarkus-app/quarkus-run.jar` -This command will run your operator locally. You can check cluster pods and deployment with the below commands. +This command will run your operator locally. You can check the cluster pods and +deployment with the following commands. `kubectl get deployment` `kubectl get pods` -Delete One of the Pod forcefully then Memcached operator will create new automatically. +Delete one of the Pod forcefully then Memcached operator will create a new one automatically. `kubectl delete pod pod-name` From b7739cf0f971db0c490875fc87b539660db3ad95 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 09:17:30 -0400 Subject: [PATCH 5/9] Fix manually running commands --- docs/tutorial.md | 189 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 135 insertions(+), 54 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 4c0cb34..7965b71 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -192,7 +192,7 @@ public class Memcached extends CustomResource You have now created the necessary classes for the API. -### Apply Custom Resource and CRD's using below command +### Creating Custom Resource and CRD There are a couple of ways to create the CRD. You can either create the file manually. Or let the quarkus extensions defined in `pom.xml` use the annotations @@ -258,8 +258,18 @@ status: #### Via Quarkus extension -TODO: there is currently an issue with the CRD generation that the schema -validation is not properly generated. +**Note** there is currently an issue with the CRD generation that the schema +validation is not properly generated. Because of this issue, we will not cover +using this portion during this tutorial. Proceed to the +[Create sample Memcached Custom Resource](#create-sample-memcached-custom-resource) section + +Running `mvn install` will invoke the CRD generator extension which will analyze +the annotations on the model objects, `Memcached`, `MemcachedSpec`, +`MemcachedStatus`, and generate the CRD in `target/kubernetes`. + +``` +mvn install +``` ### Create sample Memcached Custom Resource @@ -275,12 +285,6 @@ spec: size: 1 ``` -### TODO out of place -Create the Custom Resource: - -`kubectl apply -f memcached-sample.yaml` -################3 - ## Implement the Controller By now we have the API defined in `Memcached.java`, `MemcachedSpec.java`, @@ -552,12 +556,29 @@ You can run the operator in a couple of ways. You can run it locally where the operator runs on your development machine and talks to the cluster. Or it can build images of your operator and run it directly in the cluster. +In this section we will: + +* install the CRD +* create a Custom Resource +* run your operator + +If you want to run the operator in the cluster see the [Running the operator in +the cluster](#running-the-operator-in-the-cluster) below or if you'd prefer to +run it locally see the +[Running locally outside the cluster](#running-locally-outside-the-cluster) +section instead. + ### Running the operator in the cluster The following steps will show how to run your operator in the cluster. 1. Build and push your operator's image: +The `java-operator-plugins` project will scaffold out a Makefile to give +Operator SDK users a familiar interface. Using the `docker-*` targets you can +conveniently build your and push your operator's image to registry. In our +example, we are using `quay.io`, but any docker registry should work. + ``` make docker-build docker-push IMG=quay.io/YOURUSER/memcached-quarkus-operator:0.0.1 ``` @@ -565,9 +586,6 @@ make docker-build docker-push IMG=quay.io/YOURUSER/memcached-quarkus-operator:0. This will build the docker image `quay.io/YOURUSER/memcached-quarkus-operator:0.0.1` and push it to the registry. -**Note** You can use any docker registry you want i.e. `docker.io`, `quay.io`, -etc. - You can verify it is in your docker registry: ``` @@ -577,15 +595,20 @@ quay.io/YOURUSER/memcached-quarkus-operator 0.0.1 2. Install the CRD -Next we will install the CRD into the `default` namespace +Next we will install the CRD into the `default` namespace. Using the `crd.yaml` +you created in the [Manually created crd.yaml](#manually-create-crdyaml) +section, apply it to the cluster. + ``` -$ kubectl apply -f /tmp/crd.yaml +$ kubectl apply -f crd.yaml customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created ``` @@ -617,70 +640,67 @@ Do not apply this yet. We will do that in a later step. 4. Deploy the operator +Let's deploy your operator to the cluster. The `Makefile` has a convenience +target that can do this for you: + ``` make deploy ``` -5. Grant `cluster-admin` to service account +5. Grant `cluster-admin` to service account -Let's grant the `memcached-quarkus-operator-operator` service account -the right privileges. +Once you've deployed the operator, you will need to grant the +`memcached-quarkus-operator-operator` service account the right privileges. ``` kubectl apply -f rbac.yaml ``` +6. Verify the operator is running + +Ensure the `memcached-quarkus-operator-operator-XXX` pod is in a `Running` +status. + ``` $ kubectl get all -n default NAME READY STATUS RESTARTS AGE pod/memcached-quarkus-operator-operator-7db86ccf58-k4mlm 0/1 Running 0 18s - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/kubernetes ClusterIP 10.96.0.1 443/TCP 6m18s -service/memcached-quarkus-operator-operator ClusterIP 10.96.244.231 8080/TCP 18s - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/memcached-quarkus-operator-operator 0/1 1 0 18s - -NAME DESIRED CURRENT READY AGE -replicaset.apps/memcached-quarkus-operator-operator-7db86ccf58 1 1 0 18s +... ``` -6. Apply the memcached-sample to see the operator create the memcached-sample - pod. +7. Apply the memcached-sample + +Apply the memcached-sample to see the operator create the memcached-sample pod. ``` $ k apply -f memcached-sample.yaml memcached.cache.example.com/memcached-sample created ``` +8. Verify the sample + +Now check the cluster to see if the pod has started. Keep watching until the +`memcached-sample-XXX` pod reaches a `Running` status. + ``` $ k get all NAME READY STATUS RESTARTS AGE pod/memcached-quarkus-operator-operator-7b766f4896-kxnzt 1/1 Running 1 79s pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 18s - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/kubernetes ClusterIP 10.96.0.1 443/TCP 8h -service/memcached-quarkus-operator-operator ClusterIP 10.96.236.214 8080/TCP 79s - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/memcached-quarkus-operator-operator 1/1 1 1 80s -deployment.apps/memcached-sample 1/1 1 1 19s - -NAME DESIRED CURRENT READY AGE -replicaset.apps/memcached-quarkus-operator-operator-7b766f4896 1 1 1 80s -replicaset.apps/memcached-sample-6c765df685 1 1 1 19s +... ``` - ### Running locally outside the cluster -The following steps will show how to run your operator locally. +For development purposes, you may want to run your operator locally for faster +iteration. In the following steps, we will show how to run your operator +locally. -Compile your operator with the below command +1. Compile your operator with the below command -`mvn clean install` +``` +mvn clean install +``` You should see a nice `BUILD SUCCESS` method like the one below: @@ -691,27 +711,88 @@ You should see a nice `BUILD SUCCESS` method like the one below: [INFO] Total time: 11.193 s [INFO] Finished at: 2021-05-26T12:16:54-04:00 [INFO] ------------------------------------------------------------------------ +``` + +2. Install the CRD +Next we will install the CRD into the `default` namespace. Using the `crd.yaml` +you created in the [Manually created crd.yaml](#manually-create-crdyaml) +section, apply it to the cluster. + +``` +$ kubectl apply -f crd.yaml +customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created ``` +3. Create and apply rbac.yaml file + +The RBAC generated in the `kubernetes.yml` only has [view +permissions](https://quarkus.io/guides/deploying-to-kubernetes#using-the-kubernetes-client) +which is not enough to run the operator. For this example, we will simply grant +cluster-admin to the `memcached-quarkus-operator-operator` service account. + +Create a file called `rbac.yaml` with the following contents: + +``` +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: memcached-operator-admin +subjects: +- kind: ServiceAccount + name: memcached-quarkus-operator-operator + namespace: default +roleRef: + kind: ClusterRole + name: cluster-admin + apiGroup: "" +``` + +Let's apply the rbac role to the cluster: + +``` +kubectl apply -f rbac.yaml +``` + +4. Run the operator + After running this command, notice there is now a `target` directory. That directory may be a bit overwhelming, but the key thing to know for running locally is the `target/memcached-quarkus-operator-0.0.1.jar` file created for your operator and the `quarkus-run.jar` in `target/quarkus-app` directory. -Now, run the `jar` file using the below command. +Now, run the `jar` file using the below command. This command will run your +opeator locally. + +``` +java -jar target/quarkus-app/quarkus-run.jar +``` + +**Note** the above will run the operator and remain running until you kill it. +You will need another terminal to complete the rest of these commands. + +5. Apply the memcached-sample -`java -jar target/quarkus-app/quarkus-run.jar` +Apply the memcached-sample to see the operator create the memcached-sample pod. -This command will run your operator locally. You can check the cluster pods and -deployment with the following commands. +``` +$ k apply -f memcached-sample.yaml +memcached.cache.example.com/memcached-sample created +``` -`kubectl get deployment` +6. Verify the sample -`kubectl get pods` +Now check the cluster to see if the pod has started. Keep watching until the +`memcached-sample-XXX` pod reaches a `Running` status. -Delete one of the Pod forcefully then Memcached operator will create a new one automatically. +``` +$ k get all +NAME READY STATUS RESTARTS AGE +pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 18s +... +``` -`kubectl delete pod pod-name` +7. Trigger a reconcile -In the end, change the size from the memcached-sample.yaml file and apply it to the cluster. After these steps, an operator will make sure that the cluster has an updated number of pods in it. +If you modify the size field of the `memcached-sample.yaml` and re-apply it. The +operator will trigger a reconcile and adjust the sample pods to the size given. From 1b00907a449aaebbae85071eb18b2119efcb7f7c Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 09:20:58 -0400 Subject: [PATCH 6/9] Fix MemcachedStatus sample formatting --- docs/tutorial.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 7965b71..2708c90 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -154,7 +154,7 @@ import java.util.List; public class MemcachedStatus { // Add Status information here - // Nodes are the names of the memcached pods + // Nodes are the names of the memcached pods private List nodes; public List getNodes() { From 8b6082624af340cb34eb2653b14d7d0e3d8d34c8 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 09:22:10 -0400 Subject: [PATCH 7/9] Fix the Memcached sample to match actual generation --- docs/tutorial.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/tutorial.md b/docs/tutorial.md index 2708c90..754a04b 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -186,6 +186,8 @@ The `Memcached` is the Schema for the Memcacheds API. @Version("v1") @Group("cache.example.com") +@Kind("Memcached") +@Plural("memcacheds") public class Memcached extends CustomResource implements Namespaced {} ``` From 40a4a968c95842bea0a33fc528fabe0ef1dc283b Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 09:28:18 -0400 Subject: [PATCH 8/9] Use kubectl instead of my alias --- docs/tutorial.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 754a04b..282eb4e 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -675,7 +675,7 @@ pod/memcached-quarkus-operator-operator-7db86ccf58-k4mlm 0/1 Running 0 Apply the memcached-sample to see the operator create the memcached-sample pod. ``` -$ k apply -f memcached-sample.yaml +$ kubectl apply -f memcached-sample.yaml memcached.cache.example.com/memcached-sample created ``` @@ -685,7 +685,7 @@ Now check the cluster to see if the pod has started. Keep watching until the `memcached-sample-XXX` pod reaches a `Running` status. ``` -$ k get all +$ kubectl get all NAME READY STATUS RESTARTS AGE pod/memcached-quarkus-operator-operator-7b766f4896-kxnzt 1/1 Running 1 79s pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 18s @@ -778,7 +778,7 @@ You will need another terminal to complete the rest of these commands. Apply the memcached-sample to see the operator create the memcached-sample pod. ``` -$ k apply -f memcached-sample.yaml +$ kubectl apply -f memcached-sample.yaml memcached.cache.example.com/memcached-sample created ``` @@ -788,7 +788,7 @@ Now check the cluster to see if the pod has started. Keep watching until the `memcached-sample-XXX` pod reaches a `Running` status. ``` -$ k get all +$ kubectl get all NAME READY STATUS RESTARTS AGE pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 18s ... From 5459c28a656b2856c8a63b274c6c2279599a1e50 Mon Sep 17 00:00:00 2001 From: "jesus m. rodriguez" Date: Thu, 27 May 2021 09:41:17 -0400 Subject: [PATCH 9/9] Add step to trigger reconcile using in cluster mode --- docs/tutorial.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/tutorial.md b/docs/tutorial.md index 282eb4e..fdd829a 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -692,6 +692,11 @@ pod/memcached-sample-6c765df685-mfqnz 1/1 Running 0 ... ``` +9. Trigger a reconcile + +If you modify the size field of the `memcached-sample.yaml` and re-apply it. The +operator will trigger a reconcile and adjust the sample pods to the size given. + ### Running locally outside the cluster For development purposes, you may want to run your operator locally for faster