Initial commit

.gitlab-ci.yml

+image: "transit/kubectl-plus:v1.19.4"
+
+stages:
+    - prepare
+    - build
+    - deploy
+    - delete
+
+secrets:
+  stage: prepare
+  script:
+    - kubectl delete secret gitlab-registry-credentials || true
+    - "kubectl create secret docker-registry gitlab-registry-credentials \
+            --docker-server=$CI_REGISTRY \
+            --docker-username=image-registry \ 
+            --docker-password=$CI_REGISTRY_TOKEN \ 
+        "
+
+grpc:
+  stage: build
+  image:
+    name: gcr.io/kaniko-project/executor:debug
+    entrypoint: [""]
+  only:
+    changes:
+    - .gitlab-ci.yml
+    - grpc/*
+    - grpc/*/*
+  script:
+    - mkdir -p /kaniko/.docker
+    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
+    - cd grpc
+    - /kaniko/executor --context . --destination $CI_REGISTRY_IMAGE/grpc:latest
+
+rest:
+  stage: build
+  image:
+    name: gcr.io/kaniko-project/executor:debug
+    entrypoint: [""]
+  only:
+    changes:
+    - .gitlab-ci.yml
+    - rest/*
+    - rest/*/*
+  script:
+    - mkdir -p /kaniko/.docker
+    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
+    - cd grpc
+    - /kaniko/executor --context . --destination $CI_REGISTRY_IMAGE/rest:latest
+
+grpc-svc:
+  stage: deploy
+  script:
+  - mo deploy/grpc-dep.yaml | kubectl delete -f - || true
+  - mo deploy/grpc-dep.yaml | kubectl apply -f -
+  - mo deploy/grpc-svc.yaml | kubectl apply -f -
+
+rest-svc:
+  stage: deploy
+  script:
+  - mo deploy/rest-dep.yaml | kubectl delete -f - || true
+  - mo deploy/rest-dep.yaml | kubectl apply -f -
+  - mo deploy/rest-svc.yaml | kubectl apply -f -
+
+monitor-svc:
+  stage: deploy
+  script:
+  - mo deploy/monitor-dep.yaml | kubectl delete -f - || true
+  - mo deploy/monitor-dep.yaml | kubectl apply -f -
+  - mo deploy/monitor-svc.yaml | kubectl apply -f -

README.md

-# lab-grpc
+# Lab 09: Request-Response-basierte Interaktion mit gRPC
 
+gRPC (gRPC Remote Procedure Calls) ist ein Protokoll zum Aufruf von Funktionen in verteilten Computersystemen. Es basiert auf HTTP/2 und [Protocol Buffers](https://developers.google.com/protocol-buffers). [gRPC](https://grpc.io) wird von der [Cloud Native Computing Foundation](https://www.cncf.io) als ["incubating project"](https://www.cncf.io/projects) bewertet.
+
+Dieses Lab bietet eine grundlegende Einführung in die Arbeit mit gRPC (auf Basis von Python) und umfasst:
+
+- Definieren eines Dienstes in einer .proto-Datei.
+- Server- und Client-Code mit Hilfe des Protokollpuffer-Compilers zu generieren
+- Die Python gRPC-API für ein einfaches Client-Server-Setting zu nutzen.
+
+Es basiert auf dem offiziellen [gRPC Tutorial](https://grpc.io/docs/languages/python). Diese Lab ist allerdings so vorbereitet, dass Sie automatisch in Kubernetes deployen können.
+
+## Inhalt
+- [Lab 09: Request-Response-basierte Interaktion mit gRPC](#lab-09-request-response-basierte-interaktion-mit-grpc)
+  - [Inhalt](#inhalt)
+  - [Vorbereitung](#vorbereitung)
+  - [Übung 01:](#übung-01)
+
+## Vorbereitung
+
+Sie benötigen auf Ihren lokalen Entwicklungsrechner:
+
+- [Python 3](https://www.python.org/downloads)
+- [Docker](https://www.docker.com/get-started)
+
+Führen Sie anschließend bitte folgende Schritte aus:
+
+1. [Forken](https://git.mylab.th-luebeck.de/cloud-native/lab-rest/-/forks/new) Sie bitte dieses GitLab-Repository in Ihren GitLab-Namensraum.
+2. Klonen Sie das in Schritt 1 geforkte Repository bitte anschließend auf ihrem lokalen Rechner (`git clone`).
+3. Installieren Sie anschließend bitte lokal auf Ihrem Rechner die Kubernetes-IDE [Lens](https://k8slens.dev/).
+4. Laden Sie sich Ihre `kubeconfig` Datei im [Moodle-Kurs](https://to-be-donex) herunter.
+5. Starten Sie Lens und fügen Sie der IDE die kubeconfig Datei hinzu, um auf Ihren Cluster zugreifen zu können. Sie sollten dann Ihren Namespace in dem für Sie bereitgestellten K8S-Cluster sehen.
+6. Erstellen Sie anschließend in Gitlab unter `Einstellungen -> Repository -> Bereitstellungstoken` für das in Schritt 1 geforkte Repository einen Bereitstellungstoken, um selbstgebaute Container Images deployen zu können.
+    - **Name:** `Registry read access (deployment)`
+    - **Username:** `image-registry` (bitte exakt so!)
+    - **Scope:** `read-registry` (nicht mit read repository verwechseln!)
+    - Klicken Sie anschließend auf Bereitstellungstoken erstellen und kopieren Sie sich dieses geheime Token in die Zwischenablage!
+7. Hinterlegen Sie nun für Gitlab Build-Pipelines dieses geheime Token unter `Einstellungen -> CI/CD -> Variables (Aufklappen) -> ADD VARIABLE` als CI/CD-Variable.
+    - **Key:** `CI_REGISTRY_TOKEN` (exakt so)
+    - **Value:** Fügen Sie hier das geheime Token (Schritt 2) aus der Zwischenablage ein.
+    - **Type:** `Variable` (nichts anderes)
+    - **Flags:** Selektieren Sie `Mask Variable` damit das geheime Token in Log-Dateien maskiert wird.
+8. Hinterlegen Sie in Ihrem geforkten GitLab-Repository nun die `kubeconfig`-Datei als CI-Environment-Variable mittels `Einstellungen -> CI/CI -> Variables (Aufklappen) -> ADD VARIABLE` (setzen Sie hierfür folgende Werte)
+    - **Key:** `KUBECONFIG` (Exakt so eingeben)
+    - **Value:** Inhalt der kubeconfig (z.B. mittels Copy-Paste aus Editor)
+    - **Typ:** `File` (Auswählen, WICHTIG!!!)
+
+## Übung 01:

deploy/grpc-dep.yaml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: grpc-server
+spec:
+  selector:
+    matchLabels:
+      app: grpc-server
+  template:
+    metadata:
+      labels:
+        app: grpc-server
+    spec:
+      containers:
+      - name: grpc-server
+        image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
+        ports:
+        - containerPort: 5555

deploy/grpc-svc.yaml

+apiVersion: v1
+kind: Service
+metadata:
+  name: grpc-svc
+spec:
+  selector:
+    app: grpc-server
+  ports:
+  - port: 5000
+    targetPort: 5000

deploy/monitor-dep.yaml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: monitor
+spec:
+  selector:
+    matchLabels:
+      app: monitor
+  template:
+    metadata:
+      labels:
+        app: monitor
+    spec:
+      containers:
+      - name: monitor
+        image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
+        cmd: "greeter_monitor.py"
+        env:
+        - name: GRPC_SVC
+          value: grpc-svc
+        - name: REST_SVC
+          value: rest-svc
+        ports:
+        - containerPort: 8080

deploy/monitor-svc.yaml

+apiVersion: v1
+kind: Service
+metadata:
+  name: monitor-svc
+spec:
+  selector:
+    app: monitor
+  ports:
+  - port: 8080
+    targetPort: 8080

deploy/rest-dep.yaml

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: rest-server
+spec:
+  selector:
+    matchLabels:
+      app: rest-server
+  template:
+    metadata:
+      labels:
+        app: rest-server
+    spec:
+      containers:
+      - name: rest-server
+        image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
+        ports:
+        - containerPort: 5000

deploy/rest-svc.yaml

+apiVersion: v1
+kind: Service
+metadata:
+  name: rest-svc
+spec:
+  selector:
+    app: rest-server
+  ports:
+  - port: 5000
+    targetPort: 5000

grpc/Dockerfile

+FROM python:3.9-slim
+
+COPY Requirements.txt /app/Requirements.txt
+RUN pip3 install -r /app/Requirements.txt
+
+COPY . /app/
+WORKDIR /app
+
+RUN python3 -m grpc_tools.protoc --proto_path=. --python_out=. --grpc_python_out=. helloworld.proto
+
+EXPOSE 5555
+EXPOSE 8080
+
+ENTRYPOINT ["python3", "-u"]
+
+CMD ["greeter_server.py"]

grpc/Requirements.txt

+grpcio
+grpcio-tools
+googleapis-common-protos
+flask
+requests
\ No newline at end of file

grpc/greeter_client.py

+import requests
+import time
+from statistics import mean
+from tqdm import tqdm
+
+import grpc
+import helloworld_pb2
+import helloworld_pb2_grpc
+
+channel = grpc.insecure_channel('localhost:5555')
+stub = helloworld_pb2_grpc.GreeterStub(channel)
+
+grpc_s = []
+rest_s = []
+
+for i in tqdm(range(0, 100)):
+    start = time.time()
+    response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
+    grpc_s.append(time.time() - start)
+
+    start = time.time()
+    x = requests.get("http://localhost:5000/hello/you")
+    rest_s.append(time.time() - start)
+
+print(f"gRPC: { mean(grpc_s) * 1000 }ms")
+print(f"REST: { mean(rest_s) * 1000 }ms")
\ No newline at end of file

grpc/greeter_monitor.py

+import flask
+import os, time, requests, statistics
+import grpc
+import helloworld_pb2
+import helloworld_pb2_grpc
+
+grpc_svc = os.environ.get("GRPC_SVC", "localhost")
+grpc_port = os.environ.get("GRPC_PORT", "5555")
+
+rest_svc = os.environ.get("REST_SVC", "localhost")
+rest_port = os.environ.get("REST_PORT", "5000")
+
+channel = grpc.insecure_channel(f"{ grpc_svc }:{ grpc_port }")
+stub = helloworld_pb2_grpc.GreeterStub(channel)
+
+app = flask.Flask(__name__)
+
+@app.route('/', methods=['GET'])
+def monitor():
+    grpc_s = []
+    rest_s = []
+    n = 100
+
+    for i in range(0, n):
+        start = time.time()
+        x = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
+        grpc_s.append(time.time() - start)
+
+        start = time.time()
+        y = requests.get(f"http://{ rest_svc }:{ rest_port }/hello/you")
+        rest_s.append(time.time() - start)
+        
+    return ", ".join([
+        f"gRPC: { statistics.mean(grpc_s) * 1000 }ms",
+        f"REST: { statistics.mean(rest_s) * 1000 }ms",
+        f"n = { n }"
+    ])
+
+app.run(host="0.0.0.0", port=8080)

grpc/greeter_server.py

+from concurrent import futures
+
+import grpc
+from datetime import datetime
+
+import helloworld_pb2
+import helloworld_pb2_grpc
+
+# Handle gRPC calls
+class Greeter(helloworld_pb2_grpc.GreeterServicer):
+
+    def SayHello(self, request, context):
+        # print(f"{ context.peer() } | { datetime.now().isoformat() } | Request { request } ".strip())
+        return helloworld_pb2.HelloReply(message=f"Hello, { request.name }!")
+
+# Server setup and start
+server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
+helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
+port = 5555
+print(f"listening on port { port }")
+server.add_insecure_port(f"[::]:{ port }")
+server.start()
+server.wait_for_termination()
\ No newline at end of file

grpc/helloworld.proto

+syntax = "proto3";
+
+// The greeting service definition.
+service Greeter {
+  // Sends a greeting
+  rpc SayHello (HelloRequest) returns (HelloReply) {}
+  
+}
+
+// The request message containing the user's name.
+message HelloRequest {
+  string name = 1;
+}
+
+// The response message containing the greetings
+message HelloReply {
+  string message = 1;
+}
\ No newline at end of file

grpc/helloworld_pb2.py

+# -*- coding: utf-8 -*-
+# Generated by the protocol buffer compiler.  DO NOT EDIT!
+# source: helloworld.proto
+"""Generated protocol buffer code."""
+from google.protobuf import descriptor as _descriptor
+from google.protobuf import message as _message
+from google.protobuf import reflection as _reflection
+from google.protobuf import symbol_database as _symbol_database
+# @@protoc_insertion_point(imports)
+
+_sym_db = _symbol_database.Default()
+
+
+
+
+DESCRIPTOR = _descriptor.FileDescriptor(
+  name='helloworld.proto',
+  package='',
+  syntax='proto3',
+  serialized_options=None,
+  create_key=_descriptor._internal_create_key,
+  serialized_pb=b'\n\x10helloworld.proto\"\x1c\n\x0cHelloRequest\x12\x0c\n\x04name\x18\x01 \x01(\t\"\x1d\n\nHelloReply\x12\x0f\n\x07message\x18\x01 \x01(\t23\n\x07Greeter\x12(\n\x08SayHello\x12\r.HelloRequest\x1a\x0b.HelloReply\"\x00\x62\x06proto3'
+)
+
+
+
+
+_HELLOREQUEST = _descriptor.Descriptor(
+  name='HelloRequest',
+  full_name='HelloRequest',
+  filename=None,
+  file=DESCRIPTOR,
+  containing_type=None,
+  create_key=_descriptor._internal_create_key,
+  fields=[
+    _descriptor.FieldDescriptor(
+      name='name', full_name='HelloRequest.name', index=0,
+      number=1, type=9, cpp_type=9, label=1,
+      has_default_value=False, default_value=b"".decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
+  ],
+  extensions=[
+  ],
+  nested_types=[],
+  enum_types=[
+  ],
+  serialized_options=None,
+  is_extendable=False,
+  syntax='proto3',
+  extension_ranges=[],
+  oneofs=[
+  ],
+  serialized_start=20,
+  serialized_end=48,
+)
+
+
+_HELLOREPLY = _descriptor.Descriptor(
+  name='HelloReply',
+  full_name='HelloReply',
+  filename=None,
+  file=DESCRIPTOR,
+  containing_type=None,
+  create_key=_descriptor._internal_create_key,
+  fields=[
+    _descriptor.FieldDescriptor(
+      name='message', full_name='HelloReply.message', index=0,
+      number=1, type=9, cpp_type=9, label=1,
+      has_default_value=False, default_value=b"".decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
+  ],
+  extensions=[
+  ],
+  nested_types=[],
+  enum_types=[
+  ],
+  serialized_options=None,
+  is_extendable=False,
+  syntax='proto3',
+  extension_ranges=[],
+  oneofs=[
+  ],
+  serialized_start=50,
+  serialized_end=79,
+)
+
+DESCRIPTOR.message_types_by_name['HelloRequest'] = _HELLOREQUEST
+DESCRIPTOR.message_types_by_name['HelloReply'] = _HELLOREPLY
+_sym_db.RegisterFileDescriptor(DESCRIPTOR)
+
+HelloRequest = _reflection.GeneratedProtocolMessageType('HelloRequest', (_message.Message,), {
+  'DESCRIPTOR' : _HELLOREQUEST,
+  '__module__' : 'helloworld_pb2'
+  # @@protoc_insertion_point(class_scope:HelloRequest)
+  })
+_sym_db.RegisterMessage(HelloRequest)
+
+HelloReply = _reflection.GeneratedProtocolMessageType('HelloReply', (_message.Message,), {
+  'DESCRIPTOR' : _HELLOREPLY,
+  '__module__' : 'helloworld_pb2'
+  # @@protoc_insertion_point(class_scope:HelloReply)
+  })
+_sym_db.RegisterMessage(HelloReply)
+
+
+
+_GREETER = _descriptor.ServiceDescriptor(
+  name='Greeter',
+  full_name='Greeter',
+  file=DESCRIPTOR,
+  index=0,
+  serialized_options=None,
+  create_key=_descriptor._internal_create_key,
+  serialized_start=81,
+  serialized_end=132,
+  methods=[
+  _descriptor.MethodDescriptor(
+    name='SayHello',
+    full_name='Greeter.SayHello',
+    index=0,
+    containing_service=None,
+    input_type=_HELLOREQUEST,
+    output_type=_HELLOREPLY,
+    serialized_options=None,
+    create_key=_descriptor._internal_create_key,
+  ),
+])
+_sym_db.RegisterServiceDescriptor(_GREETER)
+
+DESCRIPTOR.services_by_name['Greeter'] = _GREETER
+
+# @@protoc_insertion_point(module_scope)

grpc/helloworld_pb2_grpc.py

+# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
+"""Client and server classes corresponding to protobuf-defined services."""
+import grpc
+
+import helloworld_pb2 as helloworld__pb2
+
+
+class GreeterStub(object):
+    """The greeting service definition.
+    """
+
+    def __init__(self, channel):
+        """Constructor.
+
+        Args:
+            channel: A grpc.Channel.
+        """
+        self.SayHello = channel.unary_unary(
+                '/Greeter/SayHello',
+                request_serializer=helloworld__pb2.HelloRequest.SerializeToString,
+                response_deserializer=helloworld__pb2.HelloReply.FromString,
+                )
+
+
+class GreeterServicer(object):
+    """The greeting service definition.
+    """
+
+    def SayHello(self, request, context):
+        """Sends a greeting
+        """
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+        context.set_details('Method not implemented!')
+        raise NotImplementedError('Method not implemented!')
+
+
+def add_GreeterServicer_to_server(servicer, server):
+    rpc_method_handlers = {
+            'SayHello': grpc.unary_unary_rpc_method_handler(
+                    servicer.SayHello,
+                    request_deserializer=helloworld__pb2.HelloRequest.FromString,
+                    response_serializer=helloworld__pb2.HelloReply.SerializeToString,
+            ),
+    }
+    generic_handler = grpc.method_handlers_generic_handler(
+            'Greeter', rpc_method_handlers)
+    server.add_generic_rpc_handlers((generic_handler,))
+
+
+ # This class is part of an EXPERIMENTAL API.
+class Greeter(object):
+    """The greeting service definition.
+    """
+
+    @staticmethod
+    def SayHello(request,
+            target,
+            options=(),
+            channel_credentials=None,
+            call_credentials=None,
+            insecure=False,
+            compression=None,
+            wait_for_ready=None,
+            timeout=None,
+            metadata=None):
+        return grpc.experimental.unary_unary(request, target, '/Greeter/SayHello',
+            helloworld__pb2.HelloRequest.SerializeToString,
+            helloworld__pb2.HelloReply.FromString,
+            options, channel_credentials,
+            insecure, call_credentials, compression, wait_for_ready, timeout, metadata)

rest/Dockerfile

+FROM python:3.9-alpine
+
+RUN pip3 install flask
+
+COPY . /app/
+WORKDIR /app
+EXPOSE 5000
+
+ENTRYPOINT ["python3", "-u", "greeter_rest.py"]

rest/Requirements.txt

+flask
\ No newline at end of file