Commit cd847ca7 authored by Nane Kratzke's avatar Nane Kratzke
Browse files

Initial commit

parent 383f288d
image: "transit/kubectl-plus:v1.19.4"
stages:
- prepare
- build
- deploy
- delete
secrets:
stage: prepare
script:
- kubectl delete secret gitlab-registry-credentials || true
- "kubectl create secret docker-registry gitlab-registry-credentials \
--docker-server=$CI_REGISTRY \
--docker-username=image-registry \
--docker-password=$CI_REGISTRY_TOKEN \
"
grpc:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
only:
changes:
- .gitlab-ci.yml
- grpc/*
- grpc/*/*
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd grpc
- /kaniko/executor --context . --destination $CI_REGISTRY_IMAGE/grpc:latest
rest:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
only:
changes:
- .gitlab-ci.yml
- rest/*
- rest/*/*
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd grpc
- /kaniko/executor --context . --destination $CI_REGISTRY_IMAGE/rest:latest
grpc-svc:
stage: deploy
script:
- mo deploy/grpc-dep.yaml | kubectl delete -f - || true
- mo deploy/grpc-dep.yaml | kubectl apply -f -
- mo deploy/grpc-svc.yaml | kubectl apply -f -
rest-svc:
stage: deploy
script:
- mo deploy/rest-dep.yaml | kubectl delete -f - || true
- mo deploy/rest-dep.yaml | kubectl apply -f -
- mo deploy/rest-svc.yaml | kubectl apply -f -
monitor-svc:
stage: deploy
script:
- mo deploy/monitor-dep.yaml | kubectl delete -f - || true
- mo deploy/monitor-dep.yaml | kubectl apply -f -
- mo deploy/monitor-svc.yaml | kubectl apply -f -
# lab-grpc
# Lab 09: Request-Response-basierte Interaktion mit gRPC
gRPC (gRPC Remote Procedure Calls) ist ein Protokoll zum Aufruf von Funktionen in verteilten Computersystemen. Es basiert auf HTTP/2 und [Protocol Buffers](https://developers.google.com/protocol-buffers). [gRPC](https://grpc.io) wird von der [Cloud Native Computing Foundation](https://www.cncf.io) als ["incubating project"](https://www.cncf.io/projects) bewertet.
Dieses Lab bietet eine grundlegende Einführung in die Arbeit mit gRPC (auf Basis von Python) und umfasst:
- Definieren eines Dienstes in einer .proto-Datei.
- Server- und Client-Code mit Hilfe des Protokollpuffer-Compilers zu generieren
- Die Python gRPC-API für ein einfaches Client-Server-Setting zu nutzen.
Es basiert auf dem offiziellen [gRPC Tutorial](https://grpc.io/docs/languages/python). Diese Lab ist allerdings so vorbereitet, dass Sie automatisch in Kubernetes deployen können.
## Inhalt
- [Lab 09: Request-Response-basierte Interaktion mit gRPC](#lab-09-request-response-basierte-interaktion-mit-grpc)
- [Inhalt](#inhalt)
- [Vorbereitung](#vorbereitung)
- [Übung 01:](#übung-01)
## Vorbereitung
Sie benötigen auf Ihren lokalen Entwicklungsrechner:
- [Python 3](https://www.python.org/downloads)
- [Docker](https://www.docker.com/get-started)
Führen Sie anschließend bitte folgende Schritte aus:
1. [Forken](https://git.mylab.th-luebeck.de/cloud-native/lab-rest/-/forks/new) Sie bitte dieses GitLab-Repository in Ihren GitLab-Namensraum.
2. Klonen Sie das in Schritt 1 geforkte Repository bitte anschließend auf ihrem lokalen Rechner (`git clone`).
3. Installieren Sie anschließend bitte lokal auf Ihrem Rechner die Kubernetes-IDE [Lens](https://k8slens.dev/).
4. Laden Sie sich Ihre `kubeconfig` Datei im [Moodle-Kurs](https://to-be-donex) herunter.
5. Starten Sie Lens und fügen Sie der IDE die kubeconfig Datei hinzu, um auf Ihren Cluster zugreifen zu können. Sie sollten dann Ihren Namespace in dem für Sie bereitgestellten K8S-Cluster sehen.
6. Erstellen Sie anschließend in Gitlab unter `Einstellungen -> Repository -> Bereitstellungstoken` für das in Schritt 1 geforkte Repository einen Bereitstellungstoken, um selbstgebaute Container Images deployen zu können.
- **Name:** `Registry read access (deployment)`
- **Username:** `image-registry` (bitte exakt so!)
- **Scope:** `read-registry` (nicht mit read repository verwechseln!)
- Klicken Sie anschließend auf Bereitstellungstoken erstellen und kopieren Sie sich dieses geheime Token in die Zwischenablage!
7. Hinterlegen Sie nun für Gitlab Build-Pipelines dieses geheime Token unter `Einstellungen -> CI/CD -> Variables (Aufklappen) -> ADD VARIABLE` als CI/CD-Variable.
- **Key:** `CI_REGISTRY_TOKEN` (exakt so)
- **Value:** Fügen Sie hier das geheime Token (Schritt 2) aus der Zwischenablage ein.
- **Type:** `Variable` (nichts anderes)
- **Flags:** Selektieren Sie `Mask Variable` damit das geheime Token in Log-Dateien maskiert wird.
8. Hinterlegen Sie in Ihrem geforkten GitLab-Repository nun die `kubeconfig`-Datei als CI-Environment-Variable mittels `Einstellungen -> CI/CI -> Variables (Aufklappen) -> ADD VARIABLE` (setzen Sie hierfür folgende Werte)
- **Key:** `KUBECONFIG` (Exakt so eingeben)
- **Value:** Inhalt der kubeconfig (z.B. mittels Copy-Paste aus Editor)
- **Typ:** `File` (Auswählen, WICHTIG!!!)
## Übung 01:
apiVersion: apps/v1
kind: Deployment
metadata:
name: grpc-server
spec:
selector:
matchLabels:
app: grpc-server
template:
metadata:
labels:
app: grpc-server
spec:
containers:
- name: grpc-server
image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
ports:
- containerPort: 5555
apiVersion: v1
kind: Service
metadata:
name: grpc-svc
spec:
selector:
app: grpc-server
ports:
- port: 5000
targetPort: 5000
apiVersion: apps/v1
kind: Deployment
metadata:
name: monitor
spec:
selector:
matchLabels:
app: monitor
template:
metadata:
labels:
app: monitor
spec:
containers:
- name: monitor
image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
cmd: "greeter_monitor.py"
env:
- name: GRPC_SVC
value: grpc-svc
- name: REST_SVC
value: rest-svc
ports:
- containerPort: 8080
apiVersion: v1
kind: Service
metadata:
name: monitor-svc
spec:
selector:
app: monitor
ports:
- port: 8080
targetPort: 8080
apiVersion: apps/v1
kind: Deployment
metadata:
name: rest-server
spec:
selector:
matchLabels:
app: rest-server
template:
metadata:
labels:
app: rest-server
spec:
containers:
- name: rest-server
image: {{ CI_REGISTRY_IMAGE }}/grpc:latest
ports:
- containerPort: 5000
apiVersion: v1
kind: Service
metadata:
name: rest-svc
spec:
selector:
app: rest-server
ports:
- port: 5000
targetPort: 5000
FROM python:3.9-slim
COPY Requirements.txt /app/Requirements.txt
RUN pip3 install -r /app/Requirements.txt
COPY . /app/
WORKDIR /app
RUN python3 -m grpc_tools.protoc --proto_path=. --python_out=. --grpc_python_out=. helloworld.proto
EXPOSE 5555
EXPOSE 8080
ENTRYPOINT ["python3", "-u"]
CMD ["greeter_server.py"]
grpcio
grpcio-tools
googleapis-common-protos
flask
requests
\ No newline at end of file
import requests
import time
from statistics import mean
from tqdm import tqdm
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
channel = grpc.insecure_channel('localhost:5555')
stub = helloworld_pb2_grpc.GreeterStub(channel)
grpc_s = []
rest_s = []
for i in tqdm(range(0, 100)):
start = time.time()
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
grpc_s.append(time.time() - start)
start = time.time()
x = requests.get("http://localhost:5000/hello/you")
rest_s.append(time.time() - start)
print(f"gRPC: { mean(grpc_s) * 1000 }ms")
print(f"REST: { mean(rest_s) * 1000 }ms")
\ No newline at end of file
import flask
import os, time, requests, statistics
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
grpc_svc = os.environ.get("GRPC_SVC", "localhost")
grpc_port = os.environ.get("GRPC_PORT", "5555")
rest_svc = os.environ.get("REST_SVC", "localhost")
rest_port = os.environ.get("REST_PORT", "5000")
channel = grpc.insecure_channel(f"{ grpc_svc }:{ grpc_port }")
stub = helloworld_pb2_grpc.GreeterStub(channel)
app = flask.Flask(__name__)
@app.route('/', methods=['GET'])
def monitor():
grpc_s = []
rest_s = []
n = 100
for i in range(0, n):
start = time.time()
x = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
grpc_s.append(time.time() - start)
start = time.time()
y = requests.get(f"http://{ rest_svc }:{ rest_port }/hello/you")
rest_s.append(time.time() - start)
return ", ".join([
f"gRPC: { statistics.mean(grpc_s) * 1000 }ms",
f"REST: { statistics.mean(rest_s) * 1000 }ms",
f"n = { n }"
])
app.run(host="0.0.0.0", port=8080)
from concurrent import futures
import grpc
from datetime import datetime
import helloworld_pb2
import helloworld_pb2_grpc
# Handle gRPC calls
class Greeter(helloworld_pb2_grpc.GreeterServicer):
def SayHello(self, request, context):
# print(f"{ context.peer() } | { datetime.now().isoformat() } | Request { request } ".strip())
return helloworld_pb2.HelloReply(message=f"Hello, { request.name }!")
# Server setup and start
server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
port = 5555
print(f"listening on port { port }")
server.add_insecure_port(f"[::]:{ port }")
server.start()
server.wait_for_termination()
\ No newline at end of file
syntax = "proto3";
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
\ No newline at end of file
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: helloworld.proto
"""Generated protocol buffer code."""
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='helloworld.proto',
package='',
syntax='proto3',
serialized_options=None,
create_key=_descriptor._internal_create_key,
serialized_pb=b'\n\x10helloworld.proto\"\x1c\n\x0cHelloRequest\x12\x0c\n\x04name\x18\x01 \x01(\t\"\x1d\n\nHelloReply\x12\x0f\n\x07message\x18\x01 \x01(\t23\n\x07Greeter\x12(\n\x08SayHello\x12\r.HelloRequest\x1a\x0b.HelloReply\"\x00\x62\x06proto3'
)
_HELLOREQUEST = _descriptor.Descriptor(
name='HelloRequest',
full_name='HelloRequest',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='HelloRequest.name', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=20,
serialized_end=48,
)
_HELLOREPLY = _descriptor.Descriptor(
name='HelloReply',
full_name='HelloReply',
filename=None,
file=DESCRIPTOR,
containing_type=None,
create_key=_descriptor._internal_create_key,
fields=[
_descriptor.FieldDescriptor(
name='message', full_name='HelloReply.message', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=50,
serialized_end=79,
)
DESCRIPTOR.message_types_by_name['HelloRequest'] = _HELLOREQUEST
DESCRIPTOR.message_types_by_name['HelloReply'] = _HELLOREPLY
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
HelloRequest = _reflection.GeneratedProtocolMessageType('HelloRequest', (_message.Message,), {
'DESCRIPTOR' : _HELLOREQUEST,
'__module__' : 'helloworld_pb2'
# @@protoc_insertion_point(class_scope:HelloRequest)
})
_sym_db.RegisterMessage(HelloRequest)
HelloReply = _reflection.GeneratedProtocolMessageType('HelloReply', (_message.Message,), {
'DESCRIPTOR' : _HELLOREPLY,
'__module__' : 'helloworld_pb2'
# @@protoc_insertion_point(class_scope:HelloReply)
})
_sym_db.RegisterMessage(HelloReply)
_GREETER = _descriptor.ServiceDescriptor(
name='Greeter',
full_name='Greeter',
file=DESCRIPTOR,
index=0,
serialized_options=None,
create_key=_descriptor._internal_create_key,
serialized_start=81,
serialized_end=132,
methods=[
_descriptor.MethodDescriptor(
name='SayHello',
full_name='Greeter.SayHello',
index=0,
containing_service=None,
input_type=_HELLOREQUEST,
output_type=_HELLOREPLY,
serialized_options=None,
create_key=_descriptor._internal_create_key,
),
])
_sym_db.RegisterServiceDescriptor(_GREETER)
DESCRIPTOR.services_by_name['Greeter'] = _GREETER
# @@protoc_insertion_point(module_scope)
# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
"""Client and server classes corresponding to protobuf-defined services."""
import grpc
import helloworld_pb2 as helloworld__pb2
class GreeterStub(object):
"""The greeting service definition.
"""
def __init__(self, channel):
"""Constructor.
Args:
channel: A grpc.Channel.
"""
self.SayHello = channel.unary_unary(
'/Greeter/SayHello',
request_serializer=helloworld__pb2.HelloRequest.SerializeToString,
response_deserializer=helloworld__pb2.HelloReply.FromString,
)
class GreeterServicer(object):
"""The greeting service definition.
"""
def SayHello(self, request, context):
"""Sends a greeting
"""
context.set_code(grpc.StatusCode.UNIMPLEMENTED)
context.set_details('Method not implemented!')
raise NotImplementedError('Method not implemented!')
def add_GreeterServicer_to_server(servicer, server):
rpc_method_handlers = {
'SayHello': grpc.unary_unary_rpc_method_handler(
servicer.SayHello,
request_deserializer=helloworld__pb2.HelloRequest.FromString,
response_serializer=helloworld__pb2.HelloReply.SerializeToString,
),
}
generic_handler = grpc.method_handlers_generic_handler(
'Greeter', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))
# This class is part of an EXPERIMENTAL API.
class Greeter(object):
"""The greeting service definition.
"""
@staticmethod
def SayHello(request,
target,
options=(),
channel_credentials=None,
call_credentials=None,
insecure=False,
compression=None,
wait_for_ready=None,
timeout=None,
metadata=None):
return grpc.experimental.unary_unary(request, target, '/Greeter/SayHello',
helloworld__pb2.HelloRequest.SerializeToString,
helloworld__pb2.HelloReply.FromString,
options, channel_credentials,
insecure, call_credentials, compression, wait_for_ready, timeout, metadata)
FROM python:3.9-alpine
RUN pip3 install flask
COPY . /app/
WORKDIR /app
EXPOSE 5000
ENTRYPOINT ["python3", "-u", "greeter_rest.py"]
flask
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment