Jekyll2020-08-31T09:01:19-05:00https://evdev.me/feed.xmlEvDevThe tech escapades of a software engineer turned SRE in the world of containerization, cloud, monitoring, tracing and many more concepts still to be explored!Evgeny YakimovSetting up Home-hosted Kubernetes Cluster2020-08-22T02:52:41-05:002020-08-22T02:52:41-05:00https://evdev.me/setting-up-home-hosted-kubernetes-cluster<p>Since my GCP hosted cluster’s free tier expired, I needed to find a new low-cost solution for hosting a generally-available Kubernetes cluster. I decided to host one at home, using some old equipment I had laying around backed by my internet connection. This post describes how I set this up with only minimal configuration but still with the capabilities of my previous cluster.</p>
<p>For more details on the needs for my Kubernetes cluster, please read my earlier post: <a href="/setting-up-development-and-production-kubernetes-cluster/#objectives">Setting up a Kubernetes Cluster for Development and Production</a>.</p>
<h2 id="objectives">Objectives</h2>
<p>Fundamentally I wanted to set up a single-node Kubernetes cluster, which should behave very similar to what you would expect from a cloud-hosted one. This includes:</p>
<ul>
<li>Be reachable externally and securely</li>
<li>Provide HTTPS-based ingress</li>
<li>Require minimum administration/maintenance</li>
</ul>
<h2 id="installing-os">Installing OS</h2>
<p>Since I’ve been using Ubuntu for almost a decade, I decided to stick with installing a basic <a href="https://ubuntu.com/download/server">ubuntu-server</a> image (20.04 LTS) via USB. I did not select any additional packages to be installed.</p>
<h2 id="installing-kubernetes">Installing Kubernetes</h2>
<p>There are several options for how to install Kubernetes, such as <a href="https://kubernetes.io/docs/tasks/tools/install-minikube/">minikube</a>, <a href="https://microk8s.io/">microk8s</a> or even manually, I decided to give <strong>microk8s</strong> a go since it was recommended for Ubuntu installs.</p>
<h3 id="what-is-microk8s">What is microk8s?</h3>
<p>Similarly to minikube, microk8s is a lightweight Kubernetes cluster with a small ecosystem of utilities and addons. It simplifies the installation of Kubernetes on top of an existing Linux distribution. It even ships with <code class="highlighter-rouge">kubectl</code> and an extension can provide <code class="highlighter-rouge">helm</code>.</p>
<h3 id="installing-microk8s">Installing microk8s</h3>
<p>Microk8s is distributed through <code class="highlighter-rouge">snap</code> and can be installed with:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt update <span class="o">&&</span> <span class="nb">sudo </span>apt <span class="nb">install </span>snapd <span class="c"># if you don't have snap</span>
<span class="nb">sudo </span>snap <span class="nb">install </span>microk8s <span class="nt">--classic</span>
</code></pre></div></div>
<p>To enable administration without sudo, simply run:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>usermod <span class="nt">-a</span> <span class="nt">-G</span> microk8s <span class="nv">$USER</span>
</code></pre></div></div>
<p>Install useful microk8s extensions:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>microk8s <span class="nb">enable </span>dns dashboard rbac storage
</code></pre></div></div>
<h3 id="accessing-microk8s-via-kubectl">Accessing microk8s via kubectl</h3>
<p>You can either administer the cluster directly on the host by using the microk8 wrappers, or simply get the kubectl config to use remotely.</p>
<h4 id="local-access">Local Access</h4>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>microk8s kubectl get nodes
</code></pre></div></div>
<h4 id="remote-access">Remote Access</h4>
<p>Generate and retrieve the kubectl config using</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>microk8s config <span class="o">></span> kubectl.config <span class="c"># download this to your local device</span>
</code></pre></div></div>
<h2 id="setting-up-the-cluster-for-external-connectivity">Setting up the cluster for external connectivity</h2>
<h3 id="configure-external-access-port-forwarding">Configure external access (port forwarding)</h3>
<p>If you’re behind a NAT router like me, you will first need to configure port-forwarding from your router to the Kubernetes host. In my case, I preconfigured ports 80 and 443 for forwarding web traffic.</p>
<h3 id="install-ingress-nginx">Install ingress-nginx</h3>
<p>I intend to use <a href="https://kubernetes.github.io/ingress-nginx/">ingress-nginx</a> to accept and dispatch traffic to pods within my cluster. Nginx will need to accept inbound connections on the host port (80 and 443 for me).</p>
<p><code class="highlighter-rouge">ingress-nginx/values.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">controller</span><span class="pi">:</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">DaemonSet</span>
<span class="na">hostPort</span><span class="pi">:</span>
<span class="na">enabled</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">service</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
</code></pre></div></div>
<p>To install ingress-nginx</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl create namespace ingress-nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm <span class="nb">install</span> <span class="se">\</span>
ingress-nginx ingress-nginx/ingress-nginx <span class="se">\</span>
<span class="nt">-n</span> ingress-nginx <span class="se">\</span>
<span class="nt">-f</span> ingress-nginx/values.yaml
</code></pre></div></div>
<p><em>NOTE: I did initially try and use the ingress addon provided with microk8s, but I never managed to get it working. Given that the <a href="https://microk8s.io/docs/addon-ingress">documentation</a> was rather limited, I went back to using ingress-nginx.</em></p>
<h3 id="install-cert-manager">Install cert-manager</h3>
<p>Since I’ll be using SSL to provide secure access, I’ll need a certificate manager installed on my cluster which will allow me to automatically acquire certificates from <a href="https://letsencrypt.org/">Let’s Encrypt</a>. The popular <a href="https://cert-manager.io/docs/">jetstack cert-manager</a> can be installed by running:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl create namespace cert-manager
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm <span class="nb">install</span> <span class="se">\</span>
cert-manager jetstack/cert-manager <span class="se">\</span>
<span class="nt">--namespace</span> cert-manager <span class="se">\</span>
<span class="nt">--version</span> v0.16.1 <span class="se">\</span>
<span class="nt">--set</span> <span class="nv">installCRDs</span><span class="o">=</span><span class="nb">true</span>
</code></pre></div></div>
<p>This should be followed up by installing a certificate issuer, for example as described in <a href="https://cert-manager.io/docs/tutorials/acme/ingress/#step-6-configure-let-s-encrypt-issuer">Configure Let’s Encrypt Issuer</a>.</p>
<h3 id="exposing-kube-apiserver-publicly">Exposing kube-apiserver publicly</h3>
<p>The remote kubectl config above will work as long as the node can be reached directly or possibly by adding another port-forward. However, I prefer to expose it as a dedicated end-point on my ingress with a publicly valid SSL certificate. To achieve this I’ve applied the following ingress configuration:</p>
<p><code class="highlighter-rouge">kubernetes-ingress.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">networking.k8s.io/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Ingress</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">kubernetes</span>
<span class="na">annotations</span><span class="pi">:</span>
<span class="s">kubernetes.io/ingress.class</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="s">cert-manager.io/cluster-issuer</span><span class="pi">:</span> <span class="s">letsencrypt-prod</span>
<span class="s">nginx.ingress.kubernetes.io/backend-protocol</span><span class="pi">:</span> <span class="s2">"</span><span class="s">HTTPS"</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">rules</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">host</span><span class="pi">:</span> <span class="s">kubernetes.my.domain.name</span>
<span class="na">http</span><span class="pi">:</span>
<span class="na">paths</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">path</span><span class="pi">:</span> <span class="s">/</span>
<span class="na">backend</span><span class="pi">:</span>
<span class="na">serviceName</span><span class="pi">:</span> <span class="s">kubernetes</span>
<span class="na">servicePort</span><span class="pi">:</span> <span class="s">443</span>
<span class="na">tls</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">hosts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">kubernetes.my.domain.name</span>
<span class="na">secretName</span><span class="pi">:</span> <span class="s">kubernetes-tls</span>
</code></pre></div></div>
<p>I can now update my kubectl configuration to hit this end-point directly without any extra port-forwards or custom certificate authority entries.</p>
<h2 id="conclusion">Conclusion</h2>
<p>At this point, my home-hosted cluster appears and behaves like my previous cloud-hosted one with the caveat of being on a slower internet connection and being in a non-scalable single-node configuration. I then continued to set up my baseline applications including <a href="https://brigade.sh">brigade</a>, following the same <a href="/setting-up-development-and-production-kubernetes-cluster/#setting-up-brigade">steps</a> as I did in my previous cluster. I found that microk8s, was a very easy way to get Kubernetes up and running, although I did struggle with the ingress addon initially, overall it was a fairly straight forward to set up.</p>
<p>You can find a dense summary of this set-up process as well as most of my configuration on my cluster set-up git project: <a href="https://github.com/eyjohn/homelab">homelab</a>.</p>Evgeny YakimovSince my GCP hosted cluster’s free tier expired, I needed to find a new low-cost solution for hosting a generally-available Kubernetes cluster. I decided to host one at home, using some old equipment I had laying around backed by my internet connection. This post describes how I set this up with only minimal configuration but still with the capabilities of my previous cluster.Playing with WebSockets on Kubernetes using Go2019-12-08T15:08:43-06:002019-12-08T15:08:43-06:00https://evdev.me/websockets-on-kubernetes<p>Given my background in real-time data streaming, I wanted to experiment with building an application with such characteristics for the Web and try to run it on Kubernetes. This post explores how I built a simple WebSockets based application in Go and hosted it on my Kubernetes cluster.</p>
<h2 id="websockets">WebSockets</h2>
<p>Let’s start with a quick recap of what WebSockets are and how they work.</p>
<h3 id="what-are-websockets">What are WebSockets?</h3>
<p>WebSockets are a mechanism to create a TCP like connection in the web stack. They are an extension to the HTTP protocol on supported web servers and clients.</p>
<h3 id="how-websockets-work">How WebSockets work?</h3>
<p>WebSockets start with a normal HTTP connection which requests a connection “Upgrade” to the type of “websocket”. After the handshake exchange, the connection remains open, allowing bidirectional messages similar to a normal network socket.</p>
<p>Since the underlying protocol is HTTP and the rest of the protocol is basically a long-living payload exchange, WebSockets inherit many characteristics such as:</p>
<ul>
<li>SSL/TLS support</li>
<li>Distribution middleware (proxies and load balancers)</li>
<li>Rely on the same L3/L4 firewall rules as normal web traffic</li>
<li>Supports HTTP headers such as cookies or compression settings</li>
</ul>
<h2 id="the-application">The Application</h2>
<p>I’m planning on building a simple application that exhibits the characteristics of real-time data streaming. It will consist of a server-side component and HTML/JavaScript client-side app which will communicate over a WebSocket.</p>
<p>To make things easier, I’ll make the server-side component serve the client-side application as a static asset over HTTP. Since its basically a web server, I’ll throw in a health end-point too to allow HTTP probes on the application.</p>
<p>For the WebSocket connection, I’ll make the server do a couple of “real-time-ey” things, firstly it will echo back any messages it receives from the client, secondly, it’ll send messages to the client at random times.</p>
<p>I’ll be using <strong>Go</strong> for this application as I’ve been meaning to find more opportunities to use it. For a container development and deployment workflow, I’ll be using <a href="https://draft.sh/">Draft</a>.</p>
<h3 id="bootstrapping-the-project">Bootstrapping the project</h3>
<p>We’ll start by running <code class="highlighter-rouge">draft create -p go</code> to bootstrap the draft project, and then create a couple of files <code class="highlighter-rouge">main.go</code> for the main server-side component and <code class="highlighter-rouge">index.html</code> for the client-side app.</p>
<p>Once set up, I can simply run <code class="highlighter-rouge">draft up</code> (and then <code class="highlighter-rouge">draft connect</code>) to connect to my development Kubernetes instance.</p>
<h2 id="the-server-side-application">The server-side application</h2>
<h3 id="serving-static-content">Serving static content</h3>
<p>Let’s create a basic web server that serves only a single page. In this case, the file: <code class="highlighter-rouge">index.html</code>.</p>
<p><code class="highlighter-rouge">main.go</code></p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span>
<span class="k">import</span> <span class="p">(</span>
<span class="s">"io/ioutil"</span>
<span class="s">"log"</span>
<span class="s">"net/http"</span>
<span class="p">)</span>
<span class="k">func</span> <span class="n">defaultHandler</span><span class="p">(</span><span class="n">w</span> <span class="n">http</span><span class="o">.</span><span class="n">ResponseWriter</span><span class="p">,</span> <span class="n">r</span> <span class="o">*</span><span class="n">http</span><span class="o">.</span><span class="n">Request</span><span class="p">)</span> <span class="p">{</span>
<span class="n">content</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">ioutil</span><span class="o">.</span><span class="n">ReadFile</span><span class="p">(</span><span class="s">"index.html"</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">Header</span><span class="p">()</span><span class="o">.</span><span class="n">Set</span><span class="p">(</span><span class="s">"Content-Type"</span><span class="p">,</span> <span class="s">"text/html"</span><span class="p">)</span>
<span class="n">w</span><span class="o">.</span><span class="n">Write</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">http</span><span class="o">.</span><span class="n">HandleFunc</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span> <span class="n">defaultHandler</span><span class="p">)</span>
<span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">ListenAndServe</span><span class="p">(</span><span class="s">":8080"</span><span class="p">,</span> <span class="no">nil</span><span class="p">))</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">index.html</code></p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><!DOCTYPE html></span>
<span class="nt"><html></span>
<span class="nt"><body></span>
HELLO WORLD
<span class="nt"></body></span>
<span class="nt"></html></span>
</code></pre></div></div>
<p>Now for a quick test:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>draft connect
Connect to gowebsockettest:8080 on localhost:64993
</code></pre></div></div>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>curl http://localhost:64993
<<span class="o">!</span>DOCTYPE html>
<html>
<body>
HELLO WORLD
</body>
</html>
</code></pre></div></div>
<p>Great! we’ve built a… web server, one that only serves a single file.</p>
<h3 id="adding-a-health-end-point">Adding a health end-point</h3>
<p>We can now add a simple health end-point which we’ll register as the convention <code class="highlighter-rouge">/healthz</code> by adding the following to <code class="highlighter-rouge">main.go</code>.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">healthzHandler</span><span class="p">(</span><span class="n">w</span> <span class="n">http</span><span class="o">.</span><span class="n">ResponseWriter</span><span class="p">,</span> <span class="n">r</span> <span class="o">*</span><span class="n">http</span><span class="o">.</span><span class="n">Request</span><span class="p">)</span> <span class="p">{</span>
<span class="n">w</span><span class="o">.</span><span class="n">Write</span><span class="p">([]</span><span class="kt">byte</span><span class="p">(</span><span class="s">"OK"</span><span class="p">))</span>
<span class="p">}</span>
<span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">http</span><span class="o">.</span><span class="n">HandleFunc</span><span class="p">(</span><span class="s">"/healthz"</span><span class="p">,</span> <span class="n">healthzHandler</span><span class="p">)</span>
<span class="c">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>curl http://localhost:65400/healthz
OK
</code></pre></div></div>
<h3 id="accepting-websocket-connections">Accepting WebSocket connections</h3>
<p>There are multiple options for adding WebSocket support in Go, at the time of writing, it seemed that the <a href="https://github.com/gorilla/websocket">Gorilla WebSocket</a> implementation provided the best features. This package works on top of <code class="highlighter-rouge">net/http</code> by taking an existing connection and “upgrading” it to a WebSocket.</p>
<p><code class="highlighter-rouge">main.go</code></p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span>
<span class="c">// ...</span>
<span class="s">"github.com/gorilla/websocket"</span>
<span class="p">}</span>
<span class="c">// The upgrader which will perform the HTTP connection upgrade to WebSocket</span>
<span class="k">var</span> <span class="n">upgrader</span> <span class="o">=</span> <span class="n">websocket</span><span class="o">.</span><span class="n">Upgrader</span><span class="p">{</span>
<span class="n">ReadBufferSize</span><span class="o">:</span> <span class="m">1024</span><span class="p">,</span>
<span class="n">WriteBufferSize</span><span class="o">:</span> <span class="m">1024</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">func</span> <span class="n">websocketHandler</span><span class="p">(</span><span class="n">w</span> <span class="n">http</span><span class="o">.</span><span class="n">ResponseWriter</span><span class="p">,</span> <span class="n">r</span> <span class="o">*</span><span class="n">http</span><span class="o">.</span><span class="n">Request</span><span class="p">)</span> <span class="p">{</span>
<span class="n">conn</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">upgrader</span><span class="o">.</span><span class="n">Upgrade</span><span class="p">(</span><span class="n">w</span><span class="p">,</span> <span class="n">r</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="n">log</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="k">return</span>
<span class="p">}</span>
<span class="n">log</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"Connection from %v"</span><span class="p">,</span> <span class="n">conn</span><span class="o">.</span><span class="n">RemoteAddr</span><span class="p">())</span>
<span class="p">}</span>
<span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">http</span><span class="o">.</span><span class="n">HandleFunc</span><span class="p">(</span><span class="s">"/websocket"</span><span class="p">,</span> <span class="n">websocketHandler</span><span class="p">)</span>
<span class="c">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now that the server is capable of accepting WebSocket connections, let’s switch to the client-side application.</p>
<h2 id="the-client-side-application">The client-side application</h2>
<p>To keep things simple, I’ll put all the HTML and JavaScript in a single file, a literal single page application if you will.</p>
<h3 id="creating-a-basic-web-page">Creating a basic web page</h3>
<p>We’ll start with some basic HTML.</p>
<p><code class="highlighter-rouge">index.html</code></p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><!DOCTYPE html></span>
<span class="nt"><html></span>
<span class="nt"><head></span>
<span class="nt"><title></span>Go WebSocket Test<span class="nt"></title></span>
<span class="nt"><script></span>
<span class="c1">// JavaScript hoes here</span>
<span class="nt"></script></span>
<span class="nt"></head></span>
<span class="nt"><body></span>
<span class="nt"><input</span> <span class="na">id=</span><span class="s">"payload"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="nt">/></span>
<span class="nt"><button</span> <span class="na">id=</span><span class="s">"send"</span><span class="nt">></span>Send<span class="nt"></button></span>
<span class="nt"><pre</span> <span class="na">id=</span><span class="s">"log"</span><span class="nt">></pre></span>
<span class="nt"></body></span>
<span class="nt"></html></span>
</code></pre></div></div>
<p>Next, we’ll add some helpful utility functions.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Helper function to log messages</span>
<span class="kd">function</span> <span class="nx">log</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">logElement</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="dl">"</span><span class="s2">log</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">logElement</span><span class="p">.</span><span class="nx">innerText</span> <span class="o">+=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">().</span><span class="nx">toISOString</span><span class="p">()</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">: </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">message</span> <span class="o">+</span> <span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Helper function to generate a WebSocket from the current location</span>
<span class="kd">function</span> <span class="nx">getWsUrl</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">loc</span> <span class="o">=</span> <span class="nb">window</span><span class="p">.</span><span class="nx">location</span><span class="p">,</span> <span class="nx">new_uri</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">loc</span><span class="p">.</span><span class="nx">protocol</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">https:</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">new_uri</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">wss:</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">new_uri</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">ws:</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">new_uri</span> <span class="o">+=</span> <span class="dl">"</span><span class="s2">//</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">loc</span><span class="p">.</span><span class="nx">host</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">/websocket</span><span class="dl">"</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">new_uri</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="creating-a-websocket-connection">Creating a WebSocket connection</h3>
<p>The JavaScript WebSocket API is pretty straight-forward, taking a target URL on construction and exposing several properties to bind to events. To create a WebSocket and handle events through its lifetime:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">ws</span><span class="p">;</span>
<span class="kd">function</span> <span class="nx">connect</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Connecting</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">ws</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">WebSocket</span><span class="p">(</span><span class="nx">getWsUrl</span><span class="p">());</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">onopen</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Connected</span><span class="dl">"</span><span class="p">);</span>
<span class="p">};</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">onmessage</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Received: </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">event</span><span class="p">.</span><span class="nx">data</span><span class="p">);</span>
<span class="p">};</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">onclose</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// We'll reconnect indefinitely</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Connection Closed, Reconnecting</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">setTimeout</span><span class="p">(</span><span class="nx">connect</span><span class="p">,</span> <span class="mi">1000</span><span class="p">);</span>
<span class="p">};</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">onerror</span> <span class="o">=</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// On any error, just close and cause a reconnect</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Error, Closing</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">close</span><span class="p">();</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And finally to connect everything together:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">window</span><span class="p">.</span><span class="nx">onload</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// Connect button and input to send messages</span>
<span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="dl">'</span><span class="s1">send</span><span class="dl">'</span><span class="p">).</span><span class="nx">onclick</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">ws</span><span class="p">.</span><span class="nx">send</span><span class="p">(</span><span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="dl">"</span><span class="s2">payload</span><span class="dl">"</span><span class="p">).</span><span class="nx">value</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Start a connection</span>
<span class="nx">connect</span><span class="p">();</span>
<span class="p">};</span>
</code></pre></div></div>
<h2 id="the-real-time-streaming-part">The real-time streaming part</h2>
<p>Now, let’s get back to the server-side application and make it actually do something in real-time.</p>
<h3 id="1-echo-messages">1. Echo messages</h3>
<p>Firstly we’ll create a function which will read messages and try to echo them back. Upon any failure (including due to the client-side application disconnecting), we’ll simply close the connection.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">websocketEcho</span><span class="p">(</span><span class="n">conn</span> <span class="o">*</span><span class="n">websocket</span><span class="o">.</span><span class="n">Conn</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">{</span>
<span class="n">mt</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">conn</span><span class="o">.</span><span class="n">ReadMessage</span><span class="p">()</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="n">log</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="s">"Read Error:"</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>
<span class="k">break</span>
<span class="p">}</span>
<span class="n">log</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"Received Message: %s from %v"</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">conn</span><span class="o">.</span><span class="n">RemoteAddr</span><span class="p">())</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">WriteMessage</span><span class="p">(</span><span class="n">mt</span><span class="p">,</span> <span class="n">message</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="n">log</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="s">"Write Error:"</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>
<span class="k">break</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">conn</span><span class="o">.</span><span class="n">Close</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="2-send-messages-at-random-intervals">2. Send messages at random intervals</h3>
<p>Next, we will send messages at random intervals of up to 3 seconds based on a randomised sleep. To keep things simple, we’ll stop on error, and leave the clean-up of closing the connection to the echo function.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">websocketRandPing</span><span class="p">(</span><span class="n">conn</span> <span class="o">*</span><span class="n">websocket</span><span class="o">.</span><span class="n">Conn</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">{</span>
<span class="n">err</span> <span class="o">:=</span> <span class="n">conn</span><span class="o">.</span><span class="n">WriteMessage</span><span class="p">(</span><span class="n">websocket</span><span class="o">.</span><span class="n">TextMessage</span><span class="p">,</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">(</span><span class="s">"randping"</span><span class="p">))</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="n">log</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="k">return</span>
<span class="p">}</span>
<span class="n">time</span><span class="o">.</span><span class="n">Sleep</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">Duration</span><span class="p">(</span><span class="n">rand</span><span class="o">.</span><span class="n">Intn</span><span class="p">(</span><span class="kt">int</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">Second</span> <span class="o">*</span> <span class="m">3</span><span class="p">))))</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="putting-it-all-together">Putting it all together</h3>
<p>To connect everything together, we can then use “Goroutines” which are great for running such async tasks in the background. We will extend the <code class="highlighter-rouge">websocketHandler</code> function to invoke the two functions we added earlier as Goroutines.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">websocketHandler</span><span class="p">(</span><span class="n">w</span> <span class="n">http</span><span class="o">.</span><span class="n">ResponseWriter</span><span class="p">,</span> <span class="n">r</span> <span class="o">*</span><span class="n">http</span><span class="o">.</span><span class="n">Request</span><span class="p">)</span> <span class="p">{</span>
<span class="c">// ...</span>
<span class="k">go</span> <span class="n">websocketEcho</span><span class="p">(</span><span class="n">conn</span><span class="p">)</span>
<span class="k">go</span> <span class="n">websocketRandPing</span><span class="p">(</span><span class="n">conn</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="deployment">Deployment</h2>
<p>With this simple application built, how would we run such an application on Kubernetes? Well, as it turns out, very easily. Since WebSockets are based on HTTP, all the standard HTTP distribution features such as ingress and load balancers work exactly the same.</p>
<p>Luckily, Draft has already generated a Helm chart which creates a deployment and exposes a service. It can optionally be configured to use an ingress. For my cluster, I will be using the following configuration:</p>
<p><code class="highlighter-rouge">values.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">replicaCount</span><span class="pi">:</span> <span class="s">1</span>
<span class="na">image</span><span class="pi">:</span>
<span class="c1"># My personal repo that draft pushes to</span>
<span class="na">pullPolicy</span><span class="pi">:</span> <span class="s">Always</span>
<span class="na">repository</span><span class="pi">:</span> <span class="s">eyjohn/gowebsockettest</span>
<span class="na">tag</span><span class="pi">:</span> <span class="s">latest</span>
<span class="na">service</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">gowebsockettest</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">externalPort</span><span class="pi">:</span> <span class="s">80</span>
<span class="na">internalPort</span><span class="pi">:</span> <span class="s">8080</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">limits</span><span class="pi">:</span>
<span class="na">cpu</span><span class="pi">:</span> <span class="s">100m</span>
<span class="na">memory</span><span class="pi">:</span> <span class="s">128Mi</span>
<span class="na">requests</span><span class="pi">:</span>
<span class="na">cpu</span><span class="pi">:</span> <span class="s">100m</span>
<span class="na">memory</span><span class="pi">:</span> <span class="s">128Mi</span>
<span class="na">ingress</span><span class="pi">:</span>
<span class="c1"># My cluster has an ingress controller</span>
<span class="na">enabled</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">hosts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">gowebsockettest.homelab.evdev.me</span>
<span class="na">annotations</span><span class="pi">:</span>
<span class="c1"># I can use my certificate manager to ass https/wss</span>
<span class="s">kubernetes.io/ingress.class</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nginx"</span>
<span class="s">cert-manager.io/cluster-issuer</span><span class="pi">:</span> <span class="s2">"</span><span class="s">letsencrypt-prod"</span>
<span class="na">tls</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">hosts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">gowebsockettest.homelab.evdev.me</span>
<span class="na">secretName</span><span class="pi">:</span> <span class="s">gowebsockettest-tls</span>
<span class="c1"># I had to extend the helm charts to support probes</span>
<span class="na">readinessProbe</span><span class="pi">:</span>
<span class="na">httpGet</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/healthz</span>
<span class="na">port</span><span class="pi">:</span> <span class="s">8080</span>
<span class="na">initialDelaySeconds</span><span class="pi">:</span> <span class="s">5</span>
<span class="na">periodSeconds</span><span class="pi">:</span> <span class="s">10</span>
<span class="na">livenessProbe</span><span class="pi">:</span>
<span class="na">httpGet</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/healthz</span>
<span class="na">port</span><span class="pi">:</span> <span class="s">8080</span>
<span class="na">initialDelaySeconds</span><span class="pi">:</span> <span class="s">15</span>
<span class="na">periodSeconds</span><span class="pi">:</span> <span class="s">20</span>
</code></pre></div></div>
<p>To install this, I can simply run:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm <span class="nb">install</span> <span class="nt">-n</span> gowebsockettest charts/gowebsockettest/ <span class="nt">-f</span> values.yaml
</code></pre></div></div>
<p>Once the deployment has completed, we can now test it on the end-point: <a href="https://gowebsockettest.homelab.evdev.me">gowebsockettest.homelab.evdev.me</a>.</p>
<h2 id="demo">Demo</h2>
<p>Here is a quick demo of the client-side application and server side logs:</p>
<p style="text-align: center;"><img src="/assets/posts/websockets-on-kubernetes/demo.gif" alt="Screen capture of the client side app and server logs" /></p>
<h2 id="conclusion">Conclusion</h2>
<p>WebSockets were incredibly easy to get started with, especially in Go where Goroutines made asynchronous event handling very trivial. They fit well within the web-stack, so all the HTTP related set up worked just like with any normal web application. Therefore they allow a natural Kubernetes development and deployment workflow and I had no issues with any development tools or when deploying the applications to my Kubernetes cluster. This definitely goes to show that a web-based platform has a lot to offer for writing real-time data streaming applications and that it fits well within a Kubernetes environment.</p>
<p>You can find a working version of the project repository on my GitHub repository <a href="https://github.com/eyjohn/gowebsockettest">eyjohn/gowebsockettest</a> and a live version of this app on <a href="https://gowebsockettest.homelab.evdev.me">gowebsockettest.homelab.evdev.me</a> for however long that I keep it running.</p>Evgeny YakimovGiven my background in real-time data streaming, I wanted to experiment with building an application with such characteristics for the Web and try to run it on Kubernetes. This post explores how I built a simple WebSockets based application in Go and hosted it on my Kubernetes cluster.Automated deployments of this site with Brigade2019-11-09T04:00:47-06:002019-11-09T04:00:47-06:00https://evdev.me/automated-deployment-with-brigade<p>With the site up and running, I wanted to make my life easier by setting up an automated deployment pipeline powered by Brigade. Brigade is ideal for an event-driven workflow (such as on push) which will form the basis of my continuous deployment for my projects. In this post, I will cover how to create a brigade project and configure it to deploy this website automatically.</p>
<h2 id="background">Background</h2>
<p>Before getting into the setup, let’s cover how this website is built and deployed today, as well as a recap of my Brigade setup.</p>
<h3 id="building-this-website">Building this website</h3>
<p>This website is built using Jekyll, which is essentially the process of running <code class="highlighter-rouge">jekyll build</code> to convert the source files (found on <a href="https://github.com/eyjohn/evdev.me">evdev.me GitHub repository</a>) into static assets that can be deployed into any web hosting provider. You can find more details on how this website is built on my earlier post <a href="/making-this-website-with-jekyll/">Making this website with Jekyll</a>.</p>
<h3 id="hosting-and-deployment">Hosting and Deployment</h3>
<p>I use <a href="https://firebase.google.com/products/hosting/">Google’s Firebase Hosting</a> for this website as it offers secure (HTTPS) web-hosting, caches your content globally for faster access and most importantly, it comes with a free tier that’s more than enough for me. Although this post uses Firebase CLI for deployment, using an alternative deployment hosting solution, such as uploading files over FTP/SCP should be fairly similar. It’s worth noting that this type of website can easily be hosted on GitHub pages, but I chose to set up the deployment and hosting myself as a learning exercise.</p>
<h3 id="brigade">Brigade</h3>
<p><a href="https://brigade.sh/">Brigade</a> is an event-driven scripting framework designed to work with containers and is well suited to continuous integration or to initiate continuous deployment. This post uses the Brigade instance configured with GitHub integration as set up in my earlier post <a href="/setting-up-development-and-production-kubernetes-cluster/">Setting up a Kubernetes Cluster for Development and Production</a>.</p>
<h2 id="continuous-integration-workflow">Continuous integration workflow</h2>
<p>For this project I decided to use a simple workflow of building and deploying my website upon every push to some predetermined branches:</p>
<ul>
<li><strong>master</strong> - main website</li>
<li><strong>staging</strong> - staging version for manually previewing changes</li>
</ul>
<p>Brigade lets you define more complex workflows such as pull request validation by running test suites or other tools. However, these are unnecessary for this type of project and I only really need Brigade to trigger my deployment pipeline.</p>
<h2 id="setting-up-brigade-project">Setting up Brigade Project</h2>
<p>Brigade provides the CLI tool <a href="https://docs.brigade.sh/topics/brig/">brig</a> which makes it very easy to create a project:</p>
<h3 id="1-create-the-project">1. Create the project</h3>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ brig project create
? VCS or no-VCS project? VCS
? Project Name eyjohn/evdev.me
? Full repository name github.com/eyjohn/evdev.me
? Clone URL (https://github.com/your/repo.git) https://github.com/eyjohn/evdev.me.git
? Add secrets? Yes --- SEE BELOW ---
...
? Where should the project's shared secret come from? Specify my own
? Shared Secret **********
? Configure GitHub Access? No
? Configure advanced options No
Project ID: brigade-2809670f5c85efb78f808c7446e312340c1893136c869db51aa557
</code></pre></div></div>
<p>You can now verify that your project has been created by running:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>brig project list
NAME ID REPO
eyjohn/evdev.me brigade-2809670f5c85efb78f808c7446e312340c1893136c869db51aa557 github.com/eyjohn/evdev.me
</code></pre></div></div>
<p>Or alternatively by launching the “Kashti” dashboard provided by Brigade:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>brig dashboard
2019/11/05 22:32:41 Connecting to kashti at http://localhost:8081...
2019/11/05 22:32:42 Connected! When you are finished with this session, enter CTRL+C.
</code></pre></div></div>
<p style="text-align: center;"><img src="/assets/posts/automated-deployment-with-brigade/brigade_new_project.png" alt="Screenshot of fresh project in Kashti" /></p>
<h3 id="2-configure-secrets">2. Configure secrets</h3>
<p>You can define secrets that would be available to your <code class="highlighter-rouge">brigade.js</code> script, allowing you to store sensitive credentials without having to place them in the repo or the containers.</p>
<p>In this case, I plan to store my Firebase authentication credentials, but these could likewise be used to store an SSH key or FTP credentials.</p>
<p>The easiest way to do this is at the time of the creation of the project.</p>
<h3 id="3-enable-github-app-access">3. Enable GitHub app access</h3>
<p>To allow the GitHub app used by Brigade to access the repository, it will need to be configured in <a href="https://github.com/settings/apps">GitHub Apps</a> -> Edit -> Install App.</p>
<p style="text-align: center;"><img src="/assets/posts/automated-deployment-with-brigade/github_add_brigade.png" alt="Screenshot of GitHub app installation for project" /></p>
<h2 id="creating-jobs">Creating jobs</h2>
<p>With the project set up, let’s start adding some jobs.</p>
<h3 id="1-creating-a-test-job">1. Creating a test job</h3>
<p>First, let’s create a simple job to test that simply prints the branch:</p>
<p><code class="highlighter-rouge">brigade.js</code></p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="p">{</span> <span class="nx">events</span><span class="p">,</span> <span class="nx">Job</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">"</span><span class="s2">brigadier</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">events</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">"</span><span class="s2">push</span><span class="dl">"</span><span class="p">,</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">branch</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">revision</span><span class="p">.</span><span class="nx">ref</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">job</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Job</span><span class="p">(</span><span class="dl">"</span><span class="s2">test-job</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">alpine</span><span class="dl">"</span><span class="p">,</span> <span class="p">[</span><span class="s2">`echo </span><span class="p">${</span><span class="nx">branch</span><span class="p">}</span><span class="s2">`</span><span class="p">]);</span>
<span class="nx">job</span><span class="p">.</span><span class="nx">run</span><span class="p">();</span>
<span class="p">});</span>
</code></pre></div></div>
<p>After pushing we can verify that it’s working by using the Kashti dashboard:</p>
<p style="text-align: center;"><img src="/assets/posts/automated-deployment-with-brigade/test_job.png" alt="Screenshot of test job build in Kashti" /></p>
<h3 id="2-creating-a-build-job">2. Creating a build job</h3>
<p>To build this website, Brigade will need to pull the source code, run the <code class="highlighter-rouge">jekyll build</code> command and finally output the built website for the deployment step.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">createBuildJob</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">buildJob</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Job</span><span class="p">(</span><span class="dl">"</span><span class="s2">build-job</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">image</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">jekyll/jekyll</span><span class="dl">"</span><span class="p">;</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">tasks</span> <span class="o">=</span> <span class="p">[</span>
<span class="dl">"</span><span class="s2">cd /src</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">jekyll build</span><span class="dl">"</span><span class="p">,</span>
<span class="p">];</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">streamLogs</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">buildJob</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>To export the build artefacts to the deployment job, the build artefacts will need to be copied to a job <strong>storage</strong> which is shared across jobs and can be configured using:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">buildJob</span><span class="p">.</span><span class="nx">tasks</span> <span class="o">=</span> <span class="p">[</span>
<span class="dl">"</span><span class="s2">cd /src</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">jekyll build</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">cp -r firebase.json _site /build</span><span class="dl">"</span> <span class="c1">// firebase.json required for deployment</span>
<span class="p">];</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">storage</span><span class="p">.</span><span class="nx">enabled</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">storage</span><span class="p">.</span><span class="nx">path</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">/build</span><span class="dl">'</span><span class="p">;</span>
</code></pre></div></div>
<p>Since my Jekyll configuration requires some modules to be installed, it would be great to not have to install them every time I run the build and re-use them from the previous builds. This can be configured using the job <strong>cache</strong> which is shared between builds for the same job and can be configured using:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">buildJob</span><span class="p">.</span><span class="nx">cache</span><span class="p">.</span><span class="nx">size</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">100Mi</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">cache</span><span class="p">.</span><span class="nx">enabled</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="nx">buildJob</span><span class="p">.</span><span class="nx">cache</span><span class="p">.</span><span class="nx">path</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">/usr/local/bundle</span><span class="dl">'</span><span class="p">;</span> <span class="c1">// Jekyll image cache location</span>
</code></pre></div></div>
<h3 id="3-creating-a-deployment-job">3. Creating a deployment job</h3>
<p>To deploy this website to firebase, I’ll use an image with the firebase CLI tool pre-installed. Since I plan on having two different deployments: staging and production, I will make the deployment destination a configurable parameter. I will be using project secrets to store the credentials for my deployment destinations. This job will need to pull in the build artefacts from earlier and deploy them.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">createDeployJob</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">,</span> <span class="nx">staging</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">firebaseProject</span> <span class="o">=</span> <span class="nx">staging</span>
<span class="p">?</span> <span class="nx">project</span><span class="p">.</span><span class="nx">secrets</span><span class="p">.</span><span class="nx">FIREBASE_PROJECT_STAGING</span>
<span class="p">:</span> <span class="nx">project</span><span class="p">.</span><span class="nx">secrets</span><span class="p">.</span><span class="nx">FIREBASE_PROJECT_PRODUCTION</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">deployJob</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Job</span><span class="p">(</span><span class="dl">"</span><span class="s2">deploy</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">andreysenov/firebase-tools</span><span class="dl">"</span><span class="p">,</span> <span class="p">[</span>
<span class="dl">'</span><span class="s1">cd /build</span><span class="dl">'</span><span class="p">,</span>
<span class="s2">`firebase deploy --project </span><span class="p">${</span><span class="nx">firebaseProject</span><span class="p">}</span><span class="s2"> --token </span><span class="p">${</span><span class="nx">project</span><span class="p">.</span><span class="nx">secrets</span><span class="p">.</span><span class="nx">FIREBASE_TOKEN</span><span class="p">}</span><span class="s2">`</span>
<span class="p">]);</span>
<span class="nx">deployJob</span><span class="p">.</span><span class="nx">storage</span><span class="p">.</span><span class="nx">enabled</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="nx">deployJob</span><span class="p">.</span><span class="nx">storage</span><span class="p">.</span><span class="nx">path</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">/build</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">deployJob</span><span class="p">.</span><span class="nx">streamLogs</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">deployJob</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="4-putting-it-all-together">4. Putting it all together</h3>
<p>Finally, let’s hook up these jobs to the push event. For my workflow, I will need to detect the branch of the push to determine whether the build should be deployed to the production or staging destination.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="kd">function</span> <span class="nx">runBuildAndDeploy</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">buildJob</span> <span class="o">=</span> <span class="nx">createBuildJob</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">staging</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">revision</span><span class="p">.</span><span class="nx">ref</span> <span class="o">==</span> <span class="dl">"</span><span class="s2">refs/heads/master</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">staging</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">revision</span><span class="p">.</span><span class="nx">ref</span> <span class="o">==</span> <span class="dl">"</span><span class="s2">refs/heads/staging</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">staging</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span><span class="p">;</span> <span class="c1">// Nothing to do for other branches</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">deployJob</span> <span class="o">=</span> <span class="nx">createDeployJob</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">project</span><span class="p">,</span> <span class="nx">staging</span><span class="p">);</span>
<span class="k">await</span> <span class="nx">buildJob</span><span class="p">.</span><span class="nx">run</span><span class="p">();</span>
<span class="k">await</span> <span class="nx">deployJob</span><span class="p">.</span><span class="nx">run</span><span class="p">();</span>
<span class="p">}</span>
<span class="nx">events</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">"</span><span class="s2">push</span><span class="dl">"</span><span class="p">,</span> <span class="nx">runBuildAndDeploy</span><span class="p">);</span>
</code></pre></div></div>
<h3 id="5-testing-the-pipeline">5. Testing the pipeline</h3>
<p>By pushing to the <code class="highlighter-rouge">staging</code> branch I can now test that both jobs are working.</p>
<p style="text-align: center;"><img src="/assets/posts/automated-deployment-with-brigade/pipeline.png" alt="Screenshot of pipeline in Kashti" /></p>
<p>The final version of my <code class="highlighter-rouge">brigade.js</code> can be found on the GitHub repository of this website <a href="https://github.com/eyjohn/evdev.me">eyjohn/evdev.me</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Using Brigade was a big change to what I’m normally used to with other continuous integration tools such as Jenkins, which normally requires me to spend more time configuring my Jenkins instance and development environment on any nodes. Brigade, on the other hand, has very few configuration options and instead relies on users to express their jobs in a single <code class="highlighter-rouge">brigade.js</code> configuration file, backed by a deployment environment executed only through containers.</p>
<p>This required a huge mindset shift that emphasises the use of containers to execute the build, test and deployment process. This eliminates the need for a large or complex development or testing environment and encourages the use of isolated containers for each job. Furthermore, this makes it easier to run tools that might not be compatible with each other or to use new tools as everything is isolated.</p>
<p>After getting to grips with the Brigade fundamentals (jobs, storage/cache, secrets) I found it very intuitive to express the different jobs required for my project. When combined with GitHub integration, it can be used to create a range of workflows without much difficulty. Finally, with all the resources of my Kubernetes cluster, Brigade can start the containers whenever or wherever it needs. This certainly seems like the next step in the evolution of continuous integration and I’m looking forward to using it again for my future projects!</p>Evgeny YakimovWith the site up and running, I wanted to make my life easier by setting up an automated deployment pipeline powered by Brigade. Brigade is ideal for an event-driven workflow (such as on push) which will form the basis of my continuous deployment for my projects. In this post, I will cover how to create a brigade project and configure it to deploy this website automatically.Setting up a Kubernetes Cluster for Development and Production2019-10-27T17:18:00-05:002019-10-27T17:18:00-05:00https://evdev.me/setting-up-development-and-production-kubernetes-cluster<p>For my upcoming projects, I wanted to use a container-based platform both for my development workflow and production hosting needs. This post describes how I configured a Kubernetes cluster from Google’s cloud platform at a reasonable cost.</p>
<p>Why <a href="https://kubernetes.io/">Kubernetes</a>?</p>
<ul>
<li>It’s popular and is being actively developed</li>
<li>Great tooling: <code class="highlighter-rouge">helm</code>, <code class="highlighter-rouge">draft</code>, <code class="highlighter-rouge">brigade</code>, <code class="highlighter-rouge">minikube</code></li>
<li>Based on containerization</li>
<li>Supported by major cloud vendors</li>
<li>Everything is a resource <em>(more on this later)</em></li>
</ul>
<h2 id="objectives">Objectives</h2>
<p>Given that my Kubernetes cluster was intended for “hobby” projects, I had somewhat specialised needs:</p>
<ul>
<li>Keep costs low (milk the free tier!)</li>
<li>Host production apps</li>
<li>Hosted development workflow: CI, CD, GitHub integration</li>
<li>Somewhat available (redundancy with at least 1 backup node)</li>
</ul>
<p>Since this profile isn’t the priority for cloud providers (especially the low-cost part), I often opted to self-host my infrastructure, sacrificing simplicity to keep costs low.</p>
<h2 id="background">Background</h2>
<p>I expect that you already have a basic understanding of <a href="https://kubernetes.io/">Kubernetes</a> and containers in general. Here is a quick introduction of the additional tools that I plan on using.</p>
<h3 id="helm-for-package-management">Helm for Package Management</h3>
<p><a href="https://helm.sh/">Helm</a> is a great tool for managing packages and configuration through the use of <em>charts</em>. Related packages are assembled in charts which then expose a single set of configurable parameters. I can then simply maintain a list of the charts I wish to install and their configurations.</p>
<h3 id="brigade-for-continuous-integration">Brigade for Continuous Integration</h3>
<p><a href="https://brigade.sh/">Brigade</a> is an event-driven scripting framework designed to work for containers which is well suited to continuous integration. It comes with great support for GitHub integration.</p>
<h2 id="setting-up-a-cluster">Setting up a cluster</h2>
<p>Google’s cloud platform has great tooling which makes setting up a Kubernetes cluster a simple command. After installing and authenticating with the tools, all I had to do is:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud container clusters create evkube <span class="nt">--num-nodes</span> 2 <span class="nt">--disk-size</span> 15 <span class="nt">-m</span> g1-small <span class="nt">--no-enable-cloud-logging</span> <span class="nt">--no-enable-cloud-monitoring</span>
</code></pre></div></div>
<p><em>I went with the g1-small (non-free tier) configuration as the free tier f1-micro with only 600 MB ram was not sufficient for my needs, even with the g1-small I had to disable some observability features to support my workload.</em></p>
<p>Once set up, you can configure <code class="highlighter-rouge">kubectl</code> to use your new cluster by:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud container clusters get-credentials evkube
<span class="c"># Then you can run:</span>
kubectl describe nodes
</code></pre></div></div>
<p>And that’s it, you have a working cluster, let’s make it do something!</p>
<h2 id="setting-up-helm">Setting up Helm</h2>
<p>Helm is required for all of the next components that I need to install.</p>
<p><em>NOTE: Before installing helm, you may need to configure RBAC, see the documentation <a href="https://github.com/helm/helm/blob/master/docs/rbac.md">Role-based Access Control Documentation</a>.</em></p>
<p>To install helm simply:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm init <span class="nt">--service-account</span> tiller
</code></pre></div></div>
<h2 id="installing-self-hosted-infrastructure">Installing self-hosted infrastructure</h2>
<p>Why? Kubernetes treats everything as a <strong>resource</strong>: load balancer, storage or certificates. Your cloud vendor will likely pre-configure your cluster to already use their own products which are generally very easy to use. However, if you’re trying to keep costs low and don’t mind a bit of complexity, then you can probably set up everything you need yourself.</p>
<h3 id="load-balancersingress">Load Balancers/Ingress</h3>
<p>I chose to use <a href="https://kubernetes.github.io/ingress-nginx/">nginx-ingress</a> to handle and route incoming connections on a publicly accessible <code class="highlighter-rouge">hostPort</code> (configured on my firewall) rather than the cloud provider’s load balancers. To set this up:</p>
<h4 id="1-expose-the-host-port-on-your-firewall">1. Expose the host port on your firewall</h4>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gcloud compute firewall-rules create nginx-ingress <span class="nt">--allow</span> tcp:80,tcp:443
</code></pre></div></div>
<p>Whilst not covered here, you will probably want to set up DNS to point to the IP addresses which you can find by running:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl get nodes <span class="nt">-o</span> <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{ $.items[*].status.addresses[?(@.type=="ExternalIP")].address }'</span>
</code></pre></div></div>
<h4 id="2-configure-nginx-ingress">2. Configure nginx-ingress</h4>
<p><code class="highlighter-rouge">nginx-ingress/values.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">controller</span><span class="pi">:</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">DaemonSet</span> <span class="c1"># Run on every node</span>
<span class="na">daemonset</span><span class="pi">:</span>
<span class="na">useHostPort</span><span class="pi">:</span> <span class="no">true</span> <span class="c1"># Use a port on host interface</span>
<span class="na">service</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span> <span class="c1"># Don't use LoadBalancer for services</span>
</code></pre></div></div>
<h4 id="3-install-nginx-ingress">3. Install nginx-ingress</h4>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm <span class="nb">install</span> <span class="nt">-n</span> nginx-ingress <span class="nt">--namespace</span> nginx-ingress stable/nginx-ingress <span class="nt">-f</span> nginx-ingress/values.yaml
</code></pre></div></div>
<h3 id="certificate-manager">Certificate Manager</h3>
<p>Now that we can accept HTTP connections, lets set up a certificate manager to create certificates for HTTPS. I decided to use the <a href="https://github.com/jetstack/cert-manager">Jetstack Cert Manager</a> configured to use <a href="https://letsencrypt.org/">Let’s Encrypt</a> certificates (which are free). To install and configure the certificate manager:</p>
<h4 id="1-add-certificate-manager-resource-definitions">1. Add certificate manager resource definitions</h4>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply <span class="nt">--validate</span><span class="o">=</span><span class="nb">false</span> <span class="nt">-f</span> https://raw.githubusercontent.com/jetstack/cert-manager/release-0.11/deploy/manifests/00-crds.yaml
</code></pre></div></div>
<h4 id="2-add-an-issuer">2. Add an issuer</h4>
<p><strong>NOTE: Please use a staging issuer first before attempting to use a production one to avoid getting banned/throttled by Let’s Encrypt by accident!</strong></p>
<p>Create the following configuration file (and replace your email).</p>
<p><code class="highlighter-rouge">cert-manager/production-issuer.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">certmanager.k8s.io/v1alpha1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ClusterIssuer</span> <span class="c1"># This issuer can be used across namespaces on this cluster</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">letsencrypt-prod</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">acme</span><span class="pi">:</span>
<span class="na">server</span><span class="pi">:</span> <span class="s">https://acme-v02.api.letsencrypt.org/directory</span>
<span class="na">email</span><span class="pi">:</span> <span class="s">MY.EMAIL@MY.DOMAIN</span>
<span class="na">privateKeySecretRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">letsencrypt-prod</span>
<span class="na">http01</span><span class="pi">:</span> <span class="pi">{}</span>
</code></pre></div></div>
<p>Then install this issuer on your cluster.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply <span class="nt">-f</span> cert-manager/production-issuer.yaml
</code></pre></div></div>
<h4 id="3-install-cert-manager">3. Install cert-manager</h4>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm repo add jetstack https://charts.jetstack.io
helm repo update
helm <span class="nb">install</span> <span class="nt">-n</span> cert-manager <span class="nt">--namespace</span> cert-manager jetstack/cert-manager
</code></pre></div></div>
<p>For a complete installation guide, please see <a href="https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html">Cert Manager - Installing on Kubernetes</a>.</p>
<h3 id="storage-provisioner">Storage Provisioner</h3>
<p>I chose the NFS storage provisioner, as it was easy to set up and provides the <code class="highlighter-rouge">ReadWriteMany</code> mode which will be required by Brigade.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm <span class="nb">install</span> <span class="nt">-n</span> nfs-server-provisioner <span class="nt">--namespace</span> nfs-server-provisioner stable/nfs-server-provisioner
</code></pre></div></div>
<h2 id="setting-up-brigade">Setting up Brigade</h2>
<p>This post describes how to install Brigade with GitHub integration which requires you to host a web service that receives events for configured GitHub hooks. GitHub app registration is not covered but you can find more detail about this on the official <a href="https://github.com/brigadecore/brigade-github-app">brigade-github-app documentation</a>. After creating the GitHub app, you can install Brigade by following these steps:</p>
<h3 id="1-configure-brigade">1. Configure Brigade</h3>
<p>Create the following configuration file and fill in the details of your GitHub app and domain.</p>
<p><code class="highlighter-rouge">brigade/values.yaml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">brigade-github-app</span><span class="pi">:</span>
<span class="na">enabled</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">service</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">ingress</span><span class="pi">:</span>
<span class="na">hosts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">DOMAIN_USED_BY_THIS_BRIGADE_GITHUB_APP</span>
<span class="na">annotations</span><span class="pi">:</span>
<span class="c1"># Use nginx-ingress for routing</span>
<span class="s">kubernetes.io/ingress.class</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nginx"</span>
<span class="c1"># Use cert-manager issuer to create a certificate</span>
<span class="s">certmanager.k8s.io/cluster-issuer</span><span class="pi">:</span> <span class="s2">"</span><span class="s">letsencrypt-prod"</span>
<span class="s">certmanager.k8s.io/acme-challenge-type</span><span class="pi">:</span> <span class="s">http01</span>
<span class="na">tls</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">hosts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">DOMAIN_USED_BY_THIS_BRIGADE_GITHUB_APP</span>
<span class="na">secretName</span><span class="pi">:</span> <span class="s">SECRET_NAME_FOR_THIS_CERTIFICATE</span>
<span class="na">github</span><span class="pi">:</span>
<span class="c1"># The GitHub app id and credentials</span>
<span class="na">appID</span><span class="pi">:</span> <span class="s">36087</span>
<span class="na">key</span><span class="pi">:</span> <span class="s">MY_SUPER_SECRET_KEY</span>
<span class="na">worker</span><span class="pi">:</span>
<span class="c1"># Use nfs-server-provisioner for provisioning storage</span>
<span class="na">defaultBuildStorageClass</span><span class="pi">:</span> <span class="s">nfs</span>
<span class="na">defaultCacheStorageClass</span><span class="pi">:</span> <span class="s">nfs</span>
</code></pre></div></div>
<h3 id="2-install-brigade">2. Install Brigade</h3>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>helm repo add brigade https://brigadecore.github.io/charts
helm repo update
helm <span class="nb">install</span> <span class="nt">-n</span> brigade brigade/brigade <span class="nt">--namespace</span><span class="o">=</span>brigade <span class="nt">-f</span> brigade/values.yaml
</code></pre></div></div>
<h3 id="3-verify-installation">3. Verify installation</h3>
<p>Once Brigade is installed, you can use the <a href="https://docs.brigade.sh/topics/brig/">brig</a> CLI to interact with your Brigade instance.</p>
<p><em>NOTE: Since Brigade was installed to a specific namespace, you will need to set this when using <code class="highlighter-rouge">brig</code>.</em></p>
<p>To test Brigade, we can bring up the dashboard with:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">BRIGADE_NAMESPACE</span><span class="o">=</span>brigade
brig dashboard
</code></pre></div></div>
<h3 id="4-verify-github-hooks-are-working">4. Verify GitHub hooks are working</h3>
<p>Firstly verify that your HTTP endpoint is reachable and you do not receive any errors. You can check the following URL: <code class="highlighter-rouge">https://$domain_used_by_this_github_app/healthz</code></p>
<p>You can also check your app in GitHub under: GitHub Apps -> Edit -> Advanced -> Recent Deliveries. This should now start showing any hooks that were fired and their result.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I’ll be honest that the journey of learning about these concepts for the first time while trying to use them was a challenge. Furthermore, I had to figure out how to debug build failures which I eventually figured out were a result from insufficient resources on both my micro and small instances leading me to finally using small instances with monitoring disabled. However, the journey of setting up all these components helped me understand just what Kubernetes is really about.</p>
<p>Now that I have both my hosting and development workflow set up, I can start putting it to use. I’ve since been able to create a test project and have set up the generation and deployment of this website which I will explore in more detail in an upcoming post.</p>
<p>You can find a dense summary of this setup process as well as most of my configuration on my cluster setup git project: <a href="https://github.com/eyjohn/evkube">evkube</a>.</p>Evgeny YakimovFor my upcoming projects, I wanted to use a container-based platform both for my development workflow and production hosting needs. This post describes how I configured a Kubernetes cluster from Google’s cloud platform at a reasonable cost.Making this website with Jekyll2019-07-07T03:31:19-05:002019-07-07T03:31:19-05:00https://evdev.me/making-this-website-with-jekyll<p>A walk-through of how I made this easy to maintain but featureful static website using <strong>Jekyll</strong> based on a <strong>Docker</strong> development workflow. Although this required some ramp-up to getting started, making changes is trivial. All the steps outlined below should be easy to follow and include a brief explanation to help you learn!</p>
<p>Jekyll has become a popular tool for generating static websites for anything from blogs, project website or documentation portals. The main reason it got my attention was that it can give the appearance of a rich dynamic website but is ultimately just a set of generated HTML files. It also provides an easy way to mix <code class="highlighter-rouge">HTML</code> and <code class="highlighter-rouge">markdown</code> to get the flexibility you sometimes need with the simplicity you prefer which makes it easy to really add new content. Jekyll also powers GitHub Pages behind the scenes which allows you to serve content purely from your GitHub repository.</p>
<p>You are looking at the live version of this website based on the source repo <a href="https://github.com/eyjohn/evdev.me" target="_blank">eyjohn/evdev.me</a> but NOT hosted on GitHub Pages.</p>
<h2 id="objectives">Objectives</h2>
<p>By the end of this process, I want to:</p>
<ul>
<li>Generate a simple static website</li>
<li>Customise my website to give it my personal feel</li>
<li>Be able to easily add posts without worrying about databases</li>
<li>Use a <code class="highlighter-rouge">docker</code> powered development workflow for <code class="highlighter-rouge">jekyll</code></li>
</ul>
<h2 id="prerequisites">Prerequisites</h2>
<p>Before trying this, you will need to know about the following:</p>
<ul>
<li>General understanding of static websites and how they are hosted</li>
<li>Familiar with <code class="highlighter-rouge">markdown</code> and <code class="highlighter-rouge">HTML</code></li>
<li>(Optional) A basic understanding of how <code class="highlighter-rouge">docker</code> works and a working setup</li>
</ul>
<h2 id="getting-started">Getting Started</h2>
<p>Firstly, let’s create and initialise a workspace and install the necessary tools.</p>
<h3 id="creating-the-repository">Creating the repository</h3>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir </span>evdev.me
<span class="nb">cd </span>evdev.me
git init
</code></pre></div></div>
<h3 id="setup-jekyll">Setup Jekyll</h3>
<p>I will be using <code class="highlighter-rouge">docker</code> to run <code class="highlighter-rouge">jekyll</code> rather than installing it on my system. This is because I plan on creating a CI/CD pipeline built on top of containers which I will explore in a future post. You can, however, install it natively on your system which I will not be covering.</p>
<p>I chose the <code class="highlighter-rouge">jekyll/jekyll</code> dockerhub images which are provided by this repo <a href="https://github.com/envygeeks/jekyll-docker/blob/master/README.md" target="_blank">jekyll-docker</a> and has great documentation. Let’s write a quick wrapper script to easier to invoke the container while we develop:</p>
<p><code class="highlighter-rouge">docker_jekyll</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
docker run <span class="nt">--rm</span> <span class="se">\</span>
<span class="nt">--volume</span><span class="o">=</span><span class="s2">"</span><span class="nv">$PWD</span><span class="s2">:/srv/jekyll"</span> <span class="se">\</span>
<span class="nt">--volume</span><span class="o">=</span><span class="s2">"jekyll_cache_</span><span class="si">$(</span><span class="nb">basename</span> <span class="nv">$PWD</span><span class="si">)</span><span class="s2">:/usr/local/bundle"</span> <span class="se">\</span>
<span class="nt">-it</span> jekyll/jekyll <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span>
</code></pre></div></div>
<p>A fixed version can also be specified, although I just chose to use the latest available at the time (3.8.5).</p>
<p><em>NOTE: it is important that your Docker volume mounts work correctly, initially my setup: WSL + docker (minikube) did not preserve Unix execution mode correctly, and I lost HOURS to this as my gems would fail to install!</em></p>
<h3 id="get-jekyll-to-create-a-new-site">Get Jekyll to create a new site</h3>
<p>I was impressed by the default website generated by Jekyll, had it not been for my preference of dark themes, I would have stuck with it. To get started simply run this command inside your directory:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./docker_run jekyll new <span class="nb">.</span>
</code></pre></div></div>
<p><em>Running it for the first time might take some time, also you may need to use –force if the directory is non-empty.</em></p>
<p>This generates the following files:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>evdev.me/
├── 404.html
├── Gemfile
├── Gemfile.lock
├── _config.yml
├── _posts
│ └── 2019-06-23-welcome-to-jekyll.markdown
├── about.md
├── docker_jekyll
└── index.md
</code></pre></div></div>
<h3 id="build-and-serve-website">Build and Serve website</h3>
<p>You can now <code class="highlighter-rouge">build</code> (generate static files) or <code class="highlighter-rouge">serve</code> (build and serve over HTTP) this project. A <code class="highlighter-rouge">jekyll serve --watch</code> workflow is a popular development workflow. In my case I simply ran:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./docker_run jekyll serve <span class="nt">--watch</span> <span class="c"># takes a few seconds to start</span>
</code></pre></div></div>
<p>Output:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux-musl]
Configuration file: /srv/jekyll/_config.yml
Source: /srv/jekyll
Destination: /srv/jekyll/_site
Incremental build: disabled. Enable with --incremental
Generating...
Jekyll Feed: Generating feed for posts
done in 2.409 seconds.
Auto-regeneration: enabled for '/srv/jekyll'
Server address: http://0.0.0.0:4000/
Server running... press ctrl-c to stop.
</code></pre></div></div>
<p>Which looks like:</p>
<p style="text-align: center;"><img src="/assets/posts/making-this-website-with-jekyll/jekyll_new_screenshot.png" alt="Screen shot of jekyll new website" /></p>
<p>Now that the boilerplate is out of the way, I can start personalising it.</p>
<h2 id="customising-the-website">Customising the website</h2>
<p>Unless you plan on using the default theme, <strong>DO NOT</strong> start modifying any config files or templates yet as some themes may use different config or layout structure so you may end up losing any progress you make.</p>
<h3 id="changing-themelayout">Changing Theme/Layout</h3>
<p>I regretted starting to customise the layout of the default theme <strong>minima</strong> and the config as I had to scrap many of the settings when I installed a new theme. I chose a free theme <a href="https://github.com/niklasbuschmann/contrast/" target="_blank">contrast</a> which includes a dark variant that I like and I used this as my starting point. Although the customisation process may be different between themes, it is likely to follow similar steps:</p>
<h4 id="1-installing-your-theme">1. Installing your theme</h4>
<p>The <strong>contrast</strong> theme had the option to use a “remote” install (not copying all the layouts) which is documented <a href="https://github.com/niklasbuschmann/contrast#installation-jekyll-remote-theme-method" target="_blank">here</a>.</p>
<p>Alternatively, some themes would require you to start with their template repository.</p>
<p>Some features may require you to install Jekyll plugins which can be done by changing the <code class="highlighter-rouge">jekyll_plugins</code> section in your <code class="highlighter-rouge">Gemfile</code>.</p>
<h4 id="2-configuring-your-site">2. Configuring your site</h4>
<p>Edit the <code class="highlighter-rouge">_config.yml</code> file with your own settings. In the cast of <strong>contrast</strong>, it came with good documentation for the options available.</p>
<p>In some cases, I had to update <code class="highlighter-rouge">Gemfile</code> and edit or overload layout files.</p>
<h4 id="3-adjusting-layout-assets">3. Adjusting layout assets</h4>
<p>Depending on how you installed your theme, you will have to either:</p>
<ul>
<li><strong>Local Install (all theme assets in repo)</strong> <br /> Adjust existing layouts</li>
<li><strong>Remote Install (theme assets in bundle/remote)</strong> <br /> Copy existing layouts into <code class="highlighter-rouge">_layout</code> and modify as appropriate</li>
</ul>
<p>Jekyll will search for <code class="highlighter-rouge">_layouts</code> and <code class="highlighter-rouge">_includes</code> in your base folder first, which allows any file from themes to be simply overridden by creating a copy in your base folder.</p>
<h3 id="adding-new-posts">Adding new posts</h3>
<p>This was incredibly easy, simply add a new file in <code class="highlighter-rouge">_posts</code> folder, either as <code class="highlighter-rouge">markdown</code> or <code class="highlighter-rouge">HTML</code>. You simply need to add the Jekyll <a href="https://jekyllrb.com/docs/front-matter/">Front Matter</a> (top YAML section between the two <code class="highlighter-rouge">---</code>). Here is a simple example of an article in markdown:</p>
<div class="language-markdown highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">layout</span><span class="pi">:</span> <span class="s">post</span>
<span class="na">title</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Making</span><span class="nv"> </span><span class="s">this</span><span class="nv"> </span><span class="s">website</span><span class="nv"> </span><span class="s">with</span><span class="nv"> </span><span class="s">Jekyll"</span>
<span class="na">date</span><span class="pi">:</span> <span class="s">2019-06-23 13:26:01 +0100</span>
<span class="na">categories</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">jekyll</span><span class="pi">,</span> <span class="nv">docker</span><span class="pi">]</span>
<span class="na">tags</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">jekyll</span><span class="pi">,</span> <span class="nv">website</span><span class="pi">,</span> <span class="nv">docker</span><span class="pi">,</span> <span class="nv">favicon</span><span class="pi">]</span>
<span class="nn">---</span>
A walk-through of how I made this easy to maintain but featureful static website using <span class="gs">**Jekyll**</span> based on a <span class="gs">**Docker**</span> development workflow. Although this required some ramp-up to getting started, making changes is trivial. All the steps outlined below should be easy to follow and include a brief explanation to help you learn!
And now for another paragraph...
</code></pre></div></div>
<p>Unless otherwise specified in the <code class="highlighter-rouge">excerpt_separator</code> setting of the configuration file, the first paragraph will be generally used as the excerpt when generating article list pages.</p>
<h3 id="adding-or-updating-pages">Adding or updating pages</h3>
<p>Both the default <strong>minima</strong> theme and the <strong>contrast</strong> themes both included a default <code class="highlighter-rouge">index</code> page which was easily customisable. In my case, I chose to enable the pagination version which I felt provided a better experience and also rendered an article excerpt in the listing.</p>
<p>I also wanted to add a couple of extra pages, such as an <strong>about</strong>, <strong>search</strong> and <strong>contact</strong> page. This was even easier than adding a post, simply add the <em>Front Matter</em> as before, followed by your <code class="highlighter-rouge">HTML</code> or <code class="highlighter-rouge">markdown</code> content. For example:</p>
<div class="language-markdown highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">layout</span><span class="pi">:</span> <span class="s">page</span>
<span class="na">title</span><span class="pi">:</span> <span class="s">About</span>
<span class="na">permalink</span><span class="pi">:</span> <span class="s">/about/</span>
<span class="nn">---</span>
Hey, here is a page about me!
</code></pre></div></div>
<p>In addition, I had to add the page to the <code class="highlighter-rouge">navigation</code> section of my <code class="highlighter-rouge">_config.yml</code> in order for it to be listed in my navigation.</p>
<h2 id="adding-dynamic-components">Adding dynamic components</h2>
<p>The drawback of having a statically generated website is that you are unable to have dynamic content such as comments, search or forms.</p>
<p>The <strong>benefit</strong> of statically generated websites is that you do not have to! Why not rely on external services to provide this for you?</p>
<p>I was able to easily find and implement three different services that allowed me to have comments, search and a contact form within half an hour!</p>
<h3 id="comments">Comments</h3>
<p>My theme <strong>contrast</strong> came with a <code class="highlighter-rouge">_config.yml</code> with a comment section providing support for two options out of the box: <a href="https://disqus.com/">disqus</a> and <a href="https://posativ.org/isso/">isso</a>. I chose to use disqus as it came with a hosted free tier package as opposed to isso which required self-hosting.</p>
<p>After registering on <a href="https://disqus.com/">disqus.com</a> and creating a “site” with the short name “evdev”, I simply updated the <code class="highlighter-rouge">disqus</code> option of the <code class="highlighter-rouge">_config.yaml</code> to “evdev” and that was all.</p>
<h3 id="search">Search</h3>
<p>Google offers a free service called <a href="https://cse.google.com/">Custom Search Engine</a> which will automatically crawl your website and build a search index for it. You can then expose this as a search form on your own website.</p>
<p>You will need to first register your website and set a URL matching pattern after which you will be given the embed code. After that, I just added a <code class="highlighter-rouge">search.html</code> page as follows:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
layout: page
title: Search this site
permalink: /search/
---
<span class="nt"><script </span><span class="na">async</span> <span class="na">src=</span><span class="s">"https://cse.google.com/cse.js?cx=001363150109788008944:nocet6mejo4"</span><span class="nt">></script></span>
<span class="nt"><div</span> <span class="na">class=</span><span class="s">"gcse-search"</span><span class="nt">></div></span>
</code></pre></div></div>
<h3 id="contact">Contact</h3>
<p>For my contact form, I decided to use <a href="https://formspree.io/">formspree</a> as it is free and is ultimately just a form.</p>
<p>While you can pay for better integration where the user feels like they never left your website, I was happy with the free tier which cases the user to be redirected to formspree to pass a captcha and be shown a branded message before being given a link to return to my website.</p>
<p>I simply added a <code class="highlighter-rouge">contact.html</code> as follows:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
layout: page
title: Contact
permalink: /contact/
---
<span class="nt"><form</span> <span class="na">action=</span><span class="s">"https://formspree.io/MY@EMAIL.ADDRESS"</span> <span class="na">method=</span><span class="s">"POST"</span><span class="nt">></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"hidden"</span> <span class="na">name=</span><span class="s">"_subject"</span> <span class="na">value=</span><span class="s">"EvDev Contact Form"</span><span class="nt">></span>
<span class="nt"><label</span> <span class="na">for=</span><span class="s">"name"</span><span class="nt">></span>Name: <span class="nt"></label></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">name=</span><span class="s">"name"</span> <span class="na">required</span><span class="nt">></span>
<span class="nt"><br/></span>
<span class="nt"><label</span> <span class="na">for=</span><span class="s">"email"</span><span class="nt">></span>Email: <span class="nt"></label></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"email"</span> <span class="na">name=</span><span class="s">"_replyto"</span> <span class="na">required</span><span class="nt">></span>
<span class="nt"><br/></span>
<span class="nt"><label</span> <span class="na">for=</span><span class="s">"comments"</span><span class="nt">></span>Comments: <span class="nt"></label></span>
<span class="nt"><textarea</span> <span class="na">name=</span><span class="s">"comments"</span> <span class="na">required</span><span class="nt">></textarea></span>
<span class="nt"><br/></span>
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"submit"</span> <span class="na">value=</span><span class="s">"Submit"</span><span class="nt">></span>
<span class="nt"></form></span>
</code></pre></div></div>
<h2 id="conclusion">Conclusion</h2>
<p>Overall Jekyll proved to be a very easy tool to get started with, and even though documentation was easily accessible, most of the time I didn’t feel that I needed it. The fact that at the end of the process, I’m just dealing with static assets made it much easier to host and not have to worry about databases or keeping my website up-to-date.</p>
<p>My biggest pain-points during the whole process were as a result of my Docker powered workflow, or more specifically with my windows powered setup. The main issues were:</p>
<ul>
<li>Mounting the cache volume was problematic due to the need for compiled modules</li>
<li>The <code class="highlighter-rouge">jekyll serve --watch</code> workflow could not detect file changes on my working directory, I had to use the slower <code class="highlighter-rouge">--force_polling</code></li>
<li>Starting Jekyll in the container was considerably slower than running it on my WSL host</li>
</ul>Evgeny YakimovA walk-through of how I made this easy to maintain but featureful static website using Jekyll based on a Docker development workflow. Although this required some ramp-up to getting started, making changes is trivial. All the steps outlined below should be easy to follow and include a brief explanation to help you learn!