Running Kubernetes on Windows

by Sander

When I started down this path I've had quite a few people ask me "WHY?? DEAR $DIETY WHY DO YOU WANT THIS?"

Fair enough, given the whole ecosystem around containers and especially Docker and Kubernetes is largely focused on Linux and *nix's we are a .Net shop and most of our developers work on Windows machines. Switching people over purely for the sake of running a kubernetes (k8s) environment is not something I'd encourage.

One might say: "But you can create a k8s cluster in the cloud right?", well, yes that is certainly possible and most cloud providers give you free credits to do that. The drawback is that it forces you to always have an Internet connection available (repeat after me: "The network is always reliable, there is no latency"). I've found that having resources available locally works well for me, but this is definitely something you'll need to decide for yourself.

Having said that, this is about setting up a k8s cluster on Windows 10.

Here be dragons

I want to be honest with you, figuring this out has given me a few more grey hairs and I fear I've worn down some of my more choiciest swear words. This was probably more due to my own stubornness than documentation not being available. Still, it's always a challenge to piece together the right information and discarding whatever knowledge you have from working on different platforms (I used to code on a Mac).

Common approach

What you'll find is when searching for k8s on Windows is that people will recommend minikube and that is usually a good option. Unfortunately I've had a lot of trouble getting to get this setup to work reliably on my machine. The main reason for this seems to be in the networking area and IPv4/IPv6 configurations. At home it works like a charm (IPv4 only network), however at the office we have a IPv4/IPv6 combined network and things break down rapidly. I've tried disabling IPv6 on the network adapters (both host and the k8s VM) but to no avail.

"Use docker"

Luckily we have the Internet. After searching how to resolve the IPv4/IPv6 problems I came across this comment that made me go: "well.... BLEEP..."

So I set out to see if I could get the Docker way to work. (Narrator: it worked)

Starting out

There are a few things you'll need before setting off on this quest of getting a k8s cluster working on your machine:

  • Windows 10 Enterprise, Pro or Education edition with Hyper-V enabled (the link documents how to do that)
  • Docker Desktop (I tested with Version 2.0.0.2 (30215), stable channel and build 0b030e1) previous versions may work but YMMV. According to the docs it should work from version 18.06 Stable.
  • kubectl (I recommend to install this using Chocolatey)

Configuring docker

After installing Docker (and rebooting! It's Windows after all) you will need to enable k8s. Open the Docker settings application and select the Kubernetes option, you should see this:



Make sure the Enable Kubernetes and Show system containers (advanced) options are selected.

Docker will prompt you that it needs to install k8s support, click Ok when it does.

After the installation is complete, the Docker configuration is complete.

Verifying Kubernetes works

As Docker already manages the k8s cluster we need to check if it is properly running and that you can access it.

From a PowerShell or Command window use kubectl to check connectivity:

__aSyNcId_<_nkTDnav___gt; kubectl cluster-info

That should provide you with output similar to the following:

Kubernetes master is running at https://localhost:6445
KubeDNS is running at https://localhost:6445/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

If you're a command line hero, you'll be fine from this point on. However I recommend also deploying the dashboard so you can have a bit quicker look at your k8s environment.

Setting up the k8s dashboard

Open a PowerShell console and run the following commands (from the official docs):

  • kubectl create secret generic kubernetes-dashboard-certs --from-file=$HOME/certs -n kube-system
  • kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
  • kubectl proxy

At this point you can access the dashboard on http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login but you'll be greeted with this:



Select the Kubeconfig option and select the config file from your %USERPROFILE%\.kube directory. After clicking Sign In you should see the dashboard. If not please check the docs.

NOTE: The dashboard deployment is now using v1.10.1. You may want to check here if that's the latest version before deploying.

Man the helm

At this point you'll have a working k8s cluster running on your Windows machine, without minikube. The next step is deploying a helm chart to your cluster.

I've provided a sample helm chart that you get here, clone the repository and cd into the directory. Next, run helm install . and wait for the command to complete and it should end with something like:

NOTES:

1. Get the application URL by running these commands: http://k8s.demo.net/

Now when you try to access that URL you'll be sadly disappointed because it will report a connection refused error because there is nothing listening at that address.

Knock-knock

This is weird, the helm chart contains an ingress specification so it should be accessible right? When we check the dashboard it even shows listed under the ingresses. What gives?

Well by default there is no ingress controller active so there is nothing that will accept HTTP(s) traffic and route it to our service. Fortunately we can solve this problem by adding the nginx ingress controller.

Open a PowerShell console and run the following command:

__aSyNcId_<_nkTDnav___gt; helm install stable/nginx-ingress --name ingress-nginx --namespace ingress-nginx --wait

Once this command completes you should be able to connect to the service. Let's see:

__aSyNcId_<_nkTDnav___gt; curl.exe -v "http://k8s.demo.net/"
* Could not resolve host: k8s.demo.net
* Closing connection 0
curl: (6) Could not resolve host: k8s.demo.net

Ok, well great. It still doesn't work. Actually this was to be expected, your local machine is not using a DNS server that knows how to map the domain name k8s.demo.net to 127.0.0.1 where the ingress controller is listening at.

The easiest way to fix this is by adding an entry to the HOSTS file. Open up a PowerShell window with administrative privileges (you'll need this to edit the file) and run:

__aSyNcId_<_nkTDnav___gt; notepad c:\windows\system32\drivers\etc\hosts

and add the following entry:

127.0.0.1    k8s.demo.net

Save the file and run curl again:

__aSyNcId_<_nkTDnav___gt; curl.exe -v "http://k8s.demo.net/"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to k8s.demo.net (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: k8s.demo.net
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.15.6
< Date: Mon, 28 Jan 2019 13:59:07 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 6
< Connection: keep-alive
< X-App-Name: http-echo
< X-App-Version: 0.2.3
<
hello
* Connection #0 to host k8s.demo.net left intact

Success!

The finish line

At this point you have a fully working kubernetes cluster running on your machine and you have deployed a service that you can call from your machine using a proper domain name.

There are a few things to keep in mind. To run the dashboard you'll have to run kubectl proxy, it's possible to expose the dashboard using NodePort but I haven't tested that yet.

The HOSTS file "hack" isn't very user friendly. If you're working with the same services all the time this shouldn't be too big of a deal but there should be a way to do this easier with a local DNS server for example.

For now, happy k8s'ing. If you have questions feel free to reach out via Twitter @Codenizer.

Acknowledgements

This wouldn't have been possible without a lot of people writing Github issues, official documentation and blog posts. I've added a list below that hopefully captures most of the resources I've used for this journey.