Deploying Services with Proxmox, Kubernetes, and Cloudflare: An Alternative to Traditional Colocation

As a Cloud Infrastructure, DevOps, and Observability engineer, my goal is to optimize system performance and efficiency while minimizing costs. In this article, I will describe a personal project where I implemented a solution based on Proxmox, Kubernetes, and Cloudflare, hosted on my own hardware, as a viable alternative to expensive colocation services in Mexico.

Architecture Diagram

graph LR
    A[Users] --> B[Cloudflare]
    B --> |CDN, WAF| C[cloudflared Tunnel]
    
    subgraph Local Server
        C --> D[VM1 - Reverse Proxy]
        D --> E[K8s LoadBalancer]
        subgraph Kubernetes Cluster
          E --> F[VM2 - Master Node]
          E --> G[VM3 - Worker Node]
          F --> |Control Plane| F
          G --> |Pods| G
        end
    end

Hardware: A Robust Platform for Virtualization

The project was executed on a server with the following specifications:

• CPU: 36 cores.
• Storage: 24 TB.
• RAM: 256 GB.
• GPU: Tesla card for accelerated computing workloads.

This hardware provides a solid foundation for virtualization and running demanding workloads.

Architecture: Virtual Machines, Containers, and Service Exposure

The architecture is based on virtualization with Proxmox VE as the hypervisor. The decision to use Proxmox is based on its robustness, its open-source nature, and its efficient resource management. Within this environment, three virtual machines (VMs) were configured:

1. Reverse Proxy with cloudflared (VM1): This VM acts as the entry point for all external traffic, utilizing the Cloudflare tunnel, cloudflared. Instead of manually configuring a reverse proxy like HAProxy or Nginx, the cloudflared daemon was installed and configured. It establishes a secure and persistent connection between the Cloudflare network and the local infrastructure via a tunnel (Cloudflare Tunnel). When creating the tunnel, cloudflared resolves incoming requests from Cloudflare, routing them to the internal IP address of the LoadBalancer or Ingress service within the Kubernetes cluster.

2. Kubernetes Master Node (VM2): This VM houses the Kubernetes control plane. Critical components like the API Server, Scheduler, Controller Manager, and etcd reside here.

3. Kubernetes Worker Node (VM3): This VM is dedicated to running containerized workloads. This is where the Pods containing the applications are deployed.

Kubernetes was chosen for its ability to orchestrate and manage containerized applications in an automated, flexible, and scalable manner. The integration of LXD within the Kubernetes cluster can be considered as an option for system-level container management if needed, although the main management is done with containerd or CRI-O.

External connectivity is managed through Cloudflare. By configuring cloudflared on VM1 to receive traffic from Cloudflare, the benefits of its global network are leveraged: DDoS attack protection, CDN for static content distribution, web application firewall (WAF), and performance optimization. It also simplifies configuration by not having to open ports on the local firewall.

Implementation and Testing: Performance Verification

The implementation consisted of installing and configuring Proxmox, creating the three VMs, installing the operating system on each (it is recommended to specify the Linux distribution used, for example, Ubuntu Server 20.04), configuring cloudflared on VM1 to establish the tunnel with Cloudflare and point to the LoadBalancer or Ingress service in the Kubernetes cluster, and configuring the cluster itself on VMs 2 and 3.

Extensive tests were performed to validate the stability and performance of the system. Benchmarking tools like ApacheBench or wrk were used to simulate different load levels and measure the system's responsiveness. Key metrics such as CPU, memory, network, and disk usage were monitored on all VMs, as well as the TTFB (Time To First Byte) and the Total Time of requests.

The test results were highly satisfactory, demonstrating the efficiency of the configuration:

These values indicate minimal latency in request processing, with a fast TTFB and a near-identical Total Time.

Critical Analysis of TTFB in a Blog Context:

While a TTFB of 286.011 ms is a good result in general terms, it's important to analyze it in the specific context of a blog system. For a blog, where the content is mostly static, a TTFB ideally below 200 ms could be expected.

In this case, we must consider that we are talking about a home infrastructure, with its own limitations. Factors such as the quality of the local server's internet connection, the latency introduced by the cloudflared tunnel, and the configuration of the web server and the blog's content management system (CMS) (for example, WordPress, Ghost, etc.) may be influencing this value.

It would be interesting to conduct additional tests to identify potential bottlenecks. For example:

• Measure the TTFB directly on the local server: This would help determine how much of the TTFB is due to network latency and how much to internal server processing.
• Analyze the CMS configuration: Optimizing the CMS configuration, the use of caching, resource compression, and image optimization can have a significant impact on the TTFB.
• Evaluate the impact of cloudflared: Comparing the TTFB with and without the cloudflared tunnel could reveal whether this component introduces considerable latency.

In summary, although the TTFB of 286.011 ms is acceptable for a home setup, there is room for improvement, especially considering that it is a blog. A deeper analysis of the mentioned factors could lead to optimizations that bring the TTFB closer to more competitive values.

Cost and Stability Analysis: A Viable Alternative

This project demonstrates that, for certain use cases, a local infrastructure based on Proxmox and Kubernetes can compete with traditional colocation services, offering full control over the environment and significant cost reductions.

The system's stability has been excellent during testing. However, it is crucial to consider the dependence on a stable internet connection and the implementation of a power backup solution (UPS) to ensure service continuity in the event of potential outages.

Conclusions: An Efficient and Controllable Approach

The combination of Proxmox, Kubernetes, and Cloudflare, running on owned hardware and using cloudflared for secure service exposure, offers a robust, efficient, and scalable solution for application deployment. This approach provides a high degree of control, flexibility, and cost optimization compared to traditional colocation.

Although it is not a universal solution, it is ideal for personal projects, startups in early stages, or companies with specific needs that seek to optimize their IT infrastructure without incurring excessive expenses.

This project highlights the potential of open-source software, in this case, Proxmox, Kubernetes, and cloudflared, as well as the ability to create powerful infrastructures with a "do-it-yourself" approach. I will continue to explore and share new optimizations and improvements in future publications.