Hosting a CTF for approximately 400 participants, with around 100 concurrent users, presented a practical infrastructure and system-design challenge. The platform needed to remain responsive under concurrent traffic while supporting the CTF application, database, caching layer, and multiple challenge services.

This post focuses on the architecture and design decisions I made while building and deploying the platform, how the different components were connected, and the engineering trade-offs involved. It also covers the shortcomings I encountered in the architecture and the lessons I took away from running the system under real-world usage.

Architecture Overview

The CTF platform was deployed on a virtual machine, with Docker used to run and manage the different services. The architecture consisted of two main paths: the CTF platform itself and the individual challenge services.

Participant traffic to the platform was routed through Cloudflare → Nginx → CTFd. CTFd handled the competition logic and used MariaDB for persistent data and Redis for supporting application operations. The challenges were packaged as separate Docker containers and exposed independently, allowing each challenge to have its own environment and dependencies.

The goal was to keep the application layer centralized while allowing challenge services to operate independently.

Design Decisions and Implementation

Several architectural decisions shaped the platform. Each was driven by a trade-off between operational simplicity, performance, isolation, and the complexity of running the competition.

Single-Host Architecture vs. Distributed Infrastructure

The first decision was to run the platform on a single virtual machine rather than distributing the system across multiple servers. This kept the infrastructure relatively simple to deploy and operate: the application, supporting services, and challenge workloads could all be managed from one environment.

The trade-off was isolation. While a single host reduced operational overhead, it also created a larger shared failure domain. Resource contention, configuration errors, or a compromise affecting the host could potentially impact multiple parts of the platform.

Stateful Core vs. Stateless Application Layer

The platform was divided conceptually between the application layer and the services responsible for maintaining state. CTFd handled the competition logic, while MariaDB persisted application data and Redis provided supporting in-memory state.

                CTF Platform
                     |
          +----------+----------+
          |                     |
   Application Layer       Stateful Services
          |                 /           \
        CTFd           MariaDB         Redis

This separation meant that the application could be restarted independently without losing the underlying competition state. It also established clear dependencies between the application and the services responsible for persistence and caching.

Centralized Entry Point vs. Direct Service Exposure

The main web application followed a centralized path through Cloudflare and Nginx before reaching CTFd. This provided a single entry point for normal web traffic.

Challenge services, however, had a different requirement. Participants needed to connect to individual challenges directly, so those services had to be reachable through their respective ports.

                     ┌──────────────┐
                     │   INTERNET   │
                     └──────┬───────┘
                            │
                     ┌──────▼───────┐
                     │  CLOUDFLARE  │
                     └──────┬───────┘
                            │
                     ┌──────▼───────┐
                     │     NGINX    │
                     └──────┬───────┘
                            │
                     ┌──────▼───────┐
                     │     CTFd     │
                     └──────┬───────┘
                            │
                ┌───────────┴───────────┐
                ▼                       ▼
          ┌──────────┐            ┌──────────┐
          │ MariaDB  │            │  Redis   │
          └──────────┘            └──────────┘


                     CHALLENGE NETWORK

                     ┌──────────────┐
                     │   INTERNET   │
                     └──────┬───────┘
                            │
          ┌─────────────────┼─────────────────┐
          ▼                 ▼                 ▼
    ┌───────────┐     ┌───────────┐     ┌───────────┐
    │ Challenge │     │ Challenge │     │ Challenge │
    │     1     │     │     2     │     │     3     │
    └───────────┘     └───────────┘     └───────────┘

This created an important distinction between the public application surface and the challenge surface, but also made network exposure a significant architectural consideration.

Isolation vs. Operational Convenience

Docker provided a convenient abstraction for packaging the individual challenges and their dependencies. Each challenge could run in its own container without requiring its dependencies to be installed directly on the host.

The trade-off was that containerization alone did not define the complete security boundary. Stronger isolation would require deliberate network segmentation, restricted communication between containers, and tighter control over what each workload could access.

The architecture therefore favored ease of deployment and management, while leaving some of the stronger isolation mechanisms as future improvements.

Static Infrastructure vs. Dynamic Challenge Deployment

The challenge services were treated as independent workloads rather than applications tightly coupled to the host environment. Packaging challenges as containers made them easier to start, stop, replace, and modify independently.

This was particularly useful during a competition, where individual challenges could require changes without requiring the entire platform to be redeployed.

                Challenge Layer
              /        |        \
             v         v         v
        Challenge 1  Challenge 2  Challenge 3
             |         |         |
         Container  Container  Container

This approach reduced deployment friction and made the challenge layer more modular.

Availability vs. Blast Radius

The final trade-off was between keeping the system simple and available and limiting the consequences of a failure.

Running multiple components on the same host made the platform easier to operate and reduced the infrastructure required to run the competition. At the same time, it meant that the platform, supporting services, management interfaces, and challenge workloads existed within a relatively small infrastructure boundary.

This created a larger potential blast radius: an issue affecting the underlying host could have consequences beyond a single challenge or application.

The architecture therefore optimized heavily for operational simplicity and availability, but the incident later exposed where those boundaries were not strong enough.

What Went Wrong!!?

The infrastructure handled the competition as expected, but the server was eventually compromised. Investigating the incident brought the network configuration back into focus. While the main application was routed through Cloudflare and Nginx, the VM also had several other services exposed directly through publicly accessible ports. These included infrastructure and management services that did not need to be reachable by participants. At the time, those exposures were largely a result of making deployment and debugging easier.

The bigger issue was that I had treated Docker as the primary isolation boundary without giving enough attention to the network boundary around the host. The CTF platform, supporting services, management interfaces, and intentionally vulnerable challenge containers were running within the same infrastructure environment. This meant that the compromise of one exposed component had the potential to affect more of the system than it should have.

In retrospect, the architecture was optimized heavily for operational simplicity and availability, but not enough for limiting the blast radius of a security incident. Internal services should have been isolated from the public network, administrative interfaces should have been restricted, and the challenge environment should have had a much stronger boundary from the core platform. The incident made the trade-off clear: getting a system to work under load is only one part of designing infrastructure; deciding what should be allowed to reach what is equally important.

Wrap Up

The platform ultimately handled the scale it was designed for, but the incident exposed weaknesses that aren't visible when looking at performance alone. Building the system forced me to think about deployment, state management, networking, containerization, and the trade-offs involved in keeping infrastructure simple.

The biggest takeaway was that a system can be scalable and still have architectural weaknesses. In this case, the same decisions that made the platform easy to deploy and operate also increased its potential blast radius. Designing these boundaries deliberately from the beginning is something I would approach very differently today.

This was not even live for 1 hour T_T

Acknowledgements(pls read - personal note)

A lot of what I write here is simply my way of learning. I enjoy reading engineering blogs from teams at companies like Netflix, Uber, and Spotify, and I find the way they think about large-scale systems particularly interesting. These posts are my own small attempts to apply some of those ideas to the systems I build.

I also recognize that the system designs published by these companies are often the result of years of experience and the work of many distinguished engineers operating at an entirely different scale. This is nowhere near that level. I'm simply documenting the small systems I build, the decisions I make, the things that break, and what I learn from them. If writing about these projects helps me understand system design a little better, then the exercise has already served its purpose.