Integrate redirect lookups with Nginx using lua-nginx-module and the Edge Query API

Nginx Integration Guide

This guide shows you how to integrate redirect lookups into Nginx using lua-nginx-module and the Edge Query API. You'll learn both real-time API lookup with Lua and periodic sync with map files.

Estimated time: 30 minutes

Overview

By the end of this guide, you'll have Nginx automatically handling redirects by querying the Edge Query API in real-time or using locally synced map files. Incoming requests for old paths will be redirected to new destinations without manual configuration updates.

Prerequisites

Nginx 1.24+ or OpenResty (recommended for built-in Lua support)
lua-resty-http library (for real-time pattern)
curl command-line tool (for testing and sync script)
An API key from your project's Settings > API Keys page
Your project ID (found in the project dashboard URL)

What You'll Build

flowchart LR
    A[Client Request] --> B[Nginx]
    B --> C{access_by_lua}
    C --> D[HTTP API Call]
    D --> E{Match Found?}
    E -->|Yes| F[ngx.redirect 301]
    E -->|No| G[proxy_pass Backend]
    F --> A

Pattern 1: Real-Time API Lookup

This pattern uses Nginx's Lua integration to make HTTP API calls during the access_by_lua phase. Each request triggers an API lookup to fetch redirect destinations in real-time.

Step 1: Install OpenResty or lua-nginx-module

Option A: OpenResty (Recommended)

OpenResty includes Nginx with Lua support built-in:

# Ubuntu/Debian
wget -qO - https://openresty.org/package/pubkey.gpg | sudo apt-key add -
sudo apt-get -y install software-properties-common
sudo add-apt-repository -y "deb http://openresty.org/package/ubuntu $(lsb_release -sc) main"
sudo apt-get update
sudo apt-get install openresty

# RHEL/CentOS
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo
sudo yum install -y openresty

Option B: Standard Nginx with lua-nginx-module

Compile Nginx with lua-nginx-module or install from distribution packages. Check if already installed:

nginx -V 2>&1 | grep lua-nginx-module

Step 2: Install lua-resty-http

Install the HTTP client library for Lua:

# Using OPM (OpenResty Package Manager)
opm get ledgetech/lua-resty-http

# OR using LuaRocks
luarocks install lua-resty-http

Verify installation: Check that Nginx can load the module:

nginx -t

Step 3: Configure Nginx with Lua Lookup

Edit your Nginx configuration (typically /etc/nginx/nginx.conf or /usr/local/openresty/nginx/conf/nginx.conf):

# Top-level: Allow Nginx to access environment variables
env REDIRECTS_API_KEY;
env REDIRECTS_PROJECT_ID;

http {
    # Lua shared dict for caching (optional but recommended)
    lua_shared_dict redirects_cache 10m;

    server {
        listen 80;
        server_name example.com;

        location / {
            access_by_lua_block {
                local http = require "resty.http"
                local cjson = require "cjson"

                -- Configuration
                local api_key = os.getenv("REDIRECTS_API_KEY") or "YOUR_API_KEY"
                local project_id = os.getenv("REDIRECTS_PROJECT_ID") or "YOUR_PROJECT_ID"
                local api_url = "https://api.3xx.app/v1/lookup"

                -- Get the request path
                local request_path = ngx.var.uri

                -- Create HTTP client (use cosockets, not subrequests)
                local httpc = http.new()
                httpc:set_timeout(2000)  -- 2 second timeout

                -- Make API request
                local res, err = httpc:request_uri(api_url, {
                    method = "GET",
                    query = {
                        path = request_path
                    },
                    headers = {
                        ["X-API-Key"] = api_key,
                        ["X-Project-ID"] = project_id,
                    },
                })

                -- Handle API response
                if res and res.status == 200 then
                    -- Parse JSON response
                    local ok, data = pcall(cjson.decode, res.body)
                    if ok and data.destination then
                        -- Redirect found - return 301
                        return ngx.redirect(data.destination, 301)
                    end
                end

                -- No redirect found or API error - continue to backend
                -- (request will proceed to proxy_pass)
            }

            # Backend proxy
            proxy_pass http://backend:8080;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

Why lua-resty-http (not ngx.location.capture): Cosockets allow non-blocking external HTTP calls without consuming nginx worker connections. Subrequests via ngx.location.capture are designed for internal locations only.

Step 4: Set Environment Variables

Set environment variables before starting Nginx:

export REDIRECTS_API_KEY="YOUR_API_KEY"
export REDIRECTS_PROJECT_ID="YOUR_PROJECT_ID"

For systemd services, edit /lib/systemd/system/nginx.service or /lib/systemd/system/openresty.service:

[Service]
Environment="REDIRECTS_API_KEY=YOUR_API_KEY"
Environment="REDIRECTS_PROJECT_ID=YOUR_PROJECT_ID"

Reload systemd:

sudo systemctl daemon-reload

Step 5: Test the Configuration

Verification: Test syntax:

nginx -t
# OR for OpenResty:
openresty -t

You should see: syntax is ok and test is successful

Restart Nginx:

sudo systemctl restart nginx
# OR
sudo systemctl restart openresty

Verification: Test a redirect:

curl -I http://localhost/old-path

You should see a 301 Moved Permanently response with a Location header.

Pattern 2: Periodic Sync + Map File

This pattern fetches redirect rules from the Export API periodically and generates an Nginx map file. Nginx loads this map directly for zero-latency lookups.

Step 1: Create Sync Script

Create /usr/local/bin/sync-redirects.sh:

#!/bin/bash

API_KEY="YOUR_API_KEY"
PROJECT_ID="YOUR_PROJECT_ID"
API_URL="https://api.3xx.app/v1/export"
OUTPUT_FILE="/etc/nginx/redirects.map"
TEMP_FILE="${OUTPUT_FILE}.tmp"

# Fetch map file from API
curl -s -H "X-API-Key: ${API_KEY}" -H "X-Project-ID: ${PROJECT_ID}" \
    "${API_URL}?format=nginx-map-exact" > "${TEMP_FILE}"

# Atomic replacement (prevents partial file reads during reload)
mv "${TEMP_FILE}" "${OUTPUT_FILE}"

# Reload Nginx gracefully
nginx -s reload

echo "$(date): Synced redirects map"

Make it executable:

sudo chmod +x /usr/local/bin/sync-redirects.sh

Why temp file + mv: Atomic replacement ensures Nginx never reads a partially-written map file during reload.

Step 2: Configure Nginx to Use Map File

Edit your Nginx configuration:

http {
    # Map request URI to redirect destination
    map $uri $redirect_destination {
        default "";
        include /etc/nginx/redirects.map;
    }

    server {
        listen 80;
        server_name example.com;

        location / {
            # Check if redirect exists
            if ($redirect_destination) {
                return 301 $redirect_destination;
            }

            # No redirect - pass to backend
            proxy_pass http://backend:8080;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

Verification: Run the sync script manually:

sudo /usr/local/bin/sync-redirects.sh

Check the generated map file:

cat /etc/nginx/redirects.map

You should see Nginx map syntax like:

"/old-path" "https://example.com/new-path";
"/another-old" "https://example.com/another-new";

Test a redirect:

curl -I http://localhost/old-path

Step 3: Schedule with Cron

Add to crontab (run sudo crontab -e):

# Sync redirects every 5 minutes
*/5 * * * * /usr/local/bin/sync-redirects.sh >> /var/log/redirects-sync.log 2>&1

Alternatives:

systemd timer: More reliable than cron with better logging
CI/CD webhook: Trigger sync immediately after redirect changes

Fallback Behavior

Both patterns are designed to fail gracefully:

Real-Time Lookup Pattern

If the API is unreachable or times out:

The pcall wrapper catches errors
No redirect is triggered
Request proceeds to proxy_pass backend

Timeout handling: httpc:set_timeout(2000) sets a 2-second timeout. Adjust based on your latency tolerance.

Periodic Sync Pattern

If the sync script fails:

Previous /etc/nginx/redirects.map remains in place
Redirects continue working with stale data
Check /var/log/redirects-sync.log for errors

If the map file is missing or empty:

map $uri $redirect_destination returns empty string
if ($redirect_destination) evaluates to false
All requests pass through to backend (safe default)

Testing

Test comprehensive redirect scenarios:

Test 1: Exact Match

Create a redirect rule in the dashboard: /old-page → https://example.com/new-page

curl -I http://localhost/old-page

Expected: 301 Moved Permanently with Location: https://example.com/new-page

Test 2: Prefix Match (Real-Time Pattern Only)

Create a prefix redirect: /blog/* → https://blog.example.com/*

curl -I http://localhost/blog/2024/post-title

Expected: 301 with Location: https://blog.example.com/blog/2024/post-title

Note: Map file pattern uses nginx-map-exact format (exact matches only). For prefix matching with maps, use nginx-map-prefix format and adjust map configuration.

Test 3: No Match (Pass-Through)

curl -I http://localhost/api/users

Expected: Backend response (200, 404, etc.) - not a redirect

Test 4: API Failure (Real-Time Pattern Only)

Temporarily break the API key in nginx.conf:

local api_key = "INVALID_KEY"

Reload Nginx:

sudo nginx -s reload
curl -I http://localhost/old-page

Expected: Backend response (graceful degradation)

Restore the correct API key and reload.

Advanced: Caching

Add caching to the real-time pattern to reduce API calls:

access_by_lua_block {
    local http = require "resty.http"
    local cjson = require "cjson"

    -- Get from cache first
    local cache = ngx.shared.redirects_cache
    local request_path = ngx.var.uri
    local cached_dest = cache:get(request_path)

    if cached_dest then
        if cached_dest ~= "NULL" then
            return ngx.redirect(cached_dest, 301)
        end
        -- Cached NULL means no redirect - continue to backend
        return
    end

    -- Cache miss - call API
    local api_key = os.getenv("REDIRECTS_API_KEY") or "YOUR_API_KEY"
    local project_id = os.getenv("REDIRECTS_PROJECT_ID") or "YOUR_PROJECT_ID"
    local api_url = "https://api.3xx.app/v1/lookup"

    local httpc = http.new()
    httpc:set_timeout(2000)

    local res, err = httpc:request_uri(api_url, {
        method = "GET",
        query = { path = request_path },
        headers = {
            ["X-API-Key"] = api_key,
            ["X-Project-ID"] = project_id,
        },
    })

    if res and res.status == 200 then
        local ok, data = pcall(cjson.decode, res.body)
        if ok and data.destination then
            -- Cache for 5 minutes (300 seconds)
            cache:set(request_path, data.destination, 300)
            return ngx.redirect(data.destination, 301)
        end
    end

    -- No redirect - cache NULL to avoid repeated API calls
    cache:set(request_path, "NULL", 300)
}

Troubleshooting

lua-nginx-module Not Installed

Symptom: unknown directive "access_by_lua_block"

Cause: Nginx compiled without Lua support

Fix: Verify Lua module is loaded:

nginx -V 2>&1 | grep lua-nginx-module

If not present, install OpenResty or recompile Nginx with lua-nginx-module.

lua-resty-http Not Found

Symptom: module 'resty.http' not found

Cause: Library not installed or not in Lua path

Fix:

# Check Lua path
nginx -V 2>&1 | grep lua_package_path

# Install via OPM
opm get ledgetech/lua-resty-http

# OR set custom path in nginx.conf
lua_package_path "/usr/local/openresty/lualib/?.lua;;";

Subrequests vs Cosockets Confusion

Symptom: Complex subrequest configurations that don't work with external APIs

Issue: ngx.location.capture is for internal Nginx locations, not external HTTP calls

Fix: Always use lua-resty-http with cosockets for external API calls. Cosockets are non-blocking and designed for this use case.

Map File Syntax Errors

Symptom: nginx: [emerg] unexpected end of file or parsing errors

Cause: Malformed map file from failed sync

Fix:

Check map file syntax:
```
cat /etc/nginx/redirects.map
```
Ensure format is:
```
"source-path" "destination-url";
```
Re-run sync script:
```
sudo /usr/local/bin/sync-redirects.sh
```

Environment Variables Not Accessible

Symptom: os.getenv() returns nil

Cause: Nginx doesn't inherit shell environment by default

Fix: Use env directive at top-level nginx.conf:

env REDIRECTS_API_KEY;
env REDIRECTS_PROJECT_ID;

Or hardcode values in the Lua block (less secure but simpler).

Common API Issues

For authentication errors, rate limiting, quota exhaustion, and other API-level problems, see the Troubleshooting Guide.

Next Steps

Review API Reference for advanced lookup options
Explore Export Formats for optimized sync patterns (nginx-map-exact, nginx-map-prefix)
Implement caching to reduce API quota usage
Set up monitoring for API quota in project dashboard
Consider migrating to Periodic Sync pattern for high-traffic applications

Nginx

On this page