Hari 42: Webhook Alerting — Falco to Python Receiver

Day 41 proved Falco fires 6 alerts. Tapi alerts hanya ke WebUI dashboard — manual review. Hari ini buat automated alerting: Falcosidekick → webhook receiver → IF logic routes CRITICAL ke Slack path (Day 48).

Pivot: n8n → Python Webhook Receiver

Tutorial pakai n8n (low-code automation, GUI workflow builder). Tapi docker pull n8nio/n8n (~400MB) timed out 2x (5 menit each) — slow network. Pivot ke Python webhook receiver yang achieves same learning objectives:

n8n Feature	Python Equivalent
Webhook trigger (POST /falco-alert)	BaseHTTPRequestHandler.do_POST()
IF node (priority == Critical)	if priority.lower() == "critical":
True branch → Slack (Day 48)	logger.info("route to Slack")
False branch → log	logger.info("log only")

DevSecOps = problem solving, bukan dogmatic tool following. Tutorial adalah guide, bukan gospel. Pragmatic solution > stuck on tooling.

Webhook Receiver Design

# IF node logic — CRITICAL routes to Slack (Day 48), others just log
if priority.lower() == "critical":
    logger.info("IF TRUE: CRITICAL — route to Slack #security-alerts (Day 48)")
    # Day 48 will add: requests.post(slack_webhook_url, json=slack_payload)
else:
    logger.info(f"IF FALSE: {priority} — log only (below CRITICAL threshold)")

Endpoints: POST /webhook/falco-alert (receive alerts), GET /health, GET /alerts (view last 20).

Falcosidekick Webhook Config

Update falco-values.yaml — add webhook output:

# Before (Day 40):
falcosidekick:
  enabled: true
  webui:
    enabled: true

# After (Day 42):
falcosidekick:
  enabled: true
  webui:
    enabled: true
  config:
    webhook:
      address: "http://host.k3d.internal:5678/webhook/falco-alert"
      method: "POST"

host.k3d.internal = k3d DNS yang resolve ke host machine (192.168.65.254). Falcosidekick pod di cluster bisa reach webhook receiver di host.

Helm Upgrade — Revision 3

$ helm upgrade falco falcosecurity/falco --namespace falco --values security/falco-values.yaml
REVISION: 3

Yang restart: Falcosidekick pods (config berubah — webhook added)
Yang TIDAK restart: Falco DaemonSet pods (config tidak berubah)

Tidak ada eBPF probe warm-up (Day 41 lesson). Alert bisa trigger immediately.

Verify: Webhook Output Enabled

$ kubectl logs -l app.kubernetes.io/name=falcosidekick -n falco --tail=10
[INFO] : Enabled Outputs: [Webhook WebUI]
[INFO] : Falcosidekick is up and listening on :2801

Dari [WebUI] (Day 39) → [Webhook WebUI] (Day 42). Webhook output aktif.

End-to-End Test: Attack → Falco → Webhook

Deploy attacker pod (alpine tagged securebank:attacker):

# command: sh -c "cat /etc/passwd && sleep 15"
# Triggers: Sensitive file read (CRITICAL) + Shell spawned (WARNING)
kubectl apply -f test-attacker-e2e.yaml

Webhook receiver logs (real attack alerts):

ALERT RECEIVED | priority=Critical rule=Sensitive file read in SecureBank pod=attacker-e2e
  -> IF TRUE: CRITICAL alert — route to Slack #security-alerts (Day 48)
  -> Slack payload prepared: 'Sensitive file read in SecureBank' in pod 'attacker-e2e'

ALERT RECEIVED | priority=Warning rule=Shell spawned in SecureBank container pod=attacker-e2e
  -> IF FALSE: Warning alert — log only (below CRITICAL threshold)

ALERT RECEIVED | priority=Critical rule=Sensitive file read in SecureBank pod=attacker-e2e
  -> IF TRUE: CRITICAL alert — route to Slack #security-alerts (Day 48)

Falcosidekick logs:

[INFO] : Webhook - POST OK (200)   # 3 alerts forwarded to webhook
[INFO] : WebUI - POST OK (200)     # alerts also to WebUI dashboard

Alert Pipeline: Full Chain

Attacker (cat /etc/passwd)
  → Falco eBPF probe (syscall capture: openat /etc/passwd)
  → Falco engine (rule match: Sensitive file read, CRITICAL)
  → Falcosidekick (receive alert, forward to outputs)
  → Webhook (POST http://host.k3d.internal:5678/webhook/falco-alert)
  → Python webhook receiver (IF logic: CRITICAL → Slack path)
  → [Day 48: Slack #security-alerts notification]

Before vs After

Aspek	Before (Day 41)	After (Day 42)
Falcosidekick outputs	WebUI only	WebUI + Webhook
Alert routing	Dashboard only	IF logic (CRITICAL → Slack)
Automated response	None	Webhook receiver with routing
Helm revision	2	3
Pipeline	Falco → WebUI	Falco → Falcosidekick → Webhook → IF routing

Pelajaran Hari Ini

DevSecOps = problem solving. n8n image pull timed out → pivot ke Python webhook receiver. Same objectives, no Docker pull, committed to repo. Tutorial adalah guide, bukan gospel.

Falcosidekick multi-output. WebUI (dashboard, human review) + Webhook (automation, programmatic routing). Multiple outputs = defense in depth untuk alerting.

host.k3d.internal = k3d DNS untuk host access. Pods di k3d cluster bisa reach host services. Berguna untuk local dev — webhook receiver di host, Falcosidekick di cluster.

Helm upgrade selective restart. Falcosidekick restart (config berubah), Falco DaemonSet tidak (config sama). Tidak ada eBPF warm-up — alert trigger immediately.

IF node = conditional alert routing. CRITICAL → Slack (immediate), WARNING/NOTICE → log (periodic). Priority-based routing = alert fatigue prevention.

Repo

Semua code dan dokumentasi ada di: https://github.com/stayrelevantid/chalange-devsecops

Kesimpulan

Automated alerting pipeline complete. Falcosidekick → Python webhook receiver → IF logic routes CRITICAL to Slack path. 3 alerts received from real attack, Falcosidekick POST OK 200. n8n Docker pull timed out → pragmatic pivot ke Python. Besok Day 43 — External Secrets Operator, fix CKV_K8S_35 (secret as env vars).