← All Use Cases
🌐
Plackett-Burman Design

Firewall Rule Ordering

Plackett-Burman screening of 6 iptables/nftables parameters for packet processing throughput

Summary

This experiment investigates firewall rule ordering. Plackett-Burman screening of 6 iptables/nftables parameters for packet processing throughput.

The design varies 6 factors: rule count (rules), ranging from 100 to 5000, conntrack max (entries), ranging from 65536 to 1048576, rule ordering, ranging from frequency to sequential, hashlimit burst (packets), ranging from 5 to 100, nf tables, ranging from iptables to nftables, and batch verdict, ranging from off to on. The goal is to optimize 2 responses: throughput mpps (Mpps) (maximize) and latency us (us) (minimize). Fixed conditions held constant across all runs include interface = eth0, protocol mix = 80_tcp_20_udp.

A Plackett-Burman screening design was used to efficiently test 6 factors in only 8 runs. This design assumes interactions are negligible and focuses on identifying the most influential main effects.

Key Findings

For throughput mpps, the most influential factors were rule count (42.1%), nf tables (21.1%), batch verdict (14.3%). The best observed value was 4.53 (at rule count = 100, conntrack max = 1048576, rule ordering = frequency).

For latency us, the most influential factors were rule count (55.1%), nf tables (16.6%), batch verdict (14.3%). The best observed value was 0.4 (at rule count = 100, conntrack max = 1048576, rule ordering = frequency).

Recommended Next Steps

Experimental Setup

Factors

FactorLowHighUnit
rule_count1005000rules
conntrack_max655361048576entries
rule_orderingfrequencysequential
hashlimit_burst5100packets
nf_tablesiptablesnftables
batch_verdictoffon

Fixed: interface = eth0, protocol_mix = 80_tcp_20_udp

Responses

ResponseDirectionUnit
throughput_mpps↑ maximizeMpps
latency_us↓ minimizeus

Configuration

use_cases/49_firewall_rule_ordering/config.json
{ "metadata": { "name": "Firewall Rule Ordering", "description": "Plackett-Burman screening of 6 iptables/nftables parameters for packet processing throughput" }, "factors": [ { "name": "rule_count", "levels": [ "100", "5000" ], "type": "continuous", "unit": "rules" }, { "name": "conntrack_max", "levels": [ "65536", "1048576" ], "type": "continuous", "unit": "entries" }, { "name": "rule_ordering", "levels": [ "frequency", "sequential" ], "type": "categorical", "unit": "" }, { "name": "hashlimit_burst", "levels": [ "5", "100" ], "type": "continuous", "unit": "packets" }, { "name": "nf_tables", "levels": [ "iptables", "nftables" ], "type": "categorical", "unit": "" }, { "name": "batch_verdict", "levels": [ "off", "on" ], "type": "categorical", "unit": "" } ], "fixed_factors": { "interface": "eth0", "protocol_mix": "80_tcp_20_udp" }, "responses": [ { "name": "throughput_mpps", "optimize": "maximize", "unit": "Mpps" }, { "name": "latency_us", "optimize": "minimize", "unit": "us" } ], "settings": { "operation": "plackett_burman", "test_script": "use_cases/49_firewall_rule_ordering/sim.sh" } }

Experimental Matrix

The Plackett-Burman Design produces 8 runs. Each row is one experiment with specific factor settings.

Runrule_countconntrack_maxrule_orderinghashlimit_burstnf_tablesbatch_verdict
150001048576sequential5iptablesoff
210065536sequential100iptablesoff
31001048576frequency100iptableson
450001048576sequential100nftableson
51001048576frequency5nftablesoff
6500065536frequency100nftablesoff
710065536sequential5nftableson
8500065536frequency5iptableson

Step-by-Step Workflow

1

Preview the design

Terminal
$ doe info --config use_cases/49_firewall_rule_ordering/config.json
2

Generate the runner script

Terminal
$ doe generate --config use_cases/49_firewall_rule_ordering/config.json \ --output use_cases/49_firewall_rule_ordering/results/run.sh --seed 42
3

Execute the experiments

Terminal
$ bash use_cases/49_firewall_rule_ordering/results/run.sh
4

Analyze results

Terminal
$ doe analyze --config use_cases/49_firewall_rule_ordering/config.json
5

Get optimization recommendations

Terminal
$ doe optimize --config use_cases/49_firewall_rule_ordering/config.json
6

Multi-objective optimization

With 2 competing responses, use --multi to find the best compromise via Derringer–Suich desirability.

Terminal
$ doe optimize --config use_cases/49_firewall_rule_ordering/config.json --multi
7

Generate the HTML report

Terminal
$ doe report --config use_cases/49_firewall_rule_ordering/config.json \ --output use_cases/49_firewall_rule_ordering/results/report.html

Features Exercised

FeatureValue
Design typeplackett_burman
Factor typescontinuous (3), categorical (3)
Arg styledouble-dash
Responses2 (throughput_mpps ↑, latency_us ↓)
Total runs8

Analysis Results

Generated from actual experiment runs using the DOE Helper Tool.

Response: throughput_mpps

Top factors: rule_count (42.1%), nf_tables (21.1%), batch_verdict (14.3%).

ANOVA

SourceDFSSMSFp-value
SourceDFSSMSFp-value
rule_count16.35466.35467.0390.0328
conntrack_max10.46560.46560.5160.4959
rule_ordering10.12250.12250.1360.7235
hashlimit_burst10.09900.09900.1100.7502
nf_tables11.59311.59311.7650.2257
batch_verdict10.73810.73810.8180.3959
rule_count*conntrack_max10.12250.12250.1360.7235
rule_count*rule_ordering10.46560.46560.5160.4959
rule_count*hashlimit_burst11.59311.59311.7650.2257
rule_count*nf_tables10.09900.09900.1100.7502
rule_count*batch_verdict11.08781.08781.2050.3087
conntrack_max*rule_ordering16.35466.35467.0390.0328
conntrack_max*hashlimit_burst10.73810.73810.8180.3959
conntrack_max*nf_tables11.08781.08781.2050.3087
conntrack_max*batch_verdict10.09900.09900.1100.7502
rule_ordering*hashlimit_burst11.08781.08781.2050.3087
rule_ordering*nf_tables10.73810.73810.8180.3959
rule_ordering*batch_verdict11.59311.59311.7650.2257
hashlimit_burst*nf_tables16.35466.35467.0390.0328
hashlimit_burst*batch_verdict10.46560.46560.5160.4959
nf_tables*batch_verdict10.12250.12250.1360.7235
Error(LenthPSE)76.31960.9028
Total710.46081.4944

Pareto Chart

Pareto chart for throughput_mpps

Main Effects Plot

Main effects plot for throughput_mpps

Normal Probability Plot of Effects

Normal probability plot for throughput_mpps

Half-Normal Plot of Effects

Half-normal plot for throughput_mpps

Model Diagnostics

Model diagnostics for throughput_mpps

Response: latency_us

Top factors: rule_count (55.1%), nf_tables (16.6%), batch_verdict (14.3%).

ANOVA

SourceDFSSMSFp-value
SourceDFSSMSFp-value
rule_count1664.3012664.301211.2570.0122
conntrack_max134.031334.03130.5770.4724
rule_ordering10.10120.10120.0020.9681
hashlimit_burst10.15130.15130.0030.9610
nf_tables159.951259.95121.0160.3471
batch_verdict144.651244.65120.7570.4132
rule_count*conntrack_max10.10130.10130.0020.9681
rule_count*rule_ordering134.031334.03130.5770.4724
rule_count*hashlimit_burst159.951359.95131.0160.3471
rule_count*nf_tables10.15130.15130.0030.9610
rule_count*batch_verdict195.911395.91131.6250.2430
conntrack_max*rule_ordering1664.3012664.301211.2570.0122
conntrack_max*hashlimit_burst144.651244.65120.7570.4132
conntrack_max*nf_tables195.911395.91131.6250.2430
conntrack_max*batch_verdict10.15130.15130.0030.9610
rule_ordering*hashlimit_burst195.911395.91131.6250.2430
rule_ordering*nf_tables144.651344.65130.7570.4132
rule_ordering*batch_verdict159.951359.95131.0160.3471
hashlimit_burst*nf_tables1664.3012664.301211.2570.0122
hashlimit_burst*batch_verdict134.031334.03130.5770.4724
nf_tables*batch_verdict10.10130.10130.0020.9681
Error(LenthPSE)7413.083159.0119
Total7899.0987128.4427

Pareto Chart

Pareto chart for latency_us

Main Effects Plot

Main effects plot for latency_us

Normal Probability Plot of Effects

Normal probability plot for latency_us

Half-Normal Plot of Effects

Half-normal plot for latency_us

Model Diagnostics

Model diagnostics for latency_us

Response Surface Plots

3D surfaces fitted with quadratic RSM. Red dots are observed data points.

latency us conntrack max vs hashlimit burst

RSM surface: latency us conntrack max vs hashlimit burst

latency us rule count vs conntrack max

RSM surface: latency us rule count vs conntrack max

latency us rule count vs hashlimit burst

RSM surface: latency us rule count vs hashlimit burst

throughput mpps conntrack max vs hashlimit burst

RSM surface: throughput mpps conntrack max vs hashlimit burst

throughput mpps rule count vs conntrack max

RSM surface: throughput mpps rule count vs conntrack max

throughput mpps rule count vs hashlimit burst

RSM surface: throughput mpps rule count vs hashlimit burst

Multi-Objective Optimization

When responses compete, Derringer–Suich desirability finds the best compromise. Each response is scaled to a 0–1 desirability, then combined via a weighted geometric mean.

Overall Desirability
D = 0.9956

Per-Response Desirability

ResponseWeightDesirabilityPredictedDir
throughput_mpps 1.5
0.9926
4.68 0.9926 4.68 Mpps
latency_us 1.0
1.0000
-5.06 1.0000 -5.06 us

Recommended Settings

FactorValue
rule_count4788 rules
conntrack_max1.107e+05 entries
rule_orderingfrequency
hashlimit_burst92.27 packets
nf_tablesnftables
batch_verdicton

Source: from RSM model prediction

Trade-off Summary

Sacrifice = how much worse than single-objective best.

ResponsePredictedBest ObservedSacrifice
latency_us-5.060.40-5.46

Top 3 Runs by Desirability

RunDFactor Settings
#30.8021rule_count=5000, conntrack_max=65536, rule_ordering=frequency, hashlimit_burst=100, nf_tables=nftables, batch_verdict=off
#70.7012rule_count=5000, conntrack_max=1048576, rule_ordering=sequential, hashlimit_burst=100, nf_tables=nftables, batch_verdict=on

Model Quality

ResponseType
latency_us0.8477linear

Full Multi-Objective Output

doe optimize --multi
============================================================ MULTI-OBJECTIVE OPTIMIZATION Method: Derringer-Suich Desirability Function ============================================================ Overall desirability: D = 0.9956 Response Weight Desirability Predicted Direction --------------------------------------------------------------------- throughput_mpps 1.5 0.9926 4.68 Mpps ↑ latency_us 1.0 1.0000 -5.06 us ↓ Recommended settings: rule_count = 4788 rules conntrack_max = 1.107e+05 entries rule_ordering = frequency hashlimit_burst = 92.27 packets nf_tables = nftables batch_verdict = on (from RSM model prediction) Trade-off summary: throughput_mpps: 4.68 (best observed: 4.53, sacrifice: -0.15) latency_us: -5.06 (best observed: 0.40, sacrifice: -5.46) Model quality: throughput_mpps: R² = 0.7718 (linear) latency_us: R² = 0.8477 (linear) Top 3 observed runs by overall desirability: 1. Run #5 (D=0.9545): rule_count=5000, conntrack_max=65536, rule_ordering=frequency, hashlimit_burst=5, nf_tables=iptables, batch_verdict=on 2. Run #3 (D=0.8021): rule_count=5000, conntrack_max=65536, rule_ordering=frequency, hashlimit_burst=100, nf_tables=nftables, batch_verdict=off 3. Run #7 (D=0.7012): rule_count=5000, conntrack_max=1048576, rule_ordering=sequential, hashlimit_burst=100, nf_tables=nftables, batch_verdict=on

Full Analysis Output

doe analyze
=== Main Effects: throughput_mpps === Factor Effect Std Error % Contribution -------------------------------------------------------------- rule_count 1.7825 0.4322 42.1% nf_tables -0.8925 0.4322 21.1% batch_verdict 0.6075 0.4322 14.3% conntrack_max -0.4825 0.4322 11.4% rule_ordering -0.2475 0.4322 5.8% hashlimit_burst 0.2225 0.4322 5.3% === ANOVA Table: throughput_mpps === Source DF SS MS F p-value ----------------------------------------------------------------------------- rule_count 1 6.3546 6.3546 7.039 0.0328 conntrack_max 1 0.4656 0.4656 0.516 0.4959 rule_ordering 1 0.1225 0.1225 0.136 0.7235 hashlimit_burst 1 0.0990 0.0990 0.110 0.7502 nf_tables 1 1.5931 1.5931 1.765 0.2257 batch_verdict 1 0.7381 0.7381 0.818 0.3959 rule_count*conntrack_max 1 0.1225 0.1225 0.136 0.7235 rule_count*rule_ordering 1 0.4656 0.4656 0.516 0.4959 rule_count*hashlimit_burst 1 1.5931 1.5931 1.765 0.2257 rule_count*nf_tables 1 0.0990 0.0990 0.110 0.7502 rule_count*batch_verdict 1 1.0878 1.0878 1.205 0.3087 conntrack_max*rule_ordering 1 6.3546 6.3546 7.039 0.0328 conntrack_max*hashlimit_burst 1 0.7381 0.7381 0.818 0.3959 conntrack_max*nf_tables 1 1.0878 1.0878 1.205 0.3087 conntrack_max*batch_verdict 1 0.0990 0.0990 0.110 0.7502 rule_ordering*hashlimit_burst 1 1.0878 1.0878 1.205 0.3087 rule_ordering*nf_tables 1 0.7381 0.7381 0.818 0.3959 rule_ordering*batch_verdict 1 1.5931 1.5931 1.765 0.2257 hashlimit_burst*nf_tables 1 6.3546 6.3546 7.039 0.0328 hashlimit_burst*batch_verdict 1 0.4656 0.4656 0.516 0.4959 nf_tables*batch_verdict 1 0.1225 0.1225 0.136 0.7235 Error (Lenth PSE) 7 6.3196 0.9028 Total 7 10.4608 1.4944 Note: Error estimated using Lenth's pseudo-standard-error (unreplicated design) === Interaction Effects: throughput_mpps === Factor A Factor B Interaction % Contribution ------------------------------------------------------------------------ conntrack_max rule_ordering -1.7825 16.7% hashlimit_burst nf_tables -1.7825 16.7% rule_count hashlimit_burst 0.8925 8.4% rule_ordering batch_verdict -0.8925 8.4% rule_count batch_verdict 0.7375 6.9% conntrack_max nf_tables -0.7375 6.9% rule_ordering hashlimit_burst -0.7375 6.9% conntrack_max hashlimit_burst 0.6075 5.7% rule_ordering nf_tables 0.6075 5.7% rule_count rule_ordering 0.4825 4.5% hashlimit_burst batch_verdict -0.4825 4.5% rule_count conntrack_max 0.2475 2.3% nf_tables batch_verdict -0.2475 2.3% rule_count nf_tables -0.2225 2.1% conntrack_max batch_verdict 0.2225 2.1% === Summary Statistics: throughput_mpps === rule_count: Level N Mean Std Min Max ------------------------------------------------------------ 100 4 1.6350 0.5770 0.8700 2.2700 5000 4 3.4175 1.0178 2.0700 4.5300 conntrack_max: Level N Mean Std Min Max ------------------------------------------------------------ 1048576 4 2.7675 0.9181 1.7300 3.6500 65536 4 2.2850 1.5776 0.8700 4.5300 rule_ordering: Level N Mean Std Min Max ------------------------------------------------------------ frequency 4 2.6500 1.2730 1.7300 4.5300 sequential 4 2.4025 1.3511 0.8700 3.6500 hashlimit_burst: Level N Mean Std Min Max ------------------------------------------------------------ 100 4 2.4150 0.8603 1.6700 3.6500 5 4 2.6375 1.6474 0.8700 4.5300 nf_tables: Level N Mean Std Min Max ------------------------------------------------------------ iptables 4 2.9725 1.2670 1.6700 4.5300 nftables 4 2.0800 1.1621 0.8700 3.6500 batch_verdict: Level N Mean Std Min Max ------------------------------------------------------------ off 4 2.2225 0.8175 1.6700 3.4200 on 4 2.8300 1.6039 0.8700 4.5300 === Main Effects: latency_us === Factor Effect Std Error % Contribution -------------------------------------------------------------- rule_count -18.2250 4.0069 55.1% nf_tables 5.4750 4.0069 16.6% batch_verdict -4.7250 4.0069 14.3% conntrack_max 4.1250 4.0069 12.5% hashlimit_burst -0.2750 4.0069 0.8% rule_ordering -0.2250 4.0069 0.7% === ANOVA Table: latency_us === Source DF SS MS F p-value ----------------------------------------------------------------------------- rule_count 1 664.3012 664.3012 11.257 0.0122 conntrack_max 1 34.0313 34.0313 0.577 0.4724 rule_ordering 1 0.1012 0.1012 0.002 0.9681 hashlimit_burst 1 0.1513 0.1513 0.003 0.9610 nf_tables 1 59.9512 59.9512 1.016 0.3471 batch_verdict 1 44.6512 44.6512 0.757 0.4132 rule_count*conntrack_max 1 0.1013 0.1013 0.002 0.9681 rule_count*rule_ordering 1 34.0313 34.0313 0.577 0.4724 rule_count*hashlimit_burst 1 59.9513 59.9513 1.016 0.3471 rule_count*nf_tables 1 0.1513 0.1513 0.003 0.9610 rule_count*batch_verdict 1 95.9113 95.9113 1.625 0.2430 conntrack_max*rule_ordering 1 664.3012 664.3012 11.257 0.0122 conntrack_max*hashlimit_burst 1 44.6512 44.6512 0.757 0.4132 conntrack_max*nf_tables 1 95.9113 95.9113 1.625 0.2430 conntrack_max*batch_verdict 1 0.1513 0.1513 0.003 0.9610 rule_ordering*hashlimit_burst 1 95.9113 95.9113 1.625 0.2430 rule_ordering*nf_tables 1 44.6513 44.6513 0.757 0.4132 rule_ordering*batch_verdict 1 59.9513 59.9513 1.016 0.3471 hashlimit_burst*nf_tables 1 664.3012 664.3012 11.257 0.0122 hashlimit_burst*batch_verdict 1 34.0313 34.0313 0.577 0.4724 nf_tables*batch_verdict 1 0.1013 0.1013 0.002 0.9681 Error (Lenth PSE) 7 413.0831 59.0119 Total 7 899.0987 128.4427 Note: Error estimated using Lenth's pseudo-standard-error (unreplicated design) === Interaction Effects: latency_us === Factor A Factor B Interaction % Contribution ------------------------------------------------------------------------ conntrack_max rule_ordering 18.2250 21.0% hashlimit_burst nf_tables 18.2250 21.0% rule_count batch_verdict -6.9250 8.0% conntrack_max nf_tables 6.9250 8.0% rule_ordering hashlimit_burst 6.9250 8.0% rule_count hashlimit_burst -5.4750 6.3% rule_ordering batch_verdict 5.4750 6.3% conntrack_max hashlimit_burst -4.7250 5.4% rule_ordering nf_tables -4.7250 5.4% rule_count rule_ordering -4.1250 4.7% hashlimit_burst batch_verdict 4.1250 4.7% rule_count nf_tables 0.2750 0.3% conntrack_max batch_verdict -0.2750 0.3% rule_count conntrack_max 0.2250 0.3% nf_tables batch_verdict -0.2250 0.3% === Summary Statistics: latency_us === rule_count: Level N Mean Std Min Max ------------------------------------------------------------ 100 4 25.1500 3.9619 21.7000 30.8000 5000 4 6.9250 7.9101 0.4000 17.8000 conntrack_max: Level N Mean Std Min Max ------------------------------------------------------------ 1048576 4 13.9750 10.9895 1.8000 24.7000 65536 4 18.1000 12.9455 0.4000 30.8000 rule_ordering: Level N Mean Std Min Max ------------------------------------------------------------ frequency 4 16.1500 10.8734 0.4000 24.7000 sequential 4 15.9250 13.4698 1.8000 30.8000 hashlimit_burst: Level N Mean Std Min Max ------------------------------------------------------------ 100 4 16.1750 9.8659 1.8000 23.4000 5 4 15.9000 14.2237 0.4000 30.8000 nf_tables: Level N Mean Std Min Max ------------------------------------------------------------ iptables 4 13.3000 11.1107 0.4000 23.4000 nftables 4 18.7750 12.5008 1.8000 30.8000 batch_verdict: Level N Mean Std Min Max ------------------------------------------------------------ off 4 18.4000 7.7361 7.7000 24.7000 on 4 13.6750 14.9990 0.4000 30.8000

Optimization Recommendations

doe optimize
=== Optimization: throughput_mpps === Direction: maximize Best observed run: #5 rule_count = 100 conntrack_max = 1048576 rule_ordering = frequency hashlimit_burst = 100 nf_tables = iptables batch_verdict = on Value: 4.53 RSM Model (linear, R² = 0.7718, Adj R² = -0.5974): Coefficients: intercept +2.5263 rule_count -0.3038 conntrack_max -0.2413 rule_ordering -0.1113 hashlimit_burst +0.7912 nf_tables -0.4688 batch_verdict +0.0237 Predicted optimum (from linear model, at observed points): rule_count = 100 conntrack_max = 65536 rule_ordering = sequential hashlimit_burst = 100 nf_tables = iptables batch_verdict = off Predicted value: 4.1963 Surface optimum (via L-BFGS-B, linear model): rule_count = 100 conntrack_max = 65536 rule_ordering = frequency hashlimit_burst = 100 nf_tables = iptables batch_verdict = on Predicted value: 4.4662 Model quality: Good fit — general trends are captured, some noise remains. Factor importance: 1. hashlimit_burst (effect: -1.6, contribution: 40.8%) 2. nf_tables (effect: -0.9, contribution: 24.2%) 3. rule_count (effect: -0.6, contribution: 15.7%) 4. conntrack_max (effect: 0.5, contribution: 12.4%) 5. rule_ordering (effect: -0.2, contribution: 5.7%) 6. batch_verdict (effect: 0.0, contribution: 1.2%) === Optimization: latency_us === Direction: minimize Best observed run: #5 rule_count = 100 conntrack_max = 1048576 rule_ordering = frequency hashlimit_burst = 100 nf_tables = iptables batch_verdict = on Value: 0.4 RSM Model (linear, R² = 0.8477, Adj R² = -0.0662): Coefficients: intercept +16.0375 rule_count +2.3625 conntrack_max +2.0625 rule_ordering +0.1375 hashlimit_burst -7.7125 nf_tables +4.8625 batch_verdict +1.5125 Predicted optimum (from linear model, at observed points): rule_count = 100 conntrack_max = 1048576 rule_ordering = frequency hashlimit_burst = 5 nf_tables = nftables batch_verdict = off Predicted value: 26.6625 Surface optimum (via L-BFGS-B, linear model): rule_count = 100 conntrack_max = 65536 rule_ordering = frequency hashlimit_burst = 100 nf_tables = iptables batch_verdict = off Predicted value: -2.6125 Model quality: Good fit — general trends are captured, some noise remains. Factor importance: 1. hashlimit_burst (effect: 15.4, contribution: 41.4%) 2. nf_tables (effect: 9.7, contribution: 26.1%) 3. rule_count (effect: 4.7, contribution: 12.7%) 4. conntrack_max (effect: -4.1, contribution: 11.1%) 5. batch_verdict (effect: 3.0, contribution: 8.1%) 6. rule_ordering (effect: 0.3, contribution: 0.7%)
← Previous: TLS Handshake Optimization Next: DNS Resolver Caching →