Summary
This experiment investigates vpn tunnel mtu. Box-Behnken design for MTU, fragmentation, and keepalive to optimize throughput and reconnect time.
The design varies 3 factors: tunnel mtu (bytes), ranging from 1200 to 1500, fragment size (bytes), ranging from 0 to 1400, and keepalive interval (s), ranging from 10 to 120. The goal is to optimize 2 responses: throughput mbps (Mbps) (maximize) and reconnect time s (s) (minimize). Fixed conditions held constant across all runs include protocol = wireguard, encryption = chacha20.
A Box-Behnken design was chosen because it efficiently fits quadratic models with 3 continuous factors while avoiding extreme corner combinations — requiring only 15 runs instead of the 8 needed for a full factorial at two levels.
Quadratic response surface models were fitted to capture potential curvature and factor interactions. The RSM contour plots below visualize how pairs of factors jointly affect each response.
Key Findings
For throughput mbps, the most influential factors were keepalive interval (37.6%), tunnel mtu (35.0%), fragment size (27.4%). The best observed value was 927.0 (at tunnel mtu = 1350, fragment size = 1400, keepalive interval = 10).
For reconnect time s, the most influential factors were tunnel mtu (60.4%), fragment size (32.5%), keepalive interval (7.1%). The best observed value was 2.7 (at tunnel mtu = 1350, fragment size = 1400, keepalive interval = 120).
Recommended Next Steps
- Run confirmation experiments at the predicted optimal settings to validate the model.
- Consider whether any fixed factors should be varied in a future study.
Experimental Setup
Factors
| Factor | Low | High | Unit |
|---|
tunnel_mtu | 1200 | 1500 | bytes |
fragment_size | 0 | 1400 | bytes |
keepalive_interval | 10 | 120 | s |
Fixed: protocol = wireguard, encryption = chacha20
Responses
| Response | Direction | Unit |
|---|
throughput_mbps | ↑ maximize | Mbps |
reconnect_time_s | ↓ minimize | s |
Configuration
{
"metadata": {
"name": "VPN Tunnel MTU",
"description": "Box-Behnken design for MTU, fragmentation, and keepalive to optimize throughput and reconnect time"
},
"factors": [
{
"name": "tunnel_mtu",
"levels": [
"1200",
"1500"
],
"type": "continuous",
"unit": "bytes"
},
{
"name": "fragment_size",
"levels": [
"0",
"1400"
],
"type": "continuous",
"unit": "bytes"
},
{
"name": "keepalive_interval",
"levels": [
"10",
"120"
],
"type": "continuous",
"unit": "s"
}
],
"fixed_factors": {
"protocol": "wireguard",
"encryption": "chacha20"
},
"responses": [
{
"name": "throughput_mbps",
"optimize": "maximize",
"unit": "Mbps"
},
{
"name": "reconnect_time_s",
"optimize": "minimize",
"unit": "s"
}
],
"settings": {
"operation": "box_behnken",
"test_script": "use_cases/52_vpn_tunnel_mtu/sim.sh"
}
}
Experimental Matrix
The Box-Behnken Design produces 15 runs. Each row is one experiment with specific factor settings.
| Run | tunnel_mtu | fragment_size | keepalive_interval |
|---|
| 1 | 1350 | 0 | 10 |
| 2 | 1350 | 700 | 65 |
| 3 | 1500 | 700 | 120 |
| 4 | 1500 | 700 | 10 |
| 5 | 1350 | 700 | 65 |
| 6 | 1350 | 700 | 65 |
| 7 | 1200 | 700 | 120 |
| 8 | 1500 | 0 | 65 |
| 9 | 1350 | 0 | 120 |
| 10 | 1500 | 1400 | 65 |
| 11 | 1200 | 700 | 10 |
| 12 | 1350 | 1400 | 120 |
| 13 | 1200 | 0 | 65 |
| 14 | 1200 | 1400 | 65 |
| 15 | 1350 | 1400 | 10 |
Step-by-Step Workflow
1
Preview the design
$ doe info --config use_cases/52_vpn_tunnel_mtu/config.json
2
Generate the runner script
$ doe generate --config use_cases/52_vpn_tunnel_mtu/config.json \
--output use_cases/52_vpn_tunnel_mtu/results/run.sh --seed 42
3
Execute the experiments
$ bash use_cases/52_vpn_tunnel_mtu/results/run.sh
4
Analyze results
$ doe analyze --config use_cases/52_vpn_tunnel_mtu/config.json
5
Get optimization recommendations
$ doe optimize --config use_cases/52_vpn_tunnel_mtu/config.json
6
Multi-objective optimization
With 2 competing responses, use --multi to find the best compromise via Derringer–Suich desirability.
$ doe optimize --config use_cases/52_vpn_tunnel_mtu/config.json --multi
7
Generate the HTML report
$ doe report --config use_cases/52_vpn_tunnel_mtu/config.json \
--output use_cases/52_vpn_tunnel_mtu/results/report.html
Features Exercised
| Feature | Value |
|---|
| Design type | box_behnken |
| Factor types | continuous (all 3) |
| Arg style | double-dash |
| Responses | 2 (throughput_mbps ↑, reconnect_time_s ↓) |
| Total runs | 15 |
Analysis Results
Generated from actual experiment runs using the DOE Helper Tool.
Response: throughput_mbps
Top factors: keepalive_interval (37.6%), tunnel_mtu (35.0%), fragment_size (27.4%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|
| Source | DF | SS | MS | F | p-value |
| tunnel_mtu | 2 | 7737.2429 | 3868.6214 | 0.335 | 0.7251 |
| fragment_size | 2 | 6451.8500 | 3225.9250 | 0.279 | 0.7636 |
| keepalive_interval | 2 | 11112.9929 | 5556.4964 | 0.481 | 0.6351 |
| Lack | of | Fit | 6 | 23554.8476 | 3925.8079 |
| Pure | Error | 2 | 23120.6667 | | |
| Error | 8 | 46675.5143 | 11560.3333 | | |
| Total | 14 | 71977.6000 | 5141.2571 | | |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response: reconnect_time_s
Top factors: tunnel_mtu (60.4%), fragment_size (32.5%), keepalive_interval (7.1%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|
| Source | DF | SS | MS | F | p-value |
| tunnel_mtu | 2 | 136.0998 | 68.0499 | 20.663 | 0.0007 |
| fragment_size | 2 | 55.2598 | 27.6299 | 8.390 | 0.0109 |
| keepalive_interval | 2 | 2.4530 | 1.2265 | 0.372 | 0.7004 |
| Lack | of | Fit | 6 | 53.6580 | 8.9430 |
| Pure | Error | 2 | 6.5867 | | |
| Error | 8 | 60.2446 | 3.2933 | | |
| Total | 14 | 254.0573 | 18.1470 | | |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response Surface Plots
3D surfaces fitted with quadratic RSM. Red dots are observed data points.
reconnect time s fragment size vs keepalive interval
reconnect time s tunnel mtu vs fragment size
reconnect time s tunnel mtu vs keepalive interval
throughput mbps fragment size vs keepalive interval
throughput mbps tunnel mtu vs fragment size
throughput mbps tunnel mtu vs keepalive interval
Multi-Objective Optimization
When responses compete, Derringer–Suich desirability finds the best compromise.
Each response is scaled to a 0–1 desirability, then combined via a weighted geometric mean.
Overall Desirability
D = 0.9284
Per-Response Desirability
| Response | Weight | Desirability | Predicted | Dir |
|---|
throughput_mbps |
1.5 |
|
916.00 0.9114 916.00 Mbps |
↑ |
reconnect_time_s |
1.0 |
|
2.70 0.9545 2.70 s |
↓ |
Recommended Settings
| Factor | Value |
|---|
tunnel_mtu | 1350 bytes |
fragment_size | 0 bytes |
keepalive_interval | 120 s |
Source: from observed run #1
Trade-off Summary
Sacrifice = how much worse than single-objective best.
| Response | Predicted | Best Observed | Sacrifice |
|---|
reconnect_time_s | 2.70 | 2.70 | +0.00 |
Top 3 Runs by Desirability
| Run | D | Factor Settings |
|---|
| #4 | 0.8697 | tunnel_mtu=1500, fragment_size=1400, keepalive_interval=65 |
| #8 | 0.7675 | tunnel_mtu=1350, fragment_size=0, keepalive_interval=10 |
Model Quality
| Response | R² | Type |
|---|
reconnect_time_s | 0.0443 | linear |
Full Multi-Objective Output
============================================================
MULTI-OBJECTIVE OPTIMIZATION
Method: Derringer-Suich Desirability Function
============================================================
Overall desirability: D = 0.9284
Response Weight Desirability Predicted Direction
---------------------------------------------------------------------
throughput_mbps 1.5 0.9114 916.00 Mbps ↑
reconnect_time_s 1.0 0.9545 2.70 s ↓
Recommended settings:
tunnel_mtu = 1350 bytes
fragment_size = 0 bytes
keepalive_interval = 120 s
(from observed run #1)
Trade-off summary:
throughput_mbps: 916.00 (best observed: 927.00, sacrifice: +11.00)
reconnect_time_s: 2.70 (best observed: 2.70, sacrifice: +0.00)
Model quality:
throughput_mbps: R² = 0.5886 (quadratic)
reconnect_time_s: R² = 0.0443 (linear)
Top 3 observed runs by overall desirability:
1. Run #1 (D=0.9284): tunnel_mtu=1350, fragment_size=0, keepalive_interval=120
2. Run #4 (D=0.8697): tunnel_mtu=1500, fragment_size=1400, keepalive_interval=65
3. Run #8 (D=0.7675): tunnel_mtu=1350, fragment_size=0, keepalive_interval=10
Full Analysis Output
=== Main Effects: throughput_mbps ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
keepalive_interval 65.6429 18.5135 37.6%
tunnel_mtu 61.0000 18.5135 35.0%
fragment_size 47.7500 18.5135 27.4%
=== ANOVA Table: throughput_mbps ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
tunnel_mtu 2 7737.2429 3868.6214 0.335 0.7251
fragment_size 2 6451.8500 3225.9250 0.279 0.7636
keepalive_interval 2 11112.9929 5556.4964 0.481 0.6351
Lack of Fit 6 23554.8476 3925.8079 0.340 0.8715
Pure Error 2 23120.6667 11560.3333
Error 8 46675.5143 11560.3333
Total 14 71977.6000 5141.2571
=== Summary Statistics: throughput_mbps ===
tunnel_mtu:
Level N Mean Std Min Max
------------------------------------------------------------
1200 4 812.2500 69.9351 752.0000 913.0000
1350 7 833.8571 65.0677 695.0000 890.0000
1500 4 873.2500 89.7492 739.0000 927.0000
fragment_size:
Level N Mean Std Min Max
------------------------------------------------------------
0 4 849.5000 52.0545 786.0000 911.0000
1400 4 804.2500 50.0025 739.0000 858.0000
700 7 852.0000 91.1921 695.0000 927.0000
keepalive_interval:
Level N Mean Std Min Max
------------------------------------------------------------
10 4 878.5000 44.8293 822.0000 916.0000
120 4 843.7500 72.0758 752.0000 927.0000
65 7 812.8571 80.8815 695.0000 911.0000
=== Main Effects: reconnect_time_s ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
tunnel_mtu 8.2250 1.0999 60.4%
fragment_size 4.4250 1.0999 32.5%
keepalive_interval 0.9714 1.0999 7.1%
=== ANOVA Table: reconnect_time_s ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
tunnel_mtu 2 136.0998 68.0499 20.663 0.0007
fragment_size 2 55.2598 27.6299 8.390 0.0109
keepalive_interval 2 2.4530 1.2265 0.372 0.7004
Lack of Fit 6 53.6580 8.9430 2.715 0.2934
Pure Error 2 6.5867 3.2933
Error 8 60.2446 3.2933
Total 14 254.0573 18.1470
=== Summary Statistics: reconnect_time_s ===
tunnel_mtu:
Level N Mean Std Min Max
------------------------------------------------------------
1200 4 12.9500 5.0030 5.5000 16.0000
1350 7 9.3000 2.0744 7.4000 13.1000
1500 4 4.7250 2.3838 2.7000 7.7000
fragment_size:
Level N Mean Std Min Max
------------------------------------------------------------
0 4 10.0500 3.9921 7.7000 16.0000
1400 4 5.8750 2.2096 2.9000 7.7000
700 7 10.3000 4.7669 2.7000 15.7000
keepalive_interval:
Level N Mean Std Min Max
------------------------------------------------------------
10 4 8.4000 5.3895 2.7000 15.7000
120 4 9.1500 3.8562 5.6000 14.6000
65 7 9.3714 4.4694 2.9000 16.0000
Optimization Recommendations
=== Optimization: throughput_mbps ===
Direction: maximize
Best observed run: #4
tunnel_mtu = 1350
fragment_size = 1400
keepalive_interval = 10
Value: 927.0
RSM Model (linear, R² = 0.2069, Adj R² = -0.0095):
Coefficients:
intercept +838.6000
tunnel_mtu -17.3750
fragment_size +35.8750
keepalive_interval +16.5000
RSM Model (quadratic, R² = 0.9449, Adj R² = 0.8458):
Coefficients:
intercept +728.6667
tunnel_mtu -17.3750
fragment_size +35.8750
keepalive_interval +16.5000
tunnel_mtu*fragment_size +13.2500
tunnel_mtu*keepalive_interval +17.5000
fragment_size*keepalive_interval -22.0000
tunnel_mtu^2 +61.7917
fragment_size^2 +50.7917
keepalive_interval^2 +93.5417
Curvature analysis:
keepalive_interval coef=+93.5417 convex (has a minimum)
tunnel_mtu coef=+61.7917 convex (has a minimum)
fragment_size coef=+50.7917 convex (has a minimum)
Notable interactions:
fragment_size*keepalive_interval coef=-22.0000 (antagonistic)
tunnel_mtu*keepalive_interval coef=+17.5000 (synergistic)
tunnel_mtu*fragment_size coef=+13.2500 (synergistic)
Predicted optimum (from quadratic model, at observed points):
tunnel_mtu = 1350
fragment_size = 1400
keepalive_interval = 10
Predicted value: 914.3750
Surface optimum (via L-BFGS-B, quadratic model):
tunnel_mtu = 1200
fragment_size = 1400
keepalive_interval = 10
Predicted value: 997.7917
Model quality: Excellent fit — surface predictions are reliable.
Factor importance:
1. keepalive_interval (effect: 102.0, contribution: 41.4%)
2. fragment_size (effect: 75.6, contribution: 30.7%)
3. tunnel_mtu (effect: 68.9, contribution: 27.9%)
=== Optimization: reconnect_time_s ===
Direction: minimize
Best observed run: #1
tunnel_mtu = 1350
fragment_size = 1400
keepalive_interval = 120
Value: 2.7
RSM Model (linear, R² = 0.1628, Adj R² = -0.0655):
Coefficients:
intercept +9.0533
tunnel_mtu -0.3625
fragment_size -1.1125
keepalive_interval -1.9500
RSM Model (quadratic, R² = 0.4412, Adj R² = -0.5648):
Coefficients:
intercept +9.0000
tunnel_mtu -0.3625
fragment_size -1.1125
keepalive_interval -1.9500
tunnel_mtu*fragment_size +2.1500
tunnel_mtu*keepalive_interval +2.8750
fragment_size*keepalive_interval +1.3250
tunnel_mtu^2 +1.0750
fragment_size^2 -1.3250
keepalive_interval^2 +0.3500
Curvature analysis:
fragment_size coef=-1.3250 concave (has a maximum)
tunnel_mtu coef=+1.0750 convex (has a minimum)
keepalive_interval coef=+0.3500 convex (has a minimum)
Notable interactions:
tunnel_mtu*keepalive_interval coef=+2.8750 (synergistic)
tunnel_mtu*fragment_size coef=+2.1500 (synergistic)
fragment_size*keepalive_interval coef=+1.3250 (synergistic)
Predicted optimum (from linear model, at observed points):
tunnel_mtu = 1350
fragment_size = 0
keepalive_interval = 10
Predicted value: 12.1158
Surface optimum (via L-BFGS-B, linear model):
tunnel_mtu = 1500
fragment_size = 1400
keepalive_interval = 120
Predicted value: 5.6283
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. keepalive_interval (effect: 3.9, contribution: 49.1%)
2. fragment_size (effect: 2.5, contribution: 32.0%)
3. tunnel_mtu (effect: 1.5, contribution: 19.0%)