Summary
This experiment investigates encryption pipeline optimization. Full factorial of cipher suite, key size, compression, and hardware acceleration for throughput and CPU overhead.
The design varies 4 factors: cipher suite, ranging from aes128 to aes256, key size (bits), ranging from 128 to 256, compression before encrypt, ranging from off to on, and hardware acceleration, ranging from off to on. The goal is to optimize 2 responses: throughput mbps (Mbps) (maximize) and cpu overhead pct (%) (minimize). Fixed conditions held constant across all runs include protocol = tls1.3, mode = gcm.
A full factorial design was used to explore all 16 possible combinations of the 4 factors at two levels. This guarantees that every main effect and interaction can be estimated independently, at the cost of a larger experiment (16 runs).
Quadratic response surface models were fitted to capture potential curvature and factor interactions. The RSM contour plots below visualize how pairs of factors jointly affect each response.
Key Findings
For throughput mbps, the most influential factors were compression before encrypt (53.0%), cipher suite (25.5%), key size (13.1%). The best observed value was 1329.0 (at cipher suite = aes256, key size = 256, compression before encrypt = on).
For cpu overhead pct, the most influential factors were compression before encrypt (51.4%), cipher suite (32.8%), key size (13.5%). The best observed value was 2.7 (at cipher suite = aes256, key size = 256, compression before encrypt = on).
Recommended Next Steps
- Consider whether any fixed factors should be varied in a future study.
Experimental Setup
Factors
| Factor | Low | High | Unit |
|---|---|---|---|
cipher_suite | aes128 | aes256 | |
key_size | 128 | 256 | bits |
compression_before_encrypt | off | on | |
hardware_acceleration | off | on |
Fixed: protocol = tls1.3, mode = gcm
Responses
| Response | Direction | Unit |
|---|---|---|
throughput_mbps | ↑ maximize | Mbps |
cpu_overhead_pct | ↓ minimize | % |
Configuration
use_cases/58_encryption_pipeline/config.json
{
"metadata": {
"name": "Encryption Pipeline Optimization",
"description": "Full factorial of cipher suite, key size, compression, and hardware acceleration for throughput and CPU overhead"
},
"factors": [
{
"name": "cipher_suite",
"levels": [
"aes128",
"aes256"
],
"type": "categorical",
"unit": ""
},
{
"name": "key_size",
"levels": [
"128",
"256"
],
"type": "continuous",
"unit": "bits"
},
{
"name": "compression_before_encrypt",
"levels": [
"off",
"on"
],
"type": "categorical",
"unit": ""
},
{
"name": "hardware_acceleration",
"levels": [
"off",
"on"
],
"type": "categorical",
"unit": ""
}
],
"fixed_factors": {
"protocol": "tls1.3",
"mode": "gcm"
},
"responses": [
{
"name": "throughput_mbps",
"optimize": "maximize",
"unit": "Mbps"
},
{
"name": "cpu_overhead_pct",
"optimize": "minimize",
"unit": "%"
}
],
"settings": {
"operation": "full_factorial",
"test_script": "use_cases/58_encryption_pipeline/sim.sh"
}
}
Experimental Matrix
The Full Factorial Design produces 16 runs. Each row is one experiment with specific factor settings.
| Run | cipher_suite | key_size | compression_before_encrypt | hardware_acceleration |
|---|---|---|---|---|
| 1 | aes128 | 256 | on | on |
| 2 | aes256 | 128 | off | on |
| 3 | aes128 | 256 | off | on |
| 4 | aes128 | 256 | on | off |
| 5 | aes256 | 256 | on | off |
| 6 | aes256 | 128 | on | off |
| 7 | aes256 | 256 | off | off |
| 8 | aes256 | 128 | off | off |
| 9 | aes128 | 128 | off | on |
| 10 | aes128 | 128 | on | off |
| 11 | aes256 | 256 | off | on |
| 12 | aes256 | 256 | on | on |
| 13 | aes128 | 256 | off | off |
| 14 | aes256 | 128 | on | on |
| 15 | aes128 | 128 | off | off |
| 16 | aes128 | 128 | on | on |
Step-by-Step Workflow
1
Preview the design
Terminal
$ doe info --config use_cases/58_encryption_pipeline/config.json
2
Generate the runner script
Terminal
$ doe generate --config use_cases/58_encryption_pipeline/config.json \
--output use_cases/58_encryption_pipeline/results/run.sh --seed 42
3
Execute the experiments
Terminal
$ bash use_cases/58_encryption_pipeline/results/run.sh
4
Analyze results
Terminal
$ doe analyze --config use_cases/58_encryption_pipeline/config.json
5
Get optimization recommendations
Terminal
$ doe optimize --config use_cases/58_encryption_pipeline/config.json
6
Multi-objective optimization
With 2 competing responses, use --multi to find the best compromise via Derringer–Suich desirability.
Terminal
$ doe optimize --config use_cases/58_encryption_pipeline/config.json --multi
7
Generate the HTML report
Terminal
$ doe report --config use_cases/58_encryption_pipeline/config.json \
--output use_cases/58_encryption_pipeline/results/report.html
Features Exercised
| Feature | Value |
|---|---|
| Design type | full_factorial |
| Factor types | continuous (1), categorical (3) |
| Arg style | double-dash |
| Responses | 2 (throughput_mbps ↑, cpu_overhead_pct ↓) |
| Total runs | 16 |
Analysis Results
Generated from actual experiment runs using the DOE Helper Tool.
Response: throughput_mbps
Top factors: compression_before_encrypt (53.0%), cipher_suite (25.5%), key_size (13.1%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| cipher_suite | 1 | 87616.0000 | 87616.0000 | 1.117 | 0.3390 |
| key_size | 1 | 23104.0000 | 23104.0000 | 0.294 | 0.6107 |
| compression_before_encrypt | 1 | 376996.0000 | 376996.0000 | 4.804 | 0.0799 |
| hardware_acceleration | 1 | 9409.0000 | 9409.0000 | 0.120 | 0.7432 |
| cipher_suite*key_size | 1 | 65280.2500 | 65280.2500 | 0.832 | 0.4036 |
| cipher_suite*compression_before_encrypt | 1 | 1806.2500 | 1806.2500 | 0.023 | 0.8853 |
| cipher_suite*hardware_acceleration | 1 | 82082.2500 | 82082.2500 | 1.046 | 0.3533 |
| key_size*compression_before_encrypt | 1 | 166872.2500 | 166872.2500 | 2.127 | 0.2046 |
| key_size*hardware_acceleration | 1 | 16002.2500 | 16002.2500 | 0.204 | 0.6705 |
| compression_before_encrypt*hardware_acceleration | 1 | 74256.2500 | 74256.2500 | 0.946 | 0.3753 |
| Error | 5 | 392355.2500 | 78471.0500 | ||
| Total | 15 | 1295779.7500 | 86385.3167 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response: cpu_overhead_pct
Top factors: compression_before_encrypt (51.4%), cipher_suite (32.8%), key_size (13.5%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| cipher_suite | 1 | 416.1600 | 416.1600 | 0.924 | 0.3806 |
| key_size | 1 | 70.5600 | 70.5600 | 0.157 | 0.7086 |
| compression_before_encrypt | 1 | 1020.8025 | 1020.8025 | 2.266 | 0.1926 |
| hardware_acceleration | 1 | 2.1025 | 2.1025 | 0.005 | 0.9482 |
| cipher_suite*key_size | 1 | 238.7025 | 238.7025 | 0.530 | 0.4993 |
| cipher_suite*compression_before_encrypt | 1 | 0.8100 | 0.8100 | 0.002 | 0.9678 |
| cipher_suite*hardware_acceleration | 1 | 320.4100 | 320.4100 | 0.711 | 0.4375 |
| key_size*compression_before_encrypt | 1 | 620.0100 | 620.0100 | 1.376 | 0.2936 |
| key_size*hardware_acceleration | 1 | 21.1600 | 21.1600 | 0.047 | 0.8370 |
| compression_before_encrypt*hardware_acceleration | 1 | 382.2025 | 382.2025 | 0.848 | 0.3993 |
| Error | 5 | 2252.5375 | 450.5075 | ||
| Total | 15 | 5345.4575 | 356.3638 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Multi-Objective Optimization
When responses compete, Derringer–Suich desirability finds the best compromise. Each response is scaled to a 0–1 desirability, then combined via a weighted geometric mean.
Overall Desirability
D = 0.9545
Per-Response Desirability
| Response | Weight | Desirability | Predicted | Dir |
|---|---|---|---|---|
throughput_mbps |
1.5 |
0.9545
|
1329.00 0.9545 1329.00 Mbps | ↑ |
cpu_overhead_pct |
1.0 |
0.9545
|
2.70 0.9545 2.70 % | ↓ |
Recommended Settings
| Factor | Value |
|---|---|
cipher_suite | aes128 |
key_size | 128 bits |
compression_before_encrypt | off |
hardware_acceleration | on |
Source: from observed run #16
Trade-off Summary
Sacrifice = how much worse than single-objective best.
| Response | Predicted | Best Observed | Sacrifice |
|---|---|---|---|
cpu_overhead_pct | 2.70 | 2.70 | +0.00 |
Top 3 Runs by Desirability
| Run | D | Factor Settings |
|---|---|---|
| #9 | 0.8618 | cipher_suite=aes256, key_size=256, compression_before_encrypt=off, hardware_acceleration=off |
| #1 | 0.7832 | cipher_suite=aes256, key_size=128, compression_before_encrypt=on, hardware_acceleration=on |
Model Quality
| Response | R² | Type |
|---|---|---|
cpu_overhead_pct | 0.2704 | linear |
Full Multi-Objective Output
doe optimize --multi
============================================================
MULTI-OBJECTIVE OPTIMIZATION
Method: Derringer-Suich Desirability Function
============================================================
Overall desirability: D = 0.9545
Response Weight Desirability Predicted Direction
---------------------------------------------------------------------
throughput_mbps 1.5 0.9545 1329.00 Mbps ↑
cpu_overhead_pct 1.0 0.9545 2.70 % ↓
Recommended settings:
cipher_suite = aes128
key_size = 128 bits
compression_before_encrypt = off
hardware_acceleration = on
(from observed run #16)
Trade-off summary:
throughput_mbps: 1329.00 (best observed: 1329.00, sacrifice: +0.00)
cpu_overhead_pct: 2.70 (best observed: 2.70, sacrifice: +0.00)
Model quality:
throughput_mbps: R² = 0.2208 (linear)
cpu_overhead_pct: R² = 0.2704 (linear)
Top 3 observed runs by overall desirability:
1. Run #16 (D=0.9545): cipher_suite=aes128, key_size=128, compression_before_encrypt=off, hardware_acceleration=on
2. Run #9 (D=0.8618): cipher_suite=aes256, key_size=256, compression_before_encrypt=off, hardware_acceleration=off
3. Run #1 (D=0.7832): cipher_suite=aes256, key_size=128, compression_before_encrypt=on, hardware_acceleration=on
Full Analysis Output
doe analyze
=== Main Effects: throughput_mbps ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
compression_before_encrypt -307.0000 73.4784 53.0%
cipher_suite -148.0000 73.4784 25.5%
key_size -76.0000 73.4784 13.1%
hardware_acceleration 48.5000 73.4784 8.4%
=== ANOVA Table: throughput_mbps ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
cipher_suite 1 87616.0000 87616.0000 1.117 0.3390
key_size 1 23104.0000 23104.0000 0.294 0.6107
compression_before_encrypt 1 376996.0000 376996.0000 4.804 0.0799
hardware_acceleration 1 9409.0000 9409.0000 0.120 0.7432
cipher_suite*key_size 1 65280.2500 65280.2500 0.832 0.4036
cipher_suite*compression_before_encrypt 1 1806.2500 1806.2500 0.023 0.8853
cipher_suite*hardware_acceleration 1 82082.2500 82082.2500 1.046 0.3533
key_size*compression_before_encrypt 1 166872.2500 166872.2500 2.127 0.2046
key_size*hardware_acceleration 1 16002.2500 16002.2500 0.204 0.6705
compression_before_encrypt*hardware_acceleration 1 74256.2500 74256.2500 0.946 0.3753
Error 5 392355.2500 78471.0500
Total 15 1295779.7500 86385.3167
=== Interaction Effects: throughput_mbps ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
key_size compression_before_encrypt 204.2500 29.3%
cipher_suite hardware_acceleration -143.2500 20.6%
compression_before_encrypt hardware_acceleration -136.2500 19.6%
cipher_suite key_size -127.7500 18.4%
key_size hardware_acceleration 63.2500 9.1%
cipher_suite compression_before_encrypt 21.2500 3.1%
=== Summary Statistics: throughput_mbps ===
cipher_suite:
Level N Mean Std Min Max
------------------------------------------------------------
aes128 8 884.1250 327.3493 387.0000 1329.0000
aes256 8 736.1250 255.8071 484.0000 1219.0000
key_size:
Level N Mean Std Min Max
------------------------------------------------------------
128 8 848.1250 307.0298 387.0000 1219.0000
256 8 772.1250 295.8776 484.0000 1329.0000
compression_before_encrypt:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 963.6250 308.9618 484.0000 1329.0000
on 8 656.6250 189.2020 387.0000 1012.0000
hardware_acceleration:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 785.8750 176.5522 564.0000 1108.0000
on 8 834.3750 390.6361 387.0000 1329.0000
=== Main Effects: cpu_overhead_pct ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
compression_before_encrypt 15.9750 4.7194 51.4%
cipher_suite 10.2000 4.7194 32.8%
key_size 4.2000 4.7194 13.5%
hardware_acceleration -0.7250 4.7194 2.3%
=== ANOVA Table: cpu_overhead_pct ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
cipher_suite 1 416.1600 416.1600 0.924 0.3806
key_size 1 70.5600 70.5600 0.157 0.7086
compression_before_encrypt 1 1020.8025 1020.8025 2.266 0.1926
hardware_acceleration 1 2.1025 2.1025 0.005 0.9482
cipher_suite*key_size 1 238.7025 238.7025 0.530 0.4993
cipher_suite*compression_before_encrypt 1 0.8100 0.8100 0.002 0.9678
cipher_suite*hardware_acceleration 1 320.4100 320.4100 0.711 0.4375
key_size*compression_before_encrypt 1 620.0100 620.0100 1.376 0.2936
key_size*hardware_acceleration 1 21.1600 21.1600 0.047 0.8370
compression_before_encrypt*hardware_acceleration 1 382.2025 382.2025 0.848 0.3993
Error 5 2252.5375 450.5075
Total 15 5345.4575 356.3638
=== Interaction Effects: cpu_overhead_pct ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
key_size compression_before_encrypt -12.4500 29.9%
compression_before_encrypt hardware_acceleration 9.7750 23.5%
cipher_suite hardware_acceleration 8.9500 21.5%
cipher_suite key_size 7.7250 18.5%
key_size hardware_acceleration -2.3000 5.5%
cipher_suite compression_before_encrypt 0.4500 1.1%
=== Summary Statistics: cpu_overhead_pct ===
cipher_suite:
Level N Mean Std Min Max
------------------------------------------------------------
aes128 8 29.5125 19.8697 2.7000 59.7000
aes256 8 39.7125 17.5892 7.3000 60.5000
key_size:
Level N Mean Std Min Max
------------------------------------------------------------
128 8 32.5125 18.3940 7.3000 59.7000
256 8 36.7125 20.3769 2.7000 60.5000
compression_before_encrypt:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 26.6250 20.6664 2.7000 58.4000
on 8 42.6000 13.8097 19.2000 60.5000
hardware_acceleration:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 34.9750 11.7139 13.9000 53.3000
on 8 34.2500 25.0224 2.7000 60.5000
Optimization Recommendations
doe optimize
=== Optimization: throughput_mbps ===
Direction: maximize
Best observed run: #16
cipher_suite = aes256
key_size = 256
compression_before_encrypt = on
hardware_acceleration = on
Value: 1329.0
RSM Model (linear, R² = 0.1473, Adj R² = -0.1628):
Coefficients:
intercept +810.1250
cipher_suite +30.7500
key_size +54.6250
compression_before_encrypt +79.8750
hardware_acceleration +40.2500
RSM Model (quadratic, R² = 0.5898, Adj R² = -5.1536):
Coefficients:
intercept +162.0250
cipher_suite +30.7500
key_size +54.6250
compression_before_encrypt +79.8750
hardware_acceleration +40.2500
cipher_suite*key_size -92.7500
cipher_suite*compression_before_encrypt +39.7500
cipher_suite*hardware_acceleration -75.3750
key_size*compression_before_encrypt +131.6250
key_size*hardware_acceleration +50.0000
compression_before_encrypt*hardware_acceleration -12.0000
cipher_suite^2 +162.0250
key_size^2 +162.0250
compression_before_encrypt^2 +162.0250
hardware_acceleration^2 +162.0250
Curvature analysis:
cipher_suite coef=+162.0250 convex (has a minimum)
key_size coef=+162.0250 convex (has a minimum)
compression_before_encrypt coef=+162.0250 convex (has a minimum)
hardware_acceleration coef=+162.0250 convex (has a minimum)
Notable interactions:
key_size*compression_before_encrypt coef=+131.6250 (synergistic)
cipher_suite*key_size coef=-92.7500 (antagonistic)
cipher_suite*hardware_acceleration coef=-75.3750 (antagonistic)
key_size*hardware_acceleration coef=+50.0000 (synergistic)
cipher_suite*compression_before_encrypt coef=+39.7500 (synergistic)
compression_before_encrypt*hardware_acceleration coef=-12.0000 (antagonistic)
Predicted optimum (from linear model, at observed points):
cipher_suite = aes256
key_size = 256
compression_before_encrypt = on
hardware_acceleration = on
Predicted value: 1015.6250
Surface optimum (via L-BFGS-B, linear model):
cipher_suite = aes256
key_size = 256
compression_before_encrypt = on
hardware_acceleration = on
Predicted value: 1015.6250
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. compression_before_encrypt (effect: 159.8, contribution: 38.9%)
2. key_size (effect: 109.2, contribution: 26.6%)
3. hardware_acceleration (effect: 80.5, contribution: 19.6%)
4. cipher_suite (effect: 61.5, contribution: 15.0%)
=== Optimization: cpu_overhead_pct ===
Direction: minimize
Best observed run: #16
cipher_suite = aes256
key_size = 256
compression_before_encrypt = on
hardware_acceleration = on
Value: 2.7
RSM Model (linear, R² = 0.1349, Adj R² = -0.1796):
Coefficients:
intercept +34.6125
cipher_suite -2.3625
key_size -3.3000
compression_before_encrypt -5.1000
hardware_acceleration -1.6125
RSM Model (quadratic, R² = 0.6437, Adj R² = -4.3441):
Coefficients:
intercept +6.9225
cipher_suite -2.3625
key_size -3.3000
compression_before_encrypt -5.1000
hardware_acceleration -1.6125
cipher_suite*key_size +5.0500
cipher_suite*compression_before_encrypt -1.6750
cipher_suite*hardware_acceleration +5.0375
key_size*compression_before_encrypt -10.6375
key_size*hardware_acceleration -1.7000
compression_before_encrypt*hardware_acceleration +0.5000
cipher_suite^2 +6.9225
key_size^2 +6.9225
compression_before_encrypt^2 +6.9225
hardware_acceleration^2 +6.9225
Curvature analysis:
key_size coef=+6.9225 convex (has a minimum)
compression_before_encrypt coef=+6.9225 convex (has a minimum)
hardware_acceleration coef=+6.9225 convex (has a minimum)
cipher_suite coef=+6.9225 convex (has a minimum)
Notable interactions:
key_size*compression_before_encrypt coef=-10.6375 (antagonistic)
cipher_suite*key_size coef=+5.0500 (synergistic)
cipher_suite*hardware_acceleration coef=+5.0375 (synergistic)
key_size*hardware_acceleration coef=-1.7000 (antagonistic)
cipher_suite*compression_before_encrypt coef=-1.6750 (antagonistic)
compression_before_encrypt*hardware_acceleration coef=+0.5000 (synergistic)
Predicted optimum (from linear model, at observed points):
cipher_suite = aes128
key_size = 128
compression_before_encrypt = off
hardware_acceleration = off
Predicted value: 46.9875
Surface optimum (via L-BFGS-B, linear model):
cipher_suite = aes256
key_size = 256
compression_before_encrypt = on
hardware_acceleration = on
Predicted value: 22.2375
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. compression_before_encrypt (effect: -10.2, contribution: 41.2%)
2. key_size (effect: -6.6, contribution: 26.7%)
3. cipher_suite (effect: -4.7, contribution: 19.1%)
4. hardware_acceleration (effect: -3.2, contribution: 13.0%)