Summary
This experiment investigates cdn cache hit optimization. Full factorial with categorical and continuous factors for cache hit ratio and origin bandwidth.
The design varies 4 factors: ttl hours (h), ranging from 1 to 24, cache policy, ranging from lru to lfu, cache size gb (GB), ranging from 50 to 200, and prefetch, ranging from off to on. The goal is to optimize 2 responses: hit ratio (%) (maximize) and origin bandwidth (Gbps) (minimize). Fixed conditions held constant across all runs include origin region = us-east-1, compression = brotli.
A full factorial design was used to explore all 16 possible combinations of the 4 factors at two levels. This guarantees that every main effect and interaction can be estimated independently, at the cost of a larger experiment (16 runs).
Quadratic response surface models were fitted to capture potential curvature and factor interactions. The RSM contour plots below visualize how pairs of factors jointly affect each response.
Key Findings
For hit ratio, the most influential factors were cache policy (53.9%), ttl hours (30.3%), cache size gb (8.4%). The best observed value was 99.0 (at ttl hours = 1, cache policy = lfu, cache size gb = 200).
For origin bandwidth, the most influential factors were cache policy (50.2%), cache size gb (18.4%), ttl hours (18.1%). The best observed value was 0.98 (at ttl hours = 1, cache policy = lfu, cache size gb = 200).
Recommended Next Steps
- Consider whether any fixed factors should be varied in a future study.
Experimental Setup
Factors
| Factor | Low | High | Unit |
|---|---|---|---|
ttl_hours | 1 | 24 | h |
cache_policy | lru | lfu | |
cache_size_gb | 50 | 200 | GB |
prefetch | off | on |
Fixed: origin_region = us-east-1, compression = brotli
Responses
| Response | Direction | Unit |
|---|---|---|
hit_ratio | ↑ maximize | % |
origin_bandwidth | ↓ minimize | Gbps |
Configuration
use_cases/29_cdn_cache_optimization/config.json
{
"metadata": {
"name": "CDN Cache Hit Optimization",
"description": "Full factorial with categorical and continuous factors for cache hit ratio and origin bandwidth"
},
"factors": [
{
"name": "ttl_hours",
"levels": [
"1",
"24"
],
"type": "continuous",
"unit": "h"
},
{
"name": "cache_policy",
"levels": [
"lru",
"lfu"
],
"type": "categorical",
"unit": ""
},
{
"name": "cache_size_gb",
"levels": [
"50",
"200"
],
"type": "continuous",
"unit": "GB"
},
{
"name": "prefetch",
"levels": [
"off",
"on"
],
"type": "categorical",
"unit": ""
}
],
"fixed_factors": {
"origin_region": "us-east-1",
"compression": "brotli"
},
"responses": [
{
"name": "hit_ratio",
"optimize": "maximize",
"unit": "%"
},
{
"name": "origin_bandwidth",
"optimize": "minimize",
"unit": "Gbps"
}
],
"settings": {
"operation": "full_factorial",
"test_script": "use_cases/29_cdn_cache_optimization/sim.sh"
}
}
Experimental Matrix
The Full Factorial Design produces 16 runs. Each row is one experiment with specific factor settings.
| Run | ttl_hours | cache_policy | cache_size_gb | prefetch |
|---|---|---|---|---|
| 1 | 1 | lfu | 200 | on |
| 2 | 24 | lru | 50 | on |
| 3 | 1 | lfu | 50 | on |
| 4 | 1 | lfu | 200 | off |
| 5 | 24 | lfu | 200 | off |
| 6 | 24 | lru | 200 | off |
| 7 | 24 | lfu | 50 | off |
| 8 | 24 | lru | 50 | off |
| 9 | 1 | lru | 50 | on |
| 10 | 1 | lru | 200 | off |
| 11 | 24 | lfu | 50 | on |
| 12 | 24 | lfu | 200 | on |
| 13 | 1 | lfu | 50 | off |
| 14 | 24 | lru | 200 | on |
| 15 | 1 | lru | 50 | off |
| 16 | 1 | lru | 200 | on |
Step-by-Step Workflow
1
Preview the design
Terminal
$ doe info --config use_cases/29_cdn_cache_optimization/config.json
2
Generate the runner script
Terminal
$ doe generate --config use_cases/29_cdn_cache_optimization/config.json \
--output use_cases/29_cdn_cache_optimization/results/run.sh --seed 42
3
Execute the experiments
Terminal
$ bash use_cases/29_cdn_cache_optimization/results/run.sh
4
Analyze results
Terminal
$ doe analyze --config use_cases/29_cdn_cache_optimization/config.json
5
Get optimization recommendations
Terminal
$ doe optimize --config use_cases/29_cdn_cache_optimization/config.json
6
Multi-objective optimization
With 2 competing responses, use --multi to find the best compromise via Derringer–Suich desirability.
Terminal
$ doe optimize --config use_cases/29_cdn_cache_optimization/config.json --multi
7
Generate the HTML report
Terminal
$ doe report --config use_cases/29_cdn_cache_optimization/config.json \
--output use_cases/29_cdn_cache_optimization/results/report.html
Features Exercised
| Feature | Value |
|---|---|
| Design type | full_factorial |
| Factor types | continuous (2), categorical (2) |
| Arg style | double-dash |
| Responses | 2 (hit_ratio ↑, origin_bandwidth ↓) |
| Total runs | 16 |
Analysis Results
Generated from actual experiment runs using the DOE Helper Tool.
Response: hit_ratio
Top factors: cache_policy (53.9%), ttl_hours (30.3%), cache_size_gb (8.4%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| ttl_hours | 1 | 169.6506 | 169.6506 | 1.143 | 0.3338 |
| cache_policy | 1 | 537.0806 | 537.0806 | 3.620 | 0.1155 |
| cache_size_gb | 1 | 13.1406 | 13.1406 | 0.089 | 0.7780 |
| prefetch | 1 | 10.0806 | 10.0806 | 0.068 | 0.8048 |
| ttl_hours*cache_policy | 1 | 411.0756 | 411.0756 | 2.771 | 0.1569 |
| ttl_hours*cache_size_gb | 1 | 1022.4006 | 1022.4006 | 6.891 | 0.0468 |
| ttl_hours*prefetch | 1 | 62.8056 | 62.8056 | 0.423 | 0.5440 |
| cache_policy*cache_size_gb | 1 | 2.1756 | 2.1756 | 0.015 | 0.9083 |
| cache_policy*prefetch | 1 | 74.3906 | 74.3906 | 0.501 | 0.5105 |
| cache_size_gb*prefetch | 1 | 153.1406 | 153.1406 | 1.032 | 0.3563 |
| Error | 5 | 741.8731 | 148.3746 | ||
| Total | 15 | 3197.8144 | 213.1876 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response: origin_bandwidth
Top factors: cache_policy (50.2%), cache_size_gb (18.4%), ttl_hours (18.1%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| ttl_hours | 1 | 28.4089 | 28.4089 | 0.785 | 0.4161 |
| cache_policy | 1 | 218.7441 | 218.7441 | 6.048 | 0.0573 |
| cache_size_gb | 1 | 29.4306 | 29.4306 | 0.814 | 0.4084 |
| prefetch | 1 | 15.5236 | 15.5236 | 0.429 | 0.5413 |
| ttl_hours*cache_policy | 1 | 130.0740 | 130.0740 | 3.596 | 0.1164 |
| ttl_hours*cache_size_gb | 1 | 316.1284 | 316.1284 | 8.740 | 0.0317 |
| ttl_hours*prefetch | 1 | 10.5950 | 10.5950 | 0.293 | 0.6116 |
| cache_policy*cache_size_gb | 1 | 2.3409 | 2.3409 | 0.065 | 0.8093 |
| cache_policy*prefetch | 1 | 9.5790 | 9.5790 | 0.265 | 0.6288 |
| cache_size_gb*prefetch | 1 | 67.0761 | 67.0761 | 1.854 | 0.2314 |
| Error | 5 | 180.8543 | 36.1709 | ||
| Total | 15 | 1008.7550 | 67.2503 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response Surface Plots
3D surfaces fitted with quadratic RSM. Red dots are observed data points.
hit ratio ttl hours vs cache size gb
origin bandwidth ttl hours vs cache size gb
Multi-Objective Optimization
When responses compete, Derringer–Suich desirability finds the best compromise. Each response is scaled to a 0–1 desirability, then combined via a weighted geometric mean.
Overall Desirability
D = 0.9545
Per-Response Desirability
| Response | Weight | Desirability | Predicted | Dir |
|---|---|---|---|---|
hit_ratio |
1.0 |
0.9545
|
99.00 0.9545 99.00 % | ↑ |
origin_bandwidth |
1.5 |
0.9545
|
0.98 0.9545 0.98 Gbps | ↓ |
Recommended Settings
| Factor | Value |
|---|---|
ttl_hours | 24 h |
cache_policy | lru |
cache_size_gb | 50 GB |
prefetch | off |
Source: from observed run #12
Trade-off Summary
Sacrifice = how much worse than single-objective best.
| Response | Predicted | Best Observed | Sacrifice |
|---|---|---|---|
origin_bandwidth | 0.98 | 0.98 | +0.00 |
Top 3 Runs by Desirability
| Run | D | Factor Settings |
|---|---|---|
| #14 | 0.8538 | ttl_hours=24, cache_policy=lfu, cache_size_gb=50, prefetch=off |
| #5 | 0.7953 | ttl_hours=1, cache_policy=lru, cache_size_gb=50, prefetch=off |
Model Quality
| Response | R² | Type |
|---|---|---|
origin_bandwidth | 0.4057 | linear |
Full Multi-Objective Output
doe optimize --multi
============================================================
MULTI-OBJECTIVE OPTIMIZATION
Method: Derringer-Suich Desirability Function
============================================================
Overall desirability: D = 0.9545
Response Weight Desirability Predicted Direction
---------------------------------------------------------------------
hit_ratio 1.0 0.9545 99.00 % ↑
origin_bandwidth 1.5 0.9545 0.98 Gbps ↓
Recommended settings:
ttl_hours = 24 h
cache_policy = lru
cache_size_gb = 50 GB
prefetch = off
(from observed run #12)
Trade-off summary:
hit_ratio: 99.00 (best observed: 99.00, sacrifice: +0.00)
origin_bandwidth: 0.98 (best observed: 0.98, sacrifice: +0.00)
Model quality:
hit_ratio: R² = 0.3901 (linear)
origin_bandwidth: R² = 0.4057 (linear)
Top 3 observed runs by overall desirability:
1. Run #12 (D=0.9545): ttl_hours=24, cache_policy=lru, cache_size_gb=50, prefetch=off
2. Run #14 (D=0.8538): ttl_hours=24, cache_policy=lfu, cache_size_gb=50, prefetch=off
3. Run #5 (D=0.7953): ttl_hours=1, cache_policy=lru, cache_size_gb=50, prefetch=off
Full Analysis Output
doe analyze
=== Main Effects: hit_ratio ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
cache_policy -11.5875 3.6502 53.9%
ttl_hours 6.5125 3.6502 30.3%
cache_size_gb -1.8125 3.6502 8.4%
prefetch -1.5875 3.6502 7.4%
=== ANOVA Table: hit_ratio ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
ttl_hours 1 169.6506 169.6506 1.143 0.3338
cache_policy 1 537.0806 537.0806 3.620 0.1155
cache_size_gb 1 13.1406 13.1406 0.089 0.7780
prefetch 1 10.0806 10.0806 0.068 0.8048
ttl_hours*cache_policy 1 411.0756 411.0756 2.771 0.1569
ttl_hours*cache_size_gb 1 1022.4006 1022.4006 6.891 0.0468
ttl_hours*prefetch 1 62.8056 62.8056 0.423 0.5440
cache_policy*cache_size_gb 1 2.1756 2.1756 0.015 0.9083
cache_policy*prefetch 1 74.3906 74.3906 0.501 0.5105
cache_size_gb*prefetch 1 153.1406 153.1406 1.032 0.3563
Error 5 741.8731 148.3746
Total 15 3197.8144 213.1876
=== Interaction Effects: hit_ratio ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
ttl_hours cache_size_gb 15.9875 38.7%
ttl_hours cache_policy -10.1375 24.5%
cache_size_gb prefetch 6.1875 15.0%
cache_policy prefetch 4.3125 10.4%
ttl_hours prefetch 3.9625 9.6%
cache_policy cache_size_gb 0.7375 1.8%
=== Summary Statistics: hit_ratio ===
ttl_hours:
Level N Mean Std Min Max
------------------------------------------------------------
1 8 66.9750 12.6078 47.9000 86.4000
24 8 73.4875 16.5420 53.3000 99.0000
cache_policy:
Level N Mean Std Min Max
------------------------------------------------------------
lfu 8 76.0250 15.1687 58.0000 99.0000
lru 8 64.4375 12.2480 47.9000 82.5000
cache_size_gb:
Level N Mean Std Min Max
------------------------------------------------------------
200 8 71.1375 13.6577 53.3000 89.6000
50 8 69.3250 16.3835 47.9000 99.0000
prefetch:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 71.0250 13.1701 55.8000 89.6000
on 8 69.4375 16.7911 47.9000 99.0000
=== Main Effects: origin_bandwidth ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
cache_policy 7.3950 2.0502 50.2%
cache_size_gb 2.7125 2.0502 18.4%
ttl_hours -2.6650 2.0502 18.1%
prefetch 1.9700 2.0502 13.4%
=== ANOVA Table: origin_bandwidth ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
ttl_hours 1 28.4089 28.4089 0.785 0.4161
cache_policy 1 218.7441 218.7441 6.048 0.0573
cache_size_gb 1 29.4306 29.4306 0.814 0.4084
prefetch 1 15.5236 15.5236 0.429 0.5413
ttl_hours*cache_policy 1 130.0740 130.0740 3.596 0.1164
ttl_hours*cache_size_gb 1 316.1284 316.1284 8.740 0.0317
ttl_hours*prefetch 1 10.5950 10.5950 0.293 0.6116
cache_policy*cache_size_gb 1 2.3409 2.3409 0.065 0.8093
cache_policy*prefetch 1 9.5790 9.5790 0.265 0.6288
cache_size_gb*prefetch 1 67.0761 67.0761 1.854 0.2314
Error 5 180.8543 36.1709
Total 15 1008.7550 67.2503
=== Interaction Effects: origin_bandwidth ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
ttl_hours cache_size_gb -8.8900 39.3%
ttl_hours cache_policy 5.7025 25.2%
cache_size_gb prefetch -4.0950 18.1%
ttl_hours prefetch -1.6275 7.2%
cache_policy prefetch -1.5475 6.8%
cache_policy cache_size_gb -0.7650 3.4%
=== Summary Statistics: origin_bandwidth ===
ttl_hours:
Level N Mean Std Min Max
------------------------------------------------------------
1 8 16.5712 7.5119 4.3400 28.0800
24 8 13.9062 9.1444 0.9800 25.1900
cache_policy:
Level N Mean Std Min Max
------------------------------------------------------------
lfu 8 11.5412 8.4851 0.9800 23.3300
lru 8 18.9363 6.3923 10.1700 28.0800
cache_size_gb:
Level N Mean Std Min Max
------------------------------------------------------------
200 8 13.8825 8.1076 2.5700 25.1900
50 8 16.5950 8.6122 0.9800 28.0800
prefetch:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 14.2538 8.2697 2.5700 23.3300
on 8 16.2237 8.5734 0.9800 28.0800
Optimization Recommendations
doe optimize
=== Optimization: hit_ratio ===
Direction: maximize
Best observed run: #12
ttl_hours = 1
cache_policy = lfu
cache_size_gb = 200
prefetch = off
Value: 99.0
RSM Model (linear, R² = 0.2435, Adj R² = -0.0316):
Coefficients:
intercept +70.2313
ttl_hours -0.1437
cache_policy -1.0813
cache_size_gb -3.6313
prefetch -5.8562
RSM Model (quadratic, R² = 0.5540, Adj R² = -5.6894):
Coefficients:
intercept +14.0462
ttl_hours -0.1438
cache_policy -1.0813
cache_size_gb -3.6312
prefetch -5.8562
ttl_hours*cache_policy +1.8688
ttl_hours*cache_size_gb -1.0563
ttl_hours*prefetch +7.3688
cache_policy*cache_size_gb -0.6437
cache_policy*prefetch +1.1063
cache_size_gb*prefetch +1.2312
ttl_hours^2 +14.0463
cache_policy^2 +14.0463
cache_size_gb^2 +14.0463
prefetch^2 +14.0463
Curvature analysis:
ttl_hours coef=+14.0463 convex (has a minimum)
cache_policy coef=+14.0463 convex (has a minimum)
cache_size_gb coef=+14.0463 convex (has a minimum)
prefetch coef=+14.0463 convex (has a minimum)
Notable interactions:
ttl_hours*prefetch coef=+7.3688 (synergistic)
ttl_hours*cache_policy coef=+1.8688 (synergistic)
cache_size_gb*prefetch coef=+1.2312 (synergistic)
cache_policy*prefetch coef=+1.1063 (synergistic)
ttl_hours*cache_size_gb coef=-1.0563 (antagonistic)
cache_policy*cache_size_gb coef=-0.6437 (antagonistic)
Predicted optimum (from linear model, at observed points):
ttl_hours = 1
cache_policy = lfu
cache_size_gb = 50
prefetch = off
Predicted value: 80.9438
Surface optimum (via L-BFGS-B, linear model):
ttl_hours = 1
cache_policy = lru
cache_size_gb = 50
prefetch = off
Predicted value: 80.9438
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. prefetch (effect: -11.7, contribution: 54.7%)
2. cache_size_gb (effect: 7.3, contribution: 33.9%)
3. cache_policy (effect: -2.2, contribution: 10.1%)
4. ttl_hours (effect: -0.3, contribution: 1.3%)
=== Optimization: origin_bandwidth ===
Direction: minimize
Best observed run: #12
ttl_hours = 1
cache_policy = lfu
cache_size_gb = 200
prefetch = off
Value: 0.98
RSM Model (linear, R² = 0.2968, Adj R² = 0.0411):
Coefficients:
intercept +15.2388
ttl_hours -0.5725
cache_policy +0.5062
cache_size_gb +2.2438
prefetch +3.6188
RSM Model (quadratic, R² = 0.5288, Adj R² = -6.0678):
Coefficients:
intercept +3.0477
ttl_hours -0.5725
cache_policy +0.5063
cache_size_gb +2.2437
prefetch +3.6187
ttl_hours*cache_policy -0.5050
ttl_hours*cache_size_gb +0.4175
ttl_hours*prefetch -3.4775
cache_policy*cache_size_gb +0.9038
cache_policy*prefetch -0.4388
cache_size_gb*prefetch -1.0462
ttl_hours^2 +3.0478
cache_policy^2 +3.0478
cache_size_gb^2 +3.0478
prefetch^2 +3.0478
Curvature analysis:
ttl_hours coef=+3.0478 convex (has a minimum)
cache_policy coef=+3.0478 convex (has a minimum)
cache_size_gb coef=+3.0478 convex (has a minimum)
prefetch coef=+3.0478 convex (has a minimum)
Notable interactions:
ttl_hours*prefetch coef=-3.4775 (antagonistic)
cache_size_gb*prefetch coef=-1.0462 (antagonistic)
cache_policy*cache_size_gb coef=+0.9038 (synergistic)
ttl_hours*cache_policy coef=-0.5050 (antagonistic)
cache_policy*prefetch coef=-0.4388 (antagonistic)
ttl_hours*cache_size_gb coef=+0.4175 (synergistic)
Predicted optimum (from linear model, at observed points):
ttl_hours = 1
cache_policy = lru
cache_size_gb = 200
prefetch = on
Predicted value: 22.1800
Surface optimum (via L-BFGS-B, linear model):
ttl_hours = 24
cache_policy = lru
cache_size_gb = 50
prefetch = off
Predicted value: 8.2975
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. prefetch (effect: 7.2, contribution: 52.1%)
2. cache_size_gb (effect: -4.5, contribution: 32.3%)
3. ttl_hours (effect: -1.1, contribution: 8.2%)
4. cache_policy (effect: 1.0, contribution: 7.3%)