Summary
This experiment investigates lens sharpness testing. Full factorial of aperture, focal length, focus distance, and image stabilization to maximize center sharpness and minimize corner softness.
The design varies 4 factors: aperture f (f-stop), ranging from 2.8 to 11, focal length (mm), ranging from 24 to 70, focus dist m (m), ranging from 1 to 10, and stabilization, ranging from off to on. The goal is to optimize 2 responses: center lpmm (lp/mm) (maximize) and corner falloff pct (%) (minimize). Fixed conditions held constant across all runs include body = full_frame, iso = 200.
A full factorial design was used to explore all 16 possible combinations of the 4 factors at two levels. This guarantees that every main effect and interaction can be estimated independently, at the cost of a larger experiment (16 runs).
Quadratic response surface models were fitted to capture potential curvature and factor interactions. The RSM contour plots below visualize how pairs of factors jointly affect each response.
Key Findings
For center lpmm, the most influential factors were focus dist m (65.4%), focal length (32.1%), aperture f (1.3%). The best observed value was 87.0 (at aperture f = 11, focal length = 70, focus dist m = 10).
For corner falloff pct, the most influential factors were focus dist m (66.7%), aperture f (16.7%), stabilization (12.5%). The best observed value was 16.0 (at aperture f = 2.8, focal length = 24, focus dist m = 1).
Recommended Next Steps
- Consider whether any fixed factors should be varied in a future study.
Experimental Setup
Factors
| Factor | Low | High | Unit |
|---|---|---|---|
aperture_f | 2.8 | 11 | f-stop |
focal_length | 24 | 70 | mm |
focus_dist_m | 1 | 10 | m |
stabilization | off | on |
Fixed: body = full_frame, iso = 200
Responses
| Response | Direction | Unit |
|---|---|---|
center_lpmm | ↑ maximize | lp/mm |
corner_falloff_pct | ↓ minimize | % |
Configuration
use_cases/149_lens_sharpness/config.json
{
"metadata": {
"name": "Lens Sharpness Testing",
"description": "Full factorial of aperture, focal length, focus distance, and image stabilization to maximize center sharpness and minimize corner softness"
},
"factors": [
{
"name": "aperture_f",
"levels": [
"2.8",
"11"
],
"type": "continuous",
"unit": "f-stop"
},
{
"name": "focal_length",
"levels": [
"24",
"70"
],
"type": "continuous",
"unit": "mm"
},
{
"name": "focus_dist_m",
"levels": [
"1",
"10"
],
"type": "continuous",
"unit": "m"
},
{
"name": "stabilization",
"levels": [
"off",
"on"
],
"type": "categorical",
"unit": ""
}
],
"fixed_factors": {
"body": "full_frame",
"iso": "200"
},
"responses": [
{
"name": "center_lpmm",
"optimize": "maximize",
"unit": "lp/mm"
},
{
"name": "corner_falloff_pct",
"optimize": "minimize",
"unit": "%"
}
],
"settings": {
"operation": "full_factorial",
"test_script": "use_cases/149_lens_sharpness/sim.sh"
}
}
Experimental Matrix
The Full Factorial Design produces 16 runs. Each row is one experiment with specific factor settings.
| Run | aperture_f | focal_length | focus_dist_m | stabilization |
|---|---|---|---|---|
| 1 | 2.8 | 70 | 10 | on |
| 2 | 11 | 24 | 1 | on |
| 3 | 2.8 | 70 | 1 | on |
| 4 | 2.8 | 70 | 10 | off |
| 5 | 11 | 70 | 10 | off |
| 6 | 11 | 24 | 10 | off |
| 7 | 11 | 70 | 1 | off |
| 8 | 11 | 24 | 1 | off |
| 9 | 2.8 | 24 | 1 | on |
| 10 | 2.8 | 24 | 10 | off |
| 11 | 11 | 70 | 1 | on |
| 12 | 11 | 70 | 10 | on |
| 13 | 2.8 | 70 | 1 | off |
| 14 | 11 | 24 | 10 | on |
| 15 | 2.8 | 24 | 1 | off |
| 16 | 2.8 | 24 | 10 | on |
Step-by-Step Workflow
1
Preview the design
Terminal
$ doe info --config use_cases/149_lens_sharpness/config.json
2
Generate the runner script
Terminal
$ doe generate --config use_cases/149_lens_sharpness/config.json \
--output use_cases/149_lens_sharpness/results/run.sh --seed 42
3
Execute the experiments
Terminal
$ bash use_cases/149_lens_sharpness/results/run.sh
4
Analyze results
Terminal
$ doe analyze --config use_cases/149_lens_sharpness/config.json
5
Get optimization recommendations
Terminal
$ doe optimize --config use_cases/149_lens_sharpness/config.json
6
Multi-objective optimization
With 2 competing responses, use --multi to find the best compromise via Derringer–Suich desirability.
Terminal
$ doe optimize --config use_cases/149_lens_sharpness/config.json --multi
7
Generate the HTML report
Terminal
$ doe report --config use_cases/149_lens_sharpness/config.json \
--output use_cases/149_lens_sharpness/results/report.html
Features Exercised
| Feature | Value |
|---|---|
| Design type | full_factorial |
| Factor types | continuous (3), categorical (1) |
| Arg style | double-dash |
| Responses | 2 (center_lpmm ↑, corner_falloff_pct ↓) |
| Total runs | 16 |
Analysis Results
Generated from actual experiment runs using the DOE Helper Tool.
Response: center_lpmm
Top factors: focus_dist_m (65.4%), focal_length (32.1%), aperture_f (1.3%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| aperture_f | 1 | 0.0625 | 0.0625 | 0.002 | 0.9668 |
| focal_length | 1 | 39.0625 | 39.0625 | 1.200 | 0.3233 |
| focus_dist_m | 1 | 162.5625 | 162.5625 | 4.992 | 0.0757 |
| stabilization | 1 | 0.0625 | 0.0625 | 0.002 | 0.9668 |
| aperture_f*focal_length | 1 | 14.0625 | 14.0625 | 0.432 | 0.5401 |
| aperture_f*focus_dist_m | 1 | 95.0625 | 95.0625 | 2.919 | 0.1482 |
| aperture_f*stabilization | 1 | 162.5625 | 162.5625 | 4.992 | 0.0757 |
| focal_length*focus_dist_m | 1 | 18.0625 | 18.0625 | 0.555 | 0.4899 |
| focal_length*stabilization | 1 | 22.5625 | 22.5625 | 0.693 | 0.4431 |
| focus_dist_m*stabilization | 1 | 27.5625 | 27.5625 | 0.846 | 0.3998 |
| Error | 5 | 162.8125 | 32.5625 | ||
| Total | 15 | 704.4375 | 46.9625 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response: corner_falloff_pct
Top factors: focus_dist_m (66.7%), aperture_f (16.7%), stabilization (12.5%).
ANOVA
| Source | DF | SS | MS | F | p-value |
|---|---|---|---|---|---|
| Source | DF | SS | MS | F | p-value |
| aperture_f | 1 | 4.0000 | 4.0000 | 0.037 | 0.8556 |
| focal_length | 1 | 0.2500 | 0.2500 | 0.002 | 0.9636 |
| focus_dist_m | 1 | 64.0000 | 64.0000 | 0.587 | 0.4780 |
| stabilization | 1 | 2.2500 | 2.2500 | 0.021 | 0.8913 |
| aperture_f*focal_length | 1 | 9.0000 | 9.0000 | 0.083 | 0.7853 |
| aperture_f*focus_dist_m | 1 | 56.2500 | 56.2500 | 0.516 | 0.5046 |
| aperture_f*stabilization | 1 | 196.0000 | 196.0000 | 1.799 | 0.2375 |
| focal_length*focus_dist_m | 1 | 121.0000 | 121.0000 | 1.111 | 0.3402 |
| focal_length*stabilization | 1 | 306.2500 | 306.2500 | 2.811 | 0.1545 |
| focus_dist_m*stabilization | 1 | 196.0000 | 196.0000 | 1.799 | 0.2375 |
| Error | 5 | 544.7500 | 108.9500 | ||
| Total | 15 | 1499.7500 | 99.9833 |
Pareto Chart
Main Effects Plot
Normal Probability Plot of Effects
Half-Normal Plot of Effects
Model Diagnostics
Response Surface Plots
3D surfaces fitted with quadratic RSM. Red dots are observed data points.
center lpmm aperture f vs focal length
center lpmm aperture f vs focus dist m
center lpmm focal length vs focus dist m
corner falloff pct aperture f vs focal length
corner falloff pct aperture f vs focus dist m
corner falloff pct focal length vs focus dist m
Multi-Objective Optimization
When responses compete, Derringer–Suich desirability finds the best compromise. Each response is scaled to a 0–1 desirability, then combined via a weighted geometric mean.
Overall Desirability
D = 0.9431
Per-Response Desirability
| Response | Weight | Desirability | Predicted | Dir |
|---|---|---|---|---|
center_lpmm |
1.5 |
0.9545
|
87.00 0.9545 87.00 lp/mm | ↑ |
corner_falloff_pct |
1.0 |
0.9261
|
17.00 0.9261 17.00 % | ↓ |
Recommended Settings
| Factor | Value |
|---|---|
aperture_f | 2.8 f-stop |
focal_length | 70 mm |
focus_dist_m | 1 m |
stabilization | off |
Source: from observed run #14
Trade-off Summary
Sacrifice = how much worse than single-objective best.
| Response | Predicted | Best Observed | Sacrifice |
|---|---|---|---|
corner_falloff_pct | 17.00 | 16.00 | +1.00 |
Top 3 Runs by Desirability
| Run | D | Factor Settings |
|---|---|---|
| #6 | 0.7420 | aperture_f=11, focal_length=24, focus_dist_m=1, stabilization=off |
| #2 | 0.7218 | aperture_f=2.8, focal_length=70, focus_dist_m=1, stabilization=on |
Model Quality
| Response | R² | Type |
|---|---|---|
corner_falloff_pct | 0.6256 | linear |
Full Multi-Objective Output
doe optimize --multi
============================================================
MULTI-OBJECTIVE OPTIMIZATION
Method: Derringer-Suich Desirability Function
============================================================
Overall desirability: D = 0.9431
Response Weight Desirability Predicted Direction
---------------------------------------------------------------------
center_lpmm 1.5 0.9545 87.00 lp/mm ↑
corner_falloff_pct 1.0 0.9261 17.00 % ↓
Recommended settings:
aperture_f = 2.8 f-stop
focal_length = 70 mm
focus_dist_m = 1 m
stabilization = off
(from observed run #14)
Trade-off summary:
center_lpmm: 87.00 (best observed: 87.00, sacrifice: +0.00)
corner_falloff_pct: 17.00 (best observed: 16.00, sacrifice: +1.00)
Model quality:
center_lpmm: R² = 0.4880 (linear)
corner_falloff_pct: R² = 0.6256 (linear)
Top 3 observed runs by overall desirability:
1. Run #14 (D=0.9431): aperture_f=2.8, focal_length=70, focus_dist_m=1, stabilization=off
2. Run #6 (D=0.7420): aperture_f=11, focal_length=24, focus_dist_m=1, stabilization=off
3. Run #2 (D=0.7218): aperture_f=2.8, focal_length=70, focus_dist_m=1, stabilization=on
Full Analysis Output
doe analyze
=== Main Effects: center_lpmm ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
focus_dist_m 6.3750 1.7132 65.4%
focal_length -3.1250 1.7132 32.1%
aperture_f -0.1250 1.7132 1.3%
stabilization -0.1250 1.7132 1.3%
=== ANOVA Table: center_lpmm ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
aperture_f 1 0.0625 0.0625 0.002 0.9668
focal_length 1 39.0625 39.0625 1.200 0.3233
focus_dist_m 1 162.5625 162.5625 4.992 0.0757
stabilization 1 0.0625 0.0625 0.002 0.9668
aperture_f*focal_length 1 14.0625 14.0625 0.432 0.5401
aperture_f*focus_dist_m 1 95.0625 95.0625 2.919 0.1482
aperture_f*stabilization 1 162.5625 162.5625 4.992 0.0757
focal_length*focus_dist_m 1 18.0625 18.0625 0.555 0.4899
focal_length*stabilization 1 22.5625 22.5625 0.693 0.4431
focus_dist_m*stabilization 1 27.5625 27.5625 0.846 0.3998
Error 5 162.8125 32.5625
Total 15 704.4375 46.9625
=== Interaction Effects: center_lpmm ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
aperture_f stabilization -6.3750 31.5%
aperture_f focus_dist_m -4.8750 24.1%
focus_dist_m stabilization 2.6250 13.0%
focal_length stabilization -2.3750 11.7%
focal_length focus_dist_m 2.1250 10.5%
aperture_f focal_length -1.8750 9.3%
=== Summary Statistics: center_lpmm ===
aperture_f:
Level N Mean Std Min Max
------------------------------------------------------------
11 8 74.8750 7.6240 65.0000 87.0000
2.8 8 74.7500 6.5192 62.0000 82.0000
focal_length:
Level N Mean Std Min Max
------------------------------------------------------------
24 8 76.3750 5.6805 65.0000 82.0000
70 8 73.2500 7.9237 62.0000 87.0000
focus_dist_m:
Level N Mean Std Min Max
------------------------------------------------------------
1 8 71.6250 6.9475 62.0000 82.0000
10 8 78.0000 5.3984 70.0000 87.0000
stabilization:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 74.8750 6.3118 65.0000 82.0000
on 8 74.7500 7.7965 62.0000 87.0000
=== Main Effects: corner_falloff_pct ===
Factor Effect Std Error % Contribution
--------------------------------------------------------------
focus_dist_m -4.0000 2.4998 66.7%
aperture_f -1.0000 2.4998 16.7%
stabilization -0.7500 2.4998 12.5%
focal_length 0.2500 2.4998 4.2%
=== ANOVA Table: corner_falloff_pct ===
Source DF SS MS F p-value
-----------------------------------------------------------------------------
aperture_f 1 4.0000 4.0000 0.037 0.8556
focal_length 1 0.2500 0.2500 0.002 0.9636
focus_dist_m 1 64.0000 64.0000 0.587 0.4780
stabilization 1 2.2500 2.2500 0.021 0.8913
aperture_f*focal_length 1 9.0000 9.0000 0.083 0.7853
aperture_f*focus_dist_m 1 56.2500 56.2500 0.516 0.5046
aperture_f*stabilization 1 196.0000 196.0000 1.799 0.2375
focal_length*focus_dist_m 1 121.0000 121.0000 1.111 0.3402
focal_length*stabilization 1 306.2500 306.2500 2.811 0.1545
focus_dist_m*stabilization 1 196.0000 196.0000 1.799 0.2375
Error 5 544.7500 108.9500
Total 15 1499.7500 99.9833
=== Interaction Effects: corner_falloff_pct ===
Factor A Factor B Interaction % Contribution
------------------------------------------------------------------------
focal_length stabilization 8.7500 26.1%
aperture_f stabilization 7.0000 20.9%
focus_dist_m stabilization -7.0000 20.9%
focal_length focus_dist_m -5.5000 16.4%
aperture_f focus_dist_m 3.7500 11.2%
aperture_f focal_length -1.5000 4.5%
=== Summary Statistics: corner_falloff_pct ===
aperture_f:
Level N Mean Std Min Max
------------------------------------------------------------
11 8 32.8750 10.6159 17.0000 48.0000
2.8 8 31.8750 10.0490 16.0000 47.0000
focal_length:
Level N Mean Std Min Max
------------------------------------------------------------
24 8 32.2500 8.3109 19.0000 42.0000
70 8 32.5000 12.0475 16.0000 48.0000
focus_dist_m:
Level N Mean Std Min Max
------------------------------------------------------------
1 8 34.3750 12.0823 16.0000 48.0000
10 8 30.3750 7.6893 17.0000 42.0000
stabilization:
Level N Mean Std Min Max
------------------------------------------------------------
off 8 32.7500 8.6313 16.0000 42.0000
on 8 32.0000 11.8080 17.0000 48.0000
Optimization Recommendations
doe optimize
=== Optimization: center_lpmm ===
Direction: maximize
Best observed run: #14
aperture_f = 11
focal_length = 70
focus_dist_m = 10
stabilization = on
Value: 87.0
RSM Model (linear, R² = 0.0799, Adj R² = -0.2547):
Coefficients:
intercept +74.8125
aperture_f +0.1875
focal_length -0.0625
focus_dist_m +1.1875
stabilization +1.4375
RSM Model (quadratic, R² = 0.7078, Adj R² = -3.3825):
Coefficients:
intercept +14.9625
aperture_f +0.1875
focal_length -0.0625
focus_dist_m +1.1875
stabilization +1.4375
aperture_f*focal_length +0.8125
aperture_f*focus_dist_m -0.1875
aperture_f*stabilization -2.1875
focal_length*focus_dist_m +1.0625
focal_length*stabilization +4.3125
focus_dist_m*stabilization +1.5625
aperture_f^2 +14.9625
focal_length^2 +14.9625
focus_dist_m^2 +14.9625
stabilization^2 +14.9625
Curvature analysis:
aperture_f coef=+14.9625 convex (has a minimum)
focal_length coef=+14.9625 convex (has a minimum)
focus_dist_m coef=+14.9625 convex (has a minimum)
stabilization coef=+14.9625 convex (has a minimum)
Notable interactions:
focal_length*stabilization coef=+4.3125 (synergistic)
aperture_f*stabilization coef=-2.1875 (antagonistic)
focus_dist_m*stabilization coef=+1.5625 (synergistic)
focal_length*focus_dist_m coef=+1.0625 (synergistic)
aperture_f*focal_length coef=+0.8125 (synergistic)
Predicted optimum (from linear model, at observed points):
aperture_f = 11
focal_length = 24
focus_dist_m = 10
stabilization = on
Predicted value: 77.6875
Surface optimum (via L-BFGS-B, linear model):
aperture_f = 11
focal_length = 24
focus_dist_m = 10
stabilization = on
Predicted value: 77.6875
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. stabilization (effect: 2.9, contribution: 50.0%)
2. focus_dist_m (effect: 2.4, contribution: 41.3%)
3. aperture_f (effect: -0.4, contribution: 6.5%)
4. focal_length (effect: -0.1, contribution: 2.2%)
=== Optimization: corner_falloff_pct ===
Direction: minimize
Best observed run: #6
aperture_f = 2.8
focal_length = 24
focus_dist_m = 1
stabilization = off
Value: 16.0
RSM Model (linear, R² = 0.0903, Adj R² = -0.2404):
Coefficients:
intercept +32.3750
aperture_f +1.3750
focal_length -1.1250
focus_dist_m +0.5000
stabilization -2.2500
RSM Model (quadratic, R² = 0.9088, Adj R² = -0.3677):
Coefficients:
intercept +6.4750
aperture_f +1.3750
focal_length -1.1250
focus_dist_m +0.5000
stabilization -2.2500
aperture_f*focal_length -3.6250
aperture_f*focus_dist_m +0.2500
aperture_f*stabilization -0.0000
focal_length*focus_dist_m -3.2500
focal_length*stabilization -7.2500
focus_dist_m*stabilization -0.6250
aperture_f^2 +6.4750
focal_length^2 +6.4750
focus_dist_m^2 +6.4750
stabilization^2 +6.4750
Curvature analysis:
aperture_f coef=+6.4750 convex (has a minimum)
focal_length coef=+6.4750 convex (has a minimum)
focus_dist_m coef=+6.4750 convex (has a minimum)
stabilization coef=+6.4750 convex (has a minimum)
Notable interactions:
focal_length*stabilization coef=-7.2500 (antagonistic)
aperture_f*focal_length coef=-3.6250 (antagonistic)
focal_length*focus_dist_m coef=-3.2500 (antagonistic)
focus_dist_m*stabilization coef=-0.6250 (antagonistic)
Predicted optimum (from linear model, at observed points):
aperture_f = 11
focal_length = 24
focus_dist_m = 10
stabilization = off
Predicted value: 37.6250
Surface optimum (via L-BFGS-B, linear model):
aperture_f = 2.8
focal_length = 70
focus_dist_m = 1
stabilization = on
Predicted value: 27.1250
Model quality: Weak fit — consider adding center points or using a different design.
Factor importance:
1. stabilization (effect: -4.5, contribution: 42.9%)
2. aperture_f (effect: -2.8, contribution: 26.2%)
3. focal_length (effect: -2.2, contribution: 21.4%)
4. focus_dist_m (effect: 1.0, contribution: 9.5%)