B Results Appendix

B.1 Economic Cost Breakdowns

Table B.1 gives the per-order detail behind the Economic results of Section 5.1: the cost difference by realised session order, with each order group’s own summaries and interval.

Paired cost differences by realised session order, APM minus Spec-kit, in US dollars, at 95.3% coverage for the n = 7 row and 93.8% for n = 5.

Order	n	Sign (+/−)	Median	HL	Mean	CI
APM first	7	4 / 3	+0.80	+2.74	+5.72	[−2.72, +19.49]
APM second	5	5 / 0	+10.79	+8.97	+9.12	[+0.04, +16.81]

B.2 Process Breakdowns

Table B.2 gives the model-specific paired time differences behind the Process results of Section 5.2, at session and phase scope. At session scope, APM was longer in all six pairs under each model, with a wider duration gap under Opus. Its turn-time and inter-turn-time components were higher in at least five of the six pairs in each group. At phase scope, the implementation phase was longer under APM in all six pairs of each model, with close median differences. For implementation duration, the Sonnet group had the higher Hodges-Lehmann estimate and mean. Planning remained mixed in sign under both models.

Paired time differences within each model group, by scope, APM minus Spec-kit, in seconds.

Group	Scope	Measure	n	Sign (+/−)	Median	HL	Mean	CI
Opus	Session	Duration	6	6 / 0	+2436.4	+2436.4	+2603.4	[+664.1, +5596.1]
		Turn time	6	6 / 0	+2206.9	+2139.4	+2076.9	[+519.7, +3415.5]
		Inter-turn idle	6	5 / 1	+674.4	+667.1	+526.5	[−1475.3, +2180.6]
	Planning	Duration	6	3 / 3	−46.2	−46.2	+205.5	[−885.5, +2347.0]
		Turn time	6	4 / 2	+108.6	+139.3	+168.5	[−1185.8, +1497.0]
		Inter-turn idle	6	3 / 3	+147.7	+132.4	+37.0	[−1188.3, +849.9]
	Implementation	Duration	6	6 / 0	+1820.6	+1862.6	+2259.3	[+331.7, +5752.0]
		Turn time	6	6 / 0	+1498.6	+1612.5	+1873.5	[+782.2, +3566.5]
		Inter-turn idle	6	4 / 2	+247.9	+247.9	+385.8	[−450.4, +2185.5]
Sonnet	Session	Duration	6	6 / 0	+1878.6	+2302.2	+2310.7	[+701.8, +4635.4]
		Turn time	6	5 / 1	+1199.3	+1384.4	+1643.8	[−1034.8, +3833.9]
		Inter-turn idle	6	6 / 0	+679.3	+679.3	+666.9	[+5.8, +1736.6]
	Planning	Duration	6	2 / 4	−366.6	−361.4	−321.3	[−1411.2, +821.2]
		Turn time	6	1 / 5	−344.1	−472.6	−452.3	[−1354.9, +308.5]
		Inter-turn idle	6	4 / 2	+98.6	+111.2	+130.9	[−156.0, +512.7]
	Implementation	Duration	6	6 / 0	+1862.5	+2486.9	+2592.8	[+589.7, +6046.7]
		Turn time	6	5 / 1	+1450.1	+1822.9	+2066.9	[−858.7, +5188.8]
		Inter-turn idle	6	5 / 1	+412.4	+464.9	+526.0	[−47.1, +1448.4]

Table B.3 lists the session-duration and turn-prompt differences within each realised session order. Neither order reverses either result; the central differences are comparable across the two groups.

Paired session-duration and turn-prompt differences by realised session order, APM minus Spec-kit. Duration is in seconds; the turn-prompt rows are counts. The n = 7 rows have 95.3% coverage and the n = 5 rows 93.8%.

Measure	Order	n	Sign (+/−/=)	Median	HL	Mean	CI
Duration	APM first	7	7 / 0	+2124.3	+2537.8	+2676.2	[+1201.5, +4635.4]
	APM second	5	5 / 0	+2197.1	+2197.1	+2150.3	[+664.1, +3902.6]
Turn prompts	APM first	7	6 / 0 / 1	+7.0	+7.5	+7.9	[+1.0, +15.0]
	APM second	5	5 / 0 / 0	+9.0	+10.0	+10.4	[+4.0, +19.0]

Table B.4 details each APM session’s turn-time composition, including its solo and concurrent spans, behind the concurrency reading in Section 5.2.1.

Turn-time composition of the APM sessions, in seconds, with the concurrent span as a share of the turn time. Descriptive and APM only, since Spec-kit’s single conversation has no concurrent span.

Participant	Solo	Concurrent	Concurrent share
P001	4834	0	0.0%
P002	5993	16	0.3%
P003	4458	661	12.9%
P004	5300	58	1.1%
P005	5686	0	0.0%
P006	6203	9	0.1%
P007	4765	812	14.6%
P008	3934	580	12.8%
P009	4476	1541	25.6%
P010	6556	1259	16.1%
P011	4218	577	12.0%
P012	4037	429	9.6%

Table B.5 details the turn-prompt difference by scope within each model group. The implementation difference set the session difference under both models and planning stayed level, with the gap clearer under Sonnet, where all six pairs favoured APM at the session and implementation scopes.

Paired turn-prompt count differences by scope within each model group, APM minus Spec-kit.

Group	Scope	n	Sign (+/−/=)	Median	HL	Mean	CI
Opus	Session	6	5 / 0 / 1	+6.5	+7.0	+8.0	[+0.0, +21.0]
	Planning	6	3 / 3 / 0	+0.0	+0.0	+0.0	[−10.0, +13.0]
	Implementation	6	5 / 0 / 1	+7.0	+7.0	+7.7	[+0.0, +22.0]
Sonnet	Session	6	6 / 0 / 0	+8.0	+8.5	+9.8	[+7.0, +19.0]
	Planning	6	2 / 2 / 2	+0.0	+0.0	−0.3	[−3.0, +2.0]
	Implementation	6	6 / 0 / 0	+9.5	+9.5	+10.0	[+5.0, +17.0]

Table B.6 gives the turn-prompt bucket shares within each model group, each bucket by its median, Hodges-Lehmann estimate, and mean.

Turn-prompt bucket shares within each model group, by phase and framework, each bucket by its median, Hodges-Lehmann estimate, and mean.

Group	Phase	Framework	Bucket	Median	HL	Mean
Opus	Planning	APM	Protocol	0.13	0.22	0.25
			Short	0.53	0.53	0.46
			Standard	0.31	0.30	0.29
		Spec-kit	Protocol	0.40	0.40	0.42
			Short	0.47	0.46	0.45
			Standard	0.07	0.08	0.13
	Implementation	APM	Protocol	0.74	0.74	0.76
			Short	0.10	0.12	0.12
			Standard	0.06	0.12	0.12
		Spec-kit	Protocol	0.20	0.25	0.27
			Short	0.32	0.33	0.38
			Standard	0.24	0.30	0.35
Sonnet	Planning	APM	Protocol	0.10	0.10	0.10
			Short	0.48	0.48	0.48
			Standard	0.42	0.42	0.42
		Spec-kit	Protocol	0.64	0.64	0.66
			Short	0.24	0.24	0.24
			Standard	0.10	0.10	0.11
	Implementation	APM	Protocol	0.70	0.68	0.66
			Short	0.15	0.15	0.17
			Standard	0.10	0.11	0.17
		Spec-kit	Protocol	0.38	0.38	0.38
			Short	0.12	0.25	0.29
			Standard	0.00	0.12	0.17

1. The three buckets are summarised independently, so their summaries need not sum to one; see note [a] of Table 5.10.

Table B.7 lists the corresponding turn-prompt character counts within each model group.

Turn-prompt bucket character counts within each model group, by phase and framework, each bucket by its median, Hodges-Lehmann estimate, and mean.

Group	Phase	Framework	Bucket	Median	HL	Mean
Opus	Planning	APM	Protocol	150	144	144
			Short	20	21	21
			Standard	85	139	136
		Spec-kit	Protocol	104	104	110
			Short	20	14	15
			Standard	81	81	81
	Implementation	APM	Protocol	117	121	122
			Short	26	23	23
			Standard	88	93	95
		Spec-kit	Protocol	100	100	100
			Short	29	29	29
			Standard	74	74	71
Sonnet	Planning	APM	Protocol	150	150	150
			Short	19	18	17
			Standard	167	167	155
		Spec-kit	Protocol	113	115	117
			Short	25	25	25
			Standard	82	88	88
	Implementation	APM	Protocol	130	129	128
			Short	15	17	18
			Standarda	69	134	237
		Spec-kit	Protocol	100	101	144
			Short	38	38	37
			Standard	117	117	117

1. P002’s high APM implementation standard-prompt character count is explained in note [b] of Table 5.10.

Table B.8 details the turn-prompt bucket shares and character counts for each APM agent role, pooled over the role’s agents across the sessions.

Per-role turn-prompt bucket shares and character counts across the twelve APM sessions, pooled over each role’s agents. Descriptive and APM only, since Spec-kit has no agent roles to separate.

Role	Bucket	Bucket share Median	Bucket share HL	Bucket share Mean	Magnitude (chars) Median	Magnitude (chars) HL	Magnitude (chars) Mean
Planner	Protocol	0.11	0.12	0.18	150	150	147
	Short	0.50	0.50	0.47	20	20	19
	Standard	0.38	0.37	0.35	156	139	146
Manager	Protocol	0.59	0.58	0.56	130	127	125
	Short	0.16	0.17	0.18	21	21	22
	Standard	0.15	0.19	0.18	79	88	99
Worker	Protocol	0.92	0.85	0.81	128	128	128
	Short	0.00	0.08	0.10	29	22	23
	Standarda	0.00	0.00	0.08	96	326	403

1. P002’s high standard-prompt character count is explained in note [b] of Table 5.10.
2. The three buckets are summarised independently, so their summaries need not sum to one; see note [a] of Table 5.10.

Table B.9 gives each session’s steers, separating those delivered to the working agent from those that were not.

Steers per participant and framework, at session scope.

Participant	APM Delivered	APM Not delivered	Spec-kit Delivered	Spec-kit Not delivered
P001	0	0	0	0
P002	0	0	0	2
P003	0	2	0	1
P004	0	0	0	0
P005	0	0	0	0
P006	1	0	0	1
P007	0	1	0	0
P008	0	0	0	0
P009	0	1	0	0
P010	0	1	0	0
P011	2	2	0	0
P012	0	3	0	2
Cohort	3	10	0	6

B.3 Functional Breakdowns

Table B.10 gives every session’s strict and partial score parts, total score, and Python and C section totals, behind the Functional results of Section 5.3.

Per-session scores, in points. The total is the strict part plus the partial part, and the Python and C section totals sum to it.

Participant	Framework	Strict	Partial	Total	Python	C
P001	APM	91.0	3.4	94.4	28.7	65.7
	Spec-kit	71.0	0.0	71.0	20.0	51.0
P002	APM	93.0	0.0	93.0	28.0	65.0
	Spec-kit	40.0	0.0	40.0	10.0	30.0
P003	APM	34.0	0.0	34.0	30.0	4.0
	Spec-kit	34.0	0.0	34.0	30.0	4.0
P004	APM	95.0	0.0	95.0	28.0	67.0
	Spec-kit	97.0	0.0	97.0	30.0	67.0
P005	APM	100.0	0.0	100.0	30.0	70.0
	Spec-kit	100.0	0.0	100.0	30.0	70.0
P006	APM	88.0	3.0	91.0	27.5	63.5
	Spec-kit	97.0	0.0	97.0	29.0	68.0
P007	APM	98.0	0.0	98.0	29.0	69.0
	Spec-kit	96.0	0.0	96.0	29.0	67.0
P008	APM	99.0	0.0	99.0	30.0	69.0
	Spec-kit	98.0	0.0	98.0	30.0	68.0
P009	APM	100.0	0.0	100.0	30.0	70.0
	Spec-kit	100.0	0.0	100.0	30.0	70.0
P010	APM	34.0	0.0	34.0	30.0	4.0
	Spec-kit	28.0	1.6	29.6	16.0	13.6
P011	APM	99.0	0.0	99.0	30.0	69.0
	Spec-kit	99.0	0.0	99.0	30.0	69.0
P012	APM	96.0	0.5	96.5	30.0	66.5
	Spec-kit	78.0	6.2	84.2	26.1	58.1

Table B.11 gives the total-score difference by realised session order.

Paired total-score difference by realised session order, APM minus Spec-kit, in points, at 95.3% coverage for the n = 7 row and 93.8% for n = 5.

Order	n	Sign (+/−/=)	Median	HL	Mean	CI
APM first	7	3 / 1 / 3	+0.0	+1.5	+4.5	[−3.0, +12.7]
APM second	5	3 / 1 / 1	+1.0	+2.2	+11.3	[−2.0, +53.0]

B.4 Perception Breakdowns

Table B.12 gives responses to the comparison question of Section 5.4, grouped by the framework used in the second session.

Responses to the comparison question, by the framework used in the second session. Each row is the participants whose second framework was the named one, and the counts are how that session felt against their first.

Second framework	Much harder	Harder	About the same	Easier	Much easier
APMa	0	2	1	1	1
Spec-kit	2	2	1	0	1

1. P012 is excluded from this row; their comparative response is tied to the session they labelled second, which does not match the session they actually completed second, so it cannot be assigned to either framework’s realised-order group. Their other perception ratings are reported per session and retain their classification.

Table B.13 gives each framework’s sessions across the four completion states the questionnaire offered.

Self-reported completion state against the three-hour cap, by framework.

Framework	Comfortably within	Near the cap	Hit the cap	Stopped early
APM	9	3	0	0
Spec-kit	10	2	0	0