aeo-methodology

Results by Question

Per-question scores (out of 10) across all 16 runs.

Score per Question (max 10.0)

Q# Category Type R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 Avg
1 Cortex AI Functions Explain 1.7 7.0 6.7 10.0 3.7 4.3 10.0 7.3 6.7 8.0 5.0 7.0 5.7 9.0 7.7 7.7 6.7
2 Cortex AI Functions Implement 6.0 9.0 7.3 9.7 8.7 8.7 9.3 8.0 7.7 9.3 4.3 6.3 2.3 5.0 7.7 7.7 7.3
3 Cortex AI Functions Implement 2.7 9.3 7.0 9.7 6.7 5.7 9.7 7.7 8.0 9.3 4.0 6.7 3.3 8.0 7.7 7.7 7.1
4 Cortex AI Functions Implement 5.0 5.7 6.7 9.0 7.0 8.3 8.0 5.0 6.0 6.3 3.7 4.3 6.0 4.3 6.3 6.0 6.1
5 Cortex AI Functions Debug 5.3 9.3 5.3 7.3 6.3 6.7 6.0 8.3 5.3 6.3 4.0 5.7 4.3 4.0 5.7 6.0 6.0
6 Cortex Search Explain 8.0 10.0 8.0 10.0 8.3 10.0 9.3 8.0 8.0 9.3 7.3 8.0 8.7 8.0 8.0 8.0 8.6
7 Cortex Search Implement 5.7 10.0 6.7 9.3 7.7 7.3 8.3 7.0 8.0 9.0 3.7 7.3 5.0 7.3 6.0 7.3 7.2
8 Cortex Search Implement 7.3 7.3 8.0 10.0 9.3 10.0 10.0 7.3 7.7 8.3 6.7 8.0 9.3 5.7 8.0 7.7 8.2
9 Cortex Search Debug 7.0 4.7 6.0 8.3 7.7 7.7 8.3 8.3 5.0 6.3 7.3 5.7 5.0 5.7 6.0 6.0 6.6
10 Cortex Agents Explain 6.7 7.7 7.0 9.3 9.0 9.3 9.0 7.0 6.7 7.3 8.2 6.7 6.7 9.3 7.3 7.7 7.8
11 Cortex Agents Implement 5.0 8.7 5.0 5.0 5.7 6.7 6.7 2.0 2.7 3.3 3.0 5.3 4.3 4.0 4.0 3.3 4.7
12 Cortex Agents Implement 5.7 5.0 5.0 6.3 6.0 6.0 7.0 3.3 4.0 5.0 3.0 5.3 3.7 6.0 5.0 3.3 5.0
13 Dynamic Tables Compare 6.0 6.3 8.0 10.0 7.7 8.7 10.0 8.0 7.7 7.3 6.0 8.0 6.0 6.7 7.7 7.7 7.6
14 Dynamic Tables Implement 8.0 10.0 8.0 9.3 10.0 9.7 9.0 5.7 5.3 5.7 4.7 4.3 5.3 6.7 6.0 5.3 7.1
15 Dynamic Tables Debug 5.3 9.7 8.0 10.0 8.0 4.3 9.3 7.7 8.7 9.3 7.0 6.7 7.8 6.2 8.7 8.3 7.8
16 Dynamic Tables Implement 7.3 9.7 7.7 9.0 8.7 1.0 9.0 8.0 8.7 7.7 6.0 5.0 9.3 6.3 8.0 8.7 7.5
17 Dynamic Tables Explain 8.0 10.0 8.0 10.0 7.7 10.0 10.0 7.3 8.3 7.3 7.3 7.7 8.7 8.7 7.3 7.3 8.4
18 Snowpark Explain 6.3 7.3 8.0 10.0 9.0 9.3 10.0 8.0 8.0 8.0 6.3 8.7 9.0 9.8 8.0 8.0 8.4
19 Snowpark Implement 8.0 9.0 7.7 10.0 10.0 6.7 10.0 7.0 8.0 7.7 7.7 5.7 7.7 6.3 8.0 8.0 8.0
20 Snowpark Implement 0.0 4.3 6.7 10.0 4.0 5.3 9.0 7.0 7.7 8.3 4.0 6.0 7.3 7.3 6.3 7.3 6.3
21 Snowpark Compare 6.3 3.7 7.7 9.7 2.7 4.7 10.0 7.3 6.3 7.0 2.7 4.0 5.3 6.0 6.7 6.3 6.0
22 Snowpark Debug 4.0 4.7 7.3 10.0 5.0 2.3 8.7 8.7 9.0 8.7 8.7 6.5 9.0 8.7 8.7 8.3 7.4
23 Streamlit in Snowf.. Compare 7.3 5.0 7.3 9.3 7.3 8.0 10.0 8.0 7.7 7.7 6.5 6.3 7.7 6.3 7.7 8.0 7.5
24 Streamlit in Snowf.. Implement 8.0 9.0 7.3 8.3 10.0 5.3 10.0 9.0 8.0 10.0 2.7 6.3 9.0 5.3 8.7 7.7 7.8
25 Streamlit in Snowf.. Implement 7.3 10.0 6.0 8.0 7.3 8.3 9.3 8.0 6.0 7.3 6.7 8.0 6.3 7.7 7.3 7.7 7.6
26 Streamlit in Snowf.. Compare 8.0 5.7 7.7 9.7 5.3 4.3 9.3 6.7 7.3 7.7 7.3 8.7 6.0 5.3 7.7 6.7 7.1
27 Apache Iceberg Tab.. Compare 7.3 9.7 7.3 9.3 4.3 9.3 9.0 8.3 7.0 7.3 5.7 8.0 7.0 7.3 7.0 7.0 7.6
28 Apache Iceberg Tab.. Implement 8.0 3.3 8.0 10.0 5.0 6.3 10.0 6.3 7.7 8.0 6.3 6.3 7.7 5.7 7.3 8.0 7.1
29 Apache Iceberg Tab.. Compare 7.3 4.3 7.0 10.0 5.3 6.3 10.0 7.0 6.3 7.0 5.7 6.3 7.3 5.7 7.0 6.3 6.8
30 Apache Iceberg Tab.. Explain 3.7 4.3 7.0 10.0 6.3 7.3 10.0 7.7 7.0 6.3 3.0 3.7 5.0 4.0 6.7 7.0 6.2
31 Apache Iceberg Tab.. Debug 7.3 10.0 6.7 9.0 10.0 5.7 8.7 6.7 4.7 6.0 7.0 4.7 6.3 4.0 4.8 5.2 6.7
32 Snowflake ML Explain 4.0 5.7 7.0 9.0 5.3 7.3 9.0 6.7 6.0 6.7 5.0 4.7 5.7 7.7 6.7 6.0 6.4
33 Snowflake ML Implement 8.0 9.7 8.0 10.0 5.0 5.0 10.0 8.3 7.0 8.3 5.7 6.3 5.0 5.7 7.3 6.7 7.2
34 Snowflake ML Implement 0.0 9.7 8.0 9.3 10.0 10.0 10.0 6.7 6.0 6.0 1.3 4.0 7.7 7.3 5.7 8.3 6.9
35 Snowflake ML Implement 6.7 8.3 5.7 7.7 8.3 8.7 7.7 5.3 4.3 5.3 3.3 2.3 5.3 4.3 5.0 5.3 5.9
36 Snowflake ML Explain 7.0 8.3 6.7 9.0 8.7 7.3 9.0 7.0 6.7 6.7 7.7 4.7 5.7 6.0 7.0 7.0 7.1
37 Snowpark Container.. Compare 8.0 10.0 8.0 10.0 8.0 10.0 10.0 8.0 7.7 7.3 6.0 5.3 6.0 9.3 7.7 8.0 8.1
38 Snowpark Container.. Implement 3.0 5.3 7.3 10.0 5.7 4.7 10.0 7.7 7.0 7.0 6.3 8.0 7.3 8.8 5.7 6.3 6.9
39 Snowpark Container.. Implement 8.0 8.7 7.3 10.0 10.0 10.0 9.3 8.0 7.3 7.3 7.0 7.0 6.3 6.7 7.3 7.3 8.0
40 Snowpark Container.. Compare 8.0 1.3 8.0 10.0 4.7 10.0 10.0 8.0 8.0 8.0 8.0 7.0 8.0 5.7 7.3 8.0 7.5
41 Native Apps Framew.. Explain 4.0 4.7 6.0 9.7 7.3 7.7 10.0 7.7 8.0 8.7 6.0 7.3 6.7 9.7 7.7 7.3 7.4
42 Native Apps Framew.. Implement 4.7 5.3 7.3 10.0 5.3 7.0 9.7 7.3 7.3 7.3 5.0 6.7 7.0 7.7 7.3 5.3 6.9
43 Streams, Tasks, Sn.. Explain 6.7 4.3 8.0 10.0 9.0 7.0 10.0 7.7 8.0 10.0 7.0 8.0 6.7 8.7 8.0 8.0 7.9
44 Streams, Tasks, Sn.. Implement 8.0 9.7 7.7 10.0 10.0 10.0 10.0 8.0 8.0 9.3 6.0 4.0 4.7 7.0 8.0 8.0 8.0
45 Streams, Tasks, Sn.. Compare 8.0 6.7 8.0 10.0 8.0 7.7 10.0 8.0 7.7 9.7 8.0 7.3 6.7 7.7 8.3 8.3 8.1
46 Governance and Sec.. Implement 7.7 9.3 7.3 9.3 8.0 4.0 9.7 8.0 8.0 9.3 6.3 6.3 8.8 8.3 7.7 8.0 7.9
47 Governance and Sec.. Explain 7.0 6.3 7.3 10.0 8.0 7.3 10.0 7.3 6.3 7.7 6.0 7.3 7.7 6.0 5.7 7.0 7.3
48 Architecture and F.. Explain 8.0 5.3 7.7 10.0 4.7 6.7 9.3 7.7 7.7 7.7 5.0 5.7 6.7 7.0 7.3 7.7 7.1
49 Architecture and F.. Compare 5.3 5.0 7.3 9.3 5.0 5.7 9.3 8.7 8.3 7.3 6.7 5.0 6.7 7.0 8.0 8.0 7.0
50 Architecture and F.. Implement 3.0 4.3 8.0 10.0 6.7 6.0 10.0 7.3 7.7 8.0 7.0 5.7 8.0 6.0 7.7 8.0 7.1

Must-Have Pass Count per Question (max 4)

Q# Category Type R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 Avg
1 Cortex AI Functions Explain 0.0 3.0 4.0 4.0 1.0 0.0 4.0 4.0 3.7 4.0 0.7 3.3 1.0 2.3 4.0 3.7 2.7
2 Cortex AI Functions Implement 3.0 4.0 4.0 4.0 3.0 3.0 3.0 4.0 4.0 4.0 2.7 4.0 0.3 3.7 4.0 4.0 3.4
3 Cortex AI Functions Implement 1.0 3.0 4.0 4.0 1.0 0.0 4.0 3.7 4.0 4.0 2.0 2.0 1.0 2.3 4.0 4.0 2.8
4 Cortex AI Functions Implement 2.0 3.0 3.0 4.0 1.0 1.0 3.0 3.0 3.0 3.7 1.0 1.0 1.3 1.3 3.0 3.0 2.3
5 Cortex AI Functions Debug 1.0 4.0 1.0 1.0 1.0 2.0 1.0 4.0 2.0 2.3 1.0 1.0 0.7 1.0 1.0 2.3 1.6
6 Cortex Search Explain 4.0 4.0 4.0 4.0 3.0 4.0 4.0 4.0 4.0 4.0 3.0 3.7 3.3 4.0 4.0 4.0 3.8
7 Cortex Search Implement 2.0 4.0 4.0 4.0 3.0 3.0 4.0 4.0 4.0 4.0 3.3 4.3 2.3 4.3 4.0 4.0 3.6
8 Cortex Search Implement 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.7 4.0 4.0 3.7 3.7 3.7 3.7 4.0 4.0 3.9
9 Cortex Search Debug 3.0 0.0 3.0 3.0 3.0 3.0 3.0 4.0 3.0 3.0 2.7 2.7 3.3 3.0 2.3 3.3 2.8
10 Cortex Agents Explain 3.0 4.0 4.0 4.0 3.0 3.0 4.0 4.0 4.0 4.0 3.0 4.0 3.0 3.0 4.0 4.0 3.6
11 Cortex Agents Implement 1.0 4.0 4.0 2.0 0.0 1.0 2.0 1.3 2.3 1.3 1.0 1.0 1.0 0.0 2.0 0.3 1.5
12 Cortex Agents Implement 3.0 0.0 3.0 2.0 2.0 1.0 3.0 2.0 2.3 2.7 0.3 1.0 0.0 1.0 2.7 2.0 1.8
13 Dynamic Tables Compare 3.0 2.0 4.0 4.0 1.0 3.0 4.0 4.0 4.0 4.0 2.0 4.0 2.7 2.3 4.0 4.0 3.2
14 Dynamic Tables Implement 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.3 4.0 3.3 4.0 2.7 0.3 2.7 3.3 3.3 3.4
15 Dynamic Tables Debug 2.0 4.0 4.0 4.0 2.0 0.0 4.0 4.0 4.0 4.0 2.7 3.0 1.3 1.3 4.0 4.0 3.0
16 Dynamic Tables Implement 4.0 4.0 4.0 3.0 3.0 0.0 4.0 3.7 4.0 3.7 2.7 2.3 3.3 2.3 3.7 4.0 3.2
17 Dynamic Tables Explain 4.0 4.0 4.0 4.0 2.0 4.0 4.0 3.7 3.7 3.7 3.3 3.3 3.3 3.3 4.0 3.3 3.6
18 Snowpark Explain 3.0 4.0 4.0 4.0 3.0 3.0 4.0 4.0 4.0 4.0 3.3 4.0 4.0 3.7 4.0 4.0 3.8
19 Snowpark Implement 4.0 4.0 4.0 4.0 4.0 1.0 4.0 4.0 4.0 4.0 3.3 1.7 3.7 4.0 4.0 4.0 3.6
20 Snowpark Implement 0.0 0.0 3.0 4.0 1.0 0.0 4.0 3.3 4.0 3.7 3.0 3.0 3.0 1.3 3.0 4.0 2.5
21 Snowpark Compare 3.0 0.0 4.0 3.0 1.0 1.0 4.0 3.3 3.0 3.3 2.0 1.0 1.7 2.7 3.0 3.3 2.5
22 Snowpark Debug 0.0 0.0 4.0 4.0 1.0 0.0 4.0 4.0 4.0 4.0 3.3 2.0 2.7 3.3 4.0 4.0 2.8
23 Streamlit in Snowf.. Compare 4.0 1.0 4.0 4.0 2.0 2.0 4.0 4.0 4.0 4.0 2.0 3.3 3.3 3.3 4.0 4.0 3.3
24 Streamlit in Snowf.. Implement 4.0 4.0 4.0 3.0 4.0 1.0 4.0 4.0 4.0 4.0 1.3 3.7 2.3 0.3 4.0 4.0 3.2
25 Streamlit in Snowf.. Implement 3.0 4.0 3.0 3.0 2.0 3.0 3.0 3.3 3.3 3.3 3.3 3.7 2.7 3.7 3.3 3.3 3.2
26 Streamlit in Snowf.. Compare 4.0 2.0 4.0 4.0 2.0 1.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.0 4.0 3.7 3.5
27 Apache Iceberg Tab.. Compare 3.0 3.0 4.0 3.0 2.0 3.0 3.0 3.3 3.0 3.3 3.0 4.0 2.3 3.0 3.3 3.0 3.1
28 Apache Iceberg Tab.. Implement 4.0 0.0 4.0 4.0 1.0 1.0 4.0 4.0 4.0 4.0 2.3 3.7 3.3 2.7 3.7 4.0 3.1
29 Apache Iceberg Tab.. Compare 3.0 1.0 4.0 4.0 1.0 2.0 4.0 4.0 3.3 4.0 3.0 3.3 3.3 3.0 3.3 3.0 3.1
30 Apache Iceberg Tab.. Explain 2.0 3.0 4.0 4.0 3.0 3.0 4.0 4.0 3.7 4.0 2.0 3.0 3.3 2.7 4.0 3.3 3.3
31 Apache Iceberg Tab.. Debug 4.0 4.0 3.0 2.0 4.0 0.0 3.0 2.0 1.3 2.0 4.0 0.7 0.7 2.0 1.0 0.7 2.1
32 Snowflake ML Explain 1.0 1.0 3.0 3.0 1.0 1.0 4.0 3.3 3.0 3.0 2.7 2.7 1.0 2.3 2.3 3.3 2.4
33 Snowflake ML Implement 4.0 4.0 4.0 4.0 1.0 1.0 4.0 4.0 4.0 4.0 3.0 3.0 0.7 3.3 4.0 4.0 3.2
34 Snowflake ML Implement 0.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.7 3.7 0.0 1.7 4.0 2.7 3.7 4.0 3.2
35 Snowflake ML Implement 4.0 3.0 4.0 4.0 3.0 4.0 4.0 2.7 2.7 2.7 1.7 1.0 2.0 1.7 2.7 3.7 2.9
36 Snowflake ML Explain 3.0 3.0 3.0 3.0 2.0 2.0 3.0 2.0 2.3 2.7 2.7 2.0 2.7 2.7 2.7 3.0 2.6
37 Snowpark Container.. Compare 4.0 4.0 4.0 4.0 3.0 4.0 4.0 4.0 4.0 4.0 3.0 3.0 3.3 4.0 4.0 4.0 3.8
38 Snowpark Container.. Implement 0.0 0.0 4.0 4.0 1.0 0.0 4.0 3.0 3.7 2.3 2.0 3.7 1.7 3.7 3.7 3.3 2.5
39 Snowpark Container.. Implement 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.0 2.7 4.0 4.0 4.0 3.9
40 Snowpark Container.. Compare 4.0 0.0 4.0 4.0 2.0 4.0 4.0 4.0 4.0 4.0 4.0 2.7 3.3 3.7 4.0 4.0 3.5
41 Native Apps Framew.. Explain 1.0 0.0 3.0 4.0 1.0 2.0 4.0 4.0 4.0 4.0 2.3 4.0 3.3 4.0 4.0 3.3 3.0
42 Native Apps Framew.. Implement 2.0 4.0 4.0 4.0 2.0 1.0 4.0 4.0 4.0 4.0 2.3 3.0 3.0 2.3 4.0 3.0 3.2
43 Streams, Tasks, Sn.. Explain 4.0 1.0 4.0 4.0 3.0 3.0 4.0 4.0 4.0 4.0 3.7 3.0 2.7 3.3 4.0 4.0 3.5
44 Streams, Tasks, Sn.. Implement 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 2.7 1.7 2.7 1.0 4.0 4.0 3.5
45 Streams, Tasks, Sn.. Compare 4.0 3.0 4.0 4.0 4.0 2.0 4.0 4.0 3.7 4.0 3.3 3.3 3.0 3.7 4.0 4.0 3.6
46 Governance and Sec.. Implement 4.0 4.0 4.0 4.0 1.0 0.0 4.0 4.0 4.0 4.0 3.0 3.3 1.7 3.7 4.0 4.0 3.3
47 Governance and Sec.. Explain 4.0 2.0 4.0 4.0 3.0 2.0 4.0 3.7 3.3 3.7 3.7 3.3 3.0 4.0 3.7 4.0 3.5
48 Architecture and F.. Explain 4.0 0.0 4.0 4.0 1.0 1.0 4.0 4.0 4.0 4.0 4.0 2.0 3.7 3.0 3.3 4.0 3.1
49 Architecture and F.. Compare 1.0 1.0 3.0 4.0 1.0 1.0 4.0 4.0 4.0 3.7 3.3 2.7 2.7 3.3 4.0 4.0 2.9
50 Architecture and F.. Implement 0.0 0.0 4.0 4.0 0.0 0.0 4.0 4.0 3.7 4.0 2.7 1.0 2.0 2.0 3.7 4.0 2.4