Evaluations/DecomP Fine Grained
main
qa_test.parquet
texttext
OpenAI OpenAI
openai GPT-4o
prediction
QC: What awards have movies produced by people born in 1910 won?
QS: (select) [simp_qa] Who were born in the year 1910?
A: ["Teeplemole", "Muntaril"]
QS: (project_values_flat_unique) [pos_qa] For which movies was #1 the producer?
A: ["Featsaw", "Zalate", "Premercy"]
QS: (project_values_flat_unique) [aw_qa] Which awards were given to #2?
A: ["Zorgion", "Chowwurst", "Hallowcock"]
QS: [EOQ]

QC: What movies have people from the country Stridery acted in?
QS: (select) [simp_qa] Who is from the country Stridery?
A: ["Gastrat"]
QS: (project_values_flat_unique) [pos_qa] Which movies has #1 been an actor in?
A: ["Partnershipmaker", "Nilitude", "Warpstone"]
QS: [EOQ]

QC: What awards have the actors of the Erowid winning movies received?
QS: (select) [aw_qa] Which movies were given the Erowid award?
A: ["Dewbar", "Caudacite"]
QS: (project_values_flat_unique) [pos_qa] Who are the actors in the movie #1?
A: ["Wetherality", "Lougerière", "Gigabut"]
QS: (project_values_flat_unique) [aw_qa] Which awards were given to #2?
A: ["Aniconder", "Trifogation"]
QS: [EOQ]

QC: What awards did the movies directed by the Modiparity winners receive?
QS: (select) [aw_qa] Who has been awarded the Modiparity award?
A: ["Bioperatology"]
QS: (project_values_flat_unique) [pos_qa] Which movies has #1 directed?
A: ["Pestok", "Vitrilateral"]
QS: (project_values_flat_unique) [aw_qa] Which awards were given to #2?
A: ["Gutney", "Antipositive"]
QS: [EOQ]

QC: What awards have movies written by people born in 1935 won?
QS: (select) [simp_qa] Who were born in the year 1935?
A: ["Sclerocybin", "Zayage"]
QS: (project_values_flat_unique) [pos_qa] What movies has #1 written?
A: ["Noenometer", "Tayenne", "Pneumodendron"]
QS: (project_values_flat_unique) [aw_qa] Which awards were given to #2?
A: ["Brownbeard", "Goosehead", "Handt"]
QS: [EOQ]

QC: What movies have the directors from Legault directed?
QS: (select) [simp_qa] Who is from the country Legault?
A: ["Metatoun", "Sapien"]
QS: (project_values_flat_unique) [pos_qa] What movies has #1 been the director of?
A: ["Coacheship", "Misapportionment"]
QS: [EOQ]

QC: {question}
Oct 18, 2024, 4:14 AM UTC
Oct 18, 2024, 4:17 AM UTC
100 rows
78117 tokens
100 rows processed, 78117 tokens used
completed
6 columns, 100 rows