Differences between revisions 48 and 61 (spanning 13 versions)

Model & Solve Track Final Results

Contents

Model & Solve Track Final Results

Model&Solve Track Final Results: Overall

System	Total	Instance	Time
clasp	2431	1432	999
aclasp	1953	1140	813
bpsolver	1878	1064	814
ezcsp	1773	993	780
idp	1442	918	524
fastdownward	367	217	150

Final results by categories

Overview

P				NP				Beyond NP				Optimization
System	Total Score	Inst. Score	Time Score	System	Total Score	Inst. Score	Time Score	System	Total Score	Inst. Score	Time Score	System	Total Score	Inst. Score	Time Score
clasp	497	290	290	clasp	1481	848	633	clasp	110	60	50	clasp	343	234	109
bpsolver	459	253	206	ezcsp	1419	786	633	bpsolver	86	43	43	aclasp	325	220	105
aclasp	404	240	164	aclasp	1224	680	544	idp	0	0	0	bpsolver	207	161	46
ezcsp	320	173	147	idp	1127	680	447	fastdownward	0	0	0	idp	140	121	19
fastdownward	181	100	81	bpsolver	1126	607	519	ezcsp	0	0	0	fastdownward	81	47	34
idp	175	117	58	fastdownward	105	70	35	aclasp	0	0	0	ezcsp	34	34	0

P Problems

System		Total	Reachability	Company Controls	Grammar-Based IE	Hydraulic Leaking	Hydraulic Planning	Stable Marriage	Partner Units Polynomial
clasp	Total	497	87	89	80	78	100	25	38
	Instance	290	50	50	50	50	50	17	23
	Time	207	37	39	30	28	50	8	15
bpsolver	Total	459	90	68	96	100	100	5	-
	Instance	253	50	50	50	50	50	3	-
	Time	206	40	18	46	50	50	2	-
aclasp	Total	404	-	84	80	76	100	25	39
	Instance	240	-	50	50	50	50	17	23
	Time	164	-	34	30	26	50	8	16
ezcsp	Total	320	-	-	-	99	99	87	35
	Instance	173	-	-	-	50	50	50	23
	Time	147	-	-	-	49	49	37	12
fastdownward	Total	181	-	-	-	87	94	-	-
	Instance	100	-	-	-	50	50	-	-
	Time	81	-	-	-	37	44	-	-
idp	Total	175	76	-	-	-	-	72	27
	Instance	117	50	-	-	-	-	50	17
	Time	58	26	-	-	-	-	22	10

NP Problems

System		Total	Generalized Slitherlink	Fastfood Optimality Check	Sokoban Decision	Knight Tour	Disjunctive Scheduling	Packing Problem	Labyrinth	Numberlink	Reverse Folding	Hanoi Tower	Magic Square Sets	Airport Pickup	Graph Colouring	Solitaire	Partner Units	Weight-Assignment Tree	Maze Generation	Incremental Scheduling	Tangram
clasp	Total	1481	97	97	85	97	67	81	71	72	94	81	98	-	51	73	79	56	97	85	100
	Instance	848	50	50	50	50	50	50	47	37	47	50	50	-	37	40	50	43	50	47	50
	Time	633	47	47	35	47	17	31	24	35	47	31	48	-	14	33	29	13	47	38	50
ezcsp	Total	1419	94	97	81	75	26	99	69	70	94	27	99	81	23	50	65	96	96	77	100
	Instance	786	50	50	50	40	13	50	43	37	47	23	50	50	13	30	50	50	50	40	50
	Time	633	44	47	31	35	13	49	26	33	47	4	49	31	10	20	15	46	46	37	50
aclasp	Total	1224	96	97	83	97	-	83	68	72	80	77	98	-	28	73	77	-	95	-	100
	Instance	680	50	50	50	50	-	50	43	37	40	50	50	-	20	40	50	-	50	-	50
	Time	544	46	47	33	47	-	33	25	35	40	27	48	-	8	33	27	-	45	-	50
idp	Total	1127	91	76	69	95	15	17	42	99	-	74	93	-	66	71	33	83	73	30	100
	Instance	680	50	50	43	50	13	13	27	50	-	50	47	-	47	40	30	50	50	20	50
	Time	447	41	26	26	45	2	4	15	49	-	24	46	-	19	31	3	33	23	10	50
bpsolver	Total	1126	*	92	60	97	92	82	72	73	-	-	100	93	15	28	-	97	49	76	100
	Instance	607	*	50	30	50	50	43	40	37	-	-	50	50	10	20	-	50	37	40	50
	Time	519	*	42	30	47	42	39	32	36	-	-	50	43	5	8	-	47	12	36	50
fastdownward	Total	105	-	-	*	5	-	-	-	-	-	27	-	10	-	63	-	-	-	-	-
	Instance	70	-	-	*	5	-	-	-	-	-	23	-	7	-	35	-	-	-	-	-
	Time	35	-	-	*	0	-	-	-	-	-	4	-	3	-	28	-	-	-	-	-

Beyond NP Problems

System		Total	Strategic Companies	Minimal Diagnosis
clasp	Total	110	11	99
	Instance	60	10	50
	Time	50	1	49
bpsolver	Total	86	-	86
	Instance	43	-	43
	Time	43	-	43
idp	Total	0	-	-
	Instance	0	-	-
	Time	0	-	-
fastdownward	Total	0	-	-
	Instance	0	-	-
	Time	0	-	-
ezcsp	Total	0	-	-
	Instance	0	-	-
	Time	0	-	-
aclasp	Total	0	-	-
	Instance	0	-	-
	Time	0	-	-

Optimization Problems

System		Total	Crossing Minimization	Company Controls Optimize	Fastfood Optimization	Sokoban Optimization	Tomography	Maximal Clique
clasp	Total	343	37	87	83	87	13	36
	Instance	234	35	50	50	50	13	36
	Time	109	2	37	33	37	0	0
aclasp	Total	325	16	87	83	85	18	36
	Instance	220	16	50	50	50	18	36
	Time	105	0	37	33	35	0	0
bpsolver	Total	207	13	-	48	70	39	37
	Instance	161	13	-	35	37	39	37
	Time	46	0	-	13	33	0	0
idp	Total	140	13	-	64	21	13	29
	Instance	121	13	-	48	18	13	29
	Time	19	0	-	16	3	0	0
fastdownward	Total	81	-	-	-	81	-	-
	Instance	47	-	-	-	47	-	-
	Time	34	-	-	-	34	-	-
ezcsp	Total	34	-	-	-	-	34	-
	Instance	34	-	-	-	-	34	-
	Time	0	-	-	-	-	0	-

Detailed Results

Details: raw file

Detailed results, as a raw text file, can be downloaded HERE.
The file contains a line for each instance each system has been asked to solve.

Details: tables

Detailed results are also available as separate PDF files:

Quick reference guide

Scores result from two components: the first depending on the number of instances solved within the given time-bound, the second on the actual time spent. In case of optimization problems, "quality" of the solutions plays a role as well. Please have a look at the Scoring and Participation Rules and the file and language format specifications for more details.

System: the system name (quite easy to understand)
Benchmark: the benchmark problem
Instance: the specific instance of the problem
Out|Outcome: the answer provided by the system for the specific instance
Diagnostics: result of the evaluation of the answer (essentially, whether the system answered correctly or not)
Time: the total time spent by the system while solving the specific instance
Optimum Cost: the "cost" of the witness solution returned by the system, in case of istance of an optimization problem
Total Score: the total score rewarded for the specific instance
Inst\Opt Score - Time Score: the two components of the score, as introduced above.
WF (Cost) : Witness Found (Cost)
OF (Cost) : Optimum Found (Cost)
Inc : Inconsistent
TO : Time Out
MO/F : Mem Out Fail
N/A : Not Available / Not Applicable

ASP Competition 2011: Model&SolveTrackFinalResults (last edited 2011-06-08 14:40:28 by CarmenSantoro)

-  ⇤ ← Revision 48 as of 2011-05-19 07:52:55 → 
  Size: 26361
  Editor: FrancescoCalimeri
  Comment:
+   ← Revision 61 as of 2011-05-22 09:59:49 → ⇥
  Size: 24735
  Editor: CarmenSantoro
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 7:
-== Detailed Results ==
Detailed results can be downloaded [[attachment:teamcompetition.final.results.raw.detailed.txt|HERE]] as a ''raw'' text file.

<<BR>>

== Overall ==
<<BR>>
||<tablewidth="1%"style="text-align:center;font-weight:bold;">System ||<style="font-weight:bold;text-align:center;">Total ||<style="font-weight:bold;text-align:center;">Instance ||<style="font-weight:bold;text-align:center;">Time ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|clasp]] ||2431 ||1432 ||999 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|aclasp]] ||1953 ||1140 ||813 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|bpsolver]] ||1878 ||1064 ||814 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|ezcsp]] ||1773 ||993 ||780 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|idp]] ||1442 ||918 ||524 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|fastdownward]] ||367 ||217 ||150 ||




<<BR>>
+== Model&Solve Track Final Results: Overall ==
||<tablewidth="1%"style="text-align:center;font-weight:bold;">System ||<style="text-align: center; font-weight: bold;">Total ||<style="font-weight:bold;text-align:center;">Instance ||<style="font-weight:bold;text-align:center;">Time ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|clasp]] ||<style="font-weight: bold;">2431 ||1432 ||999 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|aclasp]] ||<style="font-weight: bold;">1953 ||1140 ||813 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|bpsolver]] ||<style="font-weight: bold;">1878 ||1064 ||814 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|ezcsp]] ||<style="font-weight: bold;">1773 ||993 ||780 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|idp]] ||<style="font-weight: bold;">1442 ||918 ||524 ||
||<style="font-weight:bold;text-align:center;">[[http://goo.gl/4XXFz|fastdownward]] ||<style="font-weight: bold;">367 ||217 ||150 ||
-Line 28:
+Line 17:
-<<BR>>
+=== Overview ===
-Line 30:
+Line 19:
-||<style="font-weight:bold;text-align:center; ;text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||
||<style="font-weight:bold;text-align:center;">clasp ||497 ||290 ||290 ||<style="font-weight:bold; ;text-align:center">clasp ||1481 ||848 ||633 ||<style="font-weight:bold; ;text-align:center">clasp ||110 ||60 ||50 ||<style="font-weight:bold; ;text-align:center">clasp ||343 ||234 ||109 ||
||<style="font-weight:bold;text-align:center;">bpsolver ||459 ||253 ||206 ||<style="font-weight:bold; ;text-align:center">ezcsp ||1419 ||786 ||633 ||<style="font-weight:bold; ;text-align:center">bpsolver ||86 ||43 ||43 ||<style="font-weight:bold; ;text-align:center">aclasp ||325 ||220 ||105 ||
||<style="font-weight:bold;text-align:center;">aclasp ||404 ||240 ||164 ||<style="font-weight:bold; ;text-align:center">aclasp ||1224 ||680 ||544 ||<style="font-weight:bold; ;text-align:center">idp ||0 ||0 ||0 ||<style="font-weight:bold; ;text-align:center">bpsolver ||207 ||161 ||46 ||
||<style="font-weight:bold;text-align:center;">ezcsp ||320 ||173 ||147 ||<style="font-weight:bold; ;text-align:center">idp ||1127 ||680 ||447 ||<style="font-weight:bold; ;text-align:center">fastdownward ||0 ||0 ||0 ||<style="font-weight:bold; ;text-align:center">idp ||140 ||121 ||19 ||
||<style="font-weight:bold;text-align:center;">fastdownward ||181 ||100 ||81 ||<style="font-weight:bold; ;text-align:center">bpsolver ||1126 ||607 ||519 ||<style="font-weight:bold; ;text-align:center">ezcsp ||0 ||0 ||0 ||<style="font-weight:bold; ;text-align:center">fastdownward ||81 ||47 ||34 ||
||<style="font-weight:bold;text-align:center;">idp ||175 ||117 ||58 || ||<style="font-weight:bold; ;text-align:center">fastdownward ||105 ||70 ||35 || ||<style="font-weight:bold; ;text-align:center">aclasp ||0 ||0 ||0 || ||<style="font-weight:bold; ;text-align:center">ezcsp ||34 ||34 ||0 ||




<<BR>>
+||<style="font-weight:bold;text-align:center;  ;text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||<style="text-align:center">'''System''' ||<style="text-align:center">'''Total Score''' ||<style="text-align:center">'''Inst. Score''' ||<style="text-align:center">'''Time Score''' ||
||<style="font-weight:bold;text-align:center;">clasp ||497 ||290 ||290 ||<style="font-weight:bold;  ;text-align:center">clasp ||1481 ||848 ||633 ||<style="font-weight:bold;  ;text-align:center">clasp ||110 ||60 ||50 ||<style="font-weight:bold;  ;text-align:center">clasp ||343 ||234 ||109 ||
||<style="font-weight:bold;text-align:center;">bpsolver ||459 ||253 ||206 ||<style="font-weight:bold;  ;text-align:center">ezcsp ||1419 ||786 ||633 ||<style="font-weight:bold;  ;text-align:center">bpsolver ||86 ||43 ||43 ||<style="font-weight:bold;  ;text-align:center">aclasp ||325 ||220 ||105 ||
||<style="font-weight:bold;text-align:center;">aclasp ||404 ||240 ||164 ||<style="font-weight:bold;  ;text-align:center">aclasp ||1224 ||680 ||544 ||<style="font-weight:bold;  ;text-align:center">idp ||0 ||0 ||0 ||<style="font-weight:bold;  ;text-align:center">bpsolver ||207 ||161 ||46 ||
||<style="font-weight:bold;text-align:center;">ezcsp ||320 ||173 ||147 ||<style="font-weight:bold;  ;text-align:center">idp ||1127 ||680 ||447 ||<style="font-weight:bold;  ;text-align:center">fastdownward ||0 ||0 ||0 ||<style="font-weight:bold;  ;text-align:center">idp ||140 ||121 ||19 ||
||<style="font-weight:bold;text-align:center;">fastdownward ||181 ||100 ||81 ||<style="font-weight:bold;  ;text-align:center">bpsolver ||1126 ||607 ||519 ||<style="font-weight:bold;  ;text-align:center">ezcsp ||0 ||0 ||0 ||<style="font-weight:bold;  ;text-align:center">fastdownward ||81 ||47 ||34 ||
||<style="font-weight:bold;text-align:center;">idp ||175 ||117 ||58 || ||<style="font-weight:bold;  ;text-align:center">fastdownward ||105 ||70 ||35 || ||<style="font-weight:bold;  ;text-align:center">aclasp ||0 ||0 ||0 || ||<style="font-weight:bold;  ;text-align:center">ezcsp ||34 ||34 ||0 ||
-Line 44:
+Line 28:
-<<BR>>

{{{#!wiki comment
||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">[[Model&SolveReachabilityResults|Reachability]]||<style="font-weight: bold; text-align: center;">[[Model&SolveCompanyControlsResults|Company Controls]]||<style="font-weight: bold; text-align: center;">[[Model&SolveGrammar-BasedIEResults|Grammar-Based IE]]||<style="font-weight: bold; text-align: center;">[[Model&SolveHydraulicLeakingResults|Hydraulic Leaking]]||<style="font-weight: bold; text-align: center;">[[Model&SolveHydraulicPlanningResults|Hydraulic Planning]] ||<style="font-weight: bold; text-align: center;">[[Model&SolveStableMarriageResults|Stable Marriage]]||<style="font-weight: bold; text-align: center;">[[Model&SolvePartnerUnitsPolynomialResults|Partner Units Polynomial]]||
+||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">Reachability||<style="font-weight: bold; text-align: center;">Company Controls||<style="font-weight: bold; text-align: center;">Grammar-Based IE||<style="font-weight: bold; text-align: center;">Hydraulic Leaking||<style="font-weight: bold; text-align: center;">Hydraulic Planning||<style="font-weight: bold; text-align: center;">Stable Marriage||<style="font-weight: bold; text-align: center;">Partner Units Polynomial||
-Line 49:
+Line 30:
-}}}
||||<tablewidth="1%" tablestyle="text-align:center; "style="text-align:center;font-weight:bold;">System ||<style="font-weight:bold;text-align:center;">Total ||<style="font-weight:bold;text-align:center;">Reachability ||<style="font-weight:bold;text-align:center;">Company Controls ||<style="font-weight:bold;text-align:center;">Grammar-Based IE ||<style="font-weight:bold;text-align:center;">Hydraulic Leaking ||<style="font-weight:bold;text-align:center;">Hydraulic Planning ||<style="font-weight:bold;text-align:center;">Stable Marriage ||<style="font-weight:bold;text-align:center;">Partner Units Polynomial ||
||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">497 ||<style="font-weight:bold;">87 ||<style="font-weight:bold;">89 ||<style="font-weight:bold;">80 ||<style="font-weight:bold;">78 ||<style="font-weight:bold;">100 ||<style="font-weight:bold;">25 ||<style="font-weight:bold;">38 ||
-Line 70:
+Line 48:
-<<BR>>
-Line 74:
+Line 49:
-<<BR>>

{{{#!wiki comment
-Line 78:
+Line 50:
-}}}
||||<tablewidth="1%" tablestyle="text-align:center; "style="text-align:center;font-weight:bold;">System ||<style="font-weight:bold;text-align:center;">Total ||<style="font-weight:bold;text-align:center;">Generalized Slitherlink ||<style="font-weight:bold;text-align:center;">Fastfood Optimality Check ||<style="font-weight:bold;text-align:center;">Sokoban Decision ||<style="font-weight:bold;text-align:center;">Knight Tour ||<style="font-weight:bold;text-align:center;">Disjunctive Scheduling ||<style="font-weight:bold;text-align:center;">Packing Problem ||<style="font-weight:bold;text-align:center;">Labyrinth ||<style="font-weight:bold;text-align:center;">Numberlink ||<style="font-weight:bold;text-align:center;">Reverse Folding ||<style="font-weight:bold;text-align:center;">Hanoi Tower ||<style="font-weight:bold;text-align:center;">Magic Square Sets ||<style="font-weight:bold;text-align:center;">Airport Pickup ||<style="font-weight:bold;text-align:center;">Graph Colouring ||<style="font-weight:bold;text-align:center;">Solitaire ||<style="font-weight:bold;text-align:center;">Partner Units ||<style="font-weight:bold;text-align:center;">Weight-Assignment Tree ||<style="font-weight:bold;text-align:center;">Maze Generation ||<style="font-weight:bold;text-align:center;">Incremental Scheduling ||<style="font-weight:bold;text-align:center;">Tangram ||
||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">1481 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">85 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">67 ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">71 ||<style="font-weight: bold;">72 ||<style="font-weight: bold;">94 ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">98 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">51 ||<style="font-weight: bold;">73 ||<style="font-weight: bold;">79 ||<style="font-weight: bold;">56 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">85 ||<style="font-weight: bold;">100 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">1481 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">85 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">67 ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">71 ||<style="font-weight:bold;">72 ||<style="font-weight:bold;">94 ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">98 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">51 ||<style="font-weight:bold;">73 ||<style="font-weight:bold;">79 ||<style="font-weight:bold;">56 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">85 ||<style="font-weight:bold;">100 ||
-Line 83:
+Line 53:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">1419 ||<style="font-weight: bold;">94 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">75 ||<style="font-weight: bold;">26 ||<style="font-weight: bold;">99 ||<style="font-weight: bold;">69 ||<style="font-weight: bold;">70 ||<style="font-weight: bold;">94 ||<style="font-weight: bold;">27 ||<style="font-weight: bold;">99 ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">23 ||<style="font-weight: bold;">50 ||<style="font-weight: bold;">65 ||<style="font-weight: bold;">96 ||<style="font-weight: bold;">96 ||<style="font-weight: bold;">77 ||<style="font-weight: bold;">100 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">1419 ||<style="font-weight:bold;">94 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">75 ||<style="font-weight:bold;">26 ||<style="font-weight:bold;">99 ||<style="font-weight:bold;">69 ||<style="font-weight:bold;">70 ||<style="font-weight:bold;">94 ||<style="font-weight:bold;">27 ||<style="font-weight:bold;">99 ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">23 ||<style="font-weight:bold;">50 ||<style="font-weight:bold;">65 ||<style="font-weight:bold;">96 ||<style="font-weight:bold;">96 ||<style="font-weight:bold;">77 ||<style="font-weight:bold;">100 ||
-Line 86:
+Line 56:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">1224 ||<style="font-weight: bold;">96 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">83 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">83 ||<style="font-weight: bold;">68 ||<style="font-weight: bold;">72 ||<style="font-weight: bold;">80 ||<style="font-weight: bold;">77 ||<style="font-weight: bold;">98 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">28 ||<style="font-weight: bold;">73 ||<style="font-weight: bold;">77 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">95 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">100 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">1224 ||<style="font-weight:bold;">96 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">83 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">83 ||<style="font-weight:bold;">68 ||<style="font-weight:bold;">72 ||<style="font-weight:bold;">80 ||<style="font-weight:bold;">77 ||<style="font-weight:bold;">98 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">28 ||<style="font-weight:bold;">73 ||<style="font-weight:bold;">77 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">95 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">100 ||
-Line 89:
+Line 59:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">1127 ||<style="font-weight: bold;">91 ||<style="font-weight: bold;">76 ||<style="font-weight: bold;">69 ||<style="font-weight: bold;">95 ||<style="font-weight: bold;">15 ||<style="font-weight: bold;">17 ||<style="font-weight: bold;">42 ||<style="font-weight: bold;">99 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">74 ||<style="font-weight: bold;">93 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">66 ||<style="font-weight: bold;">71 ||<style="font-weight: bold;">33 ||<style="font-weight: bold;">83 ||<style="font-weight: bold;">73 ||<style="font-weight: bold;">30 ||<style="font-weight: bold;">100 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">1127 ||<style="font-weight:bold;">91 ||<style="font-weight:bold;">76 ||<style="font-weight:bold;">69 ||<style="font-weight:bold;">95 ||<style="font-weight:bold;">15 ||<style="font-weight:bold;">17 ||<style="font-weight:bold;">42 ||<style="font-weight:bold;">99 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">74 ||<style="font-weight:bold;">93 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">66 ||<style="font-weight:bold;">71 ||<style="font-weight:bold;">33 ||<style="font-weight:bold;">83 ||<style="font-weight:bold;">73 ||<style="font-weight:bold;">30 ||<style="font-weight:bold;">100 ||
-Line 92:
+Line 62:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">1126 ||<style="font-weight: bold;">* ||<style="font-weight: bold;">92 ||<style="font-weight: bold;">60 ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">92 ||<style="font-weight: bold;">82 ||<style="font-weight: bold;">72 ||<style="font-weight: bold;">73 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">100 ||<style="font-weight: bold;">93 ||<style="font-weight: bold;">15 ||<style="font-weight: bold;">28 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">97 ||<style="font-weight: bold;">49 ||<style="font-weight: bold;">76 ||<style="font-weight: bold;">100 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">1126 ||<style="font-weight:bold;">* ||<style="font-weight:bold;">92 ||<style="font-weight:bold;">60 ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">92 ||<style="font-weight:bold;">82 ||<style="font-weight:bold;">72 ||<style="font-weight:bold;">73 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">100 ||<style="font-weight:bold;">93 ||<style="font-weight:bold;">15 ||<style="font-weight:bold;">28 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">97 ||<style="font-weight:bold;">49 ||<style="font-weight:bold;">76 ||<style="font-weight:bold;">100 ||
-Line 95:
+Line 65:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">105 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">* ||<style="font-weight: bold;">5 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">27 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">10 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">63 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">105 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">* ||<style="font-weight:bold;">5 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">27 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">10 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">63 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 99:
+Line 69:
-<<BR>>
-Line 103:
+Line 70:
-<<BR>>

{{{#!wiki comment
-Line 107:
+Line 71:
-}}}
||||<tablewidth="1%" tablestyle="text-align:center; "style="text-align:center;font-weight:bold;">System ||<style="font-weight:bold;text-align:center;">Total ||<style="font-weight:bold;text-align:center;">Strategic Companies ||<style="font-weight:bold;text-align:center;">Minimal Diagnosis ||
||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">110 ||<style="font-weight: bold;">11 ||<style="font-weight: bold;">99 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">110 ||<style="font-weight:bold;">11 ||<style="font-weight:bold;">99 ||
-Line 112:
+Line 74:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">86 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">86 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">86 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">86 ||
-Line 115:
+Line 77:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">0 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">0 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 118:
+Line 80:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">0 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">0 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 121:
+Line 83:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">0 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">0 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 124:
+Line 86:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">0 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">0 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 128:
+Line 90:
-<<BR>>
-Line 132:
+Line 91:
-<<BR>>

{{{#!wiki comment
-Line 136:
+Line 92:
-}}}
||||<tablewidth="1%" tablestyle="text-align:center; "style="text-align:center;font-weight:bold;">System ||<style="font-weight:bold;text-align:center;">Total ||<style="font-weight:bold;text-align:center;">Crossing Minimization ||<style="font-weight:bold;text-align:center;">Company Controls Optimize ||<style="font-weight:bold;text-align:center;">Fastfood Optimization ||<style="font-weight:bold;text-align:center;">Sokoban Optimization ||<style="font-weight:bold;text-align:center;">Tomography ||<style="font-weight:bold;text-align:center;">Maximal Clique ||
||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">343 ||<style="font-weight: bold;">37 ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">83 ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">13 ||<style="font-weight: bold;">36 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>clasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">343 ||<style="font-weight:bold;">37 ||<style="font-weight:bold;">87 ||<style="font-weight:bold;">83 ||<style="font-weight:bold;">87 ||<style="font-weight:bold;">13 ||<style="font-weight:bold;">36 ||
-Line 141:
+Line 95:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">325 ||<style="font-weight: bold;">16 ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">83 ||<style="font-weight: bold;">85 ||<style="font-weight: bold;">18 ||<style="font-weight: bold;">36 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>aclasp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">325 ||<style="font-weight:bold;">16 ||<style="font-weight:bold;">87 ||<style="font-weight:bold;">83 ||<style="font-weight:bold;">85 ||<style="font-weight:bold;">18 ||<style="font-weight:bold;">36 ||
-Line 144:
+Line 98:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">207 ||<style="font-weight: bold;">13 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">48 ||<style="font-weight: bold;">70 ||<style="font-weight: bold;">39 ||<style="font-weight: bold;">37 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>bpsolver ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">207 ||<style="font-weight:bold;">13 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">48 ||<style="font-weight:bold;">70 ||<style="font-weight:bold;">39 ||<style="font-weight:bold;">37 ||
-Line 147:
+Line 101:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">140 ||<style="font-weight: bold;">13 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">64 ||<style="font-weight: bold;">21 ||<style="font-weight: bold;">13 ||<style="font-weight: bold;">29 ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>idp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">140 ||<style="font-weight:bold;">13 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">64 ||<style="font-weight:bold;">21 ||<style="font-weight:bold;">13 ||<style="font-weight:bold;">29 ||
-Line 150:
+Line 104:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">81 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>fastdownward ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">81 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||
-Line 153:
+Line 107:
-||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight: bold;">34 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">34 ||<style="font-weight: bold;">- ||
+||<style="font-weight:bold;text-align:center;vertical-align:top;" |3>ezcsp ||<style="text-align:right;">'''Total''' ||<style="font-weight:bold;">34 ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">- ||<style="font-weight:bold;">34 ||<style="font-weight:bold;">- ||
-Line 156:
+Line 110:
+== Detailed Results ==
=== Details: raw file ===
Detailed results, as a ''raw'' text file, can be downloaded  [[attachment:teamcompetition.final.results.raw.detailed.txt|HERE]].
<<BR>>
The file contains a line for each instance each system has been asked to solve. 

=== Details: tables ===
Detailed results are also available as separate PDF files: 
 * [[attachment:Model&Solve-Results-P-Problems-detailed.pdf||System Track - Details - P problems]]
 * [[attachment:Model&Solve-Results-NP-Problems-detailed.pdf||System Track - Details - NP problems]]
 * [[attachment:Model&Solve-Results-BndNP-Problems-detailed.pdf||System Track - Details - Beyond P problems]]
 * [[attachment:Model&Solve-Results-Opt-Problems-detailed.pdf||System Track - Details - Beyond P problems]]

== Quick reference guide ==
Scores result from two components: the first depending on the number of instances solved within the given time-bound, the second on the actual time spent. In case of optimization problems, "quality" of the solutions plays a role as well. Please have a look at the [[ParticipationRules|Scoring and Participation Rules]] and the [[http://www.mat.unical.it/aspcomp2011/files/LanguageSpecifications.pdf|file and language format specifications]] for more details.

 * System: the system name (quite easy to understand) 
 * Benchmark: the benchmark problem
 * Instance: the specific instance of the problem
 * Out|Outcome: the answer provided by the system for the specific instance
 * Diagnostics: result of the evaluation of the answer (essentially, whether the system answered correctly or not)
 * Time: the total time spent by the system while solving the specific instance
 * Optimum Cost: the "cost" of the witness solution returned by the system, in case of istance of an optimization problem
 * Total Score: the total score rewarded for the specific instance
 * Inst\Opt Score -  Time Score: the two components of the score, as introduced above.

 * WF (Cost) : Witness Found (Cost)
 * OF (Cost) : Optimum Found (Cost)
 * Inc       : Inconsistent
 * TO        : Time Out
 * MO/F      : Mem Out Fail
 * N/A       : Not Available / Not Applicable

ASP Competition 2011

Quick Links

Search Wiki

Page Tools