#acl EditorsGroup:read,write,delete,revert,admin All:read = Model & Solve Track Final Results = <
> <> <
> == Model&Solve Track Final Results: Overall == ||System ||Total ||Instance ||Time || ||[[http://goo.gl/4XXFz|clasp]] ||2431 ||1432 ||999 || ||[[http://goo.gl/4XXFz|bpsolver]] ||1970 ||1111 ||859 || ||[[http://goo.gl/4XXFz|aclasp]] ||1953 ||1140 ||813 || ||[[http://goo.gl/4XXFz|ezcsp]] ||1773 ||993 ||780 || ||[[http://goo.gl/4XXFz|idp]] ||1442 ||918 ||524 || ||[[http://goo.gl/4XXFz|fastdownward]] ||367 ||217 ||150 || == Final results by categories == === Overview === ||||||||'''P''' || ||||||||'''NP''' || ||||||||'''Beyond NP''' || ||||||||'''Optimization''' || ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' || ||clasp ||497 ||290 ||290 ||clasp ||1481 ||848 ||633 ||clasp ||110 ||60 ||50 ||clasp ||343 ||234 ||109 || ||bpsolver ||459 ||253 ||206 ||ezcsp ||1419 ||786 ||633 ||bpsolver ||86 ||43 ||43 ||aclasp ||325 ||220 ||105 || ||aclasp ||404 ||240 ||164 ||bpsolver ||1218 ||654 ||564 ||idp ||0 ||0 ||0 ||bpsolver ||207 ||161 ||46 || ||ezcsp ||320 ||173 ||147 ||aclasp ||1224 ||680 ||544 ||fastdownward ||0 ||0 ||0 ||idp ||140 ||121 ||19 || ||fastdownward ||181 ||100 ||81 ||idp ||1127 ||680 ||447 ||ezcsp ||0 ||0 ||0 ||fastdownward ||81 ||47 ||34 || ||idp ||175 ||117 ||58 || ||fastdownward ||105 ||70 ||35 || ||aclasp ||0 ||0 ||0 || ||ezcsp ||34 ||34 ||0 || === P Problems === ||||System ||Total ||Reachability ||Company Controls ||Grammar-Based IE ||Hydraulic Leaking ||Hydraulic Planning ||Stable Marriage ||Partner Units Polynomial || ||clasp ||'''Total''' ||497 ||87 ||89 ||80 ||78 ||100 ||25 ||38 || ||Instance ||290 ||50 ||50 ||50 ||50 ||50 ||17 ||23 || ||Time ||207 ||37 ||39 ||30 ||28 ||50 ||8 ||15 || ||bpsolver ||'''Total''' ||459 ||90 ||68 ||96 ||100 ||100 ||5 ||- || ||Instance ||253 ||50 ||50 ||50 ||50 ||50 ||3 ||- || ||Time ||206 ||40 ||18 ||46 ||50 ||50 ||2 ||- || ||aclasp ||'''Total''' ||404 ||- ||84 ||80 ||76 ||100 ||25 ||39 || ||Instance ||240 ||- ||50 ||50 ||50 ||50 ||17 ||23 || ||Time ||164 ||- ||34 ||30 ||26 ||50 ||8 ||16 || ||ezcsp ||'''Total''' ||320 ||- ||- ||- ||99 ||99 ||87 ||35 || ||Instance ||173 ||- ||- ||- ||50 ||50 ||50 ||23 || ||Time ||147 ||- ||- ||- ||49 ||49 ||37 ||12 || ||fastdownward ||'''Total''' ||181 ||- ||- ||- ||87 ||94 ||- ||- || ||Instance ||100 ||- ||- ||- ||50 ||50 ||- ||- || ||Time ||81 ||- ||- ||- ||37 ||44 ||- ||- || ||idp ||'''Total''' ||175 ||76 ||- ||- ||- ||- ||72 ||27 || ||Instance ||117 ||50 ||- ||- ||- ||- ||50 ||17 || ||Time ||58 ||26 ||- ||- ||- ||- ||22 ||10 || === NP Problems === ||||System ||Total ||Generalized Slitherlink ||Fastfood Optimality Check ||Sokoban Decision ||Knight Tour ||Disjunctive Scheduling ||Packing Problem ||Labyrinth ||Numberlink ||Reverse Folding ||Hanoi Tower ||Magic Square Sets ||Airport Pickup ||Graph Colouring ||Solitaire ||Partner Units ||Weight-Assignment Tree ||Maze Generation ||Incremental Scheduling ||Tangram || ||clasp ||'''Total''' ||1481 ||97 ||97 ||85 ||97 ||67 ||81 ||71 ||72 ||94 ||81 ||98 ||- ||51 ||73 ||79 ||56 ||97 ||85 ||100 || ||Instance ||848 ||50 ||50 ||50 ||50 ||50 ||50 ||47 ||37 ||47 ||50 ||50 ||- ||37 ||40 ||50 ||43 ||50 ||47 ||50 || ||Time ||633 ||47 ||47 ||35 ||47 ||17 ||31 ||24 ||35 ||47 ||31 ||48 ||- ||14 ||33 ||29 ||13 ||47 ||38 ||50 || ||ezcsp ||'''Total''' ||1419 ||94 ||97 ||81 ||75 ||26 ||99 ||69 ||70 ||94 ||27 ||99 ||81 ||23 ||50 ||65 ||96 ||96 ||77 ||100 || ||Instance ||786 ||50 ||50 ||50 ||40 ||13 ||50 ||43 ||37 ||47 ||23 ||50 ||50 ||13 ||30 ||50 ||50 ||50 ||40 ||50 || ||Time ||633 ||44 ||47 ||31 ||35 ||13 ||49 ||26 ||33 ||47 ||4 ||49 ||31 ||10 ||20 ||15 ||46 ||46 ||37 ||50 || ||aclasp ||'''Total''' ||1224 ||96 ||97 ||83 ||97 ||- ||83 ||68 ||72 ||80 ||77 ||98 ||- ||28 ||73 ||77 ||- ||95 ||- ||100 || ||Instance ||680 ||50 ||50 ||50 ||50 ||- ||50 ||43 ||37 ||40 ||50 ||50 ||- ||20 ||40 ||50 ||- ||50 ||- ||50 || ||Time ||544 ||46 ||47 ||33 ||47 ||- ||33 ||25 ||35 ||40 ||27 ||48 ||- ||8 ||33 ||27 ||- ||45 ||- ||50 || ||idp ||'''Total''' ||1127 ||91 ||76 ||69 ||95 ||15 ||17 ||42 ||99 ||- ||74 ||93 ||- ||66 ||71 ||33 ||83 ||73 ||30 ||100 || ||Instance ||680 ||50 ||50 ||43 ||50 ||13 ||13 ||27 ||50 ||- ||50 ||47 ||- ||47 ||40 ||30 ||50 ||50 ||20 ||50 || ||Time ||447 ||41 ||26 ||26 ||45 ||2 ||4 ||15 ||49 ||- ||24 ||46 ||- ||19 ||31 ||3 ||33 ||23 ||10 ||50 || ||bpsolver ||'''Total''' ||1126 ||* ||92 ||60 ||97 ||92 ||82 ||72 ||73 ||- ||94* ||100 ||93 ||15 ||28 ||- ||97 ||49 ||76 ||100 || ||Instance ||607 ||* ||50 ||30 ||50 ||50 ||43 ||40 ||37 ||- ||47*||50 ||50 ||10 ||20 ||- ||50 ||37 ||40 ||50 || ||Time ||519 ||* ||42 ||30 ||47 ||42 ||39 ||32 ||36 ||- ||47*||50 ||43 ||5 ||8 ||- ||47 ||12 ||36 ||50 || ||fastdownward ||'''Total''' ||105 ||- ||- ||* ||5 ||- ||- ||- ||- ||- ||27 ||- ||10 ||- ||63 ||- ||- ||- ||- ||- || ||Instance ||70 ||- ||- ||* ||5 ||- ||- ||- ||- ||- ||23 ||- ||7 ||- ||35 ||- ||- ||- ||- ||- || ||Time ||35 ||- ||- ||* ||0 ||- ||- ||- ||- ||- ||4 ||- ||3 ||- ||28 ||- ||- ||- ||- ||- || === Beyond NP Problems === ||||System ||Total ||Strategic Companies ||Minimal Diagnosis || ||clasp ||'''Total''' ||110 ||11 ||99 || ||Instance ||60 ||10 ||50 || ||Time ||50 ||1 ||49 || ||bpsolver ||'''Total''' ||86 ||- ||86 || ||Instance ||43 ||- ||43 || ||Time ||43 ||- ||43 || ||idp ||'''Total''' ||0 ||- ||- || ||Instance ||0 ||- ||- || ||Time ||0 ||- ||- || ||fastdownward ||'''Total''' ||0 ||- ||- || ||Instance ||0 ||- ||- || ||Time ||0 ||- ||- || ||ezcsp ||'''Total''' ||0 ||- ||- || ||Instance ||0 ||- ||- || ||Time ||0 ||- ||- || ||aclasp ||'''Total''' ||0 ||- ||- || ||Instance ||0 ||- ||- || ||Time ||0 ||- ||- || === Optimization Problems === ||||System ||Total ||Crossing Minimization ||Company Controls Optimize ||Fastfood Optimization ||Sokoban Optimization ||Tomography ||Maximal Clique || ||clasp ||'''Total''' ||343 ||37 ||87 ||83 ||87 ||13 ||36 || ||Instance ||234 ||35 ||50 ||50 ||50 ||13 ||36 || ||Time ||109 ||2 ||37 ||33 ||37 ||0 ||0 || ||aclasp ||'''Total''' ||325 ||16 ||87 ||83 ||85 ||18 ||36 || ||Instance ||220 ||16 ||50 ||50 ||50 ||18 ||36 || ||Time ||105 ||0 ||37 ||33 ||35 ||0 ||0 || ||bpsolver ||'''Total''' ||207 ||13 ||- ||48 ||70 ||39 ||37 || ||Instance ||161 ||13 ||- ||35 ||37 ||39 ||37 || ||Time ||46 ||0 ||- ||13 ||33 ||0 ||0 || ||idp ||'''Total''' ||140 ||13 ||- ||64 ||21 ||13 ||29 || ||Instance ||121 ||13 ||- ||48 ||18 ||13 ||29 || ||Time ||19 ||0 ||- ||16 ||3 ||0 ||0 || ||fastdownward ||'''Total''' ||81 ||- ||- ||- ||81 ||- ||- || ||Instance ||47 ||- ||- ||- ||47 ||- ||- || ||Time ||34 ||- ||- ||- ||34 ||- ||- || ||ezcsp ||'''Total''' ||34 ||- ||- ||- ||- ||34 ||- || ||Instance ||34 ||- ||- ||- ||- ||34 ||- || ||Time ||0 ||- ||- ||- ||- ||0 ||- || == Detailed Results == === Details: raw file === Detailed results, as a ''raw'' text file, can be downloaded [[attachment:teamcompetition.final.results.raw.detailed.txt|HERE]]. <
> The file contains a line for each instance each system has been asked to solve. === Details: tables === Detailed results are also available as separate PDF files: * [[attachment:Model&Solve-Results-P-Problems-detailed.pdf]] * [[attachment:Model&Solve-Results-NP-Problems-detailed.pdf]] * [[attachment:Model&Solve-Results-BndNP-Problems-detailed.pdf]] * [[attachment:Model&Solve-Results-Opt-Problems-detailed.pdf]] == Quick reference guide == Scores result from two components: the first depending on the number of instances solved within the given time-bound, the second on the actual time spent. In case of optimization problems, "quality" of the solutions plays a role as well. Please have a look at the [[ParticipationRules|Scoring and Participation Rules]] and the [[http://www.mat.unical.it/aspcomp2011/files/LanguageSpecifications.pdf|file and language format specifications]] for more details. * System: the system name (quite easy to understand) * Benchmark: the benchmark problem * Instance: the specific instance of the problem * Out|Outcome: the answer provided by the system for the specific instance * Diagnostics: result of the evaluation of the answer (essentially, whether the system answered correctly or not) * Time: the total time spent by the system while solving the specific instance * Optimum Cost: the "cost" of the witness solution returned by the system, in case of istance of an optimization problem * Total Score: the total score rewarded for the specific instance * Inst\Opt Score - Time Score: the two components of the score, as introduced above. * WF (Cost) : Witness Found (Cost) * OF (Cost) : Optimum Found (Cost) * Inc : Inconsistent * TO : Time Out * MO/F : Mem Out Fail * N/A : Not Available / Not Applicable * * : Score obtained on a late submission of the problem's solution