welcome: please sign in
location: Diff for "Model&SolveTrackFinalResults"
Differences between revisions 1 and 23 (spanning 22 versions)
⇤ ← Revision 1 as of 2011-05-11 14:41:53 →
Size: 295
Editor: CarmenSantoro
Comment:
← Revision 23 as of 2011-05-12 09:56:01 → ⇥
Size: 13699
Editor: CarmenSantoro
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
= System Track Final Results = = Model & Solve Track Final Results =
Line 10: Line 10:
<<BR>>
||<tablewidth="1%"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">Instance ||<style="font-weight: bold; text-align: center;">Time ||
||<style="font-weight: bold; text-align: center;">clasp ||2431 ||1432 ||999 ||
||<style="font-weight: bold; text-align: center;">schroeder ||1953 ||1140 ||813 ||
||<style="font-weight: bold; text-align: center;">bpsolver ||1878 ||1064 ||814 ||
||<style="font-weight: bold; text-align: center;">ezcsp ||1773 ||993 ||780 ||
||<style="font-weight: bold; text-align: center;">idp ||1442 ||918 ||524 ||
||<style="font-weight: bold; text-align: center;">fastdownward ||367 ||217 ||150 ||
Line 11: Line 19:
<<BR>>
Line 15: Line 24:
||||||||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center;">'''P''' ||<style="text-align: center;" |13> ||||||||<style="text-align: center;">'''NP''' ||<style="text-align: center;" |13> ||||||||<style="text-align: center;">'''Beyond NP''' ||||||||<style="text-align: center;">'''Optimization''' ||
||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||<style="text-align: center;">'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||<style="text-align: center;">'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||<style="text-align: center;">'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||



||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' || ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' || ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' || ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||
||clasp ||497 ||290 ||207 ||clasp ||1481 ||848 ||633 ||clasp ||110 ||60 ||50 ||clasp ||343 ||234 ||109 ||


<<BR>>
||||||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center;">'''P''' ||<style="text-align: center;" |13> ||||||||<style="text-align: center;">'''NP''' ||<style="text-align: center;" |13> ||||||||<style="text-align: center;">'''Beyond NP''' ||||||||<style="text-align: center;">'''Optimization''' ||
||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||'''System''' ||'''Total Score''' ||'''Inst. Score''' ||'''Time Score''' ||
||clasp ||497 ||290 ||207 ||clasp ||1481 ||848 ||633 ||clasp ||110 ||60 ||50 ||clasp ||343 ||234 ||109 ||
||bpsolver ||459 ||253 ||206 ||ezcsp ||1419 ||786 ||633 ||bpsolver ||86 ||43 ||43 ||schroeder ||325 ||220 ||105 ||
||schroeder ||404 ||240 ||164 ||schroeder ||1224 ||680 ||544 ||idp ||0 ||0 ||0 ||bpsolver ||207 ||161 ||46 ||
||ezcsp ||320 ||173 ||147 ||idp ||1127 ||680 ||447 ||fastdownward ||0 ||0 ||0 ||idp ||140 ||121 ||19 ||
||fastdownward ||181 ||100 ||81 ||bpsolver ||1126 ||607 ||519 ||ezcsp ||0 ||0 ||0 ||fastdownward ||81 ||47 ||34 ||
||idp ||175 ||117 ||58 ||fastdownward ||105 ||70 ||35 ||schroeder ||0 ||0 ||0 ||ezcsp ||34 ||34 ||0 ||
Line 19: Line 48:
||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">Reachability ||<style="font-weight: bold; text-align: center;">Company Controls ||<style="font-weight: bold; text-align: center;">Grammar-Based IE ||<style="font-weight: bold; text-align: center;">Hydraulic Leaking ||<style="font-weight: bold; text-align: center;">Hydraulic Planning ||<style="font-weight: bold; text-align: center;">Stable Marriage ||<style="font-weight: bold; text-align: center;">Partner Units Polynomial ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>clasp ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">497 ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">89 ||<style="font-weight: bold;">80 ||<style="font-weight: bold;">78 ||<style="font-weight: bold;">100 ||<style="font-weight: bold;">25 ||<style="font-weight: bold;">38 ||
||<style="text-align: right;">Instance ||290 ||50 ||50 ||50 ||50 ||50 ||17 ||23 ||
||<style="text-align: right;">Time ||207 ||37 ||39 ||30 ||28 ||50 ||8 ||15 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>bpsolver ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">459 ||<style="font-weight: bold;">90 ||<style="font-weight: bold;">68 ||<style="font-weight: bold;">96 ||<style="font-weight: bold;">100 ||<style="font-weight: bold;">100 ||<style="font-weight: bold;">5 ||<style="font-weight: bold;">- ||
||<style="text-align: right;">Instance ||253 ||50 ||50 ||50 ||50 ||50 ||3 ||- ||
||<style="text-align: right;">Time ||206 ||40 ||18 ||46 ||50 ||50 ||2 ||- ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>schroeder ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">404 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">84 ||<style="font-weight: bold;">80 ||<style="font-weight: bold;">76 ||<style="font-weight: bold;">100 ||<style="font-weight: bold;">25 ||<style="font-weight: bold;">39 ||
||<style="text-align: right;">Instance ||240 ||- ||50 ||50 ||50 ||50 ||17 ||23 ||
||<style="text-align: right;">Time ||164 ||- ||34 ||30 ||26 ||50 ||8 ||16 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>ezcsp ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">320 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">99 ||<style="font-weight: bold;">99 ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">35 ||
||<style="text-align: right;">Instance ||173 ||- ||- ||- ||50 ||50 ||50 ||23 ||
||<style="text-align: right;">Time ||147 ||- ||- ||- ||49 ||49 ||37 ||12 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>fastdownward ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">181 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">87 ||<style="font-weight: bold;">94 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||
||<style="text-align: right;">Instance ||100 ||- ||- ||- ||50 ||50 ||- ||- ||
||<style="text-align: right;">Time ||81 ||- ||- ||- ||37 ||44 ||- ||- ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>idp ||<style="text-align: right;">'''Total''' ||<style="font-weight: bold;">175 ||<style="font-weight: bold;">76 ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">- ||<style="font-weight: bold;">72 ||<style="font-weight: bold;">27 ||
||<style="text-align: right;">Instance ||117 ||50 ||- ||- ||- ||- ||50 ||17 ||
||<style="text-align: right;">Time ||58 ||26 ||- ||- ||- ||- ||22 ||10 ||
Line 22: Line 73:
||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">Generalized Slitherlink ||<style="font-weight: bold; text-align: center;">Fastfood Optimality Check ||<style="font-weight: bold; text-align: center;">Sokoban Decision ||<style="font-weight: bold; text-align: center;">Knight Tour ||<style="font-weight: bold; text-align: center;">Disjunctive Scheduling ||<style="font-weight: bold; text-align: center;">Packing Problem ||<style="font-weight: bold; text-align: center;">Labyrinth ||<style="font-weight: bold; text-align: center;">Numberlink ||<style="font-weight: bold; text-align: center;">Reverse Folding ||<style="font-weight: bold; text-align: center;">Hanoi Tower ||<style="font-weight: bold; text-align: center;">Magic Square Sets ||<style="font-weight: bold; text-align: center;">Airport Pickup ||<style="font-weight: bold; text-align: center;">Graph Colouring ||<style="font-weight: bold; text-align: center;">Solitaire ||<style="font-weight: bold; text-align: center;">Partner Units ||<style="font-weight: bold; text-align: center;">Weight-Assignment Tree ||<style="font-weight: bold; text-align: center;">Maze Generation ||<style="font-weight: bold; text-align: center;">Incremental Scheduling ||<style="font-weight: bold; text-align: center;">Tangram ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>clasp ||<style="text-align: right;">'''Total''' ||1481 ||97 ||97 ||85 ||97 ||67 ||81 ||71 ||72 ||94 ||81 ||98 ||- ||51 ||73 ||79 ||56 ||97 ||85 ||100 ||
||<style="text-align: right;">Instance ||848 ||50 ||50 ||50 ||50 ||50 ||50 ||47 ||37 ||47 ||50 ||50 ||- ||37 ||40 ||50 ||43 ||50 ||47 ||50 ||
||<style="text-align: right;">Time ||633 ||47 ||47 ||35 ||47 ||17 ||31 ||24 ||35 ||47 ||31 ||48 ||- ||14 ||33 ||29 ||13 ||47 ||38 ||50 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>ezcsp ||<style="text-align: right;">'''Total''' ||1419 ||94 ||97 ||81 ||75 ||26 ||99 ||69 ||70 ||94 ||27 ||99 ||81 ||23 ||50 ||65 ||96 ||96 ||77 ||100 ||
||<style="text-align: right;">Instance ||786 ||50 ||50 ||50 ||40 ||13 ||50 ||43 ||37 ||47 ||23 ||50 ||50 ||13 ||30 ||50 ||50 ||50 ||40 ||50 ||
||<style="text-align: right;">Time ||633 ||44 ||47 ||31 ||35 ||13 ||49 ||26 ||33 ||47 ||4 ||49 ||31 ||10 ||20 ||15 ||46 ||46 ||37 ||50 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>schroeder ||<style="text-align: right;">'''Total''' ||1224 ||96 ||97 ||83 ||97 ||- ||83 ||68 ||72 ||80 ||77 ||98 ||- ||28 ||73 ||77 ||- ||95 ||- ||100 ||
||<style="text-align: right;">Instance ||680 ||50 ||50 ||50 ||50 ||- ||50 ||43 ||37 ||40 ||50 ||50 ||- ||20 ||40 ||50 ||- ||50 ||- ||50 ||
||<style="text-align: right;">Time ||544 ||46 ||47 ||33 ||47 ||- ||33 ||25 ||35 ||40 ||27 ||48 ||- ||8 ||33 ||27 ||- ||45 ||- ||50 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>idp ||<style="text-align: right;">'''Total''' ||1127 ||91 ||76 ||69 ||95 ||15 ||17 ||42 ||99 ||- ||74 ||93 ||- ||66 ||71 ||33 ||83 ||73 ||30 ||100 ||
||<style="text-align: right;">Instance ||680 ||50 ||50 ||43 ||50 ||13 ||13 ||27 ||50 ||- ||50 ||47 ||- ||47 ||40 ||30 ||50 ||50 ||20 ||50 ||
||<style="text-align: right;">Time ||447 ||41 ||26 ||26 ||45 ||2 ||4 ||15 ||49 ||- ||24 ||46 ||- ||19 ||31 ||3 ||33 ||23 ||10 ||50 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>bpsolver ||<style="text-align: right;">'''Total''' ||1126 ||* ||92 ||60 ||97 ||92 ||82 ||72 ||73 ||- ||- ||100 ||93 ||15 ||28 ||- ||97 ||49 ||76 ||100 ||
||<style="text-align: right;">Instance ||607 ||* ||50 ||30 ||50 ||50 ||43 ||40 ||37 ||- ||- ||50 ||50 ||10 ||20 ||- ||50 ||37 ||40 ||50 ||
||<style="text-align: right;">Time ||519 ||* ||42 ||30 ||47 ||42 ||39 ||32 ||36 ||- ||- ||50 ||43 ||5 ||8 ||- ||47 ||12 ||36 ||50 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>fastdownward ||<style="text-align: right;">'''Total''' ||105 ||- ||- ||* ||5 ||- ||- ||- ||- ||- ||27 ||- ||10 ||- ||63 ||- ||- ||- ||- ||- ||
||<style="text-align: right;">Instance ||70 ||- ||- ||* ||5 ||- ||- ||- ||- ||- ||23 ||- ||7 ||- ||35 ||- ||- ||- ||- ||- ||
||<style="text-align: right;">Time ||35 ||- ||- ||* ||0 ||- ||- ||- ||- ||- ||4 ||- ||3 ||- ||28 ||- ||- ||- ||- ||- ||
Line 25: Line 98:
||||<tablewidth="1%" tablestyle="text-align: center;"style="text-align: center; font-weight: bold;">System ||<style="font-weight: bold; text-align: center;">Total ||<style="font-weight: bold; text-align: center;">Strategic Companies ||<style="font-weight: bold; text-align: center;">Minimal Diagnosis ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>clasp ||<style="text-align: right;">'''Total''' ||110 ||11 ||99 ||
||<style="text-align: right;">Instance ||60 ||10 ||50 ||
||<style="text-align: right;">Time ||50 ||1 ||49 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>bpsolver ||<style="text-align: right;">'''Total''' ||86 ||- ||86 ||
||<style="text-align: right;">Instance ||43 ||- ||43 ||
||<style="text-align: right;">Time ||43 ||- ||43 ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>idp ||<style="text-align: right;">'''Total''' ||0 ||- ||- ||
||<style="text-align: right;">Instance ||0 ||- ||- ||
||<style="text-align: right;">Time ||0 ||- ||- ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>fastdownward ||<style="text-align: right;">'''Total''' ||0 ||- ||- ||
||<style="text-align: right;">Instance ||0 ||- ||- ||
||<style="text-align: right;">Time ||0 ||- ||- ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>ezcsp ||<style="text-align: right;">'''Total''' ||0 ||- ||- ||
||<style="text-align: right;">Instance ||0 ||- ||- ||
||<style="text-align: right;">Time ||0 ||- ||- ||
||<style="font-weight: bold; text-align: center; vertical-align: top;" |3>schroeder ||<style="text-align: right;">'''Total''' ||0 ||- ||- ||
||<style="text-align: right;">Instance ||0 ||- ||- ||
||<style="text-align: right;">Time ||0 ||- ||- ||




<<BR>>

== Optimization Problems ==
<<BR>>

Model & Solve Track Final Results


Contents

  1. Model & Solve Track Final Results
    1. Overall
    2. Final results by categories
    3. P Problems
    4. NP Problems
    5. Beyond NP Problems
    6. Optimization Problems


Overall


System

Total

Instance

Time

clasp

2431

1432

999

schroeder

1953

1140

813

bpsolver

1878

1064

814

ezcsp

1773

993

780

idp

1442

918

524

fastdownward

367

217

150

Final results by categories


P

NP

Beyond NP

Optimization

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

clasp

497

290

207

clasp

1481

848

633

clasp

110

60

50

clasp

343

234

109


P

NP

Beyond NP

Optimization

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

System

Total Score

Inst. Score

Time Score

clasp

497

290

207

clasp

1481

848

633

clasp

110

60

50

clasp

343

234

109

bpsolver

459

253

206

ezcsp

1419

786

633

bpsolver

86

43

43

schroeder

325

220

105

schroeder

404

240

164

schroeder

1224

680

544

idp

0

0

0

bpsolver

207

161

46

ezcsp

320

173

147

idp

1127

680

447

fastdownward

0

0

0

idp

140

121

19

fastdownward

181

100

81

bpsolver

1126

607

519

ezcsp

0

0

0

fastdownward

81

47

34

idp

175

117

58

fastdownward

105

70

35

schroeder

0

0

0

ezcsp

34

34

0

P Problems


System

Total

Reachability

Company Controls

Grammar-Based IE

Hydraulic Leaking

Hydraulic Planning

Stable Marriage

Partner Units Polynomial

clasp

Total

497

87

89

80

78

100

25

38

Instance

290

50

50

50

50

50

17

23

Time

207

37

39

30

28

50

8

15

bpsolver

Total

459

90

68

96

100

100

5

-

Instance

253

50

50

50

50

50

3

-

Time

206

40

18

46

50

50

2

-

schroeder

Total

404

-

84

80

76

100

25

39

Instance

240

-

50

50

50

50

17

23

Time

164

-

34

30

26

50

8

16

ezcsp

Total

320

-

-

-

99

99

87

35

Instance

173

-

-

-

50

50

50

23

Time

147

-

-

-

49

49

37

12

fastdownward

Total

181

-

-

-

87

94

-

-

Instance

100

-

-

-

50

50

-

-

Time

81

-

-

-

37

44

-

-

idp

Total

175

76

-

-

-

-

72

27

Instance

117

50

-

-

-

-

50

17

Time

58

26

-

-

-

-

22

10

NP Problems


System

Total

Generalized Slitherlink

Fastfood Optimality Check

Sokoban Decision

Knight Tour

Disjunctive Scheduling

Packing Problem

Labyrinth

Numberlink

Reverse Folding

Hanoi Tower

Magic Square Sets

Airport Pickup

Graph Colouring

Solitaire

Partner Units

Weight-Assignment Tree

Maze Generation

Incremental Scheduling

Tangram

clasp

Total

1481

97

97

85

97

67

81

71

72

94

81

98

-

51

73

79

56

97

85

100

Instance

848

50

50

50

50

50

50

47

37

47

50

50

-

37

40

50

43

50

47

50

Time

633

47

47

35

47

17

31

24

35

47

31

48

-

14

33

29

13

47

38

50

ezcsp

Total

1419

94

97

81

75

26

99

69

70

94

27

99

81

23

50

65

96

96

77

100

Instance

786

50

50

50

40

13

50

43

37

47

23

50

50

13

30

50

50

50

40

50

Time

633

44

47

31

35

13

49

26

33

47

4

49

31

10

20

15

46

46

37

50

schroeder

Total

1224

96

97

83

97

-

83

68

72

80

77

98

-

28

73

77

-

95

-

100

Instance

680

50

50

50

50

-

50

43

37

40

50

50

-

20

40

50

-

50

-

50

Time

544

46

47

33

47

-

33

25

35

40

27

48

-

8

33

27

-

45

-

50

idp

Total

1127

91

76

69

95

15

17

42

99

-

74

93

-

66

71

33

83

73

30

100

Instance

680

50

50

43

50

13

13

27

50

-

50

47

-

47

40

30

50

50

20

50

Time

447

41

26

26

45

2

4

15

49

-

24

46

-

19

31

3

33

23

10

50

bpsolver

Total

1126

*

92

60

97

92

82

72

73

-

-

100

93

15

28

-

97

49

76

100

Instance

607

*

50

30

50

50

43

40

37

-

-

50

50

10

20

-

50

37

40

50

Time

519

*

42

30

47

42

39

32

36

-

-

50

43

5

8

-

47

12

36

50

fastdownward

Total

105

-

-

*

5

-

-

-

-

-

27

-

10

-

63

-

-

-

-

-

Instance

70

-

-

*

5

-

-

-

-

-

23

-

7

-

35

-

-

-

-

-

Time

35

-

-

*

0

-

-

-

-

-

4

-

3

-

28

-

-

-

-

-

Beyond NP Problems


System

Total

Strategic Companies

Minimal Diagnosis

clasp

Total

110

11

99

Instance

60

10

50

Time

50

1

49

bpsolver

Total

86

-

86

Instance

43

-

43

Time

43

-

43

idp

Total

0

-

-

Instance

0

-

-

Time

0

-

-

fastdownward

Total

0

-

-

Instance

0

-

-

Time

0

-

-

ezcsp

Total

0

-

-

Instance

0

-

-

Time

0

-

-

schroeder

Total

0

-

-

Instance

0

-

-

Time

0

-

-


Optimization Problems


ASP Competition 2011: Model&SolveTrackFinalResults (last edited 2011-06-08 14:40:28 by CarmenSantoro)