Handling Non-local Dead-ends in Agent Planning Programs

Luk´aˇs Chrpa1 1 Czech

Sebastian Sardina3

Technical University in Prague & Charles University in Prague

2 University 3 RMIT

Nir Lipovetzky2

of Melbourne

University

Agent Planning Programs (APPs)

[AAMAS’10, AIJ’16]

A novel approach to represent and synthesize complex behaviors: 1. Can encode some know-how information available. 2. Supports automatic decision making for filling the “details.” • depends on underlying infrastructure; too complex by hand.

3. Allows continuous operation & repeated tasks. 4. Accommodates external decision (e.g., human expert).

1

Definition of APPs

APPs are a state transition system consisting of: • Program states • Transitions, directed edges between program states consisting of: • a guard, a precondition for “taking” the transition, • a maintenance formula, an invariant to be maintained during “transiting”, • a goal to be achieved after completing the transition

A decision which transition to take from a current program state is done externally by user (only transtions whose guard holds can be chosen)

2

An example of an APP: The life of an academic 1. An academic travels between home, uni, and pub in his/her life. • Can go to places in many ways (drive, bus, walk). • Modeled via an underlying theory of action (e.g., PDDL).

2. At each point, the academic has several next goal options. • S/he chooses at run-time which goal to purse next.

3. System then provides a plan for the chosen goal. (at agent work)

p1

p2 (at agent home)

(at agent home)

p3

(at agent pub)

(had agent drinks) 3

Solving APPs Offline via Planning

[ICAPS’11, AIJ’16]

A realization solution is a policy that maps pairs hworld state, transitioni to plans achieving transition goal. • There will always be a plan for any run of the APP...

4

Solving APPs Offline via Planning

[ICAPS’11, AIJ’16]

A realization solution is a policy that maps pairs hworld state, transitioni to plans achieving transition goal. • There will always be a plan for any run of the APP... Domain-independent planning engines are used to generate plans (starting in the initial world and program states). • Preferred states – world states that are known for corresponding program states. • Tabu states – world states from which some future goals cannot be achieved.

4

Dead-ends in APPs Definition (Dead-end) A dead-end is a world state from where the goal cannot be achieved. Lipovetzky et al. (2016) introduced dead-end traps: • k-DNF formula representing dead-ends for a planning problem.

5

Dead-ends in APPs Definition (Dead-end) A dead-end is a world state from where the goal cannot be achieved. Lipovetzky et al. (2016) introduced dead-end traps: • k-DNF formula representing dead-ends for a planning problem.

But, in APPs, dead-ends can be non-local (or global)! • i.e., some potentially “future” goal becomes unachievable. Existing offline solver does not reason these “global” dead-ends: • ... it only backtracks when a current goal becomes unachievable. 5

A modified example of an APP Constraints • Academic can use a car or public transport for moving around. • At the end of the day, the academic’s car must be at home. • When the academic is intoxicated (after drinking), s/he cannot drive: must not drink if car is not at home. (at agent work)

home

work (at agent home) (at car home)

(at agent home)

pub

(at agent pub)

(at car home) (had agent drinks) 6

Reasoning about non-local dead-ends in APPs

Prior to each planning episode for a pair hworld state, transitioni: 1. Compute set Gf of all future goals (i.e., reachable in APP). 2. Compute a dead-end trap for each goal in Gf . 3. Incorporate the dead-end traps (by negating them) into the maintenance formula of the current transition. Finally, generate a plan with new amended maintenance formula.

7

Online realization of APPs Online execution of APP: solve only selected transitions. APP Online Execution Cycle 1. APP user selects an enabled outgoing transition. 2. Amend the maintenance formula by reasoning about non-local dead-ends (as in the previous slide). 3. Generate a plan. • if goal is unachievable, then fail.

4. Execute the plan: • evolve world state; • evolve program state.

5. Go to step 1. 8

Experimental Evaluation Purpose: test usefulness of non-local dead-ends reasoning. Benchmark: 6 domains from the IPC benchmarks used: • Airport, Floortile, “Glued” BlocksWorld (modified BlocksWorld), Matching BlocksWorld, Logistics, Woodworking. • 20 APPs were generated for each domain. Systems: 3 planners were used: • LPG (not used for dead-ends reasoning). • Lm-Cut. • DFS+. Dead-end traps: 2-DNF dead-end traps were considered. (Time limit of 3 hours per APP was enforced.) 9

√

√

Floortile

Logistics

Glued-Bw

Match-Bw

Airport

Woodwork

Total

Online

Plain (LPG) Plain (DFS+) Plain (LmCut) Traps (DFS+) Traps (LmCut)

4 8 8 16 18

0 1 16 4 19

0 1 0 20 20

0 1 0 20 18

15 17 17 13 13

0 3 2 20 20

19 31 43 93 108

Offline

Results

Plain (LPG) Plain (DFS+) Plain (LmCut) Traps (DFS+) Traps (LmCut)

10 8 9 17 19

7 1 15 4 19

1 1 1 20 20

3 1 2 12 11

16 17 16 13 13

14 8 11 13 9

51 36 54 79 91

Non-local dead-end reasoning provides general improvement! - In Airport, more complex APPs 100000s of atoms were used to describe world states introducing huge overheads for the dead-end reasoning tool.

The “plain” offline approach outperformed the “naive” online approach when LPG was used (quick to generate plans), and in Woodworking (no “far” backtracks) √ The “dead-end” online approach outperforms the “dead-end” offline one in APPs with branching (e.g. Woodworking)

10

Summary of the contributions

We have: 1. Adapted the local dead-ends reasoning tool [Lipovetzky et al., 2016] for reasoning about non-local dead-end in Agent Planning Programs. 2. Introduced an online approach for realizing Agent Planning Programs. 3. Provided theoretical results on unsolvability of APPs under global dead-ends. 4. Empirically evaluated benefits (and drawbacks) of non-local dead-end reasoning and the online approach for realizing APPs

11

Thank you !

12

References Giuseppe De Giacomo, Fabio Patrizi, and Sebastian Sardina. Agent programming via planning programs. In Proc. of AAMAS, pages 491–498, 2010. Giuseppe De Giacomo, Alfonso Gerevini, Fabio Patrizi, Alessandro Saetti, and Sebastian Sardina. Agent planning programs. Artificial Intelligence, 231:64–106, 2016. Alfonso Gerevini, Fabio Patrizi, and Alessandro Saetti. An effective approach to realizing planning programs. In Proc. of ICAPS, pages 323–326, 2011. Nir Lipovetzky, Christian J. Muise, and Hector Geffner. Traps, invariants, and dead-ends. In Proc. of ICAPS, pages 211–215, 2016. 13