                    REVIEW OF THE STRATEGY TESTS

The purpose of this review is to collect comments about the 
various tests, particularly to classify which ones must be fixed 
before 3.0 release, and to track which ones pass for which
versions of GNU Go.

2.7.178 fails the following tests:

strategy.tst  # 8, 11, 17, 18, 20, 23, 34, 38, 40, 41, 43, 44, 45, 48
strategy2.tst # 52, 54, 55, 59, 62, 65, 67, 68, 69, 70, 78, 80
neurogo.tst # 2, 3, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24
nicklas.tst # 102, 401, 502, 601, 602, 701, 802, 803, 804, 805, 809, 810, 811,
              812, 902, 903, 1002, 1003, 1005, 1006, 1101, 1102, 1103, 1106,
              1108, 1110, 1201, 1203, 1204, 1205, 1206, 1207, 1208, 1211, 1212
              1214, 1215, 1217, 1301, 1304, 1401, 1402, 1403, 1404, 1405
              1407, 1408, 1501, 1502, 1503, 1507, 1508, 1510, 1512

2.7.205 fails the following tests:

strategy.tst  # 11, 15, 17, 18, 20, 34, 36, 44, 45, 48
strategy2.tst # 52, 65, 67, 68, 70, 80, 82, 83, 84, 86, 87, 88, 90, 93, 94,
                98, 99, 100, 101, 103, 105, 108, 109, 110, 111
neurogo.tst # 3, 4, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 19, 29, 21, 22, 24
nicklas.tst # 102, 401, 601, 602, 701, 802, 803, 804, 805, 809, 811, 812,
              902, 903, 904, 1001, 1102, 1103, 1106, 1107, 1111, 1201, 1203,
              1204, 1205, 1206, 1211, 1212, 1215, 1217, 1401, 1402, 1403,
              1407, 1501, 1701, 1802, 1803, 1904, 2001, 2002, 2103, 2104,
              2201, 2201, 2301, 2302, 2303, 2305


                           STRATEGY.TST

#11: ACCEPTABLE
gnugo -l incident108b.sgf --quiet -L178 -t

    2.7.178 makes the move at P17 which is better than the game
    move at P18 but probably inferior to the recommended move at P15.
    It is not totally clear to me that the move at P15 is really
    best since the 3-3 point is still open. 

#15: ACCEPTABLE.
gnugo -l incident121.sgf --quiet -L60 -t

     It is unclear that the recommended E9 is the best move. 
     W cannot cut at E11, or even threaten to cut and keep sente. 
     GNU Go 2.7.205 plays at G10 which is a good move.

#17 HIGH PRIORITY; EASY
gnugo -l incident165.sgf --quiet -L108 -t

    2.7.179: This can surely be fixed with the addition of a
    pattern. After N11, the strings at M12 and N10 cannot both be 
    saved. This is not necessarily the biggest move on the board but 
    at least this move should be found.

# 18 LOW PRIORITY; DEFER Reading code error
gnugo -l incident165.sgf --quiet -L142 -t

    2.7.179: Fix reading 124 first.

# 20 HIGH PRIORITY; UNKNOWN DIFFICULTY; Tuning
gnugo -l incident290.sgf --quiet -L30 -t

    2.7.179: Owl code finds Q11 but it remains undervalued at 17 points.

# 23 MODERATE PRIORITY; UNKNOWN DIFFICULTY; Reading code horizon effect
gnugo -l incident291.sgf --quiet -L88 -t

    2.7.179: Backfilling to create a group with 5 liberties. What should
    be done about this?

    2.7.205: This test passes.

# 27 MODERATE PRIORITY; UNKNOWN DIFFICULTY. Tuning.
gnugo -l strategy5.sgf --quiet -L14 -t

    This test passed before 2.7.183, but for the wrong reason: 
    the move at C3 was allegedly an owl attack/defense point for 
    the dragon at D3, which of course it is not.

    Thus this example should be regarded as having been already
    broken.

# 31 MODERATE PRIORITY; UNKNOWN DIFFICULTY. Debugging.

    This was broken in 2.7.183. We should try to find out
    the reason.

# 34 HIGH PRIORITY; EASY; Tuning
gnugo -l nicklas8.sgf --quiet -L72 -t
    
    2.7.179: This can be fixed with the addition of a J pattern.

# 36
Not yet analyzed

# 38 MODERATE PRIORITY; DIFFICULT; Optics
gnugo -l nicklas8.sgf --quiet -L86 -t

    2.7.179: There may be a way to fix this but it would require
    some coding. Some patterns in eyes.db can be either 2 eyes or 1
    eye depending on the position, and this is one of them. Such a
    position could trigger the life code which is currently turned
    off by default.  The relevant pattern is 501 or more precisely
    .XXX. The point is that only one of the . is actually a liberty.

    2.7.205: This test passes.

# 40 MODERATE PRIORITY; UNKNOWN DIFFICULTY; Reading and Tuning
gnugo -l nicklas8.sgf --quiet -L98 -t

    2.7.179: Three spurious defenses are found for D19, and E5 is
    undervalued at 11 points.

    2.7.180: after adding patterns CB326 and CB327, E5 is worth
    19 points, which seems about right. The spurious defenses
    for D19 remain.

    2.7.205: This test passes.

# 41 LOW PRIORITY; DEFER; Reading and Tuning (Fix 40 first)
gnugo -l nicklas8.sgf --quiet -L106 -t

    2.7.179: After fixing strategy 40, the spurious defenses of D19
    will be removed. Now N16 will be the most valuable move. So 
    this should be fixed automatically when 40 is fixed.

    2.7.180: this test passes.

# 43 MODERATE PRIORITY; MODERATE DIFFICULTY; Defend against double atari
gnugo -l strategy10.sgf --quiet -t

    2.7.179: Moves are proposed which save the A13 worm. But it is
    overlooked that these do not defend the double atari at B13. 
    This is probably not hard to fix though I'm not exactly sure 
    what the best method is.

    2.7.205: This test passes.

# 44 MODERATE PRIORITY; DIFFICULT; Owl mistake, Semeai
gnugo -l strategy11.sgf --quiet -L77 -t

    2.7.179: Whether or not the recommended move at G2 is found, the 
    move at A6 is egregiously bad. In this situation an owl critical 
    dragon adjoins an owl dead one. We need to revisit our policy for 
    such situations. The reading code thinks the worm at B4 can be 
    defended, so this is an owl lunch.

    2.7.180: After revision of semeai.c, the move at A6 is no longer
    found. Now GNU plays at M16 on the top. Arguably stabilizing
    the bottom at G2 or (worse but still OK) at J2 is better but
    I would class this result as ACCEPTABLE.

# 45 HIGH PRIORITY; DIFFICULT; Tuning, Semeai, Owl mistake
gnugo -l strategy11.sgf --quiet -L127 -t -d0x040

    2.7.179: What's with the enormous valuation given to N8?  This
    needs investigation. Apart from that, the move at C1 is found
    and this may be worthwhile.  This is another situation where an
    owl critical dragon adjoins an owl live one, but in this case
    the semeai seems worth fighting, unlike test 44. There is
    another owl mistake at the top, since the dragon at A18 still
    has some life in it.

    2.7.180: After revising semeai(), the move at C1 prevails.
    The heuristics give both dragons in the semeai at bottom six
    liberties, so GNU chooses to fight.

    2.7.204: C1 gets value 53, acceptable since GNU plausibly
    regards it as a semeai move. The recommended move G17 gets 
    value 36, also acceptable. What is bad is that N8 gets value
    47 broken down as follows:
    
    N8: 86.000000 - block or expand territory
    N8: -43.090000 - influence
    N8: 4.398273 - shape (shape values +2.00(1) -0.00(0), shape factor 1.102)

# 46 HIGH PRIORITY; EASY; Tuning
gnugo -l strategy12.sgf --quiet -L19 -t

    2.7.179: The highest value found is 22, so this could
    be fixed with the addition of an appropriate J pattern. 
    B17, B14 or C14 all seem acceptable moves. (C14 involves 
    a sacrifice since W may cut on either side, after which 
    B captures the cutting stone. This is a takamoku joseki.)
    B17 is the simplest idea, maybe best for now.

# 48 and #49; MODERATE PRIORITY, UNKNOWN DIFFICULTY.

    Broken in 2.7.183.

                         STRATEGY2.TST

# 52 ACCEPTABLE; Tuning
gnugo -l strategy14.sgf --quiet -L33 -t

    2.7.179: It may be better to atari at H5, then
    connect at J3. On the other hand the descent to H2
    is a good move in the right part of the board. This
    does not seem to be a priority for fixing. if we
    want to change this behavior it can be done by
    adding a pattern.

# 54 MODERATE PRIORITY; MODERATE DIFFICULTY; Owl Tuning
gnugo -l strategy14.sgf --quiet -L81 -t

    2.7.179: The recommended move at P6 is found with
    value 24, reported as an owl attack for Q6. It
    isn't noticed that this is also relevant to the
    life of P4, since B:O5 would be a strong attack.
    But the real problem in this position is that B9
    is reported as an owl attack on D9. In fact, 
    this move has little to do with the life of
    that dragon, which has some escape potential.
    Better to seal it in with E10 or (not as good) F9.

    2.7.205: This test passes, but not analyzed why.

# 55 HIGH PRIORITY; EASY; Tuning
gnugo -l strategy14.sgf --quiet -L125 -t

    No owl code involvement here. The simple extension
    to C12 is better than C14, B14 or B13, other moves
    GNU likes. This can be fixed by tuning.

    2.7.180: This test passes.

# 59 HIGH PRIORITY; UNKNOWN DIFFICULTY' Debugging
gnugo -l incident278.sgf --quiet -L322 --decidedragon A6 -o vars.sgf
gnugo -l incident278.sgf --quiet -L322 -t

    Apparent inconsistency between what decidedragon reports
    and what value_moves() reports. Decidedragon reports that
    A6 can be attacked at B1 and cannot be defended, but
    move generation assigns value 55 to C1 for owl defense of
    A6. The correct point of defense is B1.

    2.7.180: This test passes.

# 62 MODERATE PRIORITY; EASY TO MODERATE DIFFICULTY; Tuning
gnugo -l incident169.sgf --quiet -L18 -t

    The worm at C6 is not cutting, adjoins a large, live
    dragon, and saving it should be a low priority. The
    move to save it at D6 is overvalued at 25 points.

    2.7.205: This test passes.

# 65 LOW TO MODERATE PRIORITY; EASY TO MODERATE DIFFICULTY; Tuning
gnugo -l incident169.sgf --quiet -L110 -t

    In this position the clamp at M17 is a better way of
    immobilizing the L17 stone. The trick here will be
    to write a pattern with just the right amount of
    generality.

# 67 MODERATE PRIORITY; MODERATE DIFFICULTY; Owl tuning
gnugo -l incident169.sgf --quiet -L154 -t

    It may be best to steal an eye with sente at F14
    before disconnecting black at N13 or thereabouts.
    GNU misses the attack but takes profit at Q19.

# 68 LOW PRIORITY; DEFER
gnugo -l incident169.sgf --quiet -L154 -t

    Fix test 67 first.

# 69 MODERATE PRIORITY; EASY
gnugo -l incident169.sgf --quiet -L300 -t

    By filling a liberty of a dead dragon, GNU gives away
    a point. Presumably this could be fixed by replacing
    dragon.status by dragon.matcher_status in 
    analyze_neighbor() in filllib.c.

    2.7.205: This test passes.

# 70 ACCEPTABLE
gnugo -l strategy17.sgf --quiet -t

    When this test was first put up, the defensive moves
    at M15, N16, ... were drastically undervalued. This
    is no longer the case. Since it may be difficult to
    save both K16 and M18, the move at P16 is acceptable.

    2.7.180: This test passes.

# 75 and 76 ACCEPTABLE
gnugo -l semeai2.sgf --quiet -L150 -t -d0x040
gnugo -l semeai2.sgf --quiet -L152 -t -d0x040

    There are two semeai on the board. One can be won with F7
    and one can be won with Q11. So the question is which is
    bigger. GNU Go gives Q11 a large value but F7 a larger
    one. This is because in the Q11 semeai, GNU accurately
    finds that the B group at O16 cannot be killed. After
    W defends at Q11, B can make two eyes.

    Actually Q11 is a better move since it is sente. If W
    takes it, B defends, then W can take the other semeai,
    so the test is still broken but within the acceptable
    range.

# 78 HIGH PRIORITY; EASY TO MODERATE; Tuning or owl tuning
gnugo -l strategy21.sgf --quiet -L96 -t

    The dragons at C3 and F4 are not amalgamated, despite
    the fact that E2 is not safe for W. No patterns are 
    matched at H2. If a C pattern is matched at H2, and
    if the C3 and F4 dragons are amalgamated, this mistake
    may be fixed. If this is not sufficient, add a 
    pattern to owl_defendpats.db.

    gf: The C3 and F4 dragons are not amalgamated because they are not
        tactically safe. Pattern Conn509 does match at H2 but no
        connection move reason is generated due to a reading error,
        added as reading test case 131.

    2.7.205: iw: This test passes.

# 80 HIGH PRIORITY; DIFFICULTY UNKNOWN; Debugging

    Investigate why the cut/connection at Q4 is undervalued.
   
    2.7.180: This test passes.

# 86 LOW PRIORITY; DIFFICULT; semeai problem

    This requires a semeai module capable of reading. The correct
    sequence starts with a double throwin to destroy both an eye and
    some liberties. We may want to move this test case to a separate
    semeai test suite.

# 87 HIGH PRIORITY; DIFFICULTY UNKNOWN; Debugging

    Atari_atari finds an atari sequence starting with an atari (N3) on
    a dead stone. Of course white wouldn't answer this atari.

    2.7.214: This test passes

# 90 MEDIUM PRIORITY; DIFFICULTY UNKNOWN; Tuning

    B14 gives a connection underneath. Although this depends on a ko,
    it still looks like the best move. This move reason is currently
    not found. In the corner B17 is valued higher than D17, which is
    unreasonable. This is caused by the huge shape value for EB1307,
    which seems most inappropriate here.

# 91 MEDIUM PRIORITY; DIFFICULTY UNKNOWN; Tuning

    B14 is missing here too. At least it no longer plays the game move
    at B13 but instead the reasonably good move at D17.

# 93 MEDIUM PRIORITY; DIFFICULTY UNKNOWN; Tuning, owl tuning

    D11 is urgent, but the engine is far from appreciating how
    important this move is. Also the owl tuning thinks G13 will kill
    the white stones, which seems much too optimistic.

# 95 HIGH PRIORITY; DIFFICULTY UNKNOWN; Detect blunder

    The move at P13 enables a white owl attack with ko. The obvious
    solution is to enhance confirm_safety() to also consult the owl
    code and possibly the atari_atari module.

# 96 LOW PRIORITY; DIFFICULTY HARD; Owl threaten defense mistake

    O1 is assumed to eliminate some threat to live, but is totally
    ineffective and seems just to improve white's chances. Fixing it
    probably requires significant revisions of the
    owl_threaten_defense() function.

# 99 HIGH PRIORITY; DIFFICULTY HARD; Bad amalgamation

    Q2 is completely inessential and saving it should have no value at
    all. This would be solved if Q2 were classified as an inessential
    dragon, but unfortunately it's amalgamated with the black strings
    on the outside.

# 100 HIGH PRIORITY; DIFFICULTY UNKNOWN; Suboptimal defense

    M16 is the only move since it blocks off territory and defends the
    tactically threatened O13 stones. The main problem is that the
    engine never discovers that M16 is a tactical defense of O13.

# 101 HIGH PRIORITY; DIFFICULTY HARD; Suboptimal defense

    The atari_atari module spots an atari sequence starting with M18
    and therefore suggests playing there. The correct move at M16 also
    stops the atari sequence, but this is never tested.

# 102 LOW PRIORITY; DIFFICULTY HARD; Endgame mistake

    N18 is about one point better than O19, but this is beyond the owl
    code to understand. The reason for this problem is that when the
    owl code has come up with one move, further moves are never
    tested. If it was examined whether N18 also owl defended O18, the
    rest of the move valuation should be able to make the right
    choice.

# 103 HIGH PRIORITY; DIFFICULTY UNKNOWN; Owl tuning, might require
      semeai assistance

    Proper solution probably requires reading semeai code, since
    solutions giving up some stones or ending in seki are not
    acceptable.

# 104 HIGH PRIORITY; EASY; Tuning

    P4 is mandatory and urgent.

    2.7.214: This test passes

# 105 HIGH PRIORITY; DIFFICULTY UNKNOWN; Tuning, reading mistake

    There might be bigger moves on the board, but J2 is in every way
    superior to G2. The situation is complicated by the fact that the
    tactical reading fails to see that J2 captures H2 (reading test
    case 134). In the mean time an antisuji pattern for G2 might help.

# 106 HIGH PRIORITY; EASY TO MODERATE; Tuning

    F10 is immediately big and both colors have big followup moves;
    double sente.

    2.7.214: This test passes

# 107 HIGH PRIORITY; EASY; Tuning

    No reason not to block directly at F1.

    2.7.214: This test passes

# 108 HIGH PRIORITY; DIFFICULTY UNKNOWN; Incorrect amalgamation, semeai

    Block and connect at C15 is the only move.

    2.7.214: This test passes

# 109 HIGH PRIORITY; DIFFICULTY HARD; Semeai mistake

    Black can make seki by filling an outer liberty. This problem
    looks straightforward enough that the semeai module should be able
    to solve it. The problem is that D14 is amalgamated with F6
    because the tactical reading fails to see that there is a defense.
    Might be something that small_semeai should take responsibility
    for, or possibly the reading code itself.

# 110 HIGH PRIORITY; DIFFICULTY UNKNOWN; Semeai mistake

    B12 is considered inessential, which of course is unreasonable.

# 119 LOW PRIORITY; HARD; Connection mistake

    A correct analysis of this position requires understanding that
    C10 and E9 are not connected, which hardly can be done without a
    dynamic connection reader.

    2.7.210: The fully acceptable kikashi at J3 would prevail if only
        pattern CD97 didn't award L7 the completely unreasonable value
        of 30.

# 124 LOW PRIORITY; HARD; Missed chance to seki

    A fully working semeai module should find the sequence B C5, W B5,
    B C2, which seems to lead to seki.

# 125 HIGH PRIORITY; DIFFICULTY UNKNOWN; Missed atari sequence

    The atari_atari module should be capable of finding the weakness
    at K17.

# 128 HIGH PRIORITY; PROBABLY EASY; Owl reason not added

    The owl attack at O8 of Q5 is disqualified by semeai-detecting
    code in owl_reasons.c. Obviously the reasoning is wrong here since
    O8 also happens to owl-defend the critical neighbor. It shouldn't
    be too hard to detect this case, but maybe a better solution is to
    take down the semeai-detecting code entirely, since the semeai()
    module also consults the owl code now.

# 139 MEDIUM PRIORITY; DIFFICULTY UNKNOWN; Semeai mistake

    Black is way ahead in the semeai and can not allow white to escape
    by cutting at Q9. This shouldn't necessarily require reading to
    understand.


    2.7.214: It finds the move K17.
