This TURF helpfile describes the REACH.RESULTS and                      
        FREQ.RESULTS files which are produced by the TURF command.              
        This section was last updated on April 8, 2013.                         
                                                                                
        The REACH.RESULTS file shows the combinations that had the              
        best reach scores; the FREQ.RESULTS file shows the combinations         
        that had the best freq scores.                                          
                                                                                
        Most TURF runs will request a reach.results file.                       
                                                                                
              *****************************************                         
              *  in a STEP run, should the stepping   *                         
              *  be done to maximize REACH or FREQ ?  *                         
              *****************************************                         
                                                                                
        A STEP run must have either a reach.results file or a                   
        freq.results file, but not both.                                        
                                                                                
        If reach.results is in use, the best combination on reach               
        in each step is fed into the next step, and so on.                      
        As a result, the stepping will maximize the reach score                 
        in the final step.                                                      
                                                                                
        If freq.results is in use, the best combination on freq in              
        each step is fed into the next step, which maximizes freq.              
        Thus, the choice of results file indicates which of reach               
        or freq is to be maximized.                                             
                                                                                
              ****************************                                      
              *  the REACH.RESULTS file  *                                      
              ****************************                                      
                                                                                
        REACH.RESULTS rrr,                                                      
        REACH.RESULTS rrr 300,                                                  
        REACH.RESULTS rrr 5,                                                    
                                                                                
                 This is an optional output p-stat system file.                 
                                                                                
                 This file holds the combinations with the best                 
                 REACH values.                                                  
                 The combinations are in descending order on REACH.             
                 Within ties on reach, the combinations are in                  
                 descending order on FREQ.                                      
                                                                                
                 Ordinarily, the items in a combination are ordered             
                 by the reach contribution that each in turn adds.              
                 That includes FORCE items, if any.                             
                                                                                
                 However, when FORCE.FIRST is used along with FORCE,            
                 the force items are placed first, in the order in              
                 which they themselves were selected.                           
                                                                                
                    HOW MANY COMBINATIONS SHOULD BE WRITTEN ?                   
                                                                                
                 The default is to write the 100 best combinations              
                 for each size.                                                 
                 If an integer like 300 follows the file name,                  
                 then the requested 300 are written for each size.              
                                                                                
                 STEP runs, however, are treated differently.                   
                                                                                
                 In STEP runs (which must have 2 or more sizes)                 
                 the default is to write only the single best                   
                 combination at each step until the final step,                 
                 when the best 100 (or 300 or whatever) are written.            
                                                                                
                 This makes the cascading effect of the items in a              
                 STEP run much easier to see.                                   
                                                                                
                 This STEP default however can be overridden:                   
                 using SHOW.ALL.COMBOS (see below) causes the                   
                 requested amount of combinations to be written                 
                 for EVERY step.                                                
                                                                                
                    SELECTING WHICH STATS LINES ARE SHOWN:                      
                                                                                
                 The default, for each combination shown, is to                 
                 write two lines to the reach.results file.                     
                                                                                
                 The first line identifies the combination.                     
                 It provides the names of the items.                            
                 This line also contains summary information about              
                 the combination: size.and.rank, reach, freq, etc.              
                                                                                
                 The default is then to write one stats line for the            
                 items within that combination, which shows the                 
                 cumulative reach as each successive item is added.             
                                                                                
                 There can be a total of 7 such stats lines, each               
                 showing the specific item contribution in some way.            
                 It is also possible to request none of them.                   
                 These choices are controlled by the REACH.STATS                
                 identifier, described below.                                   
                                                                                
                    IDENTIFYING THE ITEMS IN EACH COMBINATION:                  
                                                                                
                 The default is to use short names to identify the              
                 items (ie, variables) that make up a combination.              
                 Short names have just the item's tag or, if no tag,            
                 the initial 16 characters.                                     
                                                                                
                 The alternative is to use the identifier FULL in               
                 the TURF command. This causes the full names to be             
                 used, which could be as much as 64 characters each.            
                                                                                
                 The LIST command mitigates this extra width somewhat.          
                 When a value exceeds 32 characters, LIST will fold             
                 it into additional lines. A different fold control             
                 can be chosen, as in     LIST ABC, FOLD 24 $                   
                                                                                
                    SELECTING WHICH SUMMARY VARIABLES ARE SHOWN:                
                                                                                
                 The default is to place 5 summary variables into               
                 the REACH.RESULTS file. These are followed by the              
                 names of the variables in the combinations.                    
                                                                                
                   (1) SIZE.AND.RANK:  the current combination size             
                                       and the rank within it.                  
                                                                                
                   (2) REACH:  the reach score for the combination.             
                                                                                
                   (3) PCT.REACHED:  the percent of the sample that             
                                     is reached by this combination.            
                                                                                
                   (4) FREQ:  the freq score for the combination.               
                                                                                
                   (5) STATS:  identifies the extra lines.                      
                               For example, cum.r identifies the                
                               cumulative.reach line.                           
                                                                                
                   (6+) These variables contain the names, short or             
                        full, of the items that make up a combination.          
                                                                                
                        The item names are ordered by their                     
                        contribution to the reach score.                        
                        I.e., the name appearing under ITEM.1                   
                        is the 'best' item in the combination.                  
                        However, using force.first along with force             
                        will cause the first force item to be placed            
                        as ITEM.1, and so on.                                   
                                                                                
                        If sizes 6 and 8 are both being done,                   
                        the file will have item.1 through item.8.               
                        The results shown for size 6 will have                  
                        blanks for item.7 and item.8.                           
                                                                                
                 It is possible to select which (if any) of those               
                 initial five variables will be shown. This is done             
                 by using either the SHOW option or the OMIT option.            
                 These are described below.                                     
                                                                                
              *********************************                                 
              *  the REACH.RESULTS file:      *                                 
              *  when not using force.first,  *                                 
              *  the items in a combination   *                                 
              *  are ordered by importance    *                                 
              *********************************                                 
                                                                                
                 Suppose 6 items, AA, BB, CC, DD, EE and FF,                    
                 make up a combination about to be written                      
                 to the reach.results file.                                     
                                                                                
                 Before writing them, they are reordered so that                
                 the leftmost item is the one with the highest                  
                 individual reach.  The next item shown has,                    
                 when paired with the leftmost item,                            
                 the largest 2-item reach score, and so on.                     
                                                                                
                 The reordering is done in this manner.                         
                                                                                
                 FIRST PHASE: Find the smallest number of items                 
                 (taken from the combination now being re-ordered)              
                 that gets a a nonzero reach, and determine the                 
                 best item or group of items at that size.                      
                 In the simple case, that will be just one item.                
                                                                                
                 When the reach threshold is 3 and there are no item            
                 or response weights, for example, we will not get              
                 a nonzero reach until groups of three are tried.               
                 We take the best of all possible groups of three.              
                                                                                
                 SECOND PHASE: We now have one or more items as                 
                 the beginning items for the reordered combination.             
                 If more remain, try each one with the beginning                
                 items, and add the item that adds the most to the              
                 reach count. Repeat this step until all items are              
                 added.                                                         
                                                                                
                 Suppose we are reordering a combination of 6 items,            
                 AA, BB, CC, DD, EE and FF.                                     
                 As described above, we start by trying each item               
                 separately. If one (or more) have a nonzero reach              
                 score, we place the item with the best reach score             
                 into the first (leftmost) position.                            
                                                                                
                 Of the remaining items, the one that adds the most             
                 to the first one is placed in the second position,             
                 and so on.                                                     
                 That is simple and obvious to show in the reach                
                 results file.                                                  
                                                                                
                 For example:  DD    CC    EE    AA    FF    BB                 
                               43    62    71    78    81    82                 
                                                                                
                 However, suppose nonzero reaches did not begin                 
                 until groups of three were tried. If BB, DD and EE             
                 were the best such group, they are shown in input              
                 order, with the reach score for the group under                
                 the last one. The symbol  -->  appears under the               
                 initial items in that group.                                   
                                                                                
                 For example:  BB    DD    EE    CC    FF    AA                 
                               -->   -->   37    55    61    64                 
                                                                                
                 This shows that no reaches occurred until groups               
                 of three items were tried, and that BB-DD-EE were              
                 the best of all 3-item groups.                                 
                                                                                
                 Since AA, CC and FF remained, BB-DD-EE-AA was                  
                 tried, then BB-DD-EE-CC, then BB-DD-EE-FF.                     
                 The output shows that CC added more than the others,           
                 bringing the reach for those four items up to 55,              
                 and so on.                                                     
                                                                                
              ***************************************                           
              *  the   REACH.RESULTS   file:        *                           
              *  using  SHOW  to control            *                           
              *  which (if any) of the initial      *                           
              *  summary statistics should be used  *                           
              ***************************************                           
                                                                                
        SHOW  size  rank  pct.of.max.reach,                                     
        SHOW  none,                                                             
                                                                                
                 The default is for the reach.results and                       
                 freq.results files to have five summary                        
                 values before the items appear. These are:                     
                                                                                
                       SIZE.AND.RANK                                            
                       REACH                                                    
                       PCT.REACHED                                              
                       FREQ                                                     
                       STATS                                                    
                                                                                
                 The SHOW option overrides that default.                        
                 It specifies which (if any) of the above summary               
                 variables should be used. In addition, four                    
                 other choices are available.                                   
                                                                                
                   (1)  SIZE: the combination size as a separate                
                        variable.                                               
                                                                                
                   (2)  RANK: the rank within the current size,                 
                        also a seperate variable.                               
                        Note, the size.and.rank variable combines               
                        both size and rank to save space.                       
                                                                                
                   (3)  PCT.OF.MAX.REACH: the percent of active                 
                        cases reached by the combination.                       
                        An active case is a usable case that                    
                        has at least one positive response;                     
                        other cases cannot possibly be reached.                 
                                                                                
                   (4)  NONE: the only variables will be the items.             
                        This allows more items per page.                        
                                                                                
                 The SHOW phrase applied to both the                            
                 reach.results and the freq.results files.                      
                                                                                
              ***************************************                           
              *  the   REACH.RESULTS   file:        *                           
              *  using  OMIT  to control            *                           
              *  which (if any) of the initial      *                           
              *  summary statistics should be used  *                           
              ***************************************                           
                                                                                
        OMIT  size.and.rank  freq,                                              
        OMIT  all,                                                              
                                                                                
                 OMIT is simply another way to do what SHOW does.               
                 It will sometimes be easier to use OMIT than SHOW.             
                                                                                
                 The default is for the reach.results and                       
                 freq.results files to have five summary                        
                 values before the items appear. These are:                     
                                                                                
                       SIZE.AND.RANK                                            
                       REACH                                                    
                       PCT.REACHED                                              
                       FREQ                                                     
                       STATS                                                    
                                                                                
                 The OMIT option overrides that default.                        
                 It specifies which of the above summary variables              
                 should be omitted.                                             
                                                                                
                 OMIT ALL is allowed.                                           
                 This causes all of the summary variables to be omitted.        
                 This allows more combinations items to appear per page.        
                                                                                
                 The OMIT phrase applied to both the                            
                 reach.results and the freq.results files.                      
                                                                                
              ***************************************                           
              *  the   REACH.RESULTS file:          *                           
              *  using REACH.STATS to select        *                           
              *  which (if any) stats lines should  *                           
              *  be written for each combination    *                           
              ***************************************                           
                                                                                
        REACH.STATS cumulative.pct unique,                                      
                                                                                
                 When a reach.results file is written,                          
                 the combination identification line can be                     
                 followed by one to seven stats lines, or by none.              
                                                                                
                 As many as seven extra lines are possible:                     
                                                                                
                   (1) cumulative, the increasing reach as each                 
                       succesive item is added. This is the default.            
                                                                                
                   (2) separate, which has the additional                       
                       reach provided by each successive item.                  
                                                                                
                   (3) cumulative.pct, the percent of the                       
                       cases reached as each item is added.                     
                                                                                
                   (4) separate.pct, the additional percent of                  
                       cases reached by each successive item.                   
                                                                                
                   (5) cumulative.freq, the summed frequency                    
                       of the items so far.                                     
                                                                                
                   (6) additional.freq, the frequency added                     
                       by the most recent item.                                 
                                                                                
                   (7) unique.reach, for each item, the amount of               
                       reach achieved by the full combination that              
                       would be lost if that item were dropped.                 
                       However, unique.reach is provided only                   
                       when the reach.threshold is one.                         
                                                                                
                 REACH.STATS can be followed by:                                
                   (1) NONE  by itself, no stats lines are written.             
                                                                                
                   (2) ALL   by itself,  7 stats lines are written.             
                                                                                
                   (3) one or more of the 7 lines described above.              
                       The requested lines would be written.                    
                                                                                
              *************************************                             
              *  the REACH.RESULTS file:          *                             
              *  TURF can be flummoxed by small,  *                             
              *  carefully constructed data sets  *                             
              *************************************                             
                                                                                
                 It should be noted that selecting the best                     
                 two items in a stepwise manner is not quite                    
                 the same as selecting the best two by trying                   
                 all possible pairs.                                            
                                                                                
                 Suppose we have a file of 14 cases with                        
                 4 items: AA, BB, CC and DD.                                    
                 We would like to find the 'best' two items.                    
                                                                                
                 AA reaches cases  1-10,                                        
                 BB reaches cases 11-13,                                        
                 CC reaches cases  1- 5 and 11-12,                              
                 DD reaches cases  6-10 and 13-14.                              
                                                                                
                 The stepwise approach selects AA and, having                   
                 AA in hand, adds BB to get its best two items.                 
                 They have a reach of 13.                                       
                                                                                
                 A non-stepwise approach tries all combinations                 
                 of size 2 and would select CC and DD.                          
                 They have a reach of 14.                                       
                                                                                
                 The TURF command uses a stepwise procedure in                  
                 the REACH.RESULTS (and FREQ.RESULTS) reordering,               
                 and in full-fledged STEP runs.                                 
                 Otherwise all runs are done by trying every                    
                 possible combination of the size being analyzed.               
                                                                                
              ***************************                                       
              *  the FREQ.RESULTS file  *                                       
              ***************************                                       
                                                                                
        FREQ.RESULTS fff,                                                       
        FREQ.RESULTS fff 500,                                                   
        FREQ.RESULTS fff 5,                                                     
                                                                                
                 optional output p-stat system file.                            
                                                                                
                 This file holds the combinations with the best                 
                 FREQ values.                                                   
                 The combinations are in descending order on FREQ.              
                 Within ties on FREQ, the combinations are in                   
                 descending order on REACH.                                     
                                                                                
                 Ordinarily, the items in a combination are                     
                 ordered by the freq contribution that each                     
                 in turn adds. That includes FORCE items, if any.               
                                                                                
                 However, when FORCE.FIRST is used along with FORCE,            
                 the force items are placed first, in the order in              
                 which they themselves were selected.                           
                                                                                
                    HOW MANY COMBINATIONS SHOULD BE WRITTEN ?                   
                                                                                
                 The default is to write the 100 best combinations              
                 for each size.                                                 
                 If an integer like 500 follows the file                        
                 name, that many are written for each size.                     
                                                                                
                 The cascading effect in STEP reach.results files,              
                 described above, also occurs in freq.results files.            
                                                                                
                    SELECTING WHICH STATS LINES ARE SHOWN:                      
                                                                                
                 The default, for each combination shown, is to                 
                 write two lines to the freq.results file.                      
                                                                                
                 The first line contains the summary variables                  
                 (size.and.rank, reach, etc), along with the                    
                 names of the items that make up the combination.               
                 The items are ordered by their freq contribution.              
                                                                                
                 The second line shows the cumulative freq as                   
                 each successive item is added.                                 
                                                                                
                 There can be 2 such stats lines, each showing                  
                 the specific item contribution in some way.                    
                 It is also possible to request none of them.                   
                 These choices are controlled by the FREQ.STATS                 
                 identifier, described below.                                   
                                                                                
                    SELECTING WHICH SUMMARY VARIABLES ARE SHOWN:                
                                                                                
                 The FREQ.RESULTS file has the same summary                     
                 variables as the REACH.RESULTS file:                           
                 size.and.rank, reach, pct.reached, freq and stats.             
                                                                                
                 The SHOW and OMIT phrases, shown above, apply to               
                 freq.results as well as to reach.results.                      
                                                                                
              ***************************************                           
              *  the   FREQ.RESULTS file:           *                           
              *  using FREQ.STATS to select         *                           
              *  which (if any) extra lines should  *                           
              *  be written for each combination    *                           
              ***************************************                           
                                                                                
        FREQ.STATS  all,                                                        
                                                                                
                 When a freq.results file is written,                           
                 the combination identification line can be                     
                 followed by one or two stats lines, or by none.                
                                                                                
                 The default is to provide a line which shows                   
                 the CUMULATIVE freq as each item is added.                     
                 A line showing the SEPARATE freq added by each                 
                 item can also be requested.                                    
                                                                                
                 Thus, FREQ.STATS can be followed by:                           
                                                                                
                   (1) NONE  by itself,  neither line is written.               
                                                                                
                   (2) ALL   by itself,  both lines are written.                
                                                                                
                   (3) one or both of CUMULATIVE and SEPARATE.                  
                       The requested lines would be written.                    
                                                                                
              *****************************************                         
              *  REACH.RESULTS and FREQ.RESULTS:      *                         
              *  in a STEP run, write the full        *                         
              *  number of combinations at each step  *                         
              *****************************************                         
                                                                                
        SHOW.ALL.COMBOS,                                                        
                                                                                
                 When writing a reach.results or freq.results                   
                 file in a STEP run, the default is to write                    
                 only the best combination from the initial                     
                 steps, and then write the requested amount                     
                 (default 100) in the final step.                               
                                                                                
                 This makes the cascading effect of the items                   
                 in a STEP run easier to see.                                   
                                                                                
                 Using SHOW.ALL.COMBOS causes the requested                     
                 amount of combinations to be written                           
                 from EVERY step.