Module 15: Evaluating and Adjusting the Plan

Module Overview

In Module 15, we will discuss how to go about evaluating your plan to see if it is working and if any changes are needed. At this point, you should be at least one week into treatment and so should have some data to work with.

Module Outline

• 15.1. Evaluating Your Plan
• 15.2. Adjusting the Plan
• 15.3. Generate a Graph of Your Data
• 15.4. Treatment Phase Token Log

Module Learning Outcomes

• Describe general strategies for evaluating your plan.
• Identify statistics to summarize your data.
• Construct a line graph to plot your data (learn how to do this and then demonstrate it for your final paper).
• Discuss ways to adjust your plan.

15.1. Evaluating Your Plan

Section Learning Objectives

• Clarify what to do if you have to change your target behavior.
• Name and discuss other situations that may necessitate adjusting your plan.
• List ways to numerically show your plan’s progress.
• Describe how to graph your data.

15.1.1. Changing the Target Behavior

As you are going through your plan it is a good idea to see how you are doing. Fortunately, you are collecting a great deal of data and so have all you need to make a determination. Though I sincerely hope this does not occur, the issue you discover right away, or within a week or so, is that you selected the wrong target behavior. You might be wondering how you could possibly make such a mistake. Maybe you are trying to increase the amount you work out but as you go through your plan you discover that you do regularly work out. Now you could add a day or work out longer possibly, but your issue is not having enough motivation to work out. The reason why your weight loss endeavor is not going anywhere is because you have crappy eating habits. I fall in this boat as I love going to the gym but fail to lose weight because I do not eat as healthy as I should. My biggest problem is portion sizes and so a behavior modification plan to work out more really does not benefit my actual long-term goals. I would need strategies linked to monitoring my caloric intake daily. I might also notice that stress and sleep are problems and change both to help with weight loss. Again, about a week into my plan I may realize that the target behavior needs to change which could result in creating a brand-new plan and quickly, as well as establishing a new baseline. You may have the same issue arise and if so, consult your instructor as soon as possible. Begin working on all 8 planning sheets for the new target behavior so at least your instructor can see you making progress toward adjust the whole plan.

15.1.2. “Seeing” Your Progress

For most of you, changes as extreme as developing a brand spanking new plan will not be needed, but you may not feel you are making progress either. It’s likely more an issue of not making progress as quickly as we would like, but progress is being made. How can you tell?

     15.1.2.1. Strategy 1: Weekly counts. You can use a few different strategies, or all of them, but the first thing you should do is get a count for each week of the treatment phase. How so? If you are trying to increase your water consumption, you might have recorded the following data for your first two weeks of the plan:

Table 15.1. Example Weekly Counts Log

Say your Week 1 goal was to drink at least 8 oz. of water each day and your Week 2 goal was to drink at minimum 16oz. of water each day. As you can see, you did well in Week1 but not as well in Week 2. We can see this from just eyeballing the data, but how much water did we actually drink each week? Our Week 1 Total is 64oz. and our Week 2 Total is 88oz. By simply counting the total amount drank each week (through simple addition), we can see a trend in the upward direction.

     15.1.2.2. Strategy 2: Daily average. We could also calculate an average or mean of total water drank each day. To do this, take the total and divide it by 7 days. Upon doing this we see that the Week 1 average is 9.14 oz., and the Week 2 average is 12.57 oz. The advantage of the average is that daily fluctuations are not as apparent, especially for Week 2, and the data smooths out.

     15.1.2.3. Strategy 3: Percentage of opportunities. We could also calculate a percentage of opportunities to gauge our success for the week. In this case, we would want to divide the total amount of water drank for the week by the total amount of water that could have been consumed. Based on our goals, we know we were shooting for 8oz. of water a day for 7 days in Week 1, or 56 total ounces, and in Week 2, we should have drunk 112 oz. (16oz. x 7 days). Since we drank 64 oz. in Week 1, we were at 114% success rate while in Week 2 we drank 88oz. which was a 78.57% success. Visual inspection of the data again shows us achieving our goal each day in Week 1, and even exceeding on Thursday, while in Week 2 we drank the required 16oz. on four days and half that for three days. Something went wrong in Week 2 and by going back and evaluating our data for Wednesday, Saturday, and Sunday, we can figure out what we did wrong. Maybe we found that on Wednesday, we left our water bottle home and so drank less as a result. Once home we drank a glass of water but did not want to drink 2 since it was close to bedtime. On Saturday and Sunday, we noticed that we were out with friends a lot and did not have a water bottle with us, and also drank soda or tea instead. Armed with this information, we can adjust our plan by using antecedent manipulations such as presenting the cue for the desirable behavior and setting our phone to give us a reminder or cue to drink, placing a water bottle in the car in case we run out which is reducing the effort for the desirable behavior, taking a water bottle to the gym with us when we work out which is an establishing operation, having a friend remind us to drink water which is social support and a verbal prompt, developing a plan for dealing with the temptation to drink soda or tea when out with friends, and keeping change on hand in case we need to buy a water bottle at school (reducing effort for making desirable behavior).

     15.1.2.4. Strategy 4: Counting behaviors. As a note, you can also count the number of behaviors that were made. Your behavioral definition likely was 1 behavior = drinking 8 oz. of water. As such, in Week 1 you made 8 total behaviors and in Week 2 you made 11 behaviors. The trend is upward but should have been 7 behaviors in Week 1 to 14 in Week 2. The extra behavior in Week 1 makes the gap narrower but still, we were not as successful as we should have been in Week 2. Outside of the strategies mentioned above, you might also revisit your criterion. Maybe we should have held at 8 oz. for two weeks before moving from Goal 1 to Goal 2.

     15.1.2.5. Strategy 5: Counting and graphing tokens earned. Another option is to find out how many tokens you earned from one week to the next. If you are making the desired behavior you should see an increase from week to week, but in the case of our example, Week 1 likely had a good number of tokens but Week 2 had not as many as it should have, and you may only have been slightly above the Week 1 total if you are using response costs for engaging in the undesirable behavior of drinking soda or tea.

These strategies to numerically summarize your data do not work for all behaviors. Determine which ones you need based on what your target behavior is and how you operationally defined it. Look at your goals and token economy too. Lastly, be sure you examine your journal each day. As I noted earlier, you may be showing progress, but not as fast as you would like. That is the case in the example I gave and analyzing why you did not meet your goal some days can identify problems with your plan, temptations you did not account for, or mistakes requiring new or clearer rules.

Be advised that for your final paper, you will be asked to use all relevant evaluation strategies given above. For most of you, this will require all 5 descriptive statistics/strategies.

     15.1.2.7. Treatment phase summary table. As during the baseline phase, you will also want to summarize your ABC charts which pools the data from the individual ABC charts for each day into one place. Record how much of the behavior you made on this table, not behavioral counts. You can conduct all analyses using this table. Note that fall and spring classes have three weeks in the treatment phase while summer courses will have two due to the shorter time the course lasts.

Table 15.2. Treatment Phase Summary Table

So, for your final project you will submit a Baseline ABC chart and a Treatment Phase ABC chart.

15.1.3. Graphing Your Data

15.1.3.1. Target behavior graph. One final way to evaluate your data is to graph it. You can do this for the behavior, whether totals, counts, averages, or a percentage; and for your token economy. To graph, you need to do the following:

1. Draw a horizontal line on your paper. This is the x-axis and you will record time on it. This could be on days or weeks, depending on how your project is set up. If weeks, you will have three tic marks. If days, you will have 21 tic marks for the three weeks of the treatment phase. Behaviors such as drinking water or achieving daily goals are best measured in days on the x-axis but if you are trying to increase the duration of workouts from week-to-week, you are best measuring in weeks. Why? If you measure days you will have some with 0 and others with the amount of time you worked out. This will look strange and as if you failed on some days and not on others, when all the 0 really means is that you did not work out that day, and were likely not scheduled to do so. You will also include baseline data on the x-axis. More on this in #5.
2. Draw a vertical line that begins at the zero point of the x-axis, or is perpendicular with it. This is the y-axis and you will record your behavior on it, depending on whether you are using totals, counts, averages, or a percentage. In the case of working out, if you are trying to increase the number of days, or frequency, of the target behavior of working out, record counts or the number of days each week that you worked out. If you are trying to increase the time, you can record the average time each week, in minutes.
3. Plot your data on the graph.
4. Connect dots from time point to time point. If your plan is generally a success, you will see an upward trend.
5. On this graph you should also plot your baseline data (the two weeks or 14 days of data you collected when doing your functional assessment and practicing your data collection). This will go before the treatment weeks and serve as a comparison. So you are not only looking for an upward trend from week-to-week during treatment, but overall, you should see an upward trend from baseline to treatment phase.
6. Your baseline to treatment transition should be indicated by a vertical dotted line in the graph itself.
7. You should also use phase labels at the top of the graph – one for Baseline and a second for Treatment.

A sample line graph is given below. Note that it is best to create this graph in Microsoft Excel. Though there is a way to create the phase line and labels in Excel, it may be best to:

1. Take a screen shot of your finished graph
2. Move it to Microsoft PowerPoint.
3. Create the line and labels in PPT.
4. Once done, be sure to group the entire finished graph with phase labels and the line.
5. Copy and paste into your final paper in the appropriate Appendix.

Figure 15.1. Example Line Graph

Again, your line graph can be easily produced in a spreadsheet program such as Excel, though statistical packages such as SPSS and SAS work too. The example above was produced in Excel and PowerPoint. Notice all key elements as described above, to include the x-axis and its labels, the y-axis and its labels, the data that was plotted and line connecting the dots, the phase line, the phase labels, and the figure title at the top.

Sometimes, it is not necessary to start at ‘0’ on the vertical axis such as in the case of plotting your weight across time. In this case, break the line which indicates you are not starting at ‘0.’ This will appear as follows:

Figure 15.2. Example Y-Axis When It Does Not Start at ‘0’

     15.1.3.2. Tokens graph. You can produce a graph for your behavior but also your token economy results. In this case, there is no baseline data to report since you did not use the token economy at this time, and so the phase line and labels are not needed. This will look like a standard graph and if your plan was generally successful, the line should slope upward from Treatment Phase Week 1 to 3. Submit it with your main graph so you can get feedback from your instructor. The graph should be straight forward.

15.2. Adjusting the Plan

Section Learning Objectives

• Generate solutions to common problems with a behavior modification plan.
• Clarify what the next steps are.

15.2.1. Common Problems and Possible Solutions

If you are at the point of needing to adjust your plan, you need to figure out what is making your plan difficult to achieve. Likely, you will have at least one issue with your antecedents, behavior, and/or consequences. Look closely at the data you gathered and your notes in the journal. Do you see a pattern? Is there a temptation you did not account for? Are you not keeping good enough records? With this in mind, let’s discuss some common issues that can be fixed (we discussed some of these in 15.1.):

1. Unrealistic Goals – Maybe your goals are just not realistic. Your distal goal may be fine, but your proximal goals along the way are too large leading to failure, or too small, leading to frustration. If too large, create more goals that are smaller and easier to achieve. Instead of going from working out 2 days to 4 days, workout 2, then 3, and then 4 days. Instead of increasing your speed on the treadmill from 5.5mph to 6pm in a matter of weeks, take baby steps and increase by 0.1 mph each week. For water consumption, move from 8 oz. to 12 oz. instead of 8 oz. to 16 oz. This subtle change may make all the difference.
2. Faulty Criterion – It is possible your goals are fine, but your criterion has you moving to the next goal too soon after the previous one is achieved. Instead of moving to 3 days working out from 2 days after just one week achieving this, increase your criterion to maintaining the current level for 2 weeks. Likewise, you may have too large of a criterion such that you are holding fast for 2 weeks when you can advance after just one week. It might be for the lower goals you can have a 1-week criterion but as you get to upper tier goals nearing your final goal, you increase the criterion to 2 weeks. This is especially important with any type of workout goal. You don’t want to risk injury so play it safe, without being so safe you become bored. Consult a trainer or your physician if you are unsure when it is safe to move up and avoiding plateauing.
3. Record Keeping Issues – If your method of record keeping does not work for you, find another way. If you cannot remember to record, ask a friend or loved one to help. Use your phone to give you reminders. It could be that you are recording, but not making quality entries in your journal or ABC chart. If this is the case, then be sure you allow yourself enough time to reflect on your target behavior for the day. You might discover that a great time to do this is while you are on the bus going home from school or right after dinner. Make any adjustments you think are necessary.
4. Problems with Strategies – If a strategy is not working, either try and adjust how you are using it or just drop it. Maybe an antecedent manipulation is not working out such as in the earlier example of leaving your clothes by your bed. Because you wake up before the sun rises you don’t see anything until you enter the bathroom. Move your clothes to the bathroom so they adequately cue you, or remind you, to work out. Make sure your prompts are helpful, your token economy awards tokens at a steady rate and that your back up reinforcers are not too high priced or too inexpensive, you are using the correct type of differential reinforcement, and you are not either underusing or overusing punishment procedures. If you are unsure if you should change a strategy, consult your instructor.
5. Unrecognized Temptations – We may be unaware of a temptation that causes us to engage in the undesirable behavior or another situation or place that leads to the temptation. If you discover one, develop an If-then plan to deal with it.
6. Inadequate or Unclear Rules – It could be your rules need work too. Look over each from the proposal and Planning Sheet 8 and see where adjustments are needed.
7. Social Support isn’t Really Supportive – Maybe your problem is the people who are supposed to be delivering reinforcers and prompts to keep you on task. If this is so, talk to them about why they seem to be undermining your success and figure out a better path going forward. It could be that they do not like the “new you” and are punishing your attempts to engage in positive change. You may need to avoid these people while you are implementing your behavior modification plan.

Of course, there can be countless other issues that you may encounter. If you cannot adjust your plan before your three weeks of the treatment phase run out, that is fine. You might be anticipating continuing with your plan and if so, make the adjustments going forward. No matter, you will write about what you discovered when you evaluated your plan in your Results and Discussion section. Be advised that the means, percentages, counts, etc. that you use for the behavior and/or token economy will be described in the Results section of the final paper while your analysis and possible solutions to issues will be covered in the Discussion section.

15.2.2. Next Steps

Once you have figured out needed changes to your plan, implement them. Continue the process of evaluation and adjustment until your plan works or you just cannot seem to reduce the unwanted behavior or increase a desirable one.

Assuming your plan works, and the behavior has changed in the hypothesized manner, you will proceed to the maintenance phase.

15.3. Generate a Graph of Your Data

Section Learning Objectives

• Create a graph of your treatment phase data and submit to your instructor for feedback by the due date in the course schedule.

Using what you learned in Section 15.1.3., generate a graph of your data so far. You will need to submit a line graph for the final paper, and this can help you get this process going. If you are not finished with your treatment phase, create the spreadsheet from which the graph is derived and add additional data until you complete the third week.

Turn your graph in to your instructor for feedback by the due date in the course schedule. This task is worth 5 points.

The Generate a Graph of Your Data Assignment can be found in Appendix 1: Self-Management Plan Documents, at the back of this book.

15.4. Treatment Phase Token Log

Section Learning Objectives

• Record the number of tokens you earned daily during the treatment phase.
• Calculate weekly totals and plan totals for the number of tokens earned.

As part of your behavior modification plan, you have been recording the number of behaviors made and subsequently the number of tokens earned per your token economy. The tokens show the success of your plan as much as any of the analyses described above and so are another DV in the study.

Turn your token log in to your instructor for feedback by the due date in the course schedule. This task is worth 5 points. You can also graph your tokens earned to date for the treatment phase and submit this as well for feedback, though note that it is not technically due at this point.

The Treatment Phase Token Log can be found in Appendix 1: Self-Management Plan Documents, at the back of this book.

Module Recap

Module 15 discussed the important issue of evaluating your plan and making adjustments as needed. We discussed hypothetical situations and what you could do about them, and methods to numerically summarize your data. We also discussed how to graph your data to generate a visual representation of its progression. Up next – maintenance and relapse prevention.