When The Correct Answer Isn’t Right

WordPress has an interesting feature that shows searches that have lead to your blog. Recently I saw a search for “what if the correct answer isn’t the right answer?” This search resonated with me because I have recently run into the same problem with a scheduling MIP model I am developing for a client. The model’s objective is to schedule as many of the available tasks as possible to minimize total cost while staying within resource constraints.

The basic functionality of the model is complete and we have started user testing with a power user to help us fine-tune the results. The model is generating good results using production-quality data, but one complaint we keep hearing is that certain task sequences desirable to the users are not found in the optimal schedule. So while the model is generating the correct answer based on its objective, constraints, and input data, the users are not 100% happy with the results. So what’s an Operation Researcher to do?

My first instinct was to try to prove to the users that the model’s globally-optimum solution is better than a schedule built around the desired task sequences. After all, if these sequences were really the best way to schedule these tasks, the model would choose them. Maybe I could show how far from optimality a schedule containing the desired sequences could be. I might even be able to show the effect each sequence has on the overall schedule. But this type of analysis is difficult to perform and even more difficult to present to users.

And even though they are presented with facts and data, users may still find it difficult to accept the model’s solution. This is a key issue because the success of a new model often comes down to one thing: Do the users accept the solution? If they don’t, no matter how fast, optimal, or cutting-edge the model is, the users are likely to work around it rather than with it.

Rather than trying to beat a square model into the users’ round hole, I decided to take a step back and reconsider some of the key aspects of the model, like the objective function and the constraints. Over the next few posts I’ll walk through a few of these keys areas that I took another look at as I continued to fine-tune my model towards my users’ needs. Hopefully I’ll find some ways to generate an optimal solution that will be more acceptable to the users.

Advertisements
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

4 Responses to When The Correct Answer Isn’t Right

  1. Paul Rubin says:

    Two things immediately come to mind: asking the users why the missing sequences are so desirable; and cranking out best-possible solutions with and without the constraint that some or all of the preferred sequences are used, then asking the users which they prefer (and explaining that the ones containing their pet sequences are the best they’re going to get with them).

    Any chance that some (all?) of the push-back is users wanting to have a voice in the decision, rather than just taking what “the computer” tells them?

    • Paul, those are great points. I think your first point (why are the sequences so desirable?) highlights something I left out of my post that could use some clarification. In general each task has a sequence-dependent setup time. But tasks sequenced together have zero setup time, hence the desire to schedule them together. Since the model is minimizing total cost (where setup time is a cost component), one would think the model would always choose task sequences with zero setup time. But there are other cost components as well which influence the scheduling. Comparing a schedule with and without these sequences could be beneficial to the users.

      You are also absolutely right about the users wanting a voice in the process. Both for this project and previous projects, I have developed and implemented models for users who have months or even years of experience doing the exact job the model will do. It is natural for the users to want to highlight gaps in the model’s abilities compared to their own abilities. In this situation we always try to emphasize new and more interesting responsibilities the users can take on now that the model is performing the more mechanical tasks for them. Sometimes people think that Operation Researchers just want to replace everyone with a machine, but in reality OR often allows them to move on to more challenging and satisfying work.

  2. Pingback: When The Correct Answer Isn’t Right Part 1: Objective | Reflections on Operations Research

  3. Thiago Serra says:

    That’s a big issue in real world projects.
    I can’t remember one in which the end user did not complain about what the system was suggesting him to do.
    Also, it has to do with many subtleties of the problem that the client usually forget to tell at early phases of the project.
    For instance, I had this case in which certain schedules were unacceptable to the client but they were afraid of forbidding them.

    I’ve written about simmilar issues on my blog, as in the following post:
    http://tatu-search.posterous.com/integer-programming-is-smarter-than-sexy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s