Three vital issues to consider when deciding which data collection method to use are reliability, validity, and practicality. Reliability has to do with the consistency of results and the freedom from error and bias in a data collection method. A method that has little or no error or bias is highly reliable, whereas the results of a method that has significant error or bias is unreliable and cannot be trusted. Decisions based on unreliable information are likely to be poor ones.

Another issue to consider in selecting a data collection method is validity. Validity is concerned with whether the data collection method actually measures what we want it to measure, that is, are we hitting the right target? Reliability and validity are complex issues, and assessing them often requires knowledge of statistics and measurement concepts.

In addition to being reliable and valid, data collection methods must also be practical, given the constraints of an organization. Practicality concerns how much time, money, and resources are available for an evaluation method. Realistic and creative trade-offs can ensure that an evaluation effort is carried out and yields useful information.