Rational choice theory has become an important perspective in many of the social sciences. The essence of rational choice theory is presented quite straightforwardly in this book by Shepsle and Bonchek. Indeed, I have placed this book on reserve in some of my classes, to introduce students to basic concepts and assumptions of the perspective.
The basic starting concepts include: (a) preferences, which represent various of humanity's want and needs; (b) self-interest, by which people behave according to their preferences. Making decisions occurs under conditions of uncertainty. As the authors put it, (page 18), "More often than not, individuals may not have an exact sense of how an instrument or behavior they might adopt relates to the outcomes they value." Beliefs, another key concept, ". . .describe the hunches an individual has concerning the efficacy of a given instrument or behavior for obtaining something he or she wants. . . ." (page 18). Beliefs, then, connect instruments to outcomes. When a person acts in agreement with both preferences and beliefs, the end result is "instrumental rationality."
For a rational choice theorist, any choice is rational if it is in agreement with one's preferences. People develop a preference ordering, with some ranking higher than others. When one uses higher ranking rather than lower ranking preferences to determine what action to take, the result is "maximization" of results. Shepsle and Bonchek put it thus (page 34): "Making a decision under conditions of risk involves choosing from among alternative lotteries. A rational choice entails choosing the 'best' lottery. The rule of rational choice is known as the Principle of Expected Utility."
In essence, one would follow this principle by selecting the choice that maximizes payoffs in terms of one's preference ordering. Rationality calls for people to make decisions on the basis of expected utility.
The key underlying assumption is that people can carry out analyses consistent with the principle of expected utility, engaging in a sort of rational means-ends analysis.
While this is a major theoretical orientation now, it is subject to considerable criticism as well. There are many well done critiques "out there." Nonetheless, if one wants an introduction to rational choice theory, this is a pretty good starting point.