There are instances in which Congress has created significant limitations on the discretion of agency officials and others where officials possess considerable discretion. As you can imagine, it is much easier for a court to review an allegation that an agency has not complied with a rule than it is to review an allegation that an agency has abused its discretion. In particular, agency decisions not to act are rarely successfully challenged. It is when an agency acts that it is most at risk of reversal or of civil liability.
Often when agencies act informally, there are few or no procedures to protect individuals. Discretion may be either unfettered or hampered by only minimal constraints. Therefore, questions of fairness are common in this area, and due process is often at issue. Judicial review of discretionary decisions, however, is limited. Be aware, however, that the law (as enforced by the judiciary) is not the only limitation on agency discretion. Political accountability is also a limitation. Angering Congress, the president, or the public can have detrimental consequences for an agency.