The short answer is that you can choose to not go forward at any point with the Fair Hearing.  The best practice is to put that in writing to the Fair Hearing office and the local area office.   It usually does not have a negative impact on your case.  It can always depend on the circumstances but then realizing that if you chose not to go forward with your Fair Hearing after it has been scheduled, you would likely not be able to request a new Fair Hearing.

There are also points where there is a procedure in order to continue a Fair Hearing if you need more time for cause and there is a process to continue the hearings themselves.