Previously started process for new clearance--not sure I want the position anymore but worried about backlash

I was offered a position with a private defense contractor working on a contract with the Agency (specifying for a reason.) I was unsuccessful crossing my clearance, so we were submitting for a new one and finally my polygraph has been scheduled after being delayed due to COVID.

Problem: I’m not sure I want the position anymore because the schedule is insane and I’ve recently gotten used to a M-F/9-5 schedule. I’m fairly confident the private contractor will understand if I back out now, but I’m not sure the Agency will.

This might be a stupid question, but does the Agency care if you end a clearance process before the polygraph? Will I be blackballed from them in the future since I’ve already submitted an SF-86 but not taken a security interview? Will it cause any issues in a future clearance process or CE down the line if I stay with my agency or pursue another position with the Agency? I know some federal agencies can be testy, but I’m not sure if the Agency is one of them.

This sort of thing happens all the time, especially when it can take months to years to get a clearance. I wouldnt worry too much about it.

I can’t speak from experience, but I would imagine that if they were going to be pissed about you backing out, they’d rather you back out before the polygraph and interview rather than after (less time and money spent on their part).

This is just a hunch on my part.