I think this can be filed under the “feasible, not advisable” department. Hence why your FSO doesn’t want to say yes or no. It really isn’t up to them and you will see more experienced people give an equally unsatisfying answer of “it depends.” I know that isn’t what one wants to hear. Can it put a clearance at risk? Yes. Is this guaranteed? No. Is it a problem you likely don’t need? I would go with yes. How often do you travel making it more worthwhile to avoid customs border clearance hassle vice having your loyalty to the US questioned? Let’s assume you have a great vacation plan and get 5 weeks vacation, travel for all of it. Is it worth risking what makes up the remaining 47 out of 52 weeks of your life?
Analogy: my RV gets 6 mpg. I studied feverishly to figure out what might get me 9 mpg. Thousands of dollars were required. I would never come close to recouping the savings in gas cost. Ever. That pretty much ended that research project for me. I travel maybe 6 or 8 times a year. So I put up with crappy mileage 6 to 8 times a year instead of invest thousands I would get zero return on. For the record, no it wasn’t me in the left lane tolling along at 50 MPH in front of you. I learned I can go 70 and still get 6 mpg.