Once you’ve waited long enough you never want to leave. The trick is to leave fast enough that you can’t see the thing you were waiting for arrive. Though there’s never any way of knowing if the elevator is going to come once your hand hits the doorknob or if the subway car is going to come when you’re halfway up the stairs. Point of advice, taking a cab when it’s really late at night is worth the extra expense.