Validations on orders are dependent on the order state so maybe it just isn’t being triggered? I am fairly sure that some kind of error will be encountered when trying to process an order to completion if it doesn’t have a shipping method, though it may not be a validation error, I can’t recall…