I think it all depends on the situation. But it is disrespectful to walk out on someone or ignore them when they are speaking to you. But let's say they got called on an emergency and lost attention on you before leaving. It will be disrespectful but understandable. So the context is important when judging right or wrong.