You don't need anyone's approval. You and your friend must share a very special understanding, being in a similar situation. What happens between you is entirely up to you and nothing to do with anyone else. No one should judge you. I hope you both get comfort from your shared time together.