Yes it does sound a bit absurd.. How can a 'brother' be a daughter? It seems to me that 'I'm so sorry, I thought your brother and sister were you children' was meant, but then it becomes a sort of speculation..
I think the writer is not a native speaker and meant to say "I'm so sorry I thought your brother or sister was your daughter". I'm guessing that he did not know the sex of the child.