In Jewish culture at that time, it was practice to refer to someone by their father’s name and not their mother’s name. This seems to be true even if the father has passed.1 As a result, it is puzzling to find the people referring to Jesus by his mother’s name here in Nazareth. The reason could still be that Mary was known to them (and Jesus' father passed away), but it seems likely that calling someone the son of a woman, in this context, is definitely a form of disrespect and an insult. Therefore, there is already an indication of the mood shift of the people of Nazareth from listening to Jesus to rejecting him.
3 Is not this the carpenter, the son of Mary and brother of James and Joses and Judas and Simon? And are not his sisters here with us?” And they took offense at him.