#but he wasn't yet mayor when fantine came along

[alicedrawslesmis]

We tend to skip this part of Valjean because we do like to focus on his better traits in fandom (or on his self-loathing) but he's also very sexist asdfghjkl

and this presents itself in the way that he's constantly trying to spare women from danger to the point of fault. He separates the genders in his factory and he separates Cosette from the world and then later from the truth (and what him and Marius are up to etc). He feels himself responsible for preserving the innocence of women. This is probably due to his past, having to provide for his sister and her kids all of his life. But this is also just a character trait of his, that remains even after Fantine dies:

Father Madeleine required of the men good will, of the women pure morals, and of all, probity. He had separated the work-rooms in order to separate the sexes, and so that the women and girls might remain discreet. On this point he was inflexible. It was the only thing in which he was in a manner intolerant. He was all the more firmly set on this severity, since M. sur M., being a garrison town, opportunities for corruption abounded.

This is what does Fantine in (much more than Victurnien's probing later, btw. I have Opinions on the way the fandom talks about her. But we'll get there when we get there). He could very well not have that rule and Fantine would not have lost her job. Like, in France at that time nobody cared if the factory worker had a bastard child, this is a factory in the early industrial revolution, they employed anybody willing and able to work. This included children and single mothers. The purity rule was Valjean's idea and weird hangup.

I think this is also part of Hugo's point. That this attempt to separate men and women and then to also cast away women who are corrupted from society, and close monitoring of everyone's morals, is bad. It doesn't protect people, and makes life worse for sex workers and for just about every woman who doesn't behave as a perfectly chaste virgin

(obviously there are good reasons why he made that choice, it's not like it was made in a vacuum. But it does place unreasonably higher standards for his female workers and in turn it's what makes Fantine's story take the biggest turn into tragedy)