Nope. The initial evacuation of Catholic Vietnamese from the north to the south in the first six months numbered a little over 300,000 people. About a quarter million followed by the end of 1955, with others moving in surges between 55 and 60. Migration continued even up to the mid 60s.
The majority of those moved in 1954 were French citizens, soldiers, and sympathizers. Which sort of mirrors the Fall of Saigon and the evacuation immediately after that: those who cast their lot with the loosing side wound up as refugees pretty quickly.
	
	
		
		
			I agree with you about the faction being the most ruthless which were the VietCong.  But it was estimated that in 59 their numbers were only about 3,000.
		
		
	 
That's mainly because in 1959 the term "Vietcong" was an umbrella term for the 10 to 15 thousand former Viet Minh and sympathizers who had remained in the south after the French partition. It grew in size and strength -- and enormously at that -- in reaction to U.S. involvement and the consolidation of disparate independence movements that until then had no common ground to speak of.
Kind of amazing how quickly the arrival of a common enemy makes former rivals into allies.
	
	
		
		
			They didn't want the unification under Ho Chi either.
		
		
	 
They didn't want Ho Chi Minh to 
win the election, no. And when it became clear that that was extremely likely to happen, Diem declared "No election for you!" and kept the partition alive for years after it was set to expire.
It's pretty much the same move the South pulled when Lincoln was elected. They couldn't stand the rule of an abolitionist, so they decided they couldn't be part of any country that would have voted for him in the first place. Interestingly, the end result -- despite U.S. interference in the conflict -- wound up being exactly the same in both cases.
	
	
		
		
			It would be like the UN saying that all the slaves were allowed to go north before the civil war and then a year later the whole country would have slavery.
		
		
	 
Well, no, it would be like a the Confederacy seceding from the Union and then saying they would rejoin if they held a new election, then changing their mind once they discovered that Lincoln would probably win the new election too.
By way of comparison: the amount of damage caused by the Vietnamese on their OWN society was ultimately far less than was caused by the U.S. attempts to suppress the communists. North Vietnam did not have anywhere near the firepower, the manpower or even the MOTIVE to wreak the kind of destruction on the South that the U.S. did, nor for that matter did the South have the ability to visit that kind of destruction on the communists. In a side by side comparison, between the U.S. war effort no the one hand and the "Unification under Ho," along with the reeducation camps, the purges, the land grabs, the often violent suppression of Catholic dissidents on the other hand, there is NO QUESTION that unification under Ho Chi Minh would actually have been the lesser of two evils. The main reason for this is that the U.S. war effort was ultimately unsuccessful and communist unification happened ANYWAY. It was, for all intents and purposes, an inevitability. The price the people of Vietnam paid for that transition in terms of blood and misery was at least two orders of magnitude higher than it should have been.