The Ha Nhi have memorized their genealogy.  It is very important that each person can recite a list of ancestors back to the progenitor of their tribe, Sm Mi O.  These lists now contain more than 60 generations.  Children are taught to recite their ancestors' names from a very early age.  Young people cannot get married unless they can recite at least seven generations before reaching a common ancestor.

According to their legends, the heavens and the earth were brought into being by a creator Apoe Miyeh (sometimes described as a "force" rather than God).  Nine powerful spirits followed the initial creation, each engendering the next:  G'ah Ne, Ne Zaw, Zaw Zeu, Zeu To, To Ma, Ma Yaw, Yaw Neh, Neh Beh, and Beh Sm.  Beh Sm gave rise to Sm Mi O.  Ha Nhi names are arranged in this way to this day, with the second part of the father's name becomes the first part of the child's name.

The Ha Nhi came from Tibet, moving south and east through China many centuries ago.  They have legends that their people came from frog's eyes.  But they also say that they once lived in 1,000 homes on a plain far to the north.

They also have a legend of a lost book, which they say was given to them by the Creator God.  It was written in their own script on buffalo skin.  But on a long journey, their ancestors got hungry and ate the book.  Ever since, they have had no written language.  But they say that they still have the wisdom in their stomachs.

There are records of at least a few Ha Nhi living in Northwestern Vietnam as early as the 8th century.  But the majority of Ha Nhi are believed to have come to Vietnam within the last three centuries.