Schleya was once a part of the wicked path. Rokushi Mirai could also be said to be from a similar background. They knew what the likes of totems meant. In essence, it meant a cursed object made from flesh or bone, containing the karma from the remains. It could be used to obtain power through vile methods, so it was generally considered a wicked path method.
In that regard, it was also similar to how a body cultivator uses inscriptions on their own body to strengthen itself. Ultimately, totems were considered external power. Although the power they gave was large, it was limited and would be spent within a few bouts, so it wasn't largely used other than to guard some sacred or unholy areas and for other blood sacrificial rituals.
Bodily inscriptions were used much more, so these kinds of totems were put on the back burner. At the very least, even wicked people wouldn't choose it as their first option.