I don’t think this really holds up, for example helium itself is chemically inert and not toxic. The main risk from inhaling helium is probably oxygen displacement at a push.
Millions of people have been using inhalers to control asthma too, this well studied and agreed to be safe. This is just off the top of my head.
I imagine if you inhaled helium several times a day for decades that it would also mess something up.