Only if you use too much. I use Neatsfoot oil on every holster I make after molding and before the final finish. This replaces the natural oils drawn out of the leather in the dying and molding processes. After that an annual thin coat of oil is plenty, brush it on and then wipe off the excess with a paper towel. Do not saturate the leather or it will soften the leather. Neatsfoot oil is my preferred leather treatment since it's cheap and I always have it on hand.
-Infidel