You should distribute complete libraries architectures not mixing them.
All you native library should complete 64bit collection, 32bit collection, x86 collection.
ie.: you can't have partial 64bit support.
You should know that the Linux kernel would try to instantiate a 64bit space to run your application and it's unable to run some 64bit lib and some 32bit lib.
If your third part libraries, apart of our, are only arm 32bit you should remove all the 64bit and x86 files.
Note about x86: some devices (ie.: Asus) devices own an arm 2 x86 translator builtin.
Performances wouldn't be so good as running native x86.