To remove a custom added token, go to the "Settings" > "Assets" screen and locate the custom token in your assets list that you would like to remove. Then click the red [x] in the custom token's box to remove it.
To instead enable/disable an asset, simply click on the token's box to toggle its status.
Note that natively integrated assets cannot be removed, however you can disable them if they have no balance. While imported tokens can be removed even if they have balance.
Removing a custom added token with a balance does not remove the balance from the chain, it simply removes the token from being displayed in your Infinity Wallet. If you have removed a custom token in error, you can always add it back again and its balance will be restored.