Singletons allow you to limit creation of your objects
This is true, but now you are mixing two different responsibilities into the same class, which is a violation of the Single Responsibility Principle. A class should not care whether or not it is a singleton. It should be concerned with its business responsibilities only. If you want to limit the ability to instantiate some class, create a factory or builder object that encapsulates creation, and in there, limit creation as you wish. Now the responsibilities of creation are partitioned away from the responsibilities of the business entity.
Singletons promote tight coupling between classes
One of the underlying properties that makes code testable is that it is loosely coupled to its surroundings. This property allows you to substitute alternate implementations for collaborators during testing to achieve specific testing goals (think mock objects). Singletons tightly couple you to the exact type of the singleton object, removing the opportunity to use polymorphism to substitute an alternative. A better alternative, as discussed in the first point above, is to alter your design to allow you to pass references to objects to your classes and methods, which will reduce the coupling issues described above.
Singletons carry state with them that last as long as the program lasts
Persistent state is the enemy of unit testing. One of the things that makes unit testing effective is that each test has to be independent of all the others. If this is not true, then the order in which the tests run affects the outcome of the tests. This can lead to cases where tests fail when they shouldn’t, and even worse, it can lead to tests that pass just because of the order in which they were run. This can hide bugs and is evil. Avoiding static variables is a good way to prevent state from being carried from test to test. Singletons, by their very nature, depend on an instance that is held in a static variable. This is an invitation for test-dependence. Avoid this by passing references to objects to your classes and methods.