With all of the other stuff out of the way, you're ready to yank the injector. Unfortunately, many people really do try to yank them out. Remember, your fuel injectors sit right in the middle of the delicate are of the engine, so any piece you break off might fall in, and this equals big pain in your fanny.
The safest way to remove a fuel injector is to use a fuel injector puller. There are other ways to remove fuel injectors, but if you aren't careful you could damage some expensive fuel injection components. Besides, the injector puller makes it so easy, just slide it underneath the lip in the injector housing and pop the injector out.
Your fuel injector goes into what amounts to a hole in the intake manifold (or your head if you're lucky). When you remove the injector, you leave this hole open. Be very careful to keep anything from falling in. This can be a real chore to fix.
In the immortal words of every auto repair tutorial, installation is the reverse of removal! You'll need to lubricate the new injector seals before they go in, and remember to go easy, don't force anything in.