In Outlook 2003 you needed to go to Tools | Options >> Other >> Advanced Options >> Add-In Manager... and COM Add-ins...; Close and Re-open Outlook to remove various Add-ins. Unfortunately if you try that in Outlook 2007 you wont find the same choices.
In Outlook 2007 go to Tools | Trust Center... >> Add-ins and you can see what Add-ins are installed, and of those which are active and which are inactive. Select COM Add-ins in the Manage drop-down and click Go.... The COM Add-Ins screen will pop-up. Uncheck the Add-Ins you want to disable (check those you want to enable). When you are completed, click OK. Close and Re-open Outlook for the changes to take affect.
Once you're satisfied that by disabling your Add-in, there was no negative effect, you can return to delete the Add-ins. Go back to Tools | Trust Center... >> Add-ins >> COM Add-ins (Go...). From the Comm Add-Ins screen, select the Add-in you want to delete and click the Remove button.