Deleting an object removes it from the file. Deleting an instance of an object on the Stage does not delete the symbol from the library.
Press Delete or Backspace.
Select Edit > Clear.
Select Edit > Cut.
Right-click (Windows) or Control-click (Macintosh) the object, and select Cut from the context menu.