Is it simply a matter of deleting the file from the Rocksmith 2014 dlc folder, then it's gone from the game, too?

 

I'm thinking of any behind the scenes things that may not be readily apparent.

yes.

Yep, delete the file and it's gone from the game. You can even separate songs by let's say artist and put them in subfolders inside the dlc folder. You could then remove those folders to say your desktop and have fewer songs in game. Remember, it will need to enumerate all new songs at game start though, so it may take a minute or 2 for a file full to show up in game.

I have a folder on my desktop that has all my DLC on it.  I select the files that I want to play and move them over to the dlc folder.  I have a shortcut on my desktop to make it easy as well.