You do most things in bash by issuing commands like cp. Most of these commands are
small programs, though some, like cd are built into the shell.
The commands come in packages, most of them from the Free Software Foundation (or GNU). Rather than list the packages here, I'll direct you to the Linux From Scratch HOWTO. It has a full and up to date list of the packages that go into a Linux system as well as instructions on how to build them.
One of the best things about Linux, in my humble opinion, is that you can get inside it and really find out how it all works. I hope that you enjoy this as much as I do. And I hope that this little note has helped you do it.