Foreign Keys give another layer of safety. They provide data integrity on data level. They prevent you from adding a record which does not fulfill the relation.

For example, if you have a table users with three Users (IDs 1, 2 and 3) and another table entries, where entries.uid has a relation to users.id, this means you can't add an entry to entries where you would add an entry.uid of 4, if you don't have an user with ID 4 defined.