This can be done one by one, or multiple categories at once.
One by one - go to Settings > Manage Categories, click the pencil icon on the right of the relevant category, the choose which Departments to assign this category to in the Accessible Departments menu.
Or, you can go from the other direction from Settings > Manage Departments, click the pencil icon, and choose which categories that Department will use.
Multiple - if you have numerous categories that need to be Assigned (or Unassigned), you can go to Settings > Manage Categories again, but this time check all the Categories you wish to Assign/Unassign, then click the Assign or Unassign button at the bottom of the screen.
You won't be allowed to Unassign a category from a department where there is data in that category.
Smart-Unassign - This button will go through every category and every department for every budget/actual data, and where there is no data in any budget for a particular category in a particular department, it will unassign that category from that department.