First, Export the product list. It is recommended you only keep the products you want to update and remove the rest from the exported list.

Add a column to the file named Custom_UnitPrice_{unit} and set the price you want. You can set the price as -1 if you want to unset the overridden price.

Eg: If you have a unit called kg, to update it add a column called Custom_UnitPrice_kg. Make sure the unit name matches exactly.