Yes, it really depends on your current situation and goal. Sometimes getting a higher damage or defense as a prey may be a better choice in order to ensure a more profitable hunt instead of picking up the loot prey. The same goes to the aspects of your hunt being one in a team or alone.
Particularly, I like to have always an exp prey or a loot prey avaialble, specially to use it when you are a person who usually hunts alone. I had immense success using the loot prey on Fire Devils (I was shocked by how it worked) with a low level paladin, but the same loot prey wasn't that great with dwarf guards using my knight (got several leather boots). The thing here is to balance the prey to a good creature that will generate most exp/loot return and try to figure out what your character needs most for the hunt. Sometimes it is damage per hit taht you elevate your profit or exp, other times you need to take care if you apply damage from distance (requiring the defensive prey)... It all really depends. Other events on world that increases exp may have you inclined to preserve an exp prey for a solitary hunt, but again, sometimes dealing more damage makes you amass higher amounts of exp.