The difference is that the BJT is not designed symmetrically. The details about the doping and dimensions of the silicon that make the collector are different than those that make the emitter.
In practice, this means the current flowing between collector-emitter responds much more to the voltage and currents between base and emitter rather than the the base-collector. Therefore, the the base-emitter is used to control the BJT rather than the base-collector.
You don't always connect the load to the collector, but you do if you want to use it as a switch (i.e. full conduction or zero conduction) because it means that as the voltage drop across the load changes due to changes in current flow or whatever other reason, it does not interfere with the voltage across the base-emitter so it is simpler to control.
If you connect a load at the emitter then the voltage drop across the load does interfere with the control signal being applied between base-emitter and sometimes you want that because it can be used as negative feedback (an amplifier or buffer which does not fully conduct).
For example, if you connect an LED load at the collector you need a current limiting resistor because the BJT acts as a switch that either fully conducts or does not conduct at all. The voltage drop across the LED cannot interfere with the what's going on between the base-emitter which is used to control the BJT.
But if you connect an LED at the emitter properly, you can have the LED voltage drop properly influence the base-emitter voltages/currents that control the transistor so it partially conducts and limit the current through the LED which means you can omit the resistor. Of course, now the heat is being dissipated in the BJT instead of the resistor.