Is there any specific definition of position of equilibrium?
No, it is a fuzzy concept and not very helpful. When a reaction is at equilibrium, there is a certain composition of the equilibrium mixture. However, this composition is not unique. There are many other compositions (sets of concentrations) that are also at equilibrium.
I will use a simple reaction to illustrate this:

The concentration of the reactant $\ce{N2O4}$ is plotted on the y-axis and the concentration of the product $\ce{NO2}$ is plotted on the x-axis. The blue curve shows reaction mixtures that are at equilibrium. If I start with pure $\ce{N2O4}$ at a partial pressure or concentration of 10 (arbitrary units, just for illustration), the reaction will proceed on the red straight line (slope -1/2 because of the stoichiometry). The reaction will proceed net forward until it reaches equilibrium, marked with the circle. However, if I initially have 8 (purple straight line) or 12 (green straight line), I will reach different equilibrium compositions. I can't reach the circled composition because I don't have the right number of atoms.
Or is the term only used when you are saying "the position of equilibrium is shifting right/left" to mean that the forward/reverse reaction is happening at a higher rate for the moment?
"The position of equilibrium is shifting right" should be replaced by "the reaction is going in a net forward direction" until it reaches equilibrium. I will use the same example to illustrate this:

In this example, we start out at equilibrium on the red line, and add some product. We are pretending the reaction is turned off as we add the product, so the reactant concentration stays the same. Then (like in a cartoon where the character notices they stepped off the cliff), we pretend the reaction is turned on again. It is not at equilibrium, and we see a net reverse reaction. You could talk about this net reverse reaction as a shift to the left, but it is describing the reaction, not the equilibrium. Equilibrium states are defined by the blue curve, which does not shift.
You could also add reactant (half as much), as the next panel shows:

This time, there is a net forward reaction, and you end up at the same equilibrium composition (because the reaction moves along on the same green line, constrained by stoichiometry).
Below are the other two cases involving removal of reactant or product. Again, I chose the example such that both cases have the reaction composition constrained to the same line, purple in this case.


Why do we say the position of equilibrium is shifted even when K is the same?
I don't know why, but we shouldn't. We should say that when we disturb an equilibrium by adding or removing a single reactant or product, the reaction will go in the direction of re-establishing an equilibrium (net forward or reverse, depending on the details). Notice I say establish "an equilibrium" because there are multiple compositions (or sets of concentrations) that satisfy the equilibrium condition. After adding or removing a single species, you can not return to the same equilibrium state (because the number of atoms changed).