This is one of those questions where, no matter how specific you get, you are best served seeing a professional in person. The issues are still too vaguely described to offer specific advice.
I outlined some of my thinking in this answer. Briefly, I am not sure how much a medical doctor would help. Now, that said, the medical discipline physiatry deals with all issues related to physical medicine and rehabilitation, so if you see an MD it should definitely be a physiatrist. If you had something like a joint replacement or serious orthopedic injury, you should often get referred to one of these folks after the acute phase. They will often work with physical therapists. You can often go to a physical therapist directly, your insurance arrangements permitting (this includes your national insurance's policies, if you're in a country that uses that). I suspect many people don't know this, but this is a potentially productive route.
Most physical therapists won't have sport-specific training. So, they may miss some stuff related to how your bicycle fits you. That said, knee pain can stem from or be aggravated by muscular imbalance issues. PTs definitely can help you determine which muscles to train or stretch.
Bike fitters are not licensed like MDs and PTs. (Mostly not, but you may encounter occasional fitters who also have PT licenses.) They deal with how your bike fits you. Moreover, good bike fitters also know how to assess your flexibility and range of motion, and they too can recommend stretches and strengthening exercises. I'm not sure how common this type of crossover knowledge is among fitters, however.
Ideally, I'd start with a bike fitter who can also advise on physical issues. They might say this is beyond my capability and ask you to concurrently see a PT. You could find both a PT and a bike fitter separately, but try to keep them apprised of what the other is recommending.
As to component-specific issues, here are some possible comments. In general, triathlon favors shorter cranks like 165mm or shorter. At 5' 4", many road bikes come stock with 170mm or 165mm cranks. You're probably in the right size range, but shorter ones exist. If your threshold power is 114W, then in the 50t ring, you are probably at low cadence. In general, if you have osteoarthritis, I suspect you want to try to aim for higher cadence. You might be better off staying in the smaller ring. If you find the cadence in that ring to be too high, then you might be able to get aftermarket big rings that have 46t or so. That might be worth considering. It might be worth considering a sub-compact crankset, e.g. a GRX 600 46-30 crank, but those are made for gravel bike chainlines, and you might also need to change the front derailleur (but it's not guaranteed that you have to).
So, unfortunately, it does appear that you will have to throw some money at the problem - most of which should involve finding out exactly what the problem is. I mention component issues not to encourage you to change cranks, but to alert you that this is a possible issue.
